system

Abstract

We provide the system. [Solution] A means for detecting newly installed artificial intelligence technology, A means for collecting the operation logs and generated products of the aforementioned artificial intelligence technology in real time, A means of analyzing the collected data and determining whether or not fraudulent activity has occurred, A means for issuing a warning when the aforementioned fraudulent activity is detected, A means of providing guidance to users to take countermeasures based on the information of the aforementioned warning, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] While conventional artificial intelligence technologies provide many conveniences to users, there is also a possibility of taking illegal actions. In particular, there are risks such as malicious AI being installed without the user noticing, leakage of personal information, execution of phishing fraud, or virus infection of the device.*

Means for Solving the Problems

[0005] This invention includes means for detecting newly installed artificial intelligence technology and collecting its operation logs and generated data in real time. Furthermore, it includes means for analyzing the collected data and determining whether or not fraudulent activity has occurred. If fraudulent activity is detected based on the analysis results, a warning is promptly issued, and the user is notified visually or audibly. This provides the user with guidance to recognize the problem and take appropriate countermeasures. By integrating these means, this system reduces the potential threat of AI technology and realizes a safe user environment.

[0006] "Newly installed artificial intelligence technology" refers to an artificial intelligence-based application or system that has been newly added to a user's device and has not yet undergone security evaluation.

[0007] An "operation log" is data that records various operational events and process states when artificial intelligence technology is executed.

[0008] "Generated products" refers to all digital content, including text, images, audio, and other materials, generated by artificial intelligence technology.

[0009] "Means of real-time collection" refers to a system that has the function of instantly recording and collecting the operations and products of artificial intelligence technology as soon as they occur.

[0010] "Means of analyzing data and determining the presence or absence of fraudulent activity" refers to a system that enables a process of analyzing collected activity logs and generated data to identify potential fraud or risks contained therein.

[0011] "Means of issuing warnings" refers to a notification process and the function that executes it to inform users of the existence of detected fraudulent activity.

[0012] "Visual or auditory notifications" refer to methods of communicating warnings to users using visual displays (e.g., pop-ups or dialog boxes) or audio notifications.

[0013] "Means of providing guidance to users" refers to a system that guides users through detailed explanations of detected fraudulent activity and countermeasures. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention provides a monitoring system to enable the safe use of artificial intelligence technology. Specifically, it monitors newly installed artificial intelligence technology on a user's terminal, analyzes its operation and output to detect potential malicious activity, and issues a warning to the user. A specific embodiment of the system is described below.

[0036] Operation on the device

[0037] The device automatically detects when a user installs new artificial intelligence (AI) technology. Upon detecting the AI ​​technology, the device registers it for security monitoring and begins collecting operation logs and generated data. An agent running on the device prepares to send the collected data to the server.

[0038] Server roles and processing

[0039] The server receives data sent from the terminal and performs analysis. During the analysis, it checks whether the AI-generated content and communication content match known malicious patterns. For example, if the text generated by the AI ​​contains suspicious links or phrases that suggest phishing, the server identifies them and performs a detailed information and risk assessment. Based on these results, the server generates necessary countermeasures.

[0040] Notifications and feedback to users

[0041] Users receive analysis results from the server on their devices. This includes the details and risk level of any detected fraudulent activity. Based on this information, users take the measures recommended by the system (e.g., disabling the relevant AI model or applying corrective patches). The results of the measures taken by the user are fed back to the server, which is used to improve the accuracy of future analyses.

[0042] In this way, a system is realized that allows users to effectively manage potential risks while maintaining an environment in which they can use new artificial intelligence technologies with peace of mind. This invention makes it possible to enjoy the benefits of AI technology while minimizing the security risks associated with its use.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The device detects when new artificial intelligence technology is installed and collects basic information about that AI technology. This includes the AI's identification information and the date and time of its activation.

[0046] Step 2:

[0047] The terminal monitors the operation logs and generated products of registered artificial intelligence technologies in real time and collects related data. This data is stored securely and made available for subsequent analysis.

[0048] Step 3:

[0049] The operation logs and generated data collected from the terminal are sent to the server. The server receives this data and begins analysis.

[0050] Step 4:

[0051] The server verifies the operation of the artificial intelligence technology based on the received data. Specifically, it checks for matches with known fraudulent patterns and anomalous patterns based on past cases.

[0052] Step 5:

[0053] If the analysis reveals fraudulent activity or potential risks, the server performs a risk assessment and sends it as a warning to the terminal. This warning includes details of the detected problem along with recommended countermeasures.

[0054] Step 6:

[0055] The terminal receives a warning from the server and notifies the user. The user is informed of the warning visually or audibly, and is presented with recommended actions from the system.

[0056] Step 7:

[0057] Users take appropriate action based on the notified warnings. For example, they can take immediate action, such as temporarily suspending a suspicious AI model.

[0058] Step 8:

[0059] The terminal feeds back the results of the measures taken by the user to the server. This information is recorded on the server and used to improve the accuracy of future analysis processes.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] In recent years, artificial intelligence technology has been utilized in various fields, but its use has also increased the risk of fraudulent activity and malicious creations. Because such risks can cause significant harm to users, it is essential to use artificial intelligence technology in a safe and secure environment. However, conventional technologies are insufficient for real-time monitoring and fraud detection, necessitating more effective countermeasures.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for detecting newly installed artificial intelligence technology, means for registering the artificial intelligence technology as a target for monitoring, and means for collecting operation records and products in real time. This promotes the safe use of artificial intelligence technology and enables early detection and response to fraudulent activity.

[0065] "Detection" is the process of recognizing and identifying newly introduced artificial intelligence technologies in the user's system.

[0066] "Registration" is the procedure for adding detected artificial intelligence technologies to the security management system as targets for monitoring.

[0067] "Operation logging" refers to the process of meticulously recording a series of operations and resource usage when artificial intelligence technology is running.

[0068] "Products" refer to the output results generated by artificial intelligence technology, and specifically include generated images, text, and other similar elements.

[0069] "Encryption" is a technology that transforms the content of information to securely protect it and prevent unauthorized access during transmission and reception.

[0070] A "remote processing device" refers to a server or central management system that receives and analyzes information transmitted from a terminal.

[0071] "Analysis" is the process of examining received data in detail and comparing it to known patterns of fraudulent behavior.

[0072] "Anomaly" refers to inappropriate AI behavior or its products that could potentially cause harm to the system or users.

[0073] "Risk assessment" is a procedure that quantifies potential risks based on analysis results and determines their severity.

[0074] A "proposed course of action" refers to specific guidelines recommended based on the evaluation results, including safety measures that users should take.

[0075] "Instructions" are pieces of information designed to prompt users to immediately take the measures recommended by the system.

[0076] "Feedback" refers to the process where users report the results of the measures they have taken to the server, and this data is used to improve the accuracy of future analyses.

[0077] This invention provides a system that supports the secure use of artificial intelligence technology by linking a terminal and a server. This system has the function of immediately detecting newly installed artificial intelligence technology on a terminal and collecting its operation records and output in real time.

[0078] When a user installs a new artificial intelligence (AI) technology, the terminal automatically detects it and registers it as a monitored target within the system. For registered AI technologies, an agent begins operating to collect operation logs and product data. This ensures that data is recorded continuously and securely stored within the terminal.

[0079] The collected data is encrypted and sent to the server. The server passively receives this data and uses an analysis engine to determine whether or not malicious activity has occurred. Specifically, it checks whether the text information created by the generative AI model contains inappropriate links or suspicious phrases. The server processes this information using analysis software such as TENSORFLOW® or PyTorch and assesses its risk.

[0080] Users receive analysis results and estimated risk levels via notifications from the server. These notifications are provided visually or audibly on the device, offering specific countermeasures. These include disabling or changing settings for artificial intelligence technology and applying security patches as needed. The actions taken by the user are returned to the server as feedback, and this feedback information is used to further improve the accuracy of the analysis.

[0081] For example, if a user inputs the prompt "Draw a summer day scene in pop art style" into a generation AI model, a similar security analysis process will be applied to the generated image and text. This allows users to use artificial intelligence technology with peace of mind while effectively managing potential risks.

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] The terminal detects newly installed artificial intelligence technologies. Specifically, the terminal monitors program installation events and retrieves software metadata. The input is information about newly detected programs, and the output is the identification of AI technologies to be registered as targets for monitoring.

[0085] Step 2:

[0086] The terminal registers the detected artificial intelligence technology with the security monitoring system. This initiates the collection of activity logs for the monitored AI technology. The input is the identified AI technology, and the output is a notification that registration to the monitoring list is complete.

[0087] Step 3:

[0088] The terminal collects operation records and output data from the running artificial intelligence technology. Specific collection items include execution time, CPU usage, and generated data (e.g., text and images). The input is operational information from the AI ​​technology, and the output is the collected detailed logs.

[0089] Step 4:

[0090] The terminal encrypts the collected data and prepares it for transmission to the server. Encryption uses methods such as TLS to protect the data. The input is the collected, unprocessed log data, and the output is the encrypted data.

[0091] Step 5:

[0092] The server receives encrypted data sent from the terminal and processes it through its analysis engine. The analysis engine compares the data against a database of known malicious patterns. The input is decrypted, clear log data, and the output is the analysis result and whether or not malicious activity occurred.

[0093] Step 6:

[0094] The server assesses the risks based on the analysis results and generates notification information for the user. Here, a risk score is calculated from the analysis results, and a detailed report including recommended countermeasures is created. The input is the analysis results, and the output is the final evaluation report.

[0095] Step 7:

[0096] The terminal provides the user with notification information from the server. The notifications are displayed visually through the user interface (UI). Input is a detailed evaluation report from the server, and output is a warning message for the user.

[0097] Step 8:

[0098] Users take action based on the notification information. For example, they might temporarily stop using problematic AI technology or change its settings. The input is the recommended action from the server, and the output is the action that was taken.

[0099] Step 9:

[0100] The terminal collects the results of actions taken by the user and sends them to the server as feedback. The collected data will be used to improve the accuracy of future analyses. The input is the user's action result, and the output is feedback data.

[0101] (Application Example 1)

[0102] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0103] In recent years, applications utilizing artificial intelligence technology have rapidly become widespread, but the security risks associated with their use have also increased. In particular, the risk of generating malicious links and phishing attacks can pose a significant threat to general users. To address this challenge, there is a need for a reliable monitoring system that constantly monitors the operation of artificial intelligence technology, immediately detects suspicious activity, and recommends appropriate responses.

[0104] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0105] This invention includes a server that monitors operation logs and generated data and transmits them to an external analysis device via a communication network; a server that checks warning information from the external analysis device and presents detailed information and countermeasures if malicious activity is detected; and a server that collects the results after countermeasures are implemented by the user's specific device and provides feedback to improve the accuracy of the analysis. This minimizes the security risks associated with the use of artificial intelligence technology and allows users to utilize these technologies with peace of mind.

[0106] "Newly installed artificial intelligence technology" refers to AI-related software or applications that have been newly added to the user's device.

[0107] An "operation log" is data that records how artificial intelligence technology operates on a device and what operations are performed.

[0108] "Products" refer to the output generated by artificial intelligence technology during its operation, specifically such as text and images.

[0109] "Abusive behavior" refers to actions performed by artificial intelligence technology that could pose a threat to security and privacy.

[0110] A "warning" is a notification sent to inform users of the risks when fraudulent activity is detected.

[0111] "Guidelines for users to take action" refers to information that shows specific actions and recommended responses that users should take after receiving a warning.

[0112] "Monitoring operation logs and output" means tracking and recording the operation and output of AI technology in real time.

[0113] A "communication network" is a network infrastructure used to transmit data from a terminal to an external analysis device.

[0114] An "external analysis device" is a computer system that analyzes data transmitted from a terminal to determine fraudulent activity.

[0115] "Feedback" is the process of collecting the results of user-submitted actions and using them to inform future analysis and system improvements.

[0116] The system of this invention is designed to effectively implement new applications related to artificial intelligence technology, and is based on the premise of collaboration between a smartphone and an external server.

[0117] The device, in this case a smartphone, monitors the operation logs and output of the installed artificial intelligence technology in real time. The device uses a watchdog library to continuously detect the activity of the AI ​​application and, if any suspicious activity is detected, sends the information to an external analysis device. This allows users to leverage monitoring functions in their daily operations.

[0118] On the server side, a process of analyzing the received data takes place. The server uses the requests library to collect data from smartphones and thoroughly examines the content generated by AI technology. In particular, it identifies suspicious links and phishing words and analyzes them against known fraudulent patterns. Based on the results of this analysis, it assesses the importance and risk and presents the user with detailed warnings and solutions. The analyzed information is fed back to the user's smartphone, and the implementation of countermeasures and the results are used to improve accuracy in the future.

[0119] For example, if a newly installed AI chatbot app regularly sends links to extract personal information from a user, the device records the frequency and content of these links and sends them to a server. The server then analyzes the links in detail, and if it detects a potential phishing attempt, it can warn the user and instruct them to refrain from using the links due to their inappropriateness.

[0120] An example of a prompt for a generated AI model is, "Can you analyze and report whether a newly installed AI application is potentially phishing based on its operation logs?" Using this prompt, the system can provide more detailed information about the AI ​​model's operation and support appropriate security measures.

[0121] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0122] Step 1:

[0123] The device detects newly installed artificial intelligence technologies. The input is a list of installed applications, which is used to begin monitoring activity logs. The device uses the watchdog library to track file system changes in real time and identify newly added AI-related applications. The output is an identified list of monitored applications.

[0124] Step 2:

[0125] The terminal collects the operation logs and generated products of detected artificial intelligence technologies and prepares them for transmission to an external server. In this step, various activity logs obtained from the monitored application are taken as input, and the log information is stored on a temporary storage medium. The output is log data processed into a format that can be transferred to the server.

[0126] Step 3:

[0127] The terminal sends the collected data to an external server. The input is the processed data prepared in step 2, and communication with the server is performed via the HTTP protocol using the requests library. The output is a transmission completion status, indicating that the data has been successfully transferred to the server.

[0128] Step 4:

[0129] The server analyzes incoming data to detect malicious patterns. The input consists of activity logs and generated data sent from the terminal, which are then compared against known patterns in a data calculation. The server executes algorithms specifically to identify suspicious links and phishing phrases, and generates an analysis result indicating whether or not malicious activity has occurred.

[0130] Step 5:

[0131] The server issues a warning to the user based on the analysis results and provides specific countermeasures. The input is the analysis results of the fraudulent activity performed by the server, and the information is processed to provide visual or auditory notifications to the user's device. The output is a warning message and specific recommended actions to the user's device.

[0132] Step 6:

[0133] The user implements countermeasures according to the guidelines provided by the server. The input consists of warning messages and countermeasures displayed on the device, which the user then manually disables the AI ​​application or takes other security measures. The output consists of the device's status after the countermeasures have been implemented and a record of it.

[0134] Step 7:

[0135] The terminal sends the user's countermeasure results back to the server for feedback. The input is the result of the countermeasure taken in step 6, which is provided to the server as data to improve the accuracy of future analyses. The output is the feedback data sent to the server and considerations for improvements in the future based on that data.

[0136] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0137] This invention combines a monitoring system and an emotion engine to ensure the security of newly installed artificial intelligence technology. The system has the ability to detect and warn of fraudulent behavior while recognizing the user's emotional state. Specific embodiments of the system are described below.

[0138] Operation on the device

[0139] The device detects when a user installs new artificial intelligence technology and registers it with the server. Simultaneously with the registration process, the device collects real-time operation logs and generated data from the AI ​​technology. This data is securely stored and ready for transmission to the server.

[0140] Server roles and processing

[0141] The server receives data sent from the terminal and performs advanced analysis. During the analysis, it thoroughly examines the text and behavioral patterns generated by the AI ​​model to check if they match known patterns for detecting fraudulent behavior. It also utilizes an emotion engine to monitor the user's emotional state and incorporates this into the analysis.

[0142] Notifications and feedback to users

[0143] When a user receives a warning from the server, the emotion engine takes the user's emotional state into account and adjusts the notification method accordingly. For example, if the user is feeling stressed, the notification will be delivered carefully and clearly. The warning will include details of the detected problem, along with guidance and recommended actions appropriate to the user's emotional state.

[0144] Specific example

[0145] For example, suppose a user installs a new AI chatbot. The device collects the chatbot's logs and generated messages and sends them to a server. The server checks the messages for phishing links and uses an emotion engine to analyze the user's emotional response to the information they receive. If the user is feeling anxious, the server issues a warning in more reassuring language, encouraging them to take action.

[0146] In this way, the present invention realizes a system that can manage the potential risks associated with the use of AI technology while taking emotions into consideration. By providing an environment in which users can use new technologies with peace of mind, it is possible to maximize the convenience of AI technology.

[0147] The following describes the processing flow.

[0148] Step 1:

[0149] When a new artificial intelligence technology is installed on the device, it detects information about that software and initiates a system-wide monitoring process. This is an automatically registered process and does not require user intervention.

[0150] Step 2:

[0151] The device collects operation logs and generated data in real time. The generated data includes text and results generated by the AI. This collected data is sent to the server using a secure protocol.

[0152] Step 3:

[0153] The server analyzes the data received from the terminal to check whether the artificial intelligence technology is behaving maliciously or abnormally. The analysis includes matching against known malicious patterns and blacklists.

[0154] Step 4:

[0155] The emotion engine embedded in the server also evaluates the user's emotional state. Specifically, it analyzes the user's operation history and feedback to understand changes in their emotions.

[0156] Step 5:

[0157] If the analysis detects fraudulent activity, the server notifies the terminal of the nature of the risk and the countermeasures. At this time, based on the evaluation of the emotion engine, the notification is adjusted to suit the user's emotional state. For example, if the situation is urgent but the user is experiencing stress, the notification will be carefully directed to encourage calm and rational action.

[0158] Step 6:

[0159] The terminal communicates warnings sent from the server to the user. It presents the information in a visually easy-to-understand format and provides specific instructions on necessary countermeasures.

[0160] Step 7:

[0161] Users take action according to the suggested measures. For example, they may temporarily suspend AI models deemed fraudulent in accordance with warning messages, following the guidelines provided by the system.

[0162] Step 8:

[0163] The terminal feeds back the user's response to the server, where it is further analyzed using an emotion engine. This feedback information will be used to improve the accuracy of fraud detection in the future.

[0164] (Example 2)

[0165] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0166] In recent years, with the development and widespread adoption of artificial intelligence technology, concerns about the security associated with its use have increased. Specifically, there are risks such as fraudulent activity, unintentional data leaks, and inappropriate manipulation of users' emotional states. This invention aims to solve these problems and provide an environment in which users can use artificial intelligence with peace of mind.

[0167] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0168] In this invention, the server includes means for identifying newly introduced artificial intelligence technology, means for a central management device to analyze information and detect the presence or absence of fraudulent activity, and means for identifying the user's emotional state using an emotion analysis engine. This ensures user safety and enables warnings that take the user's emotions into consideration.

[0169] "Artificial intelligence technology" refers to technologies that enable intelligent behavior and judgment to be imitated by information processing systems, and encompasses the field of learning from large-scale data using algorithms and models.

[0170] "Action records" refer to data that sequentially records various processes performed by artificial intelligence technology and their results.

[0171] "Generated products" refer to information or content generated by the operation of artificial intelligence technology.

[0172] A "central management device" refers to an integrated system of hardware and software that receives information transmitted from terminals and performs analysis and interpretation.

[0173] "Fraudulent activity" refers to actions involving unauthorized operations or inappropriate behavior that may compromise security or privacy.

[0174] An "emotional analysis engine" refers to a program or algorithm that identifies a user's emotional state based on linguistic or behavioral data.

[0175] "User" refers to an entity that operates or receives services using artificial intelligence technology.

[0176] "Warning information" refers to information that informs users about fraudulent activities or other risks and encourages them to take appropriate action.

[0177] This invention constructs a system for data monitoring and sentiment analysis between terminals, servers, and users to ensure the security of newly introduced artificial intelligence technology. The specific form of this system is described below.

[0178] Operation on the device

[0179] The terminal has the function to identify when a user newly installs artificial intelligence technology and registers the identified information with a central management device. To achieve this, the terminal performs accurate monitoring using installation detection software. The terminal also sequentially collects operation records and generated products, and temporarily stores them in an encrypted form for security purposes. The data is transmitted to the server via a stable communication protocol.

[0180] Server Analysis

[0181] The server receives information sent from the terminal and analyzes it based on specific rules. First, it uses a generative AI model to scrutinize the transmitted text information and identify fraudulent activity. Next, it utilizes an emotion analysis engine to identify the user's emotional state. By combining large-scale data analysis techniques and AI models for information processing, it ensures fast and accurate results.

[0182] User notifications and feedback

[0183] Users receive analysis results sent from the server, and if fraudulent activity is detected, they are prompted to take specific actions to address the risk. Here, notifications are delivered in a way that takes into account and adjusts the user's emotional state. This system allows users to use new technologies with peace of mind.

[0184] Specific example

[0185] For example, consider a scenario where a user installs a new AI chatbot. The device collects the chatbot's activity log and generated messages and sends them to a server. The server analyzes the messages for suspicious links and simultaneously evaluates the user's emotional response. For example, if the server detects anxiety in response to a message like "Username, we are concerned about your safety," it will notify the user with an emotionally sensitive warning such as "Please stay calm, it's okay. The link is not safe, so please do not click it." An example of a prompt message would be "a method to send data collected when a new chatbot is installed, analyze its content and its impact on emotions, and display a friendly warning if the user is feeling anxious."

[0186] This invention aims to create an environment in which AI technology can be used with greater confidence while ensuring user safety.

[0187] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0188] Step 1:

[0189] The terminal monitors newly introduced artificial intelligence technologies. When a user installs a new AI application, monitoring software on the terminal identifies it and collects related information. Specifically, installation log information and application identifiers are collected and prepared to be sent to the server. The input is the installation information of the AI ​​technology, and the output is the identified installation information.

[0190] Step 2:

[0191] The terminal collects operational records and generated products of AI technology. The terminal monitors the operation of the AI ​​application in real time, sequentially recording generated messages and logs. This input data (operational logs and generated products) is temporarily stored using encryption technology and prepared for secure transmission to the server. The output is encrypted operational record and generated product data.

[0192] Step 3:

[0193] The terminal sends the collected data to the server. Using an established, secure communication protocol, the terminal sends the collected data to the server. In this process, the input is encrypted data, and the output is the data sent to the server.

[0194] Step 4:

[0195] The server analyzes the transmitted data. Based on the data received from the terminal, the server uses a generative AI model to scrutinize text and behavioral patterns. Specifically, it detects patterns of fraudulent activity and identifies known suspicious links and words. The input in this process is the data sent to the server, and the output is whether or not fraudulent activity was detected.

[0196] Step 5:

[0197] The server uses an emotion analysis engine to identify the user's emotional state. In addition to analysis, the server uses the emotion analysis engine to incorporate the user's emotional state as part of the data analysis. The input is text information sent to the server, and the output is the user's emotional information.

[0198] Step 6:

[0199] The system provides warnings and feedback to users. If fraudulent activity is detected, the server sends a warning message to the user. The notification content is adjusted according to the user's emotional state, providing appropriate feedback. The input consists of analyzed data and user emotional information, while the output is the notification and feedback delivered to the user.

[0200] This entire process allows users to manage the potential risks associated with using AI technology, enabling them to utilize the technology safely and with peace of mind.

[0201] (Application Example 2)

[0202] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0203] Newly installed artificial intelligence technologies may pose unknown risks to users. However, existing monitoring systems focus on detecting fraudulent behavior, lacking the ability to provide feedback and guidance that considers the user's emotional state. This lack of focus can lead to users experiencing excessive stress and anxiety when warned about fraudulent activity, making it difficult for them to take appropriate action.

[0204] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0205] In this invention, the server includes means for analyzing the user's emotional state using an emotion recognition engine and adjusting the notification method for detecting fraudulent behavior; means for analyzing information contained in the products of artificial intelligence technology and identifying suspicious elements; and means for providing notifications on the information terminal in a manner optimized to the user's emotional state. This enables feedback in a form that is less burdensome for the user and makes it possible to take more effective measures against fraudulent behavior.

[0206] "Artificial intelligence technology" refers to technology that uses computers to perform autonomous and intelligent actions.

[0207] An "operation log" is data that records the history of various actions and states that occur while a system or program is running.

[0208] "Generated products" refer to the output results generated by artificial intelligence technology, and these include text, images, audio, and other similar elements.

[0209] "Fraudulent behavior" refers to actions that differ from the intended behavior or that may harm the user.

[0210] A "warning" is a notification intended to inform the user that there is some kind of danger or malfunction.

[0211] An "emotion recognition engine" is software or an algorithm used to analyze a user's emotional state.

[0212] "Information terminals" refers to all electronic devices, including smartphones, tablets, and smart glasses.

[0213] This invention constructs a system in which a terminal and a server work together to ensure the security of newly installed artificial intelligence (AI) technology. When a user installs new AI technology, the terminal automatically detects it and registers it with the server. Subsequently, the terminal collects the operation logs and generated products of the AI ​​technology in real time and transmits them to the server in a secure manner.

[0214] The server receives the collected data and performs advanced analysis. This analysis uses an emotion recognition engine (e.g., IBM's Watson® Emotion Analysis) to analyze the user's emotional state and adjust notification methods for detecting fraudulent behavior. It also analyzes information contained in the products of artificial intelligence technology to identify suspicious elements such as phishing links.

[0215] Users receive notifications in an appropriate manner through their information devices (e.g., smartphones, smart glasses). Notifications are customized based on the user's emotional state and delivered in a reassuring way. For example, if a user traveling installs a tourist information app and that app generates a suspicious link, the system will identify it and notify them with a message such as, "This may be a phishing attempt; please delete it without clicking the link."

[0216] An example of a prompt used in a generative AI model is, "The user's current emotion is anxiety. Please suggest a way to provide reassurance through notifications." This format allows users to continue using new AI technologies with peace of mind.

[0217] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0218] Step 1:

[0219] The device detects when the user installs new artificial intelligence technology. The input is a list of installed applications, and the output is the identification information of that application sent to the server. Specifically, the device scans the operating system's installation logs to identify new AI applications.

[0220] Step 2:

[0221] The device collects real-time operation logs and output data of detected artificial intelligence technologies. Input data is related to application activity, and the collected log data is prepared for transmission to the server. Specifically, the device uses APIs and system logs to compile information on application execution status and output (e.g., output text).

[0222] Step 3:

[0223] The server receives data sent from the terminal and performs analysis. The input is log data sent from the terminal, and the output is the result regarding the presence or absence of fraudulent activity and the user's emotional state. Specifically, the server uses an emotion recognition engine to analyze the data and check for matches with known fraudulent activity patterns.

[0224] Step 4:

[0225] The server optimizes notification content and method based on the user's emotional state. The input is the analysis result from the emotion recognition engine, and the output is a customized notification message. Specifically, it sends a prompt to a generation AI model to determine the optimal notification method based on the user's current emotional state.

[0226] Step 5:

[0227] The server sends optimized notifications to information terminals, notifying the user. The input is a notification message, and the output is a visual or auditory alert to the user. Specifically, the terminal's notification function is used to display warnings using language that takes the user's emotional state into consideration.

[0228] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0229] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0230] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0231] [Second Embodiment]

[0232] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0233] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0234] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0235] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0236] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0237] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0238] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0239] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0240] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0241] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0242] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0243] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0244] This invention provides a monitoring system to enable the safe use of artificial intelligence technology. Specifically, it monitors newly installed artificial intelligence technology on a user's terminal, analyzes its operation and output to detect potential malicious activity, and issues a warning to the user. A specific embodiment of the system is described below.

[0245] Operation on the device

[0246] The device automatically detects when a user installs new artificial intelligence (AI) technology. Upon detecting the AI ​​technology, the device registers it for security monitoring and begins collecting operation logs and generated data. An agent running on the device prepares to send the collected data to the server.

[0247] Server roles and processing

[0248] The server receives data sent from the terminal and performs analysis. During the analysis, it checks whether the AI-generated content and communication content match known malicious patterns. For example, if the text generated by the AI ​​contains suspicious links or phrases that suggest phishing, the server identifies them and performs a detailed information and risk assessment. Based on these results, the server generates necessary countermeasures.

[0249] Notifications and feedback to users

[0250] Users receive analysis results from the server on their devices. This includes the details and risk level of any detected fraudulent activity. Based on this information, users take the measures recommended by the system (e.g., disabling the relevant AI model or applying corrective patches). The results of the measures taken by the user are fed back to the server, which is used to improve the accuracy of future analyses.

[0251] In this way, a system is realized that allows users to effectively manage potential risks while maintaining an environment in which they can use new artificial intelligence technologies with peace of mind. This invention makes it possible to enjoy the benefits of AI technology while minimizing the security risks associated with its use.

[0252] The following describes the processing flow.

[0253] Step 1:

[0254] The device detects when new artificial intelligence technology is installed and collects basic information about that AI technology. This includes the AI's identification information and the date and time of its activation.

[0255] Step 2:

[0256] The terminal monitors the operation logs and generated products of registered artificial intelligence technologies in real time and collects related data. This data is stored securely and made available for subsequent analysis.

[0257] Step 3:

[0258] The operation logs and generated data collected from the terminal are sent to the server. The server receives this data and begins analysis.

[0259] Step 4:

[0260] The server verifies the operation of the artificial intelligence technology based on the received data. Specifically, it checks for matches with known fraudulent patterns and anomalous patterns based on past cases.

[0261] Step 5:

[0262] If the analysis reveals fraudulent activity or potential risks, the server performs a risk assessment and sends it as a warning to the terminal. This warning includes details of the detected problem along with recommended countermeasures.

[0263] Step 6:

[0264] The terminal receives a warning from the server and notifies the user. The user is informed of the warning visually or audibly, and is presented with recommended actions from the system.

[0265] Step 7:

[0266] Users take appropriate action based on the notified warnings. For example, they can take immediate action, such as temporarily suspending a suspicious AI model.

[0267] Step 8:

[0268] The terminal feeds back the results of the measures taken by the user to the server. This information is recorded on the server and used to improve the accuracy of future analysis processes.

[0269] (Example 1)

[0270] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0271] In recent years, artificial intelligence technology has been utilized in various fields, but its use has also increased the risk of fraudulent activity and malicious creations. Because such risks can cause significant harm to users, it is essential to use artificial intelligence technology in a safe and secure environment. However, conventional technologies are insufficient for real-time monitoring and fraud detection, necessitating more effective countermeasures.

[0272] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0273] In this invention, the server includes means for detecting newly installed artificial intelligence technology, means for registering the artificial intelligence technology as a target for monitoring, and means for collecting operation records and products in real time. This promotes the safe use of artificial intelligence technology and enables early detection and response to fraudulent activity.

[0274] "Detection" is the process of recognizing and identifying newly introduced artificial intelligence technologies in the user's system.

[0275] "Registration" refers to the procedure of adding the detected artificial intelligence technology as a monitoring target to the security management system.

[0276] "Operation record" means recording one by one a series of operations and resource usage situations when the artificial intelligence technology is operating.

[0277] "Product" refers to the output result generated by the artificial intelligence technology, specifically including generated images, texts, etc.

[0278] "Encryption" is a technology that converts the content to securely protect information and prevent unauthorized access during transmission and reception.

[0279] "Remote processing device" refers to a server or a central management system that receives the information transmitted from the terminal and performs analysis.

[0280] "Analysis" is a process of examining the received data in detail and comparing it with known unauthorized behavior patterns.

[0281] "Unauthorized event" refers to inappropriate operations or products of AI that may pose potential harm to the system or users.

[0282] "Risk assessment" is a procedure of quantifying potential risks based on the analysis results and judging their severity.

[0283] "Countermeasure plan" refers to specific action guidelines recommended based on the evaluation results, including safety measures that users should take.

[0284] "Instruction" is information to prompt the user to immediately take the countermeasures recommended by the system.

[0285] "Feedback" means that the user reports the results of the countermeasures implemented to the server and is used as data to help improve the future analysis accuracy.

[0286] The present invention provides a system that supports the safe use of artificial intelligence technology by linking a terminal and a server. This system has the function of immediately detecting newly installed artificial intelligence technology on the terminal and collecting its operation records and products in real time.

[0287] When the user installs new artificial intelligence technology, the terminal automatically detects this technology and registers it as a monitoring target within the system. For the registered artificial intelligence technology, an agent for collecting operation logs and product data starts to operate. As a result, the data is recorded one by one and safely stored within the terminal.

[0288] The collected data is encrypted and sent to the server. The server receives these data passively and uses an analysis engine to determine the presence of illegal events. Specifically, it checks whether the text information created by the generated AI model contains inappropriate links or suspicious phrases. The server processes this information using analysis software such as TensorFlow or PyTorch and evaluates the risk.

[0289] The user receives the analysis result and the estimated risk level from the server's notification. The notification is provided visually or audibly on the terminal, and specific countermeasure plans are shown to the user. This includes stopping the use of the artificial intelligence technology, changing settings, and applying security patches as necessary. The countermeasures taken by the user are returned to the server as feedback, and this feedback information is used to further improve the analysis accuracy.

[0290] For example, when the user inputs a prompt sentence such as "Please draw a summer scenery in the pop art style" into the generated AI model, a similar security analysis process is applied to the generated image or text. As a result, the user can use artificial intelligence technology with confidence and at the same time effectively manage potential risks.

[0291] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0292] Step 1:

[0293] The terminal detects newly installed artificial intelligence technologies. Specifically, the terminal monitors program installation events and retrieves software metadata. The input is information about newly detected programs, and the output is the identification of AI technologies to be registered as targets for monitoring.

[0294] Step 2:

[0295] The terminal registers the detected artificial intelligence technology with the security monitoring system. This initiates the collection of activity logs for the monitored AI technology. The input is the identified AI technology, and the output is a notification that registration to the monitoring list is complete.

[0296] Step 3:

[0297] The terminal collects operation records and output data from the running artificial intelligence technology. Specific collection items include execution time, CPU usage, and generated data (e.g., text and images). The input is operational information from the AI ​​technology, and the output is the collected detailed logs.

[0298] Step 4:

[0299] The terminal encrypts the collected data and prepares it for transmission to the server. Encryption uses methods such as TLS to protect the data. The input is the collected, unprocessed log data, and the output is the encrypted data.

[0300] Step 5:

[0301] The server receives encrypted data sent from the terminal and processes it through its analysis engine. The analysis engine compares the data against a database of known malicious patterns. The input is decrypted, clear log data, and the output is the analysis result and whether or not malicious activity occurred.

[0302] Step 6:

[0303] The server evaluates the risk based on the analysis result and generates notification information for the user. Here, a risk score is calculated from the analysis result, and a detailed report including recommended countermeasures is created. The input is the analysis result, and the output is the final evaluation report.

[0304] Step 7:

[0305] The terminal provides the notification information from the server to the user. The notification is visually displayed through the UI. The input is the detailed evaluation report from the server, and the output is a warning message for the user.

[0306] Step 8:

[0307] The user implements countermeasures based on the notification information. For example, temporarily stops using the problematic AI technology or changes the settings. The input is the recommended countermeasures from the server, and the output is the implemented countermeasures.

[0308] Step 9:

[0309] The terminal collects the results of the actions implemented by the user and sends them to the server as feedback. The collected data is used to improve the future analysis accuracy. The input is the user's action result, and the output is the feedback data.

[0310] (Application Example 1)

[0311] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0312] In recent years, applications utilizing artificial intelligence technology have rapidly become widespread, but the security risks associated with their use have also increased. In particular, the risk of generating malicious links and phishing attacks can pose a significant threat to general users. To address this challenge, there is a need for a reliable monitoring system that constantly monitors the operation of artificial intelligence technology, immediately detects suspicious activity, and recommends appropriate responses.

[0313] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0314] This invention includes a server that monitors operation logs and generated data and transmits them to an external analysis device via a communication network; a server that checks warning information from the external analysis device and presents detailed information and countermeasures if malicious activity is detected; and a server that collects the results after countermeasures are implemented by the user's specific device and provides feedback to improve the accuracy of the analysis. This minimizes the security risks associated with the use of artificial intelligence technology and allows users to utilize these technologies with peace of mind.

[0315] "Newly installed artificial intelligence technology" refers to AI-related software or applications that have been newly added to the user's device.

[0316] An "operation log" is data that records how artificial intelligence technology operates on a device and what operations are performed.

[0317] "Products" refer to the output generated by artificial intelligence technology during its operation, specifically such as text and images.

[0318] "Abusive behavior" refers to actions performed by artificial intelligence technology that could pose a threat to security and privacy.

[0319] A "warning" is a notification sent to inform users of the risks when fraudulent activity is detected.

[0320] "Guidelines for users to take action" refers to information that shows specific actions and recommended responses that users should take after receiving a warning.

[0321] "Monitoring operation logs and output" means tracking and recording the operation and output of AI technology in real time.

[0322] A "communication network" is a network infrastructure used to transmit data from a terminal to an external analysis device.

[0323] An "external analysis device" is a computer system that analyzes data transmitted from a terminal to determine fraudulent activity.

[0324] "Feedback" is the process of collecting the results of user-submitted actions and using them to inform future analysis and system improvements.

[0325] The system of this invention is designed to effectively implement new applications related to artificial intelligence technology, and is based on the premise of collaboration between a smartphone and an external server.

[0326] The device, in this case a smartphone, monitors the operation logs and output of the installed artificial intelligence technology in real time. The device uses a watchdog library to continuously detect the activity of the AI ​​application and, if any suspicious activity is detected, sends the information to an external analysis device. This allows users to leverage monitoring functions in their daily operations.

[0327] On the server side, a process of analyzing the received data takes place. The server uses the requests library to collect data from smartphones and thoroughly examines the content generated by AI technology. In particular, it identifies suspicious links and phishing words and analyzes them against known fraudulent patterns. Based on the results of this analysis, it assesses the importance and risk and presents the user with detailed warnings and solutions. The analyzed information is fed back to the user's smartphone, and the implementation of countermeasures and the results are used to improve accuracy in the future.

[0328] For example, if a newly installed AI chatbot app regularly sends links to extract personal information from a user, the device records the frequency and content of these links and sends them to a server. The server then analyzes the links in detail, and if it detects a potential phishing attempt, it can warn the user and instruct them to refrain from using the links due to their inappropriateness.

[0329] An example of a prompt for a generated AI model is, "Can you analyze and report whether a newly installed AI application is potentially phishing based on its operation logs?" Using this prompt, the system can provide more detailed information about the AI ​​model's operation and support appropriate security measures.

[0330] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0331] Step 1:

[0332] The device detects newly installed artificial intelligence technologies. The input is a list of installed applications, which is used to begin monitoring activity logs. The device uses the watchdog library to track file system changes in real time and identify newly added AI-related applications. The output is an identified list of monitored applications.

[0333] Step 2:

[0334] The terminal collects the operation logs and generated products of detected artificial intelligence technologies and prepares them for transmission to an external server. In this step, various activity logs obtained from the monitored application are taken as input, and the log information is stored on a temporary storage medium. The output is log data processed into a format that can be transferred to the server.

[0335] Step 3:

[0336] The terminal sends the collected data to an external server. The input is the processed data prepared in step 2, and communication with the server is performed via the HTTP protocol using the requests library. The output is a transmission completion status, indicating that the data has been successfully transferred to the server.

[0337] Step 4:

[0338] The server analyzes incoming data to detect malicious patterns. The input consists of activity logs and generated data sent from the terminal, which are then compared against known patterns in a data calculation. The server executes algorithms specifically to identify suspicious links and phishing phrases, and generates an analysis result indicating whether or not malicious activity has occurred.

[0339] Step 5:

[0340] The server issues a warning to the user based on the analysis results and provides specific countermeasures. The input is the analysis results of the fraudulent activity performed by the server, and the information is processed to provide visual or auditory notifications to the user's device. The output is a warning message and specific recommended actions to the user's device.

[0341] Step 6:

[0342] The user implements countermeasures according to the guidelines provided by the server. The input consists of warning messages and countermeasures displayed on the device, which the user then manually disables the AI ​​application or takes other security measures. The output consists of the device's status after the countermeasures have been implemented and a record of it.

[0343] Step 7:

[0344] The terminal sends the user's countermeasure results back to the server for feedback. The input is the result of the countermeasure taken in step 6, which is provided to the server as data to improve the accuracy of future analyses. The output is the feedback data sent to the server and considerations for improvements in the future based on that data.

[0345] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0346] This invention combines a monitoring system and an emotion engine to ensure the security of newly installed artificial intelligence technology. The system has the ability to detect and warn of fraudulent behavior while recognizing the user's emotional state. Specific embodiments of the system are described below.

[0347] Operation on the device

[0348] The device detects when a user installs new artificial intelligence technology and registers it with the server. Simultaneously with the registration process, the device collects real-time operation logs and generated data from the AI ​​technology. This data is securely stored and ready for transmission to the server.

[0349] Server roles and processing

[0350] The server receives data sent from the terminal and performs advanced analysis. During the analysis, it thoroughly examines the text and behavioral patterns generated by the AI ​​model to check if they match known patterns for detecting fraudulent behavior. It also utilizes an emotion engine to monitor the user's emotional state and incorporates this into the analysis.

[0351] Notifications and feedback to users

[0352] When a user receives a warning from the server, the emotion engine takes the user's emotional state into account and adjusts the notification method accordingly. For example, if the user is feeling stressed, the notification will be delivered carefully and clearly. The warning will include details of the detected problem, along with guidance and recommended actions appropriate to the user's emotional state.

[0353] Specific example

[0354] For example, suppose a user installs a new AI chatbot. The device collects the chatbot's logs and generated messages and sends them to a server. The server checks the messages for phishing links and uses an emotion engine to analyze the user's emotional response to the information they receive. If the user is feeling anxious, the server issues a warning in more reassuring language, encouraging them to take action.

[0355] In this way, the present invention realizes a system that can manage the potential risks associated with the use of AI technology while taking emotions into consideration. By providing an environment in which users can use new technologies with peace of mind, it is possible to maximize the convenience of AI technology.

[0356] The following describes the processing flow.

[0357] Step 1:

[0358] When a new artificial intelligence technology is installed on the device, it detects information about that software and initiates a system-wide monitoring process. This is an automatically registered process and does not require user intervention.

[0359] Step 2:

[0360] The device collects operation logs and generated data in real time. The generated data includes text and results generated by the AI. This collected data is sent to the server using a secure protocol.

[0361] Step 3:

[0362] The server analyzes the data received from the terminal to check whether the artificial intelligence technology is behaving maliciously or abnormally. The analysis includes matching against known malicious patterns and blacklists.

[0363] Step 4:

[0364] The emotion engine embedded in the server also evaluates the user's emotional state. Specifically, it analyzes the user's operation history and feedback to understand changes in their emotions.

[0365] Step 5:

[0366] If the analysis detects fraudulent activity, the server notifies the terminal of the nature of the risk and the countermeasures. At this time, based on the evaluation of the emotion engine, the notification is adjusted to suit the user's emotional state. For example, if the situation is urgent but the user is experiencing stress, the notification will be carefully directed to encourage calm and rational action.

[0367] Step 6:

[0368] The terminal communicates warnings sent from the server to the user. It presents the information in a visually easy-to-understand format and provides specific instructions on necessary countermeasures.

[0369] Step 7:

[0370] Users take action according to the suggested measures. For example, they may temporarily suspend AI models deemed fraudulent in accordance with warning messages, following the guidelines provided by the system.

[0371] Step 8:

[0372] The terminal feeds back the user's response to the server, where it is further analyzed using an emotion engine. This feedback information will be used to improve the accuracy of fraud detection in the future.

[0373] (Example 2)

[0374] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0375] In recent years, with the development and widespread adoption of artificial intelligence technology, concerns about the security associated with its use have increased. Specifically, there are risks such as fraudulent activity, unintentional data leaks, and inappropriate manipulation of users' emotional states. This invention aims to solve these problems and provide an environment in which users can use artificial intelligence with peace of mind.

[0376] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0377] In this invention, the server includes means for identifying newly introduced artificial intelligence technology, means for a central management device to analyze information and detect the presence or absence of fraudulent activity, and means for identifying the user's emotional state using an emotion analysis engine. This ensures user safety and enables warnings that take the user's emotions into consideration.

[0378] "Artificial intelligence technology" refers to technologies that enable intelligent behavior and judgment to be imitated by information processing systems, and encompasses the field of learning from large-scale data using algorithms and models.

[0379] "Action records" refer to data that sequentially records various processes performed by artificial intelligence technology and their results.

[0380] "Generated products" refer to information or content generated by the operation of artificial intelligence technology.

[0381] A "central management device" refers to an integrated system of hardware and software that receives information transmitted from terminals and performs analysis and interpretation.

[0382] "Fraudulent activity" refers to actions involving unauthorized operations or inappropriate behavior that may compromise security or privacy.

[0383] An "emotional analysis engine" refers to a program or algorithm that identifies a user's emotional state based on linguistic or behavioral data.

[0384] "User" refers to an entity that operates or receives services using artificial intelligence technology.

[0385] "Warning information" refers to information that informs users about fraudulent activities or other risks and encourages them to take appropriate action.

[0386] This invention constructs a system for data monitoring and sentiment analysis between terminals, servers, and users to ensure the security of newly introduced artificial intelligence technology. The specific form of this system is described below.

[0387] Operation on the device

[0388] The terminal has the function to identify when a user newly installs artificial intelligence technology and registers the identified information with a central management device. To achieve this, the terminal performs accurate monitoring using installation detection software. The terminal also sequentially collects operation records and generated products, and temporarily stores them in an encrypted form for security purposes. The data is transmitted to the server via a stable communication protocol.

[0389] Server Analysis

[0390] The server receives information sent from the terminal and analyzes it based on specific rules. First, it uses a generative AI model to scrutinize the transmitted text information and identify fraudulent activity. Next, it utilizes an emotion analysis engine to identify the user's emotional state. By combining large-scale data analysis techniques and AI models for information processing, it ensures fast and accurate results.

[0391] User notifications and feedback

[0392] Users receive analysis results sent from the server, and if fraudulent activity is detected, they are prompted to take specific actions to address the risk. Here, notifications are delivered in a way that takes into account and adjusts the user's emotional state. This system allows users to use new technologies with peace of mind.

[0393] Specific example

[0394] For example, consider a scenario where a user installs a new AI chatbot. The device collects the chatbot's activity log and generated messages and sends them to a server. The server analyzes the messages for suspicious links and simultaneously evaluates the user's emotional response. For example, if the server detects anxiety in response to a message like "Username, we are concerned about your safety," it will notify the user with an emotionally sensitive warning such as "Please stay calm, it's okay. The link is not safe, so please do not click it." An example of a prompt message would be "a method to send data collected when a new chatbot is installed, analyze its content and its impact on emotions, and display a friendly warning if the user is feeling anxious."

[0395] This invention aims to create an environment in which AI technology can be used with greater confidence while ensuring user safety.

[0396] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0397] Step 1:

[0398] The terminal monitors newly introduced artificial intelligence technologies. When a user installs a new AI application, monitoring software on the terminal identifies it and collects related information. Specifically, installation log information and application identifiers are collected and prepared to be sent to the server. The input is the installation information of the AI ​​technology, and the output is the identified installation information.

[0399] Step 2:

[0400] The terminal collects operational records and generated products of AI technology. The terminal monitors the operation of the AI ​​application in real time, sequentially recording generated messages and logs. This input data (operational logs and generated products) is temporarily stored using encryption technology and prepared for secure transmission to the server. The output is encrypted operational record and generated product data.

[0401] Step 3:

[0402] The terminal sends the collected data to the server. Using an established, secure communication protocol, the terminal sends the collected data to the server. In this process, the input is encrypted data, and the output is the data sent to the server.

[0403] Step 4:

[0404] The server analyzes the transmitted data. Based on the data received from the terminal, the server uses a generative AI model to scrutinize text and behavioral patterns. Specifically, it detects patterns of fraudulent activity and identifies known suspicious links and words. The input in this process is the data sent to the server, and the output is whether or not fraudulent activity was detected.

[0405] Step 5:

[0406] The server uses an emotion analysis engine to identify the user's emotional state. In addition to analysis, the server uses the emotion analysis engine to incorporate the user's emotional state as part of the data analysis. The input is text information sent to the server, and the output is the user's emotional information.

[0407] Step 6:

[0408] The system provides warnings and feedback to users. If fraudulent activity is detected, the server sends a warning message to the user. The notification content is adjusted according to the user's emotional state, providing appropriate feedback. The input consists of analyzed data and user emotional information, while the output is the notification and feedback delivered to the user.

[0409] This entire process allows users to manage the potential risks associated with using AI technology, enabling them to utilize the technology safely and with peace of mind.

[0410] (Application Example 2)

[0411] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0412] Newly installed artificial intelligence technologies may pose unknown risks to users. However, existing monitoring systems focus on detecting fraudulent behavior, lacking the ability to provide feedback and guidance that considers the user's emotional state. This lack of focus can lead to users experiencing excessive stress and anxiety when warned about fraudulent activity, making it difficult for them to take appropriate action.

[0413] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0414] In this invention, the server includes means for analyzing the user's emotional state using an emotion recognition engine and adjusting the notification method for detecting fraudulent behavior; means for analyzing information contained in the products of artificial intelligence technology and identifying suspicious elements; and means for providing notifications on the information terminal in a manner optimized to the user's emotional state. This enables feedback in a form that is less burdensome for the user and makes it possible to take more effective measures against fraudulent behavior.

[0415] "Artificial intelligence technology" refers to technology that uses computers to perform autonomous and intelligent actions.

[0416] An "operation log" is data that records the history of various actions and states that occur while a system or program is running.

[0417] "Generated products" refer to the output results generated by artificial intelligence technology, and these include text, images, audio, and other similar elements.

[0418] "Fraudulent behavior" refers to actions that differ from the intended behavior or that may harm the user.

[0419] A "warning" is a notification intended to inform the user that there is some kind of danger or malfunction.

[0420] An "emotion recognition engine" is software or an algorithm used to analyze a user's emotional state.

[0421] "Information terminals" refers to all electronic devices, including smartphones, tablets, and smart glasses.

[0422] This invention constructs a system in which a terminal and a server work together to ensure the security of newly installed artificial intelligence (AI) technology. When a user installs new AI technology, the terminal automatically detects it and registers it with the server. Subsequently, the terminal collects the operation logs and generated products of the AI ​​technology in real time and transmits them to the server in a secure manner.

[0423] The server receives the collected data and performs advanced analysis. This analysis uses an emotion recognition engine (e.g., IBM's Watson Emotion Analysis) to analyze the user's emotional state and adjust notification methods for detecting fraudulent behavior. It also analyzes information contained in the products of artificial intelligence technology to identify suspicious elements such as phishing links.

[0424] Users receive notifications in an appropriate manner through their information devices (e.g., smartphones, smart glasses). Notifications are customized based on the user's emotional state and delivered in a reassuring way. For example, if a user traveling installs a tourist information app and that app generates a suspicious link, the system will identify it and notify them with a message such as, "This may be a phishing attempt; please delete it without clicking the link."

[0425] An example of a prompt used in a generative AI model is, "The user's current emotion is anxiety. Please suggest a way to provide reassurance through notifications." This format allows users to continue using new AI technologies with peace of mind.

[0426] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0427] Step 1:

[0428] The device detects when the user installs new artificial intelligence technology. The input is a list of installed applications, and the output is the identification information of that application sent to the server. Specifically, the device scans the operating system's installation logs to identify new AI applications.

[0429] Step 2:

[0430] The device collects real-time operation logs and output data of detected artificial intelligence technologies. Input data is related to application activity, and the collected log data is prepared for transmission to the server. Specifically, the device uses APIs and system logs to compile information on application execution status and output (e.g., output text).

[0431] Step 3:

[0432] The server receives data sent from the terminal and performs analysis. The input is log data sent from the terminal, and the output is the result regarding the presence or absence of fraudulent activity and the user's emotional state. Specifically, the server uses an emotion recognition engine to analyze the data and check for matches with known fraudulent activity patterns.

[0433] Step 4:

[0434] The server optimizes notification content and method based on the user's emotional state. The input is the analysis result from the emotion recognition engine, and the output is a customized notification message. Specifically, it sends a prompt to a generation AI model to determine the optimal notification method based on the user's current emotional state.

[0435] Step 5:

[0436] The server sends optimized notifications to information terminals, notifying the user. The input is a notification message, and the output is a visual or auditory alert to the user. Specifically, the terminal's notification function is used to display warnings using language that takes the user's emotional state into consideration.

[0437] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0438] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0439] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0440] [Third Embodiment]

[0441] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0442] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0443] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0444] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0445] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0446] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0447] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0448] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0449] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0450] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0451] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0452] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0453] This invention provides a monitoring system to enable the safe use of artificial intelligence technology. Specifically, it monitors newly installed artificial intelligence technology on a user's terminal, analyzes its operation and output to detect potential malicious activity, and issues a warning to the user. A specific embodiment of the system is described below.

[0454] Operation on the device

[0455] The device automatically detects when a user installs new artificial intelligence (AI) technology. Upon detecting the AI ​​technology, the device registers it for security monitoring and begins collecting operation logs and generated data. An agent running on the device prepares to send the collected data to the server.

[0456] Server roles and processing

[0457] The server receives data sent from the terminal and performs analysis. During the analysis, it checks whether the AI-generated content and communication content match known malicious patterns. For example, if the text generated by the AI ​​contains suspicious links or phrases that suggest phishing, the server identifies them and performs a detailed information and risk assessment. Based on these results, the server generates necessary countermeasures.

[0458] Notifications and feedback to users

[0459] Users receive analysis results from the server on their devices. This includes the details and risk level of any detected fraudulent activity. Based on this information, users take the measures recommended by the system (e.g., disabling the relevant AI model or applying corrective patches). The results of the measures taken by the user are fed back to the server, which is used to improve the accuracy of future analyses.

[0460] In this way, a system is realized that allows users to effectively manage potential risks while maintaining an environment in which they can use new artificial intelligence technologies with peace of mind. This invention makes it possible to enjoy the benefits of AI technology while minimizing the security risks associated with its use.

[0461] The following describes the processing flow.

[0462] Step 1:

[0463] The device detects when new artificial intelligence technology is installed and collects basic information about that AI technology. This includes the AI's identification information and the date and time of its activation.

[0464] Step 2:

[0465] The terminal monitors the operation logs and generated products of registered artificial intelligence technologies in real time and collects related data. This data is stored securely and made available for subsequent analysis.

[0466] Step 3:

[0467] The operation logs and generated data collected from the terminal are sent to the server. The server receives this data and begins analysis.

[0468] Step 4:

[0469] The server verifies the operation of the artificial intelligence technology based on the received data. Specifically, it checks for matches with known fraudulent patterns and anomalous patterns based on past cases.

[0470] Step 5:

[0471] If the analysis reveals fraudulent activity or potential risks, the server performs a risk assessment and sends it as a warning to the terminal. This warning includes details of the detected problem along with recommended countermeasures.

[0472] Step 6:

[0473] The terminal receives a warning from the server and notifies the user. The user is informed of the warning visually or audibly, and is presented with recommended actions from the system.

[0474] Step 7:

[0475] Users take appropriate action based on the notified warnings. For example, they can take immediate action, such as temporarily suspending a suspicious AI model.

[0476] Step 8:

[0477] The terminal feeds back the results of the measures taken by the user to the server. This information is recorded on the server and used to improve the accuracy of future analysis processes.

[0478] (Example 1)

[0479] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0480] In recent years, artificial intelligence technology has been utilized in various fields, but its use has also increased the risk of fraudulent activity and malicious creations. Because such risks can cause significant harm to users, it is essential to use artificial intelligence technology in a safe and secure environment. However, conventional technologies are insufficient for real-time monitoring and fraud detection, necessitating more effective countermeasures.

[0481] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0482] In this invention, the server includes means for detecting newly installed artificial intelligence technology, means for registering the artificial intelligence technology as a target for monitoring, and means for collecting operation records and products in real time. This promotes the safe use of artificial intelligence technology and enables early detection and response to fraudulent activity.

[0483] "Detection" is the process of recognizing and identifying newly introduced artificial intelligence technologies in the user's system.

[0484] "Registration" is the procedure for adding detected artificial intelligence technologies to the security management system as targets for monitoring.

[0485] "Operation logging" refers to the process of meticulously recording a series of operations and resource usage when artificial intelligence technology is running.

[0486] "Products" refer to the output results generated by artificial intelligence technology, and specifically include generated images, text, and other similar elements.

[0487] "Encryption" is a technology that transforms the content of information to securely protect it and prevent unauthorized access during transmission and reception.

[0488] A "remote processing device" refers to a server or central management system that receives and analyzes information transmitted from a terminal.

[0489] "Analysis" is the process of examining received data in detail and comparing it to known patterns of fraudulent behavior.

[0490] "Anomaly" refers to inappropriate AI behavior or its products that could potentially cause harm to the system or users.

[0491] "Risk assessment" is a procedure that quantifies potential risks based on analysis results and determines their severity.

[0492] A "proposed course of action" refers to specific guidelines recommended based on the evaluation results, including safety measures that users should take.

[0493] "Instructions" are pieces of information designed to prompt users to immediately take the measures recommended by the system.

[0494] "Feedback" refers to the process where users report the results of the measures they have taken to the server, and this data is used to improve the accuracy of future analyses.

[0495] This invention provides a system that supports the secure use of artificial intelligence technology by linking a terminal and a server. This system has the function of immediately detecting newly installed artificial intelligence technology on a terminal and collecting its operation records and output in real time.

[0496] When a user installs a new artificial intelligence (AI) technology, the terminal automatically detects it and registers it as a monitored target within the system. For registered AI technologies, an agent begins operating to collect operation logs and product data. This ensures that data is recorded continuously and securely stored within the terminal.

[0497] The collected data is encrypted and sent to the server. The server passively receives this data and uses an analysis engine to determine whether or not malicious activity has occurred. Specifically, it checks whether the text information created by the generative AI model contains inappropriate links or suspicious phrases. The server processes this information using analysis software such as TensorFlow or PyTorch and assesses its risk.

[0498] Users receive analysis results and estimated risk levels via notifications from the server. These notifications are provided visually or audibly on the device, offering specific countermeasures. These include disabling or changing settings for artificial intelligence technology and applying security patches as needed. The actions taken by the user are returned to the server as feedback, and this feedback information is used to further improve the accuracy of the analysis.

[0499] For example, if a user inputs the prompt "Draw a summer day scene in pop art style" into a generation AI model, a similar security analysis process will be applied to the generated image and text. This allows users to use artificial intelligence technology with peace of mind while effectively managing potential risks.

[0500] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0501] Step 1:

[0502] The terminal detects newly installed artificial intelligence technologies. Specifically, the terminal monitors program installation events and retrieves software metadata. The input is information about newly detected programs, and the output is the identification of AI technologies to be registered as targets for monitoring.

[0503] Step 2:

[0504] The terminal registers the detected artificial intelligence technology with the security monitoring system. This initiates the collection of activity logs for the monitored AI technology. The input is the identified AI technology, and the output is a notification that registration to the monitoring list is complete.

[0505] Step 3:

[0506] The terminal collects operation records and output data from the running artificial intelligence technology. Specific collection items include execution time, CPU usage, and generated data (e.g., text and images). The input is operational information from the AI ​​technology, and the output is the collected detailed logs.

[0507] Step 4:

[0508] The terminal encrypts the collected data and prepares it for transmission to the server. Encryption uses methods such as TLS to protect the data. The input is the collected, unprocessed log data, and the output is the encrypted data.

[0509] Step 5:

[0510] The server receives encrypted data sent from the terminal and processes it through its analysis engine. The analysis engine compares the data against a database of known malicious patterns. The input is decrypted, clear log data, and the output is the analysis result and whether or not malicious activity occurred.

[0511] Step 6:

[0512] The server assesses the risks based on the analysis results and generates notification information for the user. Here, a risk score is calculated from the analysis results, and a detailed report including recommended countermeasures is created. The input is the analysis results, and the output is the final evaluation report.

[0513] Step 7:

[0514] The terminal provides the user with notification information from the server. The notifications are displayed visually through the user interface (UI). Input is a detailed evaluation report from the server, and output is a warning message for the user.

[0515] Step 8:

[0516] Users take action based on the notification information. For example, they might temporarily stop using problematic AI technology or change its settings. The input is the recommended action from the server, and the output is the action that was taken.

[0517] Step 9:

[0518] The terminal collects the results of actions taken by the user and sends them to the server as feedback. The collected data will be used to improve the accuracy of future analyses. The input is the user's action result, and the output is feedback data.

[0519] (Application Example 1)

[0520] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0521] In recent years, applications utilizing artificial intelligence technology have rapidly become widespread, but the security risks associated with their use have also increased. In particular, the risk of generating malicious links and phishing attacks can pose a significant threat to general users. To address this challenge, there is a need for a reliable monitoring system that constantly monitors the operation of artificial intelligence technology, immediately detects suspicious activity, and recommends appropriate responses.

[0522] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0523] This invention includes a server that monitors operation logs and generated data and transmits them to an external analysis device via a communication network; a server that checks warning information from the external analysis device and presents detailed information and countermeasures if malicious activity is detected; and a server that collects the results after countermeasures are implemented by the user's specific device and provides feedback to improve the accuracy of the analysis. This minimizes the security risks associated with the use of artificial intelligence technology and allows users to utilize these technologies with peace of mind.

[0524] "Newly installed artificial intelligence technology" refers to AI-related software or applications that have been newly added to the user's device.

[0525] An "operation log" is data that records how artificial intelligence technology operates on a device and what operations are performed.

[0526] "Products" refer to the output generated by artificial intelligence technology during its operation, specifically such as text and images.

[0527] "Abusive behavior" refers to actions performed by artificial intelligence technology that could pose a threat to security and privacy.

[0528] A "warning" is a notification sent to inform users of the risks when fraudulent activity is detected.

[0529] "Guidelines for users to take action" refers to information that shows specific actions and recommended responses that users should take after receiving a warning.

[0530] "Monitoring operation logs and output" means tracking and recording the operation and output of AI technology in real time.

[0531] A "communication network" is a network infrastructure used to transmit data from a terminal to an external analysis device.

[0532] An "external analysis device" is a computer system that analyzes data transmitted from a terminal to determine fraudulent activity.

[0533] "Feedback" is the process of collecting the results of user-submitted actions and using them to inform future analysis and system improvements.

[0534] The system of this invention is designed to effectively implement new applications related to artificial intelligence technology, and is based on the premise of collaboration between a smartphone and an external server.

[0535] The device, in this case a smartphone, monitors the operation logs and output of the installed artificial intelligence technology in real time. The device uses a watchdog library to continuously detect the activity of the AI ​​application and, if any suspicious activity is detected, sends the information to an external analysis device. This allows users to leverage monitoring functions in their daily operations.

[0536] On the server side, a process of analyzing the received data takes place. The server uses the requests library to collect data from smartphones and thoroughly examines the content generated by AI technology. In particular, it identifies suspicious links and phishing words and analyzes them against known fraudulent patterns. Based on the results of this analysis, it assesses the importance and risk and presents the user with detailed warnings and solutions. The analyzed information is fed back to the user's smartphone, and the implementation of countermeasures and the results are used to improve accuracy in the future.

[0537] For example, if a newly installed AI chatbot app regularly sends links to extract personal information from a user, the device records the frequency and content of these links and sends them to a server. The server then analyzes the links in detail, and if it detects a potential phishing attempt, it can warn the user and instruct them to refrain from using the links due to their inappropriateness.

[0538] An example of a prompt for a generated AI model is, "Can you analyze and report whether a newly installed AI application is potentially phishing based on its operation logs?" Using this prompt, the system can provide more detailed information about the AI ​​model's operation and support appropriate security measures.

[0539] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0540] Step 1:

[0541] The device detects newly installed artificial intelligence technologies. The input is a list of installed applications, which is used to begin monitoring activity logs. The device uses the watchdog library to track file system changes in real time and identify newly added AI-related applications. The output is an identified list of monitored applications.

[0542] Step 2:

[0543] The terminal collects the operation logs and generated products of detected artificial intelligence technologies and prepares them for transmission to an external server. In this step, various activity logs obtained from the monitored application are taken as input, and the log information is stored on a temporary storage medium. The output is log data processed into a format that can be transferred to the server.

[0544] Step 3:

[0545] The terminal sends the collected data to an external server. The input is the processed data prepared in step 2, and communication with the server is performed via the HTTP protocol using the requests library. The output is a transmission completion status, indicating that the data has been successfully transferred to the server.

[0546] Step 4:

[0547] The server analyzes incoming data to detect malicious patterns. The input consists of activity logs and generated data sent from the terminal, which are then compared against known patterns in a data calculation. The server executes algorithms specifically to identify suspicious links and phishing phrases, and generates an analysis result indicating whether or not malicious activity has occurred.

[0548] Step 5:

[0549] The server issues a warning to the user based on the analysis results and provides specific countermeasures. The input is the analysis results of the fraudulent activity performed by the server, and the information is processed to provide visual or auditory notifications to the user's device. The output is a warning message and specific recommended actions to the user's device.

[0550] Step 6:

[0551] The user implements countermeasures according to the guidelines provided by the server. The input consists of warning messages and countermeasures displayed on the device, which the user then manually disables the AI ​​application or takes other security measures. The output consists of the device's status after the countermeasures have been implemented and a record of it.

[0552] Step 7:

[0553] The terminal sends the user's countermeasure results back to the server for feedback. The input is the result of the countermeasure taken in step 6, which is provided to the server as data to improve the accuracy of future analyses. The output is the feedback data sent to the server and considerations for improvements in the future based on that data.

[0554] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0555] This invention combines a monitoring system and an emotion engine to ensure the security of newly installed artificial intelligence technology. The system has the ability to detect and warn of fraudulent behavior while recognizing the user's emotional state. Specific embodiments of the system are described below.

[0556] Operation on the device

[0557] The device detects when a user installs new artificial intelligence technology and registers it with the server. Simultaneously with the registration process, the device collects real-time operation logs and generated data from the AI ​​technology. This data is securely stored and ready for transmission to the server.

[0558] Server roles and processing

[0559] The server receives data sent from the terminal and performs advanced analysis. During the analysis, it thoroughly examines the text and behavioral patterns generated by the AI ​​model to check if they match known patterns for detecting fraudulent behavior. It also utilizes an emotion engine to monitor the user's emotional state and incorporates this into the analysis.

[0560] Notifications and feedback to users

[0561] When a user receives a warning from the server, the emotion engine takes the user's emotional state into account and adjusts the notification method accordingly. For example, if the user is feeling stressed, the notification will be delivered carefully and clearly. The warning will include details of the detected problem, along with guidance and recommended actions appropriate to the user's emotional state.

[0562] Specific example

[0563] For example, suppose a user installs a new AI chatbot. The device collects the chatbot's logs and generated messages and sends them to a server. The server checks the messages for phishing links and uses an emotion engine to analyze the user's emotional response to the information they receive. If the user is feeling anxious, the server issues a warning in more reassuring language, encouraging them to take action.

[0564] In this way, the present invention realizes a system that can manage the potential risks associated with the use of AI technology while taking emotions into consideration. By providing an environment in which users can use new technologies with peace of mind, it is possible to maximize the convenience of AI technology.

[0565] The following describes the processing flow.

[0566] Step 1:

[0567] When a new artificial intelligence technology is installed on the device, it detects information about that software and initiates a system-wide monitoring process. This is an automatically registered process and does not require user intervention.

[0568] Step 2:

[0569] The device collects operation logs and generated data in real time. The generated data includes text and results generated by the AI. This collected data is sent to the server using a secure protocol.

[0570] Step 3:

[0571] The server analyzes the data received from the terminal to check whether the artificial intelligence technology is behaving maliciously or abnormally. The analysis includes matching against known malicious patterns and blacklists.

[0572] Step 4:

[0573] The emotion engine embedded in the server also evaluates the user's emotional state. Specifically, it analyzes the user's operation history and feedback to understand changes in their emotions.

[0574] Step 5:

[0575] If the analysis detects fraudulent activity, the server notifies the terminal of the nature of the risk and the countermeasures. At this time, based on the evaluation of the emotion engine, the notification is adjusted to suit the user's emotional state. For example, if the situation is urgent but the user is experiencing stress, the notification will be carefully directed to encourage calm and rational action.

[0576] Step 6:

[0577] The terminal communicates warnings sent from the server to the user. It presents the information in a visually easy-to-understand format and provides specific instructions on necessary countermeasures.

[0578] Step 7:

[0579] Users take action according to the suggested measures. For example, they may temporarily suspend AI models deemed fraudulent in accordance with warning messages, following the guidelines provided by the system.

[0580] Step 8:

[0581] The terminal feeds back the user's response to the server, where it is further analyzed using an emotion engine. This feedback information will be used to improve the accuracy of fraud detection in the future.

[0582] (Example 2)

[0583] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0584] In recent years, with the development and widespread adoption of artificial intelligence technology, concerns about the security associated with its use have increased. Specifically, there are risks such as fraudulent activity, unintentional data leaks, and inappropriate manipulation of users' emotional states. This invention aims to solve these problems and provide an environment in which users can use artificial intelligence with peace of mind.

[0585] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0586] In this invention, the server includes means for identifying newly introduced artificial intelligence technology, means for a central management device to analyze information and detect the presence or absence of fraudulent activity, and means for identifying the user's emotional state using an emotion analysis engine. This ensures user safety and enables warnings that take the user's emotions into consideration.

[0587] "Artificial intelligence technology" refers to technologies that enable intelligent behavior and judgment to be imitated by information processing systems, and encompasses the field of learning from large-scale data using algorithms and models.

[0588] "Action records" refer to data that sequentially records various processes performed by artificial intelligence technology and their results.

[0589] "Generated products" refer to information or content generated by the operation of artificial intelligence technology.

[0590] A "central management device" refers to an integrated system of hardware and software that receives information transmitted from terminals and performs analysis and interpretation.

[0591] "Fraudulent activity" refers to actions involving unauthorized operations or inappropriate behavior that may compromise security or privacy.

[0592] An "emotional analysis engine" refers to a program or algorithm that identifies a user's emotional state based on linguistic or behavioral data.

[0593] "User" refers to an entity that operates or receives services using artificial intelligence technology.

[0594] "Warning information" refers to information that informs users about fraudulent activities or other risks and encourages them to take appropriate action.

[0595] This invention constructs a system for data monitoring and sentiment analysis between terminals, servers, and users to ensure the security of newly introduced artificial intelligence technology. The specific form of this system is described below.

[0596] Operation on the device

[0597] The terminal has the function to identify when a user newly installs artificial intelligence technology and registers the identified information with a central management device. To achieve this, the terminal performs accurate monitoring using installation detection software. The terminal also sequentially collects operation records and generated products, and temporarily stores them in an encrypted form for security purposes. The data is transmitted to the server via a stable communication protocol.

[0598] Server Analysis

[0599] The server receives information sent from the terminal and analyzes it based on specific rules. First, it uses a generative AI model to scrutinize the transmitted text information and identify fraudulent activity. Next, it utilizes an emotion analysis engine to identify the user's emotional state. By combining large-scale data analysis techniques and AI models for information processing, it ensures fast and accurate results.

[0600] User notifications and feedback

[0601] Users receive analysis results sent from the server, and if fraudulent activity is detected, they are prompted to take specific actions to address the risk. Here, notifications are delivered in a way that takes into account and adjusts the user's emotional state. This system allows users to use new technologies with peace of mind.

[0602] Specific example

[0603] For example, consider a scenario where a user installs a new AI chatbot. The device collects the chatbot's activity log and generated messages and sends them to a server. The server analyzes the messages for suspicious links and simultaneously evaluates the user's emotional response. For example, if the server detects anxiety in response to a message like "Username, we are concerned about your safety," it will notify the user with an emotionally sensitive warning such as "Please stay calm, it's okay. The link is not safe, so please do not click it." An example of a prompt message would be "a method to send data collected when a new chatbot is installed, analyze its content and its impact on emotions, and display a friendly warning if the user is feeling anxious."

[0604] This invention aims to create an environment in which AI technology can be used with greater confidence while ensuring user safety.

[0605] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0606] Step 1:

[0607] The terminal monitors newly introduced artificial intelligence technologies. When a user installs a new AI application, monitoring software on the terminal identifies it and collects related information. Specifically, installation log information and application identifiers are collected and prepared to be sent to the server. The input is the installation information of the AI ​​technology, and the output is the identified installation information.

[0608] Step 2:

[0609] The terminal collects operational records and generated products of AI technology. The terminal monitors the operation of the AI ​​application in real time, sequentially recording generated messages and logs. This input data (operational logs and generated products) is temporarily stored using encryption technology and prepared for secure transmission to the server. The output is encrypted operational record and generated product data.

[0610] Step 3:

[0611] The terminal sends the collected data to the server. Using an established, secure communication protocol, the terminal sends the collected data to the server. In this process, the input is encrypted data, and the output is the data sent to the server.

[0612] Step 4:

[0613] The server analyzes the transmitted data. Based on the data received from the terminal, the server uses a generative AI model to scrutinize text and behavioral patterns. Specifically, it detects patterns of fraudulent activity and identifies known suspicious links and words. The input in this process is the data sent to the server, and the output is whether or not fraudulent activity was detected.

[0614] Step 5:

[0615] The server uses an emotion analysis engine to identify the user's emotional state. In addition to analysis, the server uses the emotion analysis engine to incorporate the user's emotional state as part of the data analysis. The input is text information sent to the server, and the output is the user's emotional information.

[0616] Step 6:

[0617] The system provides warnings and feedback to users. If fraudulent activity is detected, the server sends a warning message to the user. The notification content is adjusted according to the user's emotional state, providing appropriate feedback. The input consists of analyzed data and user emotional information, while the output is the notification and feedback delivered to the user.

[0618] This entire process allows users to manage the potential risks associated with using AI technology, enabling them to utilize the technology safely and with peace of mind.

[0619] (Application Example 2)

[0620] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0621] Newly installed artificial intelligence technologies may pose unknown risks to users. However, existing monitoring systems focus on detecting fraudulent behavior, lacking the ability to provide feedback and guidance that considers the user's emotional state. This lack of focus can lead to users experiencing excessive stress and anxiety when warned about fraudulent activity, making it difficult for them to take appropriate action.

[0622] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0623] In this invention, the server includes means for analyzing the user's emotional state using an emotion recognition engine and adjusting the notification method for detecting fraudulent behavior; means for analyzing information contained in the products of artificial intelligence technology and identifying suspicious elements; and means for providing notifications on the information terminal in a manner optimized to the user's emotional state. This enables feedback in a form that is less burdensome for the user and makes it possible to take more effective measures against fraudulent behavior.

[0624] "Artificial intelligence technology" refers to technology that uses computers to perform autonomous and intelligent actions.

[0625] An "operation log" is data that records the history of various actions and states that occur while a system or program is running.

[0626] "Generated products" refer to the output results generated by artificial intelligence technology, and these include text, images, audio, and other similar elements.

[0627] "Fraudulent behavior" refers to actions that differ from the intended behavior or that may harm the user.

[0628] A "warning" is a notification intended to inform the user that there is some kind of danger or malfunction.

[0629] An "emotion recognition engine" is software or an algorithm used to analyze a user's emotional state.

[0630] "Information terminals" refers to all electronic devices, including smartphones, tablets, and smart glasses.

[0631] This invention constructs a system in which a terminal and a server work together to ensure the security of newly installed artificial intelligence (AI) technology. When a user installs new AI technology, the terminal automatically detects it and registers it with the server. Subsequently, the terminal collects the operation logs and generated products of the AI ​​technology in real time and transmits them to the server in a secure manner.

[0632] The server receives the collected data and performs advanced analysis. This analysis uses an emotion recognition engine (e.g., IBM's Watson Emotion Analysis) to analyze the user's emotional state and adjust notification methods for detecting fraudulent behavior. It also analyzes information contained in the products of artificial intelligence technology to identify suspicious elements such as phishing links.

[0633] Users receive notifications in an appropriate manner through their information devices (e.g., smartphones, smart glasses). Notifications are customized based on the user's emotional state and delivered in a reassuring way. For example, if a user traveling installs a tourist information app and that app generates a suspicious link, the system will identify it and notify them with a message such as, "This may be a phishing attempt; please delete it without clicking the link."

[0634] An example of a prompt used in a generative AI model is, "The user's current emotion is anxiety. Please suggest a way to provide reassurance through notifications." This format allows users to continue using new AI technologies with peace of mind.

[0635] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0636] Step 1:

[0637] The device detects when the user installs new artificial intelligence technology. The input is a list of installed applications, and the output is the identification information of that application sent to the server. Specifically, the device scans the operating system's installation logs to identify new AI applications.

[0638] Step 2:

[0639] The device collects real-time operation logs and output data of detected artificial intelligence technologies. Input data is related to application activity, and the collected log data is prepared for transmission to the server. Specifically, the device uses APIs and system logs to compile information on application execution status and output (e.g., output text).

[0640] Step 3:

[0641] The server receives data sent from the terminal and performs analysis. The input is log data sent from the terminal, and the output is the result regarding the presence or absence of fraudulent activity and the user's emotional state. Specifically, the server uses an emotion recognition engine to analyze the data and check for matches with known fraudulent activity patterns.

[0642] Step 4:

[0643] The server optimizes notification content and method based on the user's emotional state. The input is the analysis result from the emotion recognition engine, and the output is a customized notification message. Specifically, it sends a prompt to a generation AI model to determine the optimal notification method based on the user's current emotional state.

[0644] Step 5:

[0645] The server sends optimized notifications to information terminals, notifying the user. The input is a notification message, and the output is a visual or auditory alert to the user. Specifically, the terminal's notification function is used to display warnings using language that takes the user's emotional state into consideration.

[0646] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0647] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0648] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0649] [Fourth Embodiment]

[0650] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0651] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0652] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0653] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0654] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0655] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0656] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0657] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0658] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0659] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0660] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0661] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0662] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0663] This invention provides a monitoring system to enable the safe use of artificial intelligence technology. Specifically, it monitors newly installed artificial intelligence technology on a user's terminal, analyzes its operation and output to detect potential malicious activity, and issues a warning to the user. A specific embodiment of the system is described below.

[0664] Operation on the device

[0665] The device automatically detects when a user installs new artificial intelligence (AI) technology. Upon detecting the AI ​​technology, the device registers it for security monitoring and begins collecting operation logs and generated data. An agent running on the device prepares to send the collected data to the server.

[0666] Server roles and processing

[0667] The server receives data sent from the terminal and performs analysis. During the analysis, it checks whether the AI-generated content and communication content match known malicious patterns. For example, if the text generated by the AI ​​contains suspicious links or phrases that suggest phishing, the server identifies them and performs a detailed information and risk assessment. Based on these results, the server generates necessary countermeasures.

[0668] Notifications and feedback to users

[0669] Users receive analysis results from the server on their devices. This includes the details and risk level of any detected fraudulent activity. Based on this information, users take the measures recommended by the system (e.g., disabling the relevant AI model or applying corrective patches). The results of the measures taken by the user are fed back to the server, which is used to improve the accuracy of future analyses.

[0670] In this way, a system is realized that allows users to effectively manage potential risks while maintaining an environment in which they can use new artificial intelligence technologies with peace of mind. This invention makes it possible to enjoy the benefits of AI technology while minimizing the security risks associated with its use.

[0671] The following describes the processing flow.

[0672] Step 1:

[0673] The device detects when new artificial intelligence technology is installed and collects basic information about that AI technology. This includes the AI's identification information and the date and time of its activation.

[0674] Step 2:

[0675] The terminal monitors the operation logs and generated products of registered artificial intelligence technologies in real time and collects related data. This data is stored securely and made available for subsequent analysis.

[0676] Step 3:

[0677] The operation logs and generated data collected from the terminal are sent to the server. The server receives this data and begins analysis.

[0678] Step 4:

[0679] The server verifies the operation of the artificial intelligence technology based on the received data. Specifically, it checks for matches with known fraudulent patterns and anomalous patterns based on past cases.

[0680] Step 5:

[0681] If the analysis reveals fraudulent activity or potential risks, the server performs a risk assessment and sends it as a warning to the terminal. This warning includes details of the detected problem along with recommended countermeasures.

[0682] Step 6:

[0683] The terminal receives a warning from the server and notifies the user. The user is informed of the warning visually or audibly, and is presented with recommended actions from the system.

[0684] Step 7:

[0685] Users take appropriate action based on the notified warnings. For example, they can take immediate action, such as temporarily suspending a suspicious AI model.

[0686] Step 8:

[0687] The terminal feeds back the results of the measures taken by the user to the server. This information is recorded on the server and used to improve the accuracy of future analysis processes.

[0688] (Example 1)

[0689] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0690] In recent years, artificial intelligence technology has been utilized in various fields, but its use has also increased the risk of fraudulent activity and malicious creations. Because such risks can cause significant harm to users, it is essential to use artificial intelligence technology in a safe and secure environment. However, conventional technologies are insufficient for real-time monitoring and fraud detection, necessitating more effective countermeasures.

[0691] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0692] In this invention, the server includes means for detecting newly installed artificial intelligence technology, means for registering the artificial intelligence technology as a target for monitoring, and means for collecting operation records and products in real time. This promotes the safe use of artificial intelligence technology and enables early detection and response to fraudulent activity.

[0693] "Detection" is the process of recognizing and identifying newly introduced artificial intelligence technologies in the user's system.

[0694] "Registration" is the procedure for adding detected artificial intelligence technologies to the security management system as targets for monitoring.

[0695] "Operation logging" refers to the process of meticulously recording a series of operations and resource usage when artificial intelligence technology is running.

[0696] "Products" refer to the output results generated by artificial intelligence technology, and specifically include generated images, text, and other similar elements.

[0697] "Encryption" is a technology that transforms the content of information to securely protect it and prevent unauthorized access during transmission and reception.

[0698] A "remote processing device" refers to a server or central management system that receives and analyzes information transmitted from a terminal.

[0699] "Analysis" is the process of examining received data in detail and comparing it to known patterns of fraudulent behavior.

[0700] "Anomaly" refers to inappropriate AI behavior or its products that could potentially cause harm to the system or users.

[0701] "Risk assessment" is a procedure that quantifies potential risks based on analysis results and determines their severity.

[0702] A "proposed course of action" refers to specific guidelines recommended based on the evaluation results, including safety measures that users should take.

[0703] "Instructions" are pieces of information designed to prompt users to immediately take the measures recommended by the system.

[0704] "Feedback" refers to the process where users report the results of the measures they have taken to the server, and this data is used to improve the accuracy of future analyses.

[0705] This invention provides a system that supports the secure use of artificial intelligence technology by linking a terminal and a server. This system has the function of immediately detecting newly installed artificial intelligence technology on a terminal and collecting its operation records and output in real time.

[0706] When a user installs a new artificial intelligence (AI) technology, the terminal automatically detects it and registers it as a monitored target within the system. For registered AI technologies, an agent begins operating to collect operation logs and product data. This ensures that data is recorded continuously and securely stored within the terminal.

[0707] The collected data is encrypted and sent to the server. The server passively receives this data and uses an analysis engine to determine whether or not malicious activity has occurred. Specifically, it checks whether the text information created by the generative AI model contains inappropriate links or suspicious phrases. The server processes this information using analysis software such as TensorFlow or PyTorch and assesses its risk.

[0708] Users receive analysis results and estimated risk levels via notifications from the server. These notifications are provided visually or audibly on the device, offering specific countermeasures. These include disabling or changing settings for artificial intelligence technology and applying security patches as needed. The actions taken by the user are returned to the server as feedback, and this feedback information is used to further improve the accuracy of the analysis.

[0709] For example, if a user inputs the prompt "Draw a summer day scene in pop art style" into a generation AI model, a similar security analysis process will be applied to the generated image and text. This allows users to use artificial intelligence technology with peace of mind while effectively managing potential risks.

[0710] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0711] Step 1:

[0712] The terminal detects newly installed artificial intelligence technologies. Specifically, the terminal monitors program installation events and retrieves software metadata. The input is information about newly detected programs, and the output is the identification of AI technologies to be registered as targets for monitoring.

[0713] Step 2:

[0714] The terminal registers the detected artificial intelligence technology with the security monitoring system. This initiates the collection of activity logs for the monitored AI technology. The input is the identified AI technology, and the output is a notification that registration to the monitoring list is complete.

[0715] Step 3:

[0716] The terminal collects operation records and output data from the running artificial intelligence technology. Specific collection items include execution time, CPU usage, and generated data (e.g., text and images). The input is operational information from the AI ​​technology, and the output is the collected detailed logs.

[0717] Step 4:

[0718] The terminal encrypts the collected data and prepares it for transmission to the server. Encryption uses methods such as TLS to protect the data. The input is the collected, unprocessed log data, and the output is the encrypted data.

[0719] Step 5:

[0720] The server receives encrypted data sent from the terminal and processes it through its analysis engine. The analysis engine compares the data against a database of known malicious patterns. The input is decrypted, clear log data, and the output is the analysis result and whether or not malicious activity occurred.

[0721] Step 6:

[0722] The server assesses the risks based on the analysis results and generates notification information for the user. Here, a risk score is calculated from the analysis results, and a detailed report including recommended countermeasures is created. The input is the analysis results, and the output is the final evaluation report.

[0723] Step 7:

[0724] The terminal provides the user with notification information from the server. The notifications are displayed visually through the user interface (UI). Input is a detailed evaluation report from the server, and output is a warning message for the user.

[0725] Step 8:

[0726] Users take action based on the notification information. For example, they might temporarily stop using problematic AI technology or change its settings. The input is the recommended action from the server, and the output is the action that was taken.

[0727] Step 9:

[0728] The terminal collects the results of actions taken by the user and sends them to the server as feedback. The collected data will be used to improve the accuracy of future analyses. The input is the user's action result, and the output is feedback data.

[0729] (Application Example 1)

[0730] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0731] In recent years, applications utilizing artificial intelligence technology have rapidly become widespread, but the security risks associated with their use have also increased. In particular, the risk of generating malicious links and phishing attacks can pose a significant threat to general users. To address this challenge, there is a need for a reliable monitoring system that constantly monitors the operation of artificial intelligence technology, immediately detects suspicious activity, and recommends appropriate responses.

[0732] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0733] This invention includes a server that monitors operation logs and generated data and transmits them to an external analysis device via a communication network; a server that checks warning information from the external analysis device and presents detailed information and countermeasures if malicious activity is detected; and a server that collects the results after countermeasures are implemented by the user's specific device and provides feedback to improve the accuracy of the analysis. This minimizes the security risks associated with the use of artificial intelligence technology and allows users to utilize these technologies with peace of mind.

[0734] "Newly installed artificial intelligence technology" refers to AI-related software or applications that have been newly added to the user's device.

[0735] An "operation log" is data that records how artificial intelligence technology operates on a device and what operations are performed.

[0736] "Products" refer to the output generated by artificial intelligence technology during its operation, specifically such as text and images.

[0737] "Abusive behavior" refers to actions performed by artificial intelligence technology that could pose a threat to security and privacy.

[0738] A "warning" is a notification sent to inform users of the risks when fraudulent activity is detected.

[0739] "Guidelines for users to take action" refers to information that shows specific actions and recommended responses that users should take after receiving a warning.

[0740] "Monitoring operation logs and output" means tracking and recording the operation and output of AI technology in real time.

[0741] A "communication network" is a network infrastructure used to transmit data from a terminal to an external analysis device.

[0742] An "external analysis device" is a computer system that analyzes data transmitted from a terminal to determine fraudulent activity.

[0743] "Feedback" is the process of collecting the results of user-submitted actions and using them to inform future analysis and system improvements.

[0744] The system of this invention is designed to effectively implement new applications related to artificial intelligence technology, and is based on the premise of collaboration between a smartphone and an external server.

[0745] The device, in this case a smartphone, monitors the operation logs and output of the installed artificial intelligence technology in real time. The device uses a watchdog library to continuously detect the activity of the AI ​​application and, if any suspicious activity is detected, sends the information to an external analysis device. This allows users to leverage monitoring functions in their daily operations.

[0746] On the server side, a process of analyzing the received data takes place. The server uses the requests library to collect data from smartphones and thoroughly examines the content generated by AI technology. In particular, it identifies suspicious links and phishing words and analyzes them against known fraudulent patterns. Based on the results of this analysis, it assesses the importance and risk and presents the user with detailed warnings and solutions. The analyzed information is fed back to the user's smartphone, and the implementation of countermeasures and the results are used to improve accuracy in the future.

[0747] For example, if a newly installed AI chatbot app regularly sends links to extract personal information from a user, the device records the frequency and content of these links and sends them to a server. The server then analyzes the links in detail, and if it detects a potential phishing attempt, it can warn the user and instruct them to refrain from using the links due to their inappropriateness.

[0748] An example of a prompt for a generated AI model is, "Can you analyze and report whether a newly installed AI application is potentially phishing based on its operation logs?" Using this prompt, the system can provide more detailed information about the AI ​​model's operation and support appropriate security measures.

[0749] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0750] Step 1:

[0751] The device detects newly installed artificial intelligence technologies. The input is a list of installed applications, which is used to begin monitoring activity logs. The device uses the watchdog library to track file system changes in real time and identify newly added AI-related applications. The output is an identified list of monitored applications.

[0752] Step 2:

[0753] The terminal collects the operation logs and generated products of detected artificial intelligence technologies and prepares them for transmission to an external server. In this step, various activity logs obtained from the monitored application are taken as input, and the log information is stored on a temporary storage medium. The output is log data processed into a format that can be transferred to the server.

[0754] Step 3:

[0755] The terminal sends the collected data to an external server. The input is the processed data prepared in step 2, and communication with the server is performed via the HTTP protocol using the requests library. The output is a transmission completion status, indicating that the data has been successfully transferred to the server.

[0756] Step 4:

[0757] The server analyzes incoming data to detect malicious patterns. The input consists of activity logs and generated data sent from the terminal, which are then compared against known patterns in a data calculation. The server executes algorithms specifically to identify suspicious links and phishing phrases, and generates an analysis result indicating whether or not malicious activity has occurred.

[0758] Step 5:

[0759] The server issues a warning to the user based on the analysis results and provides specific countermeasures. The input is the analysis results of the fraudulent activity performed by the server, and the information is processed to provide visual or auditory notifications to the user's device. The output is a warning message and specific recommended actions to the user's device.

[0760] Step 6:

[0761] The user implements countermeasures according to the guidelines provided by the server. The input consists of warning messages and countermeasures displayed on the device, which the user then manually disables the AI ​​application or takes other security measures. The output consists of the device's status after the countermeasures have been implemented and a record of it.

[0762] Step 7:

[0763] The terminal sends the user's countermeasure results back to the server for feedback. The input is the result of the countermeasure taken in step 6, which is provided to the server as data to improve the accuracy of future analyses. The output is the feedback data sent to the server and considerations for improvements in the future based on that data.

[0764] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0765] This invention combines a monitoring system and an emotion engine to ensure the security of newly installed artificial intelligence technology. The system has the ability to detect and warn of fraudulent behavior while recognizing the user's emotional state. Specific embodiments of the system are described below.

[0766] Operation on the device

[0767] The device detects when a user installs new artificial intelligence technology and registers it with the server. Simultaneously with the registration process, the device collects real-time operation logs and generated data from the AI ​​technology. This data is securely stored and ready for transmission to the server.

[0768] Server roles and processing

[0769] The server receives data sent from the terminal and performs advanced analysis. During the analysis, it thoroughly examines the text and behavioral patterns generated by the AI ​​model to check if they match known patterns for detecting fraudulent behavior. It also utilizes an emotion engine to monitor the user's emotional state and incorporates this into the analysis.

[0770] Notifications and feedback to users

[0771] When a user receives a warning from the server, the emotion engine takes the user's emotional state into account and adjusts the notification method accordingly. For example, if the user is feeling stressed, the notification will be delivered carefully and clearly. The warning will include details of the detected problem, along with guidance and recommended actions appropriate to the user's emotional state.

[0772] Specific example

[0773] For example, suppose a user installs a new AI chatbot. The device collects the chatbot's logs and generated messages and sends them to a server. The server checks the messages for phishing links and uses an emotion engine to analyze the user's emotional response to the information they receive. If the user is feeling anxious, the server issues a warning in more reassuring language, encouraging them to take action.

[0774] In this way, the present invention realizes a system that can manage the potential risks associated with the use of AI technology while taking emotions into consideration. By providing an environment in which users can use new technologies with peace of mind, it is possible to maximize the convenience of AI technology.

[0775] The following describes the processing flow.

[0776] Step 1:

[0777] When a new artificial intelligence technology is installed on the device, it detects information about that software and initiates a system-wide monitoring process. This is an automatically registered process and does not require user intervention.

[0778] Step 2:

[0779] The device collects operation logs and generated data in real time. The generated data includes text and results generated by the AI. This collected data is sent to the server using a secure protocol.

[0780] Step 3:

[0781] The server analyzes the data received from the terminal to check whether the artificial intelligence technology is behaving maliciously or abnormally. The analysis includes matching against known malicious patterns and blacklists.

[0782] Step 4:

[0783] The emotion engine embedded in the server also evaluates the user's emotional state. Specifically, it analyzes the user's operation history and feedback to understand changes in their emotions.

[0784] Step 5:

[0785] If the analysis detects fraudulent activity, the server notifies the terminal of the nature of the risk and the countermeasures. At this time, based on the evaluation of the emotion engine, the notification is adjusted to suit the user's emotional state. For example, if the situation is urgent but the user is experiencing stress, the notification will be carefully directed to encourage calm and rational action.

[0786] Step 6:

[0787] The terminal communicates warnings sent from the server to the user. It presents the information in a visually easy-to-understand format and provides specific instructions on necessary countermeasures.

[0788] Step 7:

[0789] Users take action according to the suggested measures. For example, they may temporarily suspend AI models deemed fraudulent in accordance with warning messages, following the guidelines provided by the system.

[0790] Step 8:

[0791] The terminal feeds back the user's response to the server, where it is further analyzed using an emotion engine. This feedback information will be used to improve the accuracy of fraud detection in the future.

[0792] (Example 2)

[0793] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0794] In recent years, with the development and widespread adoption of artificial intelligence technology, concerns about the security associated with its use have increased. Specifically, there are risks such as fraudulent activity, unintentional data leaks, and inappropriate manipulation of users' emotional states. This invention aims to solve these problems and provide an environment in which users can use artificial intelligence with peace of mind.

[0795] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0796] In this invention, the server includes means for identifying newly introduced artificial intelligence technology, means for a central management device to analyze information and detect the presence or absence of fraudulent activity, and means for identifying the user's emotional state using an emotion analysis engine. This ensures user safety and enables warnings that take the user's emotions into consideration.

[0797] "Artificial intelligence technology" refers to technologies that enable intelligent behavior and judgment to be imitated by information processing systems, and encompasses the field of learning from large-scale data using algorithms and models.

[0798] "Action records" refer to data that sequentially records various processes performed by artificial intelligence technology and their results.

[0799] "Generated products" refer to information or content generated by the operation of artificial intelligence technology.

[0800] A "central management device" refers to an integrated system of hardware and software that receives information transmitted from terminals and performs analysis and interpretation.

[0801] "Fraudulent activity" refers to actions involving unauthorized operations or inappropriate behavior that may compromise security or privacy.

[0802] An "emotional analysis engine" refers to a program or algorithm that identifies a user's emotional state based on linguistic or behavioral data.

[0803] "User" refers to an entity that operates or receives services using artificial intelligence technology.

[0804] "Warning information" refers to information that informs users about fraudulent activities or other risks and encourages them to take appropriate action.

[0805] This invention constructs a system for data monitoring and sentiment analysis between terminals, servers, and users to ensure the security of newly introduced artificial intelligence technology. The specific form of this system is described below.

[0806] Operation on the device

[0807] The terminal has the function to identify when a user newly installs artificial intelligence technology and registers the identified information with a central management device. To achieve this, the terminal performs accurate monitoring using installation detection software. The terminal also sequentially collects operation records and generated products, and temporarily stores them in an encrypted form for security purposes. The data is transmitted to the server via a stable communication protocol.

[0808] Server Analysis

[0809] The server receives information sent from the terminal and analyzes it based on specific rules. First, it uses a generative AI model to scrutinize the transmitted text information and identify fraudulent activity. Next, it utilizes an emotion analysis engine to identify the user's emotional state. By combining large-scale data analysis techniques and AI models for information processing, it ensures fast and accurate results.

[0810] User notifications and feedback

[0811] Users receive analysis results sent from the server, and if fraudulent activity is detected, they are prompted to take specific actions to address the risk. Here, notifications are delivered in a way that takes into account and adjusts the user's emotional state. This system allows users to use new technologies with peace of mind.

[0812] Specific example

[0813] For example, consider a scenario where a user installs a new AI chatbot. The device collects the chatbot's activity log and generated messages and sends them to a server. The server analyzes the messages for suspicious links and simultaneously evaluates the user's emotional response. For example, if the server detects anxiety in response to a message like "Username, we are concerned about your safety," it will notify the user with an emotionally sensitive warning such as "Please stay calm, it's okay. The link is not safe, so please do not click it." An example of a prompt message would be "a method to send data collected when a new chatbot is installed, analyze its content and its impact on emotions, and display a friendly warning if the user is feeling anxious."

[0814] This invention aims to create an environment in which AI technology can be used with greater confidence while ensuring user safety.

[0815] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0816] Step 1:

[0817] The terminal monitors newly introduced artificial intelligence technologies. When a user installs a new AI application, monitoring software on the terminal identifies it and collects related information. Specifically, installation log information and application identifiers are collected and prepared to be sent to the server. The input is the installation information of the AI ​​technology, and the output is the identified installation information.

[0818] Step 2:

[0819] The terminal collects operational records and generated products of AI technology. The terminal monitors the operation of the AI ​​application in real time, sequentially recording generated messages and logs. This input data (operational logs and generated products) is temporarily stored using encryption technology and prepared for secure transmission to the server. The output is encrypted operational record and generated product data.

[0820] Step 3:

[0821] The terminal sends the collected data to the server. Using an established, secure communication protocol, the terminal sends the collected data to the server. In this process, the input is encrypted data, and the output is the data sent to the server.

[0822] Step 4:

[0823] The server analyzes the transmitted data. Based on the data received from the terminal, the server uses a generative AI model to scrutinize text and behavioral patterns. Specifically, it detects patterns of fraudulent activity and identifies known suspicious links and words. The input in this process is the data sent to the server, and the output is whether or not fraudulent activity was detected.

[0824] Step 5:

[0825] The server uses an emotion analysis engine to identify the user's emotional state. In addition to analysis, the server uses the emotion analysis engine to incorporate the user's emotional state as part of the data analysis. The input is text information sent to the server, and the output is the user's emotional information.

[0826] Step 6:

[0827] The system provides warnings and feedback to users. If fraudulent activity is detected, the server sends a warning message to the user. The notification content is adjusted according to the user's emotional state, providing appropriate feedback. The input consists of analyzed data and user emotional information, while the output is the notification and feedback delivered to the user.

[0828] This entire process allows users to manage the potential risks associated with using AI technology, enabling them to utilize the technology safely and with peace of mind.

[0829] (Application Example 2)

[0830] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0831] Newly installed artificial intelligence technologies may pose unknown risks to users. However, existing monitoring systems focus on detecting fraudulent behavior, lacking the ability to provide feedback and guidance that considers the user's emotional state. This lack of focus can lead to users experiencing excessive stress and anxiety when warned about fraudulent activity, making it difficult for them to take appropriate action.

[0832] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0833] In this invention, the server includes means for analyzing the user's emotional state using an emotion recognition engine and adjusting the notification method for detecting fraudulent behavior; means for analyzing information contained in the products of artificial intelligence technology and identifying suspicious elements; and means for providing notifications on the information terminal in a manner optimized to the user's emotional state. This enables feedback in a form that is less burdensome for the user and makes it possible to take more effective measures against fraudulent behavior.

[0834] "Artificial intelligence technology" refers to technology that uses computers to perform autonomous and intelligent actions.

[0835] An "operation log" is data that records the history of various actions and states that occur while a system or program is running.

[0836] "Generated products" refer to the output results generated by artificial intelligence technology, and these include text, images, audio, and other similar elements.

[0837] "Fraudulent behavior" refers to actions that differ from the intended behavior or that may harm the user.

[0838] A "warning" is a notification intended to inform the user that there is some kind of danger or malfunction.

[0839] An "emotion recognition engine" is software or an algorithm used to analyze a user's emotional state.

[0840] "Information terminals" refers to all electronic devices, including smartphones, tablets, and smart glasses.

[0841] This invention constructs a system in which a terminal and a server work together to ensure the security of newly installed artificial intelligence (AI) technology. When a user installs new AI technology, the terminal automatically detects it and registers it with the server. Subsequently, the terminal collects the operation logs and generated products of the AI ​​technology in real time and transmits them to the server in a secure manner.

[0842] The server receives the collected data and performs advanced analysis. This analysis uses an emotion recognition engine (e.g., IBM's Watson Emotion Analysis) to analyze the user's emotional state and adjust notification methods for detecting fraudulent behavior. It also analyzes information contained in the products of artificial intelligence technology to identify suspicious elements such as phishing links.

[0843] Users receive notifications in an appropriate manner through their information devices (e.g., smartphones, smart glasses). Notifications are customized based on the user's emotional state and delivered in a reassuring way. For example, if a user traveling installs a tourist information app and that app generates a suspicious link, the system will identify it and notify them with a message such as, "This may be a phishing attempt; please delete it without clicking the link."

[0844] An example of a prompt used in a generative AI model is, "The user's current emotion is anxiety. Please suggest a way to provide reassurance through notifications." This format allows users to continue using new AI technologies with peace of mind.

[0845] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0846] Step 1:

[0847] The device detects when the user installs new artificial intelligence technology. The input is a list of installed applications, and the output is the identification information of that application sent to the server. Specifically, the device scans the operating system's installation logs to identify new AI applications.

[0848] Step 2:

[0849] The device collects real-time operation logs and output data of detected artificial intelligence technologies. Input data is related to application activity, and the collected log data is prepared for transmission to the server. Specifically, the device uses APIs and system logs to compile information on application execution status and output (e.g., output text).

[0850] Step 3:

[0851] The server receives data sent from the terminal and performs analysis. The input is log data sent from the terminal, and the output is the result regarding the presence or absence of fraudulent activity and the user's emotional state. Specifically, the server uses an emotion recognition engine to analyze the data and check for matches with known fraudulent activity patterns.

[0852] Step 4:

[0853] The server optimizes notification content and method based on the user's emotional state. The input is the analysis result from the emotion recognition engine, and the output is a customized notification message. Specifically, it sends a prompt to a generation AI model to determine the optimal notification method based on the user's current emotional state.

[0854] Step 5:

[0855] The server sends optimized notifications to information terminals, notifying the user. The input is a notification message, and the output is a visual or auditory alert to the user. Specifically, the terminal's notification function is used to display warnings using language that takes the user's emotional state into consideration.

[0856] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0857] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0858] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0859] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0860] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0861] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0862] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0863] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0864] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0865] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0866] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0867] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0868] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0869] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0870] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0871] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0872] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0873] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0874] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0875] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0876] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0877] The following is further disclosed regarding the embodiments described above.

[0878] (Claim 1)

[0879] A means for detecting newly installed artificial intelligence technology,

[0880] A means for collecting the operation logs and generated products of the aforementioned artificial intelligence technology in real time,

[0881] A means of analyzing the collected data and determining whether or not fraudulent activity has occurred,

[0882] A means for issuing a warning when the aforementioned fraudulent activity is detected,

[0883] A means of providing guidance to users to take countermeasures based on the information of the aforementioned warning,

[0884] A system that includes this.

[0885] (Claim 2)

[0886] The system according to claim 1, characterized in that the analysis means analyzes text information contained in the products of artificial intelligence technology and identifies suspicious links or phrases.

[0887] (Claim 3)

[0888] The system according to claim 1, characterized in that the means for issuing the warning provides visual or auditory notification to the user at the terminal.

[0889] "Example 1"

[0890] (Claim 1)

[0891] A means for detecting newly installed artificial intelligence technology,

[0892] Means for registering the aforementioned artificial intelligence technology as a target for monitoring,

[0893] Means for collecting the operation records and products of the aforementioned artificial intelligence technology in real time,

[0894] A means for encrypting the collected information and transmitting it to a remote processing device,

[0895] The remote processing device analyzes the information content and determines whether or not an illegal event has occurred,

[0896] A means for evaluating the risks and formulating countermeasures based on the aforementioned analysis results,

[0897] A means of providing instructions to users to take appropriate action based on the aforementioned proposed countermeasures,

[0898] A means for collecting the results of the aforementioned appropriate actions as feedback,

[0899] A system that includes this.

[0900] (Claim 2)

[0901] The system according to claim 1, characterized in that the analysis means analyzes textual information contained in the products of artificial intelligence technology and identifies suspicious links or expressions.

[0902] (Claim 3)

[0903] The system according to claim 1, characterized in that the means for providing the instructions provides visual or auditory notification to the user at the terminal.

[0904] "Application Example 1"

[0905] (Claim 1)

[0906] A means for detecting newly installed artificial intelligence technology,

[0907] A means for collecting the operation logs and generated products of the aforementioned artificial intelligence technology in real time,

[0908] A means of analyzing the collected data and determining whether or not fraudulent activity has occurred,

[0909] A means for issuing a warning when the aforementioned fraudulent activity is detected,

[0910] A means of providing guidance to users to take countermeasures based on the information of the aforementioned warning,

[0911] A means for monitoring operation logs and generated data and transmitting them to an external analysis device via a communication network,

[0912] A means of checking warning information from external analytical devices and providing detailed information and countermeasures if fraudulent activity is detected,

[0913] A system that includes this.

[0914] (Claim 2)

[0915] The system according to claim 1, characterized in that the analysis means analyzes text information contained in the product of artificial intelligence technology, identifies suspicious links and phrases, and further transmits the analysis results to the user device.

[0916] (Claim 3)

[0917] The system according to claim 1, characterized in that the means for issuing the warning notifies the user visually or audibly at the terminal, and also collects the results after the user has taken countermeasures using a specific device and provides feedback to improve the accuracy of the analysis.

[0918] "Example 2 of combining an emotion engine"

[0919] (Claim 1)

[0920] A means of identifying newly introduced artificial intelligence technologies,

[0921] A means for sequentially collecting operational records and generated products of the aforementioned artificial intelligence technology,

[0922] Means for transmitting the collected information to a central management device using a communication path,

[0923] The aforementioned central control device includes means for analyzing information and detecting whether or not fraudulent activity has occurred,

[0924] The aforementioned central management device utilizes an emotion analysis engine as part of its analysis to identify the user's emotional state,

[0925] A means for detecting the aforementioned fraudulent activity and providing information that adjusts to warn the user while taking into consideration their emotional state,

[0926] A means of providing recommendations to users to take countermeasures based on the aforementioned warning information,

[0927] A system that includes this.

[0928] (Claim 2)

[0929] The system according to claim 1, characterized in that the analysis means analyzes textual information contained in the product generated by artificial intelligence technology, identifies suspicious hyperlinks and words, and further analyzes the user's emotional response.

[0930] (Claim 3)

[0931] The system according to claim 1, characterized in that the means for prompting attention provides visual or auditory notification to the user in the information recording device and provides feedback based on the user's emotions.

[0932] "Application example 2 when combining with an emotional engine"

[0933] (Claim 1)

[0934] A means for detecting newly installed artificial intelligence technology,

[0935] A means for collecting the operation logs and generated products of the aforementioned artificial intelligence technology in real time,

[0936] A means of analyzing the collected data and determining whether or not fraudulent activity has occurred,

[0937] A means for issuing a warning when the aforementioned fraudulent activity is detected,

[0938] A means of providing guidance to users to take countermeasures based on the information of the aforementioned warning,

[0939] A means of analyzing a user's emotional state using an emotion recognition engine and adjusting the notification method for detecting fraudulent behavior,

[0940] A system that includes this.

[0941] (Claim 2)

[0942] The system according to claim 1, characterized in that the analysis means analyzes information contained in the product of artificial intelligence technology and identifies suspicious elements.

[0943] (Claim 3)

[0944] The system according to claim 1, characterized in that the means for issuing the warning provides notification in a manner optimized to the user's emotional state on an information terminal. [Explanation of symbols]

[0945] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for detecting newly installed artificial intelligence technology, A means for collecting the operation logs and generated products of the aforementioned artificial intelligence technology in real time, A means of analyzing the collected data and determining whether or not fraudulent activity has occurred, A means for issuing a warning when the aforementioned fraudulent activity is detected, A means of providing guidance to users to take countermeasures based on the information of the aforementioned warning, A system that includes this.

2. The system according to claim 1, characterized in that the analysis means analyzes text information contained in the products of artificial intelligence technology and identifies suspicious links or phrases.

3. The system according to claim 1, characterized in that the means for issuing the warning provides visual or auditory notification to the user at the terminal.

Images