system

Abstract

We provide the system. [Solution] A means of having the company's authentication system authenticate the user, A means of obtaining the security policy from the server if authentication is successful, A means of applying the acquired security policy to the browser, A means of monitoring user input in real time and detecting confidential information, A means to block user input and display an alert if confidential information is detected, A means of sending an operation log to the server when the input of confidential information is detected, A means of collecting operation logs from the server and managing security incidents, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, the risk of information leakage in enterprises has been increasing, and in particular, there have been many cases where confidential information and source code are leaked to the outside through web browsers. Therefore, while enterprises enable employees to perform their work safely and efficiently, they are seeking effective means to prevent information leakage. However, it is difficult for conventional security measures to completely prevent improper acts by attackers with a certain level of skills, and in particular, it has also been a problem that employees accidentally transmit confidential information to the outside. Thus, an object of the present invention is to solve these problems by providing a secure browser system that cooperates with an enterprise authentication system and prevents leakage of confidential information in real time.

Means for Solving the Problems

[0005] This invention comprises means for authenticating a user to a company's authentication system, means for obtaining a security policy from a server upon successful authentication, means for applying the obtained security policy to a browser, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, means for sending an operation log to the server when confidential information is detected, and means for the server to collect the operation log and manage security incidents. This makes it possible to control employee operations based on the company's information security policy and prevent the leakage of confidential information. In addition, by notifying administrators of security incidents that occur on the server, a rapid response is possible, improving the overall security level of the company.

[0006] An "authentication system" is a mechanism that verifies a user's identity and allows that user to access authorized information and functions.

[0007] A "server" is a computer system that provides services and resources to other computers on a network.

[0008] A "security policy" is a set of rules and guidelines set out by a company that define its information security management policies and specific measures.

[0009] A "browser" is software used to view websites and web applications on the internet.

[0010] "Confidential information" refers to important information that companies and individuals must keep secret, and whose leakage could lead to negative consequences.

[0011] "Real-time monitoring" refers to instantly analyzing information entered by users and taking immediate action based on the results.

[0012] A "regular expression pattern" is a set of strings that represent rules for extracting specific strings, and is used for pattern matching.

[0013] An "alert" is a warning message that notifies users or administrators when specific conditions are met.

[0014] An "operation log" is a record of the operations performed by a user within the system, and it is saved so that it can be reviewed and analyzed later.

[0015] A "security incident" is an event in which a threat or abnormal situation related to information security occurs.

[0016] An "administrator" is a person responsible for monitoring and managing the operation of a system or network.

[0017] "Detection" is the process of confirming and discovering whether a particular condition or pattern exists.

[0018] "Applying" means reflecting the received policies and settings in the system or software. [Brief explanation of the drawing]

[0019] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6]It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying out the Invention

[0020] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0021] First, the language used in the following description will be explained.

[0022] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0023] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0024] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0025] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0027] [First Embodiment]

[0028] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0029] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0032] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0035] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0039] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0040] This invention is a system that prevents the leakage of confidential information by applying security policies to corporate browsers and monitoring user input in real time. This system mainly consists of a server, a terminal (user's browser), and the user.

[0041] Server operation

[0042] The server manages security policies and distributes them to endpoints. The server connects to the company's authentication system and verifies user credentials. Upon successful authentication, the server sends the appropriate security policy to the device. In the event of a security incident, the server collects the information and notifies the administrator.

[0043] Terminal (user's browser) operation

[0044] The device receives security policies sent from the server and monitors user input based on those policies. When a user launches a browser, authentication is performed first. If authentication is successful, the device downloads the latest security policy from the server and loads it into memory.

[0045] When a user enters text into a browser input field, the terminal monitors the input in real time. Using regular expression patterns defined in the policy, the terminal inspects the input. If sensitive information is detected, the terminal blocks the input and displays an alert to the user. It also sends an operation log to the server.

[0046] User actions

[0047] The user first launches a browser and logs into the company's authentication system. Once authentication is successful, the user begins using the browser as usual. If the user attempts to enter certain sensitive information into the browser, the input is monitored in real time, and if sensitive information is detected, the input is blocked and an alert is displayed.

[0048] Specific example

[0049] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user opens an email sending screen and attempts to enter "technical details of a patent application," the device's browser monitors this input in real time. Because a regular expression pattern for "technical details of a patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[0050] In this way, the system can apply a company's security policies to the browser and prevent the leakage of confidential information in real time. Furthermore, security incidents that occur are recorded and managed on the server, facilitating a rapid response.

[0051] The following describes the processing flow.

[0052] Step 1:

[0053] The device launches a browser. The user accesses the login screen and enters their authentication information into the company's authentication system.

[0054] Step 2:

[0055] The server receives the user's authentication information and uses it to verify the user's identity. If authentication is successful, the server generates an appropriate security policy and sends it to the terminal.

[0056] Step 3:

[0057] The device receives the security policy sent from the server and applies it within the browser. Specifically, it loads a regular expression pattern for detecting sensitive information into memory.

[0058] Step 4:

[0059] The user begins typing text within the browser. As the user enters information into the input field, that information is monitored in real time within the browser.

[0060] Step 5:

[0061] The terminal analyzes the entered text based on security policies. It checks for content that matches regular expression patterns.

[0062] Step 6:

[0063] If the device detects sensitive information, it will block the input and display an alert message to the user. The message will read, "This contains sensitive information. Input has been blocked."

[0064] Step 7:

[0065] The terminal logs the input of confidential information and sends that log to the server. The log includes information such as the user's input and the time when the alert occurred.

[0066] Step 8:

[0067] The server saves the received logs to the database. The server sends an alert to notify the administrator that a security incident has occurred.

[0068] Step 9:

[0069] The administrator receives an alert notification from the server, reviews the details of the security incident, and takes the necessary action.

[0070] In this way, a system is realized that monitors user input in real time throughout each processing step, thereby preventing the leakage of confidential information.

[0071] (Example 1)

[0072] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0073] In today's business environment, the leakage of confidential information poses a significant risk, requiring advanced security measures. However, existing security systems struggle to monitor user input in real time and immediately detect and address confidential information breaches. As a result, the risk of confidential information leakage remains, and companies' information assets cannot be adequately protected. This invention aims to solve these problems and provide a system that effectively protects companies' confidential information.

[0074] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0075] In this invention, the server includes means for authenticating the user, means for obtaining a security policy from the server upon successful authentication, and means for applying the obtained security policy to the user interface. This enables the application of a security policy to the terminal after user authentication by coordinating with the company's authentication system. The terminal also includes means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, and means for sending an operation log to the server when input of confidential information is detected. This prevents the leakage of confidential information and enables a rapid response. Furthermore, the server includes means for collecting operation logs and managing security incidents, which notifies the administrator of the occurrence of a security incident and enables appropriate countermeasures to be taken. This makes it possible to comprehensively protect the company's confidential information.

[0076] "User authentication means" refers to a function that verifies a user's authentication information and confirms whether the user has legitimate access rights.

[0077] The "means for obtaining security policies" refer to a function that downloads security policies from the server to users who have successfully authenticated and applies them to their devices.

[0078] "Means of application to the user interface" refers to a function that reflects the acquired security policy in the user's browser or other user interfaces.

[0079] "Real-time monitoring of user input" refers to a function that monitors data entered by users in a browser or other interface in real time and performs appropriate checks.

[0080] A "means for detecting confidential information" is a function for determining whether or not the monitored input data contains confidential information.

[0081] The "means for blocking user input and displaying alerts" are functions that prevent user input and display a warning message when confidential information is detected.

[0082] The "means for sending operation logs to the server" refers to a function for sending operation history, such as the detection of confidential information and the display of associated alerts, to the server.

[0083] "A means for collecting operation logs and managing security incidents" refers to a function that stores transmitted operation logs on a server and centrally manages security incidents.

[0084] A "regular expression pattern" is a format for defining a pattern of strings, and is used to determine whether a specific string is found.

[0085] "Administrator notification method" refers to a function that notifies administrators via email, alerts, or other methods when a security incident occurs.

[0086] This invention is a system that prevents the leakage of confidential information by applying security policies to a company's user interface (e.g., a browser) and monitoring user input in real time. This system mainly consists of a server, a terminal (the user's browser), and the user.

[0087] Server configuration and operation

[0088] The server implements the following functions:

[0089] 1. User Authentication Function: Receives authentication requests from users and verifies user information using the company's authentication system (e.g., LDAP or OAuth). This function is a crucial step in verifying whether the user has legitimate access rights.

[0090] 2. Security Policy Retrieval and Distribution Function: Upon successful authentication, the server retrieves the appropriate security policy from the database and sends it to the user's device. The security policy is managed as data in JSON format.

[0091] 3. Operation log collection and management function: Operation logs sent from terminals are recorded in a database, and security incidents are managed centrally.

[0092] The server can use a Linux®-based system, and the database can use a common database management system such as MySQL® or PostgreSQL.

[0093] Terminal (user's browser) configuration and operation

[0094] The device has the following features:

[0095] 1. Authentication Function: When a user launches a browser, they enter authentication information and send an authentication request to the server. This is expected to work with common web browsers such as Chrome and Firefox.

[0096] 2. Security policy reception and application function: Upon successful authentication, the security policy is downloaded from the server, loaded into memory, and applied to the user interface.

[0097] 3. Real-time monitoring function: User input is monitored in real time, and whether it contains sensitive information is checked using regular expression patterns defined by policy. For example, a script using JavaScript® can be created and executed in the browser.

[0098] 4. Input Blocking and Alert Display Function: If confidential information is detected, input will be blocked and an alert will be displayed to the user.

[0099] 5. Operation Log Transmission Function: Detected security incidents are sent to the server as operation logs.

[0100] User actions

[0101] The user performs the following actions:

[0102] 1. Launching the browser and authenticating: Launch your browser and log in to the company's authentication system.

[0103] 2. Normal browser operation: After successful authentication, the user uses the browser as usual. For example, they enter text on the email sending screen.

[0104] 3. Alert response: If an alert is displayed when attempting to enter specific confidential information, review the content of the alert and take appropriate action.

[0105] Specific example

[0106] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user attempts to enter the text "Technical details of patent application" on the email sending screen, the device's browser monitors this input in real time. Because a regular expression pattern for "Technical details of patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[0107] Examples of prompt statements

[0108] Prompt example:

[0109] "Please describe a system that monitors in real time when a user attempts to enter sensitive information in their browser, blocks the input if sensitive information is detected, and displays an alert."

[0110] This invention allows a company's security policies to be applied to the user interface, preventing the leakage of confidential information in real time. Furthermore, security incidents that occur are recorded and managed on the server, facilitating a rapid response.

[0111] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0112] Step 1:

[0113] User authentication request submission

[0114] Input: The user enters their authentication information (username and password) into the browser's login form.

[0115] Output: The entered authentication information is sent to the server.

[0116] Specific action: The user launches a browser, enters their authentication information into the login form, and clicks the "Login" button. This request is sent to the server as an HTTP POST request.

[0117] Step 2:

[0118] Authentication information verification on the server

[0119] Input: The server receives authentication information sent by the user.

[0120] Output: The result of authentication success or failure is generated.

[0121] Specific operation: The server sends a query to the company's authentication system (LDAP or OAuth) to verify the received authentication information and receives a response. If authentication is successful, an authentication token is generated and sent to the user.

[0122] Step 3:

[0123] Sending security policies

[0124] Input: A database containing user information and security policies for users who have successfully authenticated.

[0125] Output: The latest security policy is sent to the device.

[0126] Specific operation: The server retrieves the security policy for the relevant user from the database and sends it to the user's browser as an HTTP response in JSON format.

[0127] Step 4:

[0128] Download and apply security policies

[0129] Input: Security policy received by the terminal from the server.

[0130] Output: The security policy is loaded into the device's memory.

[0131] Specific operation: The browser analyzes the security policy it receives, loads the settings for real-time monitoring into memory according to its content, and applies them.

[0132] Step 5:

[0133] Real-time monitoring of user input

[0134] Input: The user enters text into the input field.

[0135] Output: Input content is checked in real time.

[0136] Specific operation: When a user starts typing text on the email sending screen, JavaScript is used to monitor the input in real time within the browser. A specific regular expression pattern is used based on policy.

[0137] Step 6:

[0138] Detection and blocking of sensitive information

[0139] Input: User input and a regular expression pattern defined by the policy.

[0140] Output: If sensitive information is detected, the input will be blocked and an alert will be displayed.

[0141] Specific operation: JavaScript checks the input content against a regular expression pattern defined in the policy, blocks user input if it contains sensitive information, and displays a warning message.

[0142] Step 7:

[0143] Sending operation logs

[0144] Input: Operation log when confidential information is detected.

[0145] Output: Operation logs are sent to the server.

[0146] Specific operation: A history of operations such as the detection of confidential information and the display of associated alerts is generated and sent to the server as an HTTP request.

[0147] Step 8:

[0148] Operation log collection and security incident management

[0149] Input: Operation log sent from the terminal.

[0150] Output: Operation logs are recorded in the database, and security incidents are managed.

[0151] Specific actions: The server saves the operation logs it receives to a database and lists them in a format viewable by the administrator. It also notifies the administrator of any security incidents that occur.

[0152] (Application Example 1)

[0153] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0154] In recent years, the risk of confidential information leaks within companies has increased, with unauthorized information sharing via the internet being a particular problem. There is a need for technological measures to prevent the leakage of confidential information by monitoring employee input in real time when using web browsers. Furthermore, the ability to quickly display alerts to users through different devices (such as head-mounted displays) is also essential.

[0155] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0156] In this invention, the server includes means for authenticating a user to the company's authentication system, means for obtaining a security policy from the server if authentication is successful, means for applying the obtained security policy to the browser, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert on a head-mounted display if confidential information is detected, means for sending an operation log to the server if input of confidential information is detected, and means for the server to collect the operation log and manage security incidents. This makes it possible to monitor user input in real time, prevent the leakage of confidential information, and immediately display warnings through different devices.

[0157] A "corporate authentication system" is a system that authenticates users in order to control access to a company's network and systems.

[0158] A "security policy" refers to a set of standards and rules that define the protection of data in information systems and the prevention of unintentional information leaks.

[0159] A "browser" is software used to connect to the internet and display and interact with web pages.

[0160] A "head-mounted display" is a wearable device that is worn on the user's head and provides information visually through a display.

[0161] User authentication is the process of verifying that a user accessing a system is a legitimate user.

[0162] "Real-time monitoring" refers to the immediate monitoring of systems and processes, and the instantaneous detection and response to changes in status or anomalies.

[0163] "Confidential information" refers to important internal company information that is not made public and which, if revealed to a third party, could cause damage to the company.

[0164] An "alert" is a means of notifying a user and issuing a warning when specific conditions occur.

[0165] An "operation log" refers to a record of a series of operations or events performed on a system, which is used later for analysis and auditing.

[0166] A "security incident" refers to an event in which a problem or risk related to information security occurs.

[0167] This invention is a system that authenticates users using a company's authentication system and retrieves a security policy from the server upon successful authentication. This system applies the retrieved security policy to the browser and monitors user input in real time to prevent the leakage of confidential information. Furthermore, a key feature is the ability to display an alert on a head-mounted display (HMD) when confidential information is detected, enabling immediate response.

[0168] First, the user logs into the company's authentication system using a browser or HMD. During this process, authentication information is sent to the server, which then authenticates the user. Upon successful authentication, the server retrieves the appropriate security policy and sends it to the user's device or HMD. The security policy includes a regular expression pattern for identifying sensitive information.

[0169] When a user enters text via a browser or HMD, the device monitors the input in real time. If the input matches a regular expression pattern defined in the security policy, it is determined to be sensitive information, and the input is immediately blocked. At this time, a warning message is displayed on the HMD, and the user is aware that the input of sensitive information has been blocked. In parallel, an operation log is sent to the server, which records it and manages security incidents.

[0170] The specific configuration of this system is as follows:

[0171] 1. Hardware to be used:

[0172] Head-mounted display (HMD)

[0173] User's device (PC or tablet)

[0174] 2. Software to use:

[0175] Authentication system: Performs user authentication.

[0176] Security policy management system: Distributes security policies that include regular expression patterns.

[0177] Real-time monitoring software: Monitors input content in real time and detects confidential information.

[0178] The following scenarios are possible as specific examples:

[0179] For example, consider a scenario where an employee using "Secure Browsing" within a company wears an HMD (Head-Mounted Display) while working. When the user attempts to type "technical details of a patent-pending application" into the email composition screen, a real-time alert is displayed on the HMD, blocking the input. The alert displays "Confidential information is contained. Input has been blocked," and this information is simultaneously sent to the server as a log.

[0180] Examples of prompts for a generative AI model include the following:

[0181] "A user is wearing an HMD (Head-Mounted Display) and browsing the web. I want to build a system that displays a real-time warning and blocks input if the user attempts to enter sensitive information. I will implement this using Python. The security policy will be provided by the server, and user input will be checked using regular expressions. Operation logs will be sent to the server, and alerts will be displayed on the HMD. Please tell me how to design the code."

[0182] In this way, the system of this invention can enhance corporate information security and prevent the leakage of confidential information.

[0183] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0184] Step 1:

[0185] The server receives the user's authentication information.

[0186] Input: User ID and password

[0187] Output: Authentication result (success / failure)

[0188] Specific operation: The user sends their entered ID and password to the server, which then uses its authentication system to perform authentication. If authentication is successful, the user proceeds to the next step.

[0189] Step 2:

[0190] The server retrieves the security policy and sends it to the terminal.

[0191] Input: Authentication result (success / failure)

[0192] Output: Security Policy

[0193] Specific operation: For a user who has successfully authenticated, the server searches for an appropriate security policy and sends it to the terminal. The security policy includes a regular expression pattern to identify sensitive information.

[0194] Step 3:

[0195] The device loads the security policy into memory.

[0196] Input: Security Policy

[0197] Output: Security policy loaded into memory

[0198] Specific operation: The terminal analyzes the security policy received from the server and loads it into memory. This prepares the terminal to monitor user input.

[0199] Step 4:

[0200] The user enters text via a browser or head-mounted display (HMD).

[0201] Input: User input text

[0202] Output: Monitored input text

[0203] Specific operation: The user enters text using a browser or HMD, and this input is monitored in real time.

[0204] Step 5:

[0205] The terminal monitors input text in real time and uses regular expression patterns to detect sensitive information.

[0206] Input: Input text, regular expression pattern

[0207] Output: Confidential information detection result (detected / not detected)

[0208] Specific operation: The terminal checks the input text against a regular expression pattern defined in the security policy to determine if it contains sensitive information.

[0209] Step 6:

[0210] If confidential information is detected, the terminal will block input and display an alert on the head-mounted display (HMD).

[0211] Input: Confidential information detection results

[0212] Output: Alert message, blocked input text

[0213] Specific operation: If the device detects sensitive information, it will block the input and display an alert message on the HMD stating, "Sensitive information is contained. Input has been blocked."

[0214] Step 7:

[0215] If the terminal detects the input of confidential information, it will send an operation log to the server.

[0216] Input: Confidential information detection results, input text

[0217] Output: Operation Log

[0218] Specific action: Send an operation log containing the detected confidential information and detection results to the server.

[0219] Step 8:

[0220] The server collects operation logs and manages them as security incidents.

[0221] Input: Operation Log

[0222] Output: Security incident log

[0223] Specific operation: The server saves the received operation logs to a database and manages them as security incidents.

[0224] Step 9:

[0225] The server notifies the administrator of the occurrence of a security incident.

[0226] Input: Security Incident Log

[0227] Output: Notification message

[0228] Specific operation: The server notifies the administrator of recorded security incidents to encourage a quick response.

[0229] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0230] This invention achieves more advanced information leakage prevention and security management by combining a browser system that applies a company's security policy and monitors user input in real time with an emotion engine that recognizes user emotions. This system mainly consists of a server, a terminal (the user's browser), and the user.

[0231] Server operation

[0232] The server manages security policies and distributes them to endpoints. It connects to the company's authentication system and verifies user credentials. Upon successful authentication, the server generates an appropriate security policy and sends it to the device. In the event of a security incident, the server collects the information and notifies the administrator. The server also manages data from the sentiment engine and uses it to improve or enforce security policies as needed.

[0233] Terminal (user's browser) operation

[0234] The device receives security policies sent from the server and monitors user input based on those policies. When a user launches a browser, authentication is performed first. If authentication is successful, the device downloads the latest security policy from the server and loads it into memory. Furthermore, the device has an emotion engine built in that recognizes emotions in real time from the user's facial expressions and vocalizations.

[0235] When a user enters text into a browser input field, that information is monitored in real time within the browser. The device inspects the input using regular expression patterns defined in the policy. If sensitive information is detected, the device blocks the input and displays an alert to the user. It also sends an operation log to the server.

[0236] Simultaneously, the emotion engine analyzes the user's facial expressions and vocalizations to recognize their emotional state in real time. This emotional data is then used to inform actions based on security policies. For example, if the system detects that a user is experiencing frustration, it might take steps such as providing appropriate psychological safety measures.

[0237] User actions

[0238] The user first launches a browser and logs into the company's authentication system. Upon successful authentication, the user begins using the browser as usual. If the user attempts to enter specific confidential information into the browser, the input is monitored in real time. If confidential information is detected, the input is blocked and an alert is displayed. Simultaneously, the user's emotions are monitored in real time, and appropriate measures are taken as needed.

[0239] Specific example

[0240] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user opens an email sending screen and attempts to enter "technical details of a patent application," the device's browser monitors this input in real time. Because a regular expression pattern for "technical details of a patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[0241] Furthermore, if the user experiences stress or displays anger during this process, the emotion engine recognizes this. The device sends this emotion data to a server, which then notifies the administrator based on that data. The administrator can then understand the circumstances under which the user experienced stress and provide appropriate support.

[0242] In this way, a sophisticated system that integrates security and user sentiment management is realized through each processing step.

[0243] The following describes the processing flow.

[0244] Step 1:

[0245] The device launches a browser. The user accesses the login screen and enters their authentication information into the company's authentication system.

[0246] Step 2:

[0247] The server receives the user's authentication information and uses it to verify the user's identity. If authentication is successful, the server generates an appropriate security policy and sends it to the terminal.

[0248] Step 3:

[0249] The device receives the security policy sent from the server and applies it within the browser. Specifically, it loads a regular expression pattern for detecting sensitive information into memory.

[0250] Step 4:

[0251] The device activates its emotion engine and prepares to analyze the user's facial expressions and vocalizations. The emotion engine uses the camera and microphone to collect the user's facial expressions and voice data.

[0252] Step 5:

[0253] The user begins typing text within the browser. As the user enters information into the input field, that information is monitored in real time within the browser.

[0254] Step 6:

[0255] The terminal analyzes the entered text based on security policies. It checks for content that matches regular expression patterns.

[0256] Step 7:

[0257] If the device detects sensitive information, it will block the input and display an alert message to the user. The message will read, "This contains sensitive information. Input has been blocked."

[0258] Step 8:

[0259] At the same time, the device uses an emotion engine to recognize the user's emotional state in real time. For example, if the user feels irritated or anxious, that emotion is detected in real time by the emotion engine.

[0260] Step 9:

[0261] The terminal logs the input of confidential information and sends that log to the server. The log includes the user's input, the time the alert occurred, and the user's emotional state.

[0262] Step 10:

[0263] The server saves the received logs to the database. The server sends an alert to notify the administrator that a security incident has occurred.

[0264] Step 11:

[0265] The server also notifies the administrator of the user's emotion data sent from the emotion engine. This allows the administrator to see what emotions the user displayed during a particular action.

[0266] Step 12:

[0267] The administrator receives an alert notification from the server, reviews the details of the security incident and the user's emotional state, and takes the necessary action.

[0268] In this way, by combining security policy-based functions with an emotion engine, a system is built that simultaneously prevents information leaks and manages user emotions.

[0269] (Example 2)

[0270] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0271] In recent years, information leaks and security incidents within companies have been increasing, and effective countermeasures are needed. Furthermore, preventing the leakage of confidential information due to user actions and responding quickly when users experience stress or anger are particularly important. However, conventional systems have not achieved comprehensive security management by combining user input monitoring with emotion recognition. Therefore, the present invention aims to provide advanced information leakage prevention and security management by monitoring both user input and emotions.

[0272] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for authenticating the user to the company's authentication system, means for obtaining a security policy from the server when authentication is successful, means for applying the obtained security policy to the terminal, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, means for sending an operation log to the server when input of confidential information is detected, means for the server to collect the operation log and manage security incidents, means for analyzing emotional data using an emotion engine that recognizes the user's emotions, and means for sending the emotional data to the server and notifying the administrator. This not only prevents the leakage of confidential information but also makes it possible to quickly address the stress and anger felt by the user.

[0273] A "corporate authentication system" refers to a system used within a company to verify the identity of users. Specific examples include LDAP and Active Directory.

[0274] A "server" is a computer that provides specific services or resources, and in this invention, it refers to a central device that manages security policies, collects operation logs, and analyzes sentiment data.

[0275] A "security policy" outlines the rules and guidelines for maintaining information security, including the rules and patterns used in monitoring input data and detecting confidential information.

[0276] A "device" refers to a device operated by a user, specifically a computer or smartphone, including a web browser.

[0277] An "emotion engine" refers to software or hardware used to analyze a user's emotional state, and includes technologies that determine emotions in real time based on the user's facial expressions and voice.

[0278] An "operation log" is data that records the history of operations performed by a user, and is used for tracking and analyzing security incidents.

[0279] A "security incident" refers to any event or action that could pose a threat to information security, including attempted data breaches and unauthorized access.

[0280] A "regular expression" is a set of string rules used to match text patterns, and is used to efficiently detect text that matches specific conditions.

[0281] "Emotional data" refers to digital information that indicates the user's emotional state as analyzed by the emotion engine, and includes data transmitted to the server in real time.

[0282] This invention integrates an emotion engine that recognizes user emotions into a browser system that applies a company's security policy to terminals and monitors user input in real time. This system mainly consists of a server, terminals (user's browser), and the user.

[0283] Server operation

[0284] The server plays a central role and executes the following processes:

[0285] 1. Connect to the enterprise authentication system to verify the user's authentication information. Specifically, use an authentication system such as LDAP or Active Directory.

[0286] 2. When authentication is successful, generate the enterprise security policy and send it to the terminal. The security policy includes a regular expression pattern for detecting sensitive information.

[0287] 3. When a security incident occurs, collect and manage the operation logs. Also manage the sentiment data from the sentiment engine and take prompt action based on it.

[0288] Operations of the terminal (user's browser)

[0289] The terminal is a device that the user uses daily and has the following functions:

[0290] 1. Receive the security policy sent from the server and load it into memory. The browser uses JavaScript to analyze this data.

[0291] 2. Monitor the user's input in real-time and detect sensitive information based on the regular expression pattern defined in the policy. If sensitive information is detected, block the input and display an alert to the user.

[0292] 3. Send the user's input content and operation logs to the server.

[0293] 4. The built-in sentiment engine analyzes the user's facial expressions and voice to recognize the emotional state in real-time. Especially when showing stress or anger, send that data to the server and notify the administrator.

[0294] User operation procedure

[0295] Users access the system through the following steps:

[0296] 1. Launch your browser and log in to your company's authentication system. If authentication is successful, the security policy will be sent from the server to your device and applied.

[0297] 2. When a user enters information in their browser, that information is monitored in real time. For example, if a user attempts to enter "technical details of a patent application" on an email sending screen, the browser will detect this content, block the input, and display an alert.

[0298] 3. Simultaneously, the emotion engine analyzes the user's emotions, and if it detects stress or anger, that data is sent to the server and notified to the administrator.

[0299] Specific example

[0300] For example, suppose a user launches a browser, authenticates, and a security policy is applied. When the user attempts to enter "technical details of a patent application," the browser monitors this text in real time and blocks the input because it matches a regular expression pattern defined in the policy. An alert appears stating, "This contains sensitive information. Input blocked," and an operation log is sent to the server.

[0301] Furthermore, if a user experiences stress or displays anger during this process, the emotion engine recognizes this. Emotional data is sent to the server, which notifies the administrator, enabling them to provide appropriate support based on the situation. This results in an advanced data leakage prevention system that integrates security and user emotion management.

[0302] Example of a prompt

[0303] 1. Please describe a system where, when a user attempts to enter patent information into a browser, the input is blocked and saved as an operation log on the server.

[0304] 2. Please specifically describe a scenario where emotions are recognized from an end - user's expression and reflected in the security policy in real - time.

[0305] Thus, the present invention provides a system that integrates enterprise security and user emotion management, enabling prevention of information leakage and prompt response.

[0306] The flow of the specific process in Example 2 will be described using FIG. 13.

[0307] Step 1:

[0308] When the user attempts to start the browser and log in to the authentication system, the server receives the authentication information (username and password) entered by the user. The server queries this authentication information to the enterprise authentication system such as LDAP or Active Directory. Upon receiving the response from the authentication system, if the authentication is successful, the server generates a session ID and transmits that session ID to the terminal.

[0309] Input: Authentication information entered by the user

[0310] Data processing: Query the authentication information to the authentication system

[0311] Output: Authentication result and session ID

[0312] Step 2:

[0313] For a user whose authentication is successful, the server retrieves the latest security policy information from the database to generate an enterprise security policy. The retrieved policy information is encoded in JSON format and transmitted to the terminal in an HTTP / S response.

[0314] Input: User information for successful authentication

[0315] Data processing: Retrieving security policies from the database and JSON encoding

[0316] Output: Security policy in JSON format

[0317] Step 3:

[0318] The terminal receives JSON data of the security policy from the server and parses it using JavaScript. It loads the parsed security policy into memory and begins monitoring user input under its influence.

[0319] Input: Security policy in JSON format received from the server

[0320] Data processing: Parsing and loading JSON data into memory.

[0321] Output: Loaded security policy

[0322] Step 4:

[0323] When a user enters text into a browser input field, JavaScript on the device monitors the input in real time. It checks whether the input matches a regular expression pattern defined in the security policy.

[0324] Input: User input

[0325] Data processing: Matching based on regular expression patterns

[0326] Output: Match result

[0327] Step 5:

[0328] If the input is detected as confidential information, the device will block the input and clear the input field using JavaScript. Then, it will display an alert to the user stating, "Confidential information is contained. Input has been blocked."

[0329] Input: Match result (confidential information)

[0330] Data processing: Clear input fields and display alerts.

[0331] Output: Alert displayed and input fields cleared

[0332] Step 6:

[0333] The terminal generates a user operation log and sends it to the server via an AJAX request. The operation log includes the user's input, the time, and the type of incident.

[0334] Input: User input and incident information

[0335] Data processing: Generating operation logs and sending AJAX requests.

[0336] Output: Operation log sent to the server

[0337] Step 7:

[0338] Simultaneously, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state in real time. In particular, if the user exhibits stress or anger, the emotion data is compiled in JSON format and sent from the device to the server.

[0339] Input: User's facial expressions and voice data

[0340] Data processing: Analysis of emotional states and conversion to JSON format.

[0341] Output: Sentiment data sent to the server

[0342] Step 8:

[0343] The server analyzes the received emotional data and, if it detects emotions such as stress or anger, notifies the administrator of this information. The administrator then takes appropriate action based on this notification.

[0344] Input: Sentiment data

[0345] Data processing: Analysis of emotional data and generation of notifications.

[0346] Output: Notification to administrator

[0347] In this way, by comprehensively monitoring and managing user input and emotional states, a system can be realized that prevents information leaks and appropriately responds to the user's psychological state.

[0348] (Application Example 2)

[0349] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0350] Existing security management systems have the capability to detect confidential information entered by users in real time, but they do not take into account the user's emotional state. Therefore, when a user experiences stress or anger, appropriate responses may not be taken quickly, resulting in increased security risks. To solve this problem, it is necessary to recognize the user's emotions in real time and take appropriate actions based on that recognition.

[0351] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for authenticating the user to the company's authentication system, means for obtaining a security policy when authentication is successful, means for applying the obtained security policy to the web browser, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, means for sending an operation log to the server when input of confidential information is detected, means for the server to collect the operation log and manage security incidents, means for recognizing the user's emotional state in real time, means for sending the recognized emotional data to the server, and means for automatically performing appropriate actions based on the emotional data. This makes it possible to detect security risks early based on the user's emotional state and take appropriate countermeasures quickly.

[0352] A "corporate authentication system" is a mechanism that identifies and verifies a user's access to authorized systems to which they have access rights.

[0353] A "server" is a computer system that provides services to other computers and terminals on a network.

[0354] A "security policy" is a set of rules and procedures established for the purpose of protecting information.

[0355] A "web browser" is software used to view web pages on the internet.

[0356] "User input" refers to information that a user provides to a computer system through a keyboard or other input devices.

[0357] "Confidential information" refers to information that, if disclosed, could potentially cause harm to a company, such as unreleased technical information or trade secrets.

[0358] An "alert" is a warning message that alerts users or administrators when certain conditions are met.

[0359] An "operation log" is a record of actions performed by a user and is used for security monitoring and problem solving.

[0360] A "security incident" refers to security-related problems or accidents, such as unauthorized access to a company's information systems or data breaches.

[0361] "Emotional state" refers to the user's psychological state or mood, and includes things like anger and stress.

[0362] "Real-time" means that a process or calculation produces results immediately, with virtually no delay.

[0363] "Appropriate action" refers to the best course of action to take in a given situation.

[0364] This invention is a system that authenticates users based on a company's authentication system, retrieves a security policy from a server upon successful authentication, and applies the retrieved security policy to the web browser. Furthermore, it monitors user input and emotional state in real time, blocks input and displays an alert if confidential information is detected. If confidential information is detected, it sends an operation log to the server, which collects the operation log and manages security incidents. In addition, it recognizes the user's emotional state in real time, sends the recognized emotional data to the server, and automatically performs appropriate actions based on the emotional data.

[0365] Server operation

[0366] The server connects to the company's authentication system and verifies the user's credentials. If authentication is successful, it generates a security policy and sends it to the terminal. It also collects operation logs and sentiment data to manage security incidents. The specific cloud services used are Amazon Web Services (AWS®) and Microsoft® Azure®.

[0367] Terminal (user's browser) operation

[0368] The device has a function to monitor user input in real time and detect sensitive information. It incorporates an emotion recognition engine that recognizes emotions based on the user's facial expressions and tone of voice. Specifically, it uses the Microsoft Azure Emotion API as the emotion recognition engine. If a user attempts to enter sensitive information, the device will detect it, block the input, and display an alert.

[0369] User actions

[0370] The user first launches a browser and logs into the company's authentication system. Upon successful authentication, the security policy is applied. If the user attempts to enter certain sensitive information into the browser, the device monitors it in real time, immediately blocking the input and displaying an alert if sensitive information is detected. Simultaneously, the user's emotions are monitored in real time, and if stress or anger is detected, emotion data is sent to the server.

[0371] Specific example

[0372] For example, if a user attempts to enter technical details of a patent application, the system immediately detects this and displays an alert stating, "This contains confidential information. Input has been blocked." If the user also exhibits feelings of stress or anger, the emotion recognition engine identifies these emotions and sends the emotion data to the server. Administrators can then receive this data and take appropriate action.

[0373] Example of a prompt

[0374] "A user is attempting to enter technical details of a patent application into an external email. Please implement a function that detects and blocks this input, and simultaneously recognizes and reports the user's emotional state (e.g., anger)."

[0375] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0376] Step 1:

[0377] The server authenticates the user to the company's authentication system. The user enters their authentication information (username and password) to log in to the authentication system and sends this information to the server. The server verifies the authentication information and determines whether authentication was successful. If authentication is successful, the user's authentication is complete and a token is issued to proceed to the next step. The input is the user's authentication information, and the output is whether authentication was successful or not (a token if successful).

[0378] Step 2:

[0379] The server generates a security policy upon successful authentication and sends it to the user's device using a token. The security policy is a set of rules containing a regular expression pattern for sensitive information. The input is the authentication token, and the output is the security policy.

[0380] Step 3:

[0381] The device applies the acquired security policy to the web browser. The applied security policy monitors user input in real time and determines whether sensitive information is detected. The input is the security policy, and the output is the browser environment to which the policy is applied.

[0382] Step 4:

[0383] The terminal monitors user input in real time and uses regular expression patterns to detect whether it contains sensitive information. When a user begins typing in the web browser, the text is inspected based on security policies. The input is the user's text, and the output is the result of the determination of whether it contains sensitive information.

[0384] Step 5:

[0385] The terminal blocks user input and displays an alert if sensitive information is detected. When a user attempts to enter text containing sensitive information, the system blocks the input and displays a warning message stating, "This contains sensitive information. Input blocked." The input is the result of the sensitive information detection, and the output is the blocked input and the displayed alert.

[0386] Step 6:

[0387] The terminal sends an operation log to the server when it detects the input of confidential information. The operation log contains details of the confidential information the user attempted to input and the actions that occurred. Input is the event of entering confidential information, and output is the operation log.

[0388] Step 7:

[0389] The server collects operation logs and manages security incidents. It analyzes the collected operation logs to enable administrators to take appropriate action. The input is operation logs, and the output is security incident management information.

[0390] Step 8:

[0391] The device utilizes an emotion recognition engine to recognize the user's emotional state in real time. Based on the user's facial expressions and voice data acquired from the camera and microphone, the emotion recognition algorithm classifies the emotion. The input is the user's facial expressions and voice data, and the output is the recognized emotional state.

[0392] Step 9:

[0393] The device sends recognized emotion data to the server. This emotion data includes the user's stress and anger levels. The input is the emotional state, and the output is the emotion data sent to the server.

[0394] Step 10:

[0395] The server automatically takes appropriate actions based on sentiment data. Based on this sentiment data, it can, for example, notify administrators or provide psychological support to users. The input is sentiment data, and the output is the appropriate action taken.

[0396] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0397] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0398] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0399] [Second Embodiment]

[0400] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0401] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0402] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0403] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0404] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0405] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0406] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0407] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0408] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0409] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0410] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0411] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0412] This invention is a system that prevents the leakage of confidential information by applying security policies to corporate browsers and monitoring user input in real time. This system mainly consists of a server, a terminal (user's browser), and the user.

[0413] Server operation

[0414] The server manages security policies and distributes them to endpoints. The server connects to the company's authentication system and verifies user credentials. Upon successful authentication, the server sends the appropriate security policy to the device. In the event of a security incident, the server collects the information and notifies the administrator.

[0415] Terminal (user's browser) operation

[0416] The device receives security policies sent from the server and monitors user input based on those policies. When a user launches a browser, authentication is performed first. If authentication is successful, the device downloads the latest security policy from the server and loads it into memory.

[0417] When a user enters text into a browser input field, the terminal monitors the input in real time. Using regular expression patterns defined in the policy, the terminal inspects the input. If sensitive information is detected, the terminal blocks the input and displays an alert to the user. It also sends an operation log to the server.

[0418] User actions

[0419] The user first launches a browser and logs into the company's authentication system. Once authentication is successful, the user begins using the browser as usual. If the user attempts to enter certain sensitive information into the browser, the input is monitored in real time, and if sensitive information is detected, the input is blocked and an alert is displayed.

[0420] Specific example

[0421] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user opens an email sending screen and attempts to enter "technical details of a patent application," the device's browser monitors this input in real time. Because a regular expression pattern for "technical details of a patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[0422] In this way, the system can apply a company's security policies to the browser and prevent the leakage of confidential information in real time. Furthermore, security incidents that occur are recorded and managed on the server, facilitating a rapid response.

[0423] The following describes the processing flow.

[0424] Step 1:

[0425] The device launches a browser. The user accesses the login screen and enters their authentication information into the company's authentication system.

[0426] Step 2:

[0427] The server receives the user's authentication information and uses it to verify the user's identity. If authentication is successful, the server generates an appropriate security policy and sends it to the terminal.

[0428] Step 3:

[0429] The device receives the security policy sent from the server and applies it within the browser. Specifically, it loads a regular expression pattern for detecting sensitive information into memory.

[0430] Step 4:

[0431] The user begins typing text within the browser. As the user enters information into the input field, that information is monitored in real time within the browser.

[0432] Step 5:

[0433] The terminal analyzes the entered text based on security policies. It checks for content that matches regular expression patterns.

[0434] Step 6:

[0435] If the device detects sensitive information, it will block the input and display an alert message to the user. The message will read, "This contains sensitive information. Input has been blocked."

[0436] Step 7:

[0437] The terminal logs the input of confidential information and sends that log to the server. The log includes information such as the user's input and the time when the alert occurred.

[0438] Step 8:

[0439] The server saves the received logs to the database. The server sends an alert to notify the administrator that a security incident has occurred.

[0440] Step 9:

[0441] The administrator receives an alert notification from the server, reviews the details of the security incident, and takes the necessary action.

[0442] In this way, a system is realized that monitors user input in real time throughout each processing step, thereby preventing the leakage of confidential information.

[0443] (Example 1)

[0444] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0445] In today's business environment, the leakage of confidential information poses a significant risk, requiring advanced security measures. However, existing security systems struggle to monitor user input in real time and immediately detect and address confidential information breaches. As a result, the risk of confidential information leakage remains, and companies' information assets cannot be adequately protected. This invention aims to solve these problems and provide a system that effectively protects companies' confidential information.

[0446] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0447] In this invention, the server includes means for authenticating the user, means for obtaining a security policy from the server upon successful authentication, and means for applying the obtained security policy to the user interface. This enables the application of a security policy to the terminal after user authentication by coordinating with the company's authentication system. The terminal also includes means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, and means for sending an operation log to the server when input of confidential information is detected. This prevents the leakage of confidential information and enables a rapid response. Furthermore, the server includes means for collecting operation logs and managing security incidents, which notifies the administrator of the occurrence of a security incident and enables appropriate countermeasures to be taken. This makes it possible to comprehensively protect the company's confidential information.

[0448] "User authentication means" refers to a function that verifies a user's authentication information and confirms whether the user has legitimate access rights.

[0449] The "means for obtaining security policies" refer to a function that downloads security policies from the server to users who have successfully authenticated and applies them to their devices.

[0450] "Means of application to the user interface" refers to a function that reflects the acquired security policy in the user's browser or other user interfaces.

[0451] "Real-time monitoring of user input" refers to a function that monitors data entered by users in a browser or other interface in real time and performs appropriate checks.

[0452] A "means for detecting confidential information" is a function for determining whether or not the monitored input data contains confidential information.

[0453] The "means for blocking user input and displaying alerts" are functions that prevent user input and display a warning message when confidential information is detected.

[0454] The "means for sending operation logs to the server" refers to a function for sending operation history, such as the detection of confidential information and the display of associated alerts, to the server.

[0455] "A means for collecting operation logs and managing security incidents" refers to a function that stores transmitted operation logs on a server and centrally manages security incidents.

[0456] A "regular expression pattern" is a format for defining a pattern of strings, and is used to determine whether a specific string is found.

[0457] "Administrator notification method" refers to a function that notifies administrators via email, alerts, or other methods when a security incident occurs.

[0458] This invention is a system that prevents the leakage of confidential information by applying security policies to a company's user interface (e.g., a browser) and monitoring user input in real time. This system mainly consists of a server, a terminal (the user's browser), and the user.

[0459] Server configuration and operation

[0460] The server implements the following functions:

[0461] 1. User Authentication Function: Receives authentication requests from users and verifies user information using the company's authentication system (e.g., LDAP or OAuth). This function is a crucial step in verifying whether the user has legitimate access rights.

[0462] 2. Security Policy Retrieval and Distribution Function: Upon successful authentication, the server retrieves the appropriate security policy from the database and sends it to the user's device. The security policy is managed as data in JSON format.

[0463] 3. Operation log collection and management function: Operation logs sent from terminals are recorded in a database, and security incidents are managed centrally.

[0464] The server can use a Linux-based system, and the database can use a common database management system such as MySQL or PostgreSQL.

[0465] Terminal (user's browser) configuration and operation

[0466] The device has the following features:

[0467] 1. Authentication Function: When a user launches a browser, they enter authentication information and send an authentication request to the server. This is expected to work with common web browsers such as Chrome and Firefox.

[0468] 2. Security policy reception and application function: Upon successful authentication, the security policy is downloaded from the server, loaded into memory, and applied to the user interface.

[0469] 3. Real-time monitoring function: User input is monitored in real time, and whether it contains sensitive information is checked using regular expression patterns defined in the policy. For example, a script using JavaScript can be created and executed in the browser.

[0470] 4. Input Blocking and Alert Display Function: If confidential information is detected, input will be blocked and an alert will be displayed to the user.

[0471] 5. Operation Log Transmission Function: Detected security incidents are sent to the server as operation logs.

[0472] User actions

[0473] The user performs the following actions:

[0474] 1. Launching the browser and authenticating: Launch your browser and log in to the company's authentication system.

[0475] 2. Normal browser operation: After successful authentication, the user uses the browser as usual. For example, they enter text on the email sending screen.

[0476] 3. Alert response: If an alert is displayed when attempting to enter specific confidential information, review the content of the alert and take appropriate action.

[0477] Specific example

[0478] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user attempts to enter the text "Technical details of patent application" on the email sending screen, the device's browser monitors this input in real time. Because a regular expression pattern for "Technical details of patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[0479] Examples of prompt statements

[0480] Prompt example:

[0481] "Please describe a system that monitors in real time when a user attempts to enter sensitive information in their browser, blocks the input if sensitive information is detected, and displays an alert."

[0482] This invention allows a company's security policies to be applied to the user interface, preventing the leakage of confidential information in real time. Furthermore, security incidents that occur are recorded and managed on the server, facilitating a rapid response.

[0483] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0484] Step 1:

[0485] User authentication request submission

[0486] Input: The user enters their authentication information (username and password) into the browser's login form.

[0487] Output: The entered authentication information is sent to the server.

[0488] Specific action: The user launches a browser, enters their authentication information into the login form, and clicks the "Login" button. This request is sent to the server as an HTTP POST request.

[0489] Step 2:

[0490] Authentication information verification on the server

[0491] Input: The server receives authentication information sent by the user.

[0492] Output: The result of authentication success or failure is generated.

[0493] Specific operation: The server sends a query to the company's authentication system (LDAP or OAuth) to verify the received authentication information and receives a response. If authentication is successful, an authentication token is generated and sent to the user.

[0494] Step 3:

[0495] Sending security policies

[0496] Input: A database containing user information and security policies for users who have successfully authenticated.

[0497] Output: The latest security policy is sent to the device.

[0498] Specific operation: The server retrieves the security policy for the relevant user from the database and sends it to the user's browser as an HTTP response in JSON format.

[0499] Step 4:

[0500] Download and apply security policies

[0501] Input: Security policy received by the terminal from the server.

[0502] Output: The security policy is loaded into the device's memory.

[0503] Specific operation: The browser analyzes the security policy it receives, loads the settings for real-time monitoring into memory according to its content, and applies them.

[0504] Step 5:

[0505] Real-time monitoring of user input

[0506] Input: The user enters text into the input field.

[0507] Output: Input content is checked in real time.

[0508] Specific operation: When a user starts typing text on the email sending screen, JavaScript is used to monitor the input in real time within the browser. A specific regular expression pattern is used based on policy.

[0509] Step 6:

[0510] Detection and blocking of sensitive information

[0511] Input: User input and a regular expression pattern defined by the policy.

[0512] Output: If sensitive information is detected, the input will be blocked and an alert will be displayed.

[0513] Specific operation: JavaScript checks the input content against a regular expression pattern defined in the policy, blocks user input if it contains sensitive information, and displays a warning message.

[0514] Step 7:

[0515] Sending operation logs

[0516] Input: Operation log when confidential information is detected.

[0517] Output: Operation logs are sent to the server.

[0518] Specific operation: A history of operations such as the detection of confidential information and the display of associated alerts is generated and sent to the server as an HTTP request.

[0519] Step 8:

[0520] Operation log collection and security incident management

[0521] Input: Operation log sent from the terminal.

[0522] Output: Operation logs are recorded in the database, and security incidents are managed.

[0523] Specific actions: The server saves the operation logs it receives to a database and lists them in a format viewable by the administrator. It also notifies the administrator of any security incidents that occur.

[0524] (Application Example 1)

[0525] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0526] In recent years, the risk of confidential information leaks within companies has increased, with unauthorized information sharing via the internet being a particular problem. There is a need for technological measures to prevent the leakage of confidential information by monitoring employee input in real time when using web browsers. Furthermore, the ability to quickly display alerts to users through different devices (such as head-mounted displays) is also essential.

[0527] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0528] In this invention, the server includes means for authenticating a user to the company's authentication system, means for obtaining a security policy from the server if authentication is successful, means for applying the obtained security policy to the browser, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert on a head-mounted display if confidential information is detected, means for sending an operation log to the server if input of confidential information is detected, and means for the server to collect the operation log and manage security incidents. This makes it possible to monitor user input in real time, prevent the leakage of confidential information, and immediately display warnings through different devices.

[0529] A "corporate authentication system" is a system that authenticates users in order to control access to a company's network and systems.

[0530] A "security policy" refers to a set of standards and rules that define the protection of data in information systems and the prevention of unintentional information leaks.

[0531] A "browser" is software used to connect to the internet and display and interact with web pages.

[0532] A "head-mounted display" is a wearable device that is worn on the user's head and provides information visually through a display.

[0533] User authentication is the process of verifying that a user accessing a system is a legitimate user.

[0534] "Real-time monitoring" refers to the immediate monitoring of systems and processes, and the instantaneous detection and response to changes in status or anomalies.

[0535] "Confidential information" refers to important internal company information that is not made public and which, if revealed to a third party, could cause damage to the company.

[0536] An "alert" is a means of notifying a user and issuing a warning when specific conditions occur.

[0537] An "operation log" refers to a record of a series of operations or events performed on a system, which is used later for analysis and auditing.

[0538] A "security incident" refers to an event in which a problem or risk related to information security occurs.

[0539] This invention is a system that authenticates users using a company's authentication system and retrieves a security policy from the server upon successful authentication. This system applies the retrieved security policy to the browser and monitors user input in real time to prevent the leakage of confidential information. Furthermore, a key feature is the ability to display an alert on a head-mounted display (HMD) when confidential information is detected, enabling immediate response.

[0540] First, the user logs into the company's authentication system using a browser or HMD. During this process, authentication information is sent to the server, which then authenticates the user. Upon successful authentication, the server retrieves the appropriate security policy and sends it to the user's device or HMD. The security policy includes a regular expression pattern for identifying sensitive information.

[0541] When a user enters text via a browser or HMD, the device monitors the input in real time. If the input matches a regular expression pattern defined in the security policy, it is determined to be sensitive information, and the input is immediately blocked. At this time, a warning message is displayed on the HMD, and the user is aware that the input of sensitive information has been blocked. In parallel, an operation log is sent to the server, which records it and manages security incidents.

[0542] The specific configuration of this system is as follows:

[0543] 1. Hardware to be used:

[0544] Head-mounted display (HMD)

[0545] User's device (PC or tablet)

[0546] 2. Software to use:

[0547] Authentication system: Performs user authentication.

[0548] Security policy management system: Distributes security policies that include regular expression patterns.

[0549] Real-time monitoring software: Monitors input content in real time and detects confidential information.

[0550] The following scenarios are possible as specific examples:

[0551] For example, consider a scenario where an employee using "Secure Browsing" within a company wears an HMD (Head-Mounted Display) while working. When the user attempts to type "technical details of a patent-pending application" into the email composition screen, a real-time alert is displayed on the HMD, blocking the input. The alert displays "Confidential information is contained. Input has been blocked," and this information is simultaneously sent to the server as a log.

[0552] Examples of prompts for a generative AI model include the following:

[0553] "A user is wearing an HMD (Head-Mounted Display) and browsing the web. I want to build a system that displays a real-time warning and blocks input if the user attempts to enter sensitive information. I will implement this using Python. The security policy will be provided by the server, and user input will be checked using regular expressions. Operation logs will be sent to the server, and alerts will be displayed on the HMD. Please tell me how to design the code."

[0554] In this way, the system of this invention can enhance corporate information security and prevent the leakage of confidential information.

[0555] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0556] Step 1:

[0557] The server receives the user's authentication information.

[0558] Input: User ID and password

[0559] Output: Authentication result (success / failure)

[0560] Specific operation: The user sends their entered ID and password to the server, which then uses its authentication system to perform authentication. If authentication is successful, the user proceeds to the next step.

[0561] Step 2:

[0562] The server retrieves the security policy and sends it to the terminal.

[0563] Input: Authentication result (success / failure)

[0564] Output: Security Policy

[0565] Specific operation: For a user who has successfully authenticated, the server searches for an appropriate security policy and sends it to the terminal. The security policy includes a regular expression pattern to identify sensitive information.

[0566] Step 3:

[0567] The device loads the security policy into memory.

[0568] Input: Security Policy

[0569] Output: Security policy loaded into memory

[0570] Specific operation: The terminal analyzes the security policy received from the server and loads it into memory. This prepares the terminal to monitor user input.

[0571] Step 4:

[0572] The user enters text via a browser or head-mounted display (HMD).

[0573] Input: User input text

[0574] Output: Monitored input text

[0575] Specific operation: The user enters text using a browser or HMD, and this input is monitored in real time.

[0576] Step 5:

[0577] The terminal monitors input text in real time and uses regular expression patterns to detect sensitive information.

[0578] Input: Input text, regular expression pattern

[0579] Output: Confidential information detection result (detected / not detected)

[0580] Specific operation: The terminal checks the input text against a regular expression pattern defined in the security policy to determine if it contains sensitive information.

[0581] Step 6:

[0582] If confidential information is detected, the terminal will block input and display an alert on the head-mounted display (HMD).

[0583] Input: Confidential information detection results

[0584] Output: Alert message, blocked input text

[0585] Specific operation: If the device detects sensitive information, it will block the input and display an alert message on the HMD stating, "Sensitive information is contained. Input has been blocked."

[0586] Step 7:

[0587] If the terminal detects the input of confidential information, it will send an operation log to the server.

[0588] Input: Confidential information detection results, input text

[0589] Output: Operation Log

[0590] Specific action: Send an operation log containing the detected confidential information and detection results to the server.

[0591] Step 8:

[0592] The server collects operation logs and manages them as security incidents.

[0593] Input: Operation Log

[0594] Output: Security incident log

[0595] Specific operation: The server saves the received operation logs to a database and manages them as security incidents.

[0596] Step 9:

[0597] The server notifies the administrator of the occurrence of a security incident.

[0598] Input: Security Incident Log

[0599] Output: Notification message

[0600] Specific operation: The server notifies the administrator of recorded security incidents to encourage a quick response.

[0601] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0602] This invention achieves more advanced information leakage prevention and security management by combining a browser system that applies a company's security policy and monitors user input in real time with an emotion engine that recognizes user emotions. This system mainly consists of a server, a terminal (the user's browser), and the user.

[0603] Server operation

[0604] The server manages security policies and distributes them to endpoints. It connects to the company's authentication system and verifies user credentials. Upon successful authentication, the server generates an appropriate security policy and sends it to the device. In the event of a security incident, the server collects the information and notifies the administrator. The server also manages data from the sentiment engine and uses it to improve or enforce security policies as needed.

[0605] Terminal (user's browser) operation

[0606] The device receives security policies sent from the server and monitors user input based on those policies. When a user launches a browser, authentication is performed first. If authentication is successful, the device downloads the latest security policy from the server and loads it into memory. Furthermore, the device has an emotion engine built in that recognizes emotions in real time from the user's facial expressions and vocalizations.

[0607] When a user enters text into a browser input field, that information is monitored in real time within the browser. The device inspects the input using regular expression patterns defined in the policy. If sensitive information is detected, the device blocks the input and displays an alert to the user. It also sends an operation log to the server.

[0608] Simultaneously, the emotion engine analyzes the user's facial expressions and vocalizations to recognize their emotional state in real time. This emotional data is then used to inform actions based on security policies. For example, if the system detects that a user is experiencing frustration, it might take steps such as providing appropriate psychological safety measures.

[0609] User actions

[0610] The user first launches a browser and logs into the company's authentication system. Upon successful authentication, the user begins using the browser as usual. If the user attempts to enter specific confidential information into the browser, the input is monitored in real time. If confidential information is detected, the input is blocked and an alert is displayed. Simultaneously, the user's emotions are monitored in real time, and appropriate measures are taken as needed.

[0611] Specific example

[0612] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user opens an email sending screen and attempts to enter "technical details of a patent application," the device's browser monitors this input in real time. Because a regular expression pattern for "technical details of a patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[0613] Furthermore, if the user experiences stress or displays anger during this process, the emotion engine recognizes this. The device sends this emotion data to a server, which then notifies the administrator based on that data. The administrator can then understand the circumstances under which the user experienced stress and provide appropriate support.

[0614] In this way, a sophisticated system that integrates security and user sentiment management is realized through each processing step.

[0615] The following describes the processing flow.

[0616] Step 1:

[0617] The device launches a browser. The user accesses the login screen and enters their authentication information into the company's authentication system.

[0618] Step 2:

[0619] The server receives the user's authentication information and uses it to verify the user's identity. If authentication is successful, the server generates an appropriate security policy and sends it to the terminal.

[0620] Step 3:

[0621] The device receives the security policy sent from the server and applies it within the browser. Specifically, it loads a regular expression pattern for detecting sensitive information into memory.

[0622] Step 4:

[0623] The device activates its emotion engine and prepares to analyze the user's facial expressions and vocalizations. The emotion engine uses the camera and microphone to collect the user's facial expressions and voice data.

[0624] Step 5:

[0625] The user begins typing text within the browser. As the user enters information into the input field, that information is monitored in real time within the browser.

[0626] Step 6:

[0627] The terminal analyzes the entered text based on security policies. It checks for content that matches regular expression patterns.

[0628] Step 7:

[0629] If the device detects sensitive information, it will block the input and display an alert message to the user. The message will read, "This contains sensitive information. Input has been blocked."

[0630] Step 8:

[0631] At the same time, the device uses an emotion engine to recognize the user's emotional state in real time. For example, if the user feels irritated or anxious, that emotion is detected in real time by the emotion engine.

[0632] Step 9:

[0633] The terminal logs the input of confidential information and sends that log to the server. The log includes the user's input, the time the alert occurred, and the user's emotional state.

[0634] Step 10:

[0635] The server saves the received logs to the database. The server sends an alert to notify the administrator that a security incident has occurred.

[0636] Step 11:

[0637] The server also notifies the administrator of the user's emotion data sent from the emotion engine. This allows the administrator to see what emotions the user displayed during a particular action.

[0638] Step 12:

[0639] The administrator receives an alert notification from the server, reviews the details of the security incident and the user's emotional state, and takes the necessary action.

[0640] In this way, by combining security policy-based functions with an emotion engine, a system is built that simultaneously prevents information leaks and manages user emotions.

[0641] (Example 2)

[0642] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0643] In recent years, information leaks and security incidents within companies have been increasing, and effective countermeasures are needed. Furthermore, preventing the leakage of confidential information due to user actions and responding quickly when users experience stress or anger are particularly important. However, conventional systems have not achieved comprehensive security management by combining user input monitoring with emotion recognition. Therefore, the present invention aims to provide advanced information leakage prevention and security management by monitoring both user input and emotions.

[0644] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for authenticating the user to the company's authentication system, means for obtaining a security policy from the server when authentication is successful, means for applying the obtained security policy to the terminal, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, means for sending an operation log to the server when input of confidential information is detected, means for the server to collect the operation log and manage security incidents, means for analyzing emotional data using an emotion engine that recognizes the user's emotions, and means for sending the emotional data to the server and notifying the administrator. This not only prevents the leakage of confidential information but also makes it possible to quickly address the stress and anger felt by the user.

[0645] A "corporate authentication system" refers to a system used within a company to verify the identity of users. Specific examples include LDAP and Active Directory.

[0646] A "server" is a computer that provides specific services or resources, and in this invention, it refers to a central device that manages security policies, collects operation logs, and analyzes sentiment data.

[0647] A "security policy" outlines the rules and guidelines for maintaining information security, including the rules and patterns used in monitoring input data and detecting confidential information.

[0648] A "device" refers to a device operated by a user, specifically a computer or smartphone, including a web browser.

[0649] An "emotion engine" refers to software or hardware used to analyze a user's emotional state, and includes technologies that determine emotions in real time based on the user's facial expressions and voice.

[0650] An "operation log" is data that records the history of operations performed by a user, and is used for tracking and analyzing security incidents.

[0651] A "security incident" refers to any event or action that could pose a threat to information security, including attempted data breaches and unauthorized access.

[0652] A "regular expression" is a set of string rules used to match text patterns, and is used to efficiently detect text that matches specific conditions.

[0653] "Emotional data" refers to digital information that indicates the user's emotional state as analyzed by the emotion engine, and includes data transmitted to the server in real time.

[0654] This invention integrates an emotion engine that recognizes user emotions into a browser system that applies a company's security policy to terminals and monitors user input in real time. This system mainly consists of a server, terminals (user's browser), and the user.

[0655] Server operation

[0656] The server plays a central role and performs the following tasks:

[0657] 1. Connect to the company's authentication system and verify the user's authentication information. Specifically, this involves using authentication systems such as LDAP or Active Directory.

[0658] 2. Upon successful authentication, the system generates and sends the company's security policy to the device. The security policy includes regular expression patterns for detecting sensitive information.

[0659] 3. In the event of a security incident, operation logs will be collected and managed. Furthermore, sentiment data from the sentiment engine will be managed, and a swift response will be initiated based on this data.

[0660] Terminal (user's browser) operation

[0661] A device is a device that users use on a daily basis and has the following functions:

[0662] 1. Receive the security policy sent from the server and load it into memory. The browser uses JavaScript to parse this data.

[0663] 2. Monitor user input in real time and detect sensitive information based on regular expression patterns defined in the policy. If sensitive information is detected, block the input and display an alert to the user.

[0664] 3. Send user input and operation logs to the server.

[0665] 4. The built-in emotion engine analyzes the user's facial expressions and voice to recognize their emotional state in real time. In particular, if the user shows signs of stress or anger, the data is sent to the server and notified to the administrator.

[0666] User operating procedures

[0667] Users access the system through the following steps:

[0668] 1. Launch your browser and log in to your company's authentication system. If authentication is successful, the security policy will be sent from the server to your device and applied.

[0669] 2. When a user enters information in their browser, that information is monitored in real time. For example, if a user attempts to enter "technical details of a patent application" on an email sending screen, the browser will detect this content, block the input, and display an alert.

[0670] 3. Simultaneously, the emotion engine analyzes the user's emotions, and if it detects stress or anger, that data is sent to the server and notified to the administrator.

[0671] Specific example

[0672] For example, suppose a user launches a browser, authenticates, and a security policy is applied. When the user attempts to enter "technical details of a patent application," the browser monitors this text in real time and blocks the input because it matches a regular expression pattern defined in the policy. An alert appears stating, "This contains sensitive information. Input blocked," and an operation log is sent to the server.

[0673] Furthermore, if a user experiences stress or displays anger during this process, the emotion engine recognizes this. Emotional data is sent to the server, which notifies the administrator, enabling them to provide appropriate support based on the situation. This results in an advanced data leakage prevention system that integrates security and user emotion management.

[0674] Example of a prompt

[0675] 1. "Describe a system in which, when a user attempts to enter patent information into a browser, the input is blocked and saved to the server as an operational log."

[0676] 2. "Please describe in detail a scenario in which emotions are recognized from the end user's facial expressions and reflected in security policies in real time."

[0677] Thus, the present invention provides a system that integrates corporate security and user sentiment management, enabling information leakage prevention and rapid response.

[0678] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0679] Step 1:

[0680] When a user launches a browser and attempts to log in to the authentication system, the server receives the authentication information (username and password) entered by the user. The server queries the company's authentication system, such as LDAP or Active Directory, with this authentication information. Upon receiving a response from the authentication system and confirming successful authentication, the server generates a session ID and sends that session ID to the terminal.

[0681] Input: Authentication information entered by the user

[0682] Data processing: Query authentication information with the authentication system.

[0683] Output: Authentication success / failure and session ID

[0684] Step 2:

[0685] The server retrieves the latest security policy information from the database for users who have successfully authenticated, in order to generate the company's security policy. It then encodes the retrieved policy information into JSON format and sends it to the terminal via an HTTP / S response.

[0686] Input: User information for successful authentication

[0687] Data processing: Retrieving security policies from the database and JSON encoding

[0688] Output: Security policy in JSON format

[0689] Step 3:

[0690] The terminal receives JSON data of the security policy from the server and parses it using JavaScript. It loads the parsed security policy into memory and begins monitoring user input under its influence.

[0691] Input: Security policy in JSON format received from the server

[0692] Data processing: Parsing and loading JSON data into memory.

[0693] Output: Loaded security policy

[0694] Step 4:

[0695] When a user enters text into a browser input field, JavaScript on the device monitors the input in real time. It checks whether the input matches a regular expression pattern defined in the security policy.

[0696] Input: User input

[0697] Data processing: Matching based on regular expression patterns

[0698] Output: Match result

[0699] Step 5:

[0700] If the input is detected as confidential information, the device will block the input and clear the input field using JavaScript. Then, it will display an alert to the user stating, "Confidential information is contained. Input has been blocked."

[0701] Input: Match result (confidential information)

[0702] Data processing: Clear input fields and display alerts.

[0703] Output: Alert displayed and input fields cleared

[0704] Step 6:

[0705] The terminal generates a user operation log and sends it to the server via an AJAX request. The operation log includes the user's input, the time, and the type of incident.

[0706] Input: User input and incident information

[0707] Data processing: Generating operation logs and sending AJAX requests.

[0708] Output: Operation log sent to the server

[0709] Step 7:

[0710] Simultaneously, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state in real time. In particular, if the user exhibits stress or anger, the emotion data is compiled in JSON format and sent from the device to the server.

[0711] Input: User's facial expressions and voice data

[0712] Data processing: Analysis of emotional states and conversion to JSON format.

[0713] Output: Sentiment data sent to the server

[0714] Step 8:

[0715] The server analyzes the received emotional data and, if it detects emotions such as stress or anger, notifies the administrator of this information. The administrator then takes appropriate action based on this notification.

[0716] Input: Sentiment data

[0717] Data processing: Analysis of emotional data and generation of notifications.

[0718] Output: Notification to administrator

[0719] In this way, by comprehensively monitoring and managing user input and emotional states, a system can be realized that prevents information leaks and appropriately responds to the user's psychological state.

[0720] (Application Example 2)

[0721] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0722] Existing security management systems have the capability to detect confidential information entered by users in real time, but they do not take into account the user's emotional state. Therefore, when a user experiences stress or anger, appropriate responses may not be taken quickly, resulting in increased security risks. To solve this problem, it is necessary to recognize the user's emotions in real time and take appropriate actions based on that recognition.

[0723] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for authenticating the user to the company's authentication system, means for obtaining a security policy when authentication is successful, means for applying the obtained security policy to the web browser, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, means for sending an operation log to the server when input of confidential information is detected, means for the server to collect the operation log and manage security incidents, means for recognizing the user's emotional state in real time, means for sending the recognized emotional data to the server, and means for automatically performing appropriate actions based on the emotional data. This makes it possible to detect security risks early based on the user's emotional state and take appropriate countermeasures quickly.

[0724] A "corporate authentication system" is a mechanism that identifies and verifies a user's access to authorized systems to which they have access rights.

[0725] A "server" is a computer system that provides services to other computers and terminals on a network.

[0726] A "security policy" is a set of rules and procedures established for the purpose of protecting information.

[0727] A "web browser" is software used to view web pages on the internet.

[0728] "User input" refers to information that a user provides to a computer system through a keyboard or other input devices.

[0729] "Confidential information" refers to information that, if disclosed, could potentially cause harm to a company, such as unreleased technical information or trade secrets.

[0730] An "alert" is a warning message that alerts users or administrators when certain conditions are met.

[0731] An "operation log" is a record of actions performed by a user and is used for security monitoring and problem solving.

[0732] A "security incident" refers to security-related problems or accidents, such as unauthorized access to a company's information systems or data breaches.

[0733] "Emotional state" refers to the user's psychological state or mood, and includes things like anger and stress.

[0734] "Real-time" means that a process or calculation produces results immediately, with virtually no delay.

[0735] "Appropriate action" refers to the best course of action to take in a given situation.

[0736] This invention is a system that authenticates users based on a company's authentication system, retrieves a security policy from a server upon successful authentication, and applies the retrieved security policy to the web browser. Furthermore, it monitors user input and emotional state in real time, blocks input and displays an alert if confidential information is detected. If confidential information is detected, it sends an operation log to the server, which collects the operation log and manages security incidents. In addition, it recognizes the user's emotional state in real time, sends the recognized emotional data to the server, and automatically performs appropriate actions based on the emotional data.

[0737] Server operation

[0738] The server connects to the company's authentication system and verifies the user's credentials. If authentication is successful, it generates a security policy and sends it to the terminal. It also collects operation logs and sentiment data to manage security incidents. The specific cloud services used are Amazon Web Services (AWS) and Microsoft Azure.

[0739] Terminal (user's browser) operation

[0740] The device has a function to monitor user input in real time and detect sensitive information. It incorporates an emotion recognition engine that recognizes emotions based on the user's facial expressions and tone of voice. Specifically, it uses the Microsoft Azure Emotion API as the emotion recognition engine. If a user attempts to enter sensitive information, the device will detect it, block the input, and display an alert.

[0741] User actions

[0742] The user first launches a browser and logs into the company's authentication system. Upon successful authentication, the security policy is applied. If the user attempts to enter certain sensitive information into the browser, the device monitors it in real time, immediately blocking the input and displaying an alert if sensitive information is detected. Simultaneously, the user's emotions are monitored in real time, and if stress or anger is detected, emotion data is sent to the server.

[0743] Specific example

[0744] For example, if a user attempts to enter technical details of a patent application, the system immediately detects this and displays an alert stating, "This contains confidential information. Input has been blocked." If the user also exhibits feelings of stress or anger, the emotion recognition engine identifies these emotions and sends the emotion data to the server. Administrators can then receive this data and take appropriate action.

[0745] Example of a prompt

[0746] "A user is attempting to enter technical details of a patent application into an external email. Please implement a function that detects and blocks this input, and simultaneously recognizes and reports the user's emotional state (e.g., anger)."

[0747] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0748] Step 1:

[0749] The server authenticates the user to the company's authentication system. The user enters their authentication information (username and password) to log in to the authentication system and sends this information to the server. The server verifies the authentication information and determines whether authentication was successful. If authentication is successful, the user's authentication is complete and a token is issued to proceed to the next step. The input is the user's authentication information, and the output is whether authentication was successful or not (a token if successful).

[0750] Step 2:

[0751] The server generates a security policy upon successful authentication and sends it to the user's device using a token. The security policy is a set of rules containing a regular expression pattern for sensitive information. The input is the authentication token, and the output is the security policy.

[0752] Step 3:

[0753] The device applies the acquired security policy to the web browser. The applied security policy monitors user input in real time and determines whether sensitive information is detected. The input is the security policy, and the output is the browser environment to which the policy is applied.

[0754] Step 4:

[0755] The terminal monitors user input in real time and uses regular expression patterns to detect whether it contains sensitive information. When a user begins typing in the web browser, the text is inspected based on security policies. The input is the user's text, and the output is the result of the determination of whether it contains sensitive information.

[0756] Step 5:

[0757] The terminal blocks user input and displays an alert if sensitive information is detected. When a user attempts to enter text containing sensitive information, the system blocks the input and displays a warning message stating, "This contains sensitive information. Input blocked." The input is the result of the sensitive information detection, and the output is the blocked input and the displayed alert.

[0758] Step 6:

[0759] The terminal sends an operation log to the server when it detects the input of confidential information. The operation log contains details of the confidential information the user attempted to input and the actions that occurred. Input is the event of entering confidential information, and output is the operation log.

[0760] Step 7:

[0761] The server collects operation logs and manages security incidents. It analyzes the collected operation logs to enable administrators to take appropriate action. The input is operation logs, and the output is security incident management information.

[0762] Step 8:

[0763] The device utilizes an emotion recognition engine to recognize the user's emotional state in real time. Based on the user's facial expressions and voice data acquired from the camera and microphone, the emotion recognition algorithm classifies the emotion. The input is the user's facial expressions and voice data, and the output is the recognized emotional state.

[0764] Step 9:

[0765] The device sends recognized emotion data to the server. This emotion data includes the user's stress and anger levels. The input is the emotional state, and the output is the emotion data sent to the server.

[0766] Step 10:

[0767] The server automatically takes appropriate actions based on sentiment data. Based on this sentiment data, it can, for example, notify administrators or provide psychological support to users. The input is sentiment data, and the output is the appropriate action taken.

[0768] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0769] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0770] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0771] [Third Embodiment]

[0772] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0773] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0774] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0775] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0776] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0777] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0778] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0779] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0780] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0781] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0782] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0783] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0784] This invention is a system that prevents the leakage of confidential information by applying security policies to corporate browsers and monitoring user input in real time. This system mainly consists of a server, a terminal (user's browser), and the user.

[0785] Server operation

[0786] The server manages security policies and distributes them to endpoints. The server connects to the company's authentication system and verifies user credentials. Upon successful authentication, the server sends the appropriate security policy to the device. In the event of a security incident, the server collects the information and notifies the administrator.

[0787] Terminal (user's browser) operation

[0788] The device receives security policies sent from the server and monitors user input based on those policies. When a user launches a browser, authentication is performed first. If authentication is successful, the device downloads the latest security policy from the server and loads it into memory.

[0789] When a user enters text into a browser input field, the terminal monitors the input in real time. Using regular expression patterns defined in the policy, the terminal inspects the input. If sensitive information is detected, the terminal blocks the input and displays an alert to the user. It also sends an operation log to the server.

[0790] User actions

[0791] The user first launches a browser and logs into the company's authentication system. Once authentication is successful, the user begins using the browser as usual. If the user attempts to enter certain sensitive information into the browser, the input is monitored in real time, and if sensitive information is detected, the input is blocked and an alert is displayed.

[0792] Specific example

[0793] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user opens an email sending screen and attempts to enter "technical details of a patent application," the device's browser monitors this input in real time. Because a regular expression pattern for "technical details of a patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[0794] In this way, the system can apply a company's security policies to the browser and prevent the leakage of confidential information in real time. Furthermore, security incidents that occur are recorded and managed on the server, facilitating a rapid response.

[0795] The following describes the processing flow.

[0796] Step 1:

[0797] The device launches a browser. The user accesses the login screen and enters their authentication information into the company's authentication system.

[0798] Step 2:

[0799] The server receives the user's authentication information and uses it to verify the user's identity. If authentication is successful, the server generates an appropriate security policy and sends it to the terminal.

[0800] Step 3:

[0801] The device receives the security policy sent from the server and applies it within the browser. Specifically, it loads a regular expression pattern for detecting sensitive information into memory.

[0802] Step 4:

[0803] The user begins typing text within the browser. As the user enters information into the input field, that information is monitored in real time within the browser.

[0804] Step 5:

[0805] The terminal analyzes the entered text based on security policies. It checks for content that matches regular expression patterns.

[0806] Step 6:

[0807] If the device detects sensitive information, it will block the input and display an alert message to the user. The message will read, "This contains sensitive information. Input has been blocked."

[0808] Step 7:

[0809] The terminal logs the input of confidential information and sends that log to the server. The log includes information such as the user's input and the time when the alert occurred.

[0810] Step 8:

[0811] The server saves the received logs to the database. The server sends an alert to notify the administrator that a security incident has occurred.

[0812] Step 9:

[0813] The administrator receives an alert notification from the server, reviews the details of the security incident, and takes the necessary action.

[0814] In this way, a system is realized that monitors user input in real time throughout each processing step, thereby preventing the leakage of confidential information.

[0815] (Example 1)

[0816] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0817] In today's business environment, the leakage of confidential information poses a significant risk, requiring advanced security measures. However, existing security systems struggle to monitor user input in real time and immediately detect and address confidential information breaches. As a result, the risk of confidential information leakage remains, and companies' information assets cannot be adequately protected. This invention aims to solve these problems and provide a system that effectively protects companies' confidential information.

[0818] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0819] In this invention, the server includes means for authenticating the user, means for obtaining a security policy from the server upon successful authentication, and means for applying the obtained security policy to the user interface. This enables the application of a security policy to the terminal after user authentication by coordinating with the company's authentication system. The terminal also includes means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, and means for sending an operation log to the server when input of confidential information is detected. This prevents the leakage of confidential information and enables a rapid response. Furthermore, the server includes means for collecting operation logs and managing security incidents, which notifies the administrator of the occurrence of a security incident and enables appropriate countermeasures to be taken. This makes it possible to comprehensively protect the company's confidential information.

[0820] "User authentication means" refers to a function that verifies a user's authentication information and confirms whether the user has legitimate access rights.

[0821] The "means for obtaining security policies" refer to a function that downloads security policies from the server to users who have successfully authenticated and applies them to their devices.

[0822] "Means of application to the user interface" refers to a function that reflects the acquired security policy in the user's browser or other user interfaces.

[0823] "Real-time monitoring of user input" refers to a function that monitors data entered by users in a browser or other interface in real time and performs appropriate checks.

[0824] A "means for detecting confidential information" is a function for determining whether or not the monitored input data contains confidential information.

[0825] The "means for blocking user input and displaying alerts" are functions that prevent user input and display a warning message when confidential information is detected.

[0826] The "means for sending operation logs to the server" refers to a function for sending operation history, such as the detection of confidential information and the display of associated alerts, to the server.

[0827] "A means for collecting operation logs and managing security incidents" refers to a function that stores transmitted operation logs on a server and centrally manages security incidents.

[0828] A "regular expression pattern" is a format for defining a pattern of strings, and is used to determine whether a specific string is found.

[0829] "Administrator notification method" refers to a function that notifies administrators via email, alerts, or other methods when a security incident occurs.

[0830] This invention is a system that prevents the leakage of confidential information by applying security policies to a company's user interface (e.g., a browser) and monitoring user input in real time. This system mainly consists of a server, a terminal (the user's browser), and the user.

[0831] Server configuration and operation

[0832] The server implements the following functions:

[0833] 1. User Authentication Function: Receives authentication requests from users and verifies user information using the company's authentication system (e.g., LDAP or OAuth). This function is a crucial step in verifying whether the user has legitimate access rights.

[0834] 2. Security Policy Retrieval and Distribution Function: Upon successful authentication, the server retrieves the appropriate security policy from the database and sends it to the user's device. The security policy is managed as data in JSON format.

[0835] 3. Operation log collection and management function: Operation logs sent from terminals are recorded in a database, and security incidents are managed centrally.

[0836] The server can use a Linux-based system, and the database can use a common database management system such as MySQL or PostgreSQL.

[0837] Terminal (user's browser) configuration and operation

[0838] The device has the following features:

[0839] 1. Authentication Function: When a user launches a browser, they enter authentication information and send an authentication request to the server. This is expected to work with common web browsers such as Chrome and Firefox.

[0840] 2. Security policy reception and application function: Upon successful authentication, the security policy is downloaded from the server, loaded into memory, and applied to the user interface.

[0841] 3. Real-time monitoring function: User input is monitored in real time, and whether it contains sensitive information is checked using regular expression patterns defined in the policy. For example, a script using JavaScript can be created and executed in the browser.

[0842] 4. Input Blocking and Alert Display Function: If confidential information is detected, input will be blocked and an alert will be displayed to the user.

[0843] 5. Operation Log Transmission Function: Detected security incidents are sent to the server as operation logs.

[0844] User actions

[0845] The user performs the following actions:

[0846] 1. Launching the browser and authenticating: Launch your browser and log in to the company's authentication system.

[0847] 2. Normal browser operation: After successful authentication, the user uses the browser as usual. For example, they enter text on the email sending screen.

[0848] 3. Alert response: If an alert is displayed when attempting to enter specific confidential information, review the content of the alert and take appropriate action.

[0849] Specific example

[0850] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user attempts to enter the text "Technical details of patent application" on the email sending screen, the device's browser monitors this input in real time. Because a regular expression pattern for "Technical details of patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[0851] Examples of prompt statements

[0852] Prompt example:

[0853] "Please describe a system that monitors in real time when a user attempts to enter sensitive information in their browser, blocks the input if sensitive information is detected, and displays an alert."

[0854] This invention allows a company's security policies to be applied to the user interface, preventing the leakage of confidential information in real time. Furthermore, security incidents that occur are recorded and managed on the server, facilitating a rapid response.

[0855] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0856] Step 1:

[0857] User authentication request submission

[0858] Input: The user enters their authentication information (username and password) into the browser's login form.

[0859] Output: The entered authentication information is sent to the server.

[0860] Specific action: The user launches a browser, enters their authentication information into the login form, and clicks the "Login" button. This request is sent to the server as an HTTP POST request.

[0861] Step 2:

[0862] Authentication information verification on the server

[0863] Input: The server receives authentication information sent by the user.

[0864] Output: The result of authentication success or failure is generated.

[0865] Specific operation: The server sends a query to the company's authentication system (LDAP or OAuth) to verify the received authentication information and receives a response. If authentication is successful, an authentication token is generated and sent to the user.

[0866] Step 3:

[0867] Sending security policies

[0868] Input: A database containing user information and security policies for users who have successfully authenticated.

[0869] Output: The latest security policy is sent to the device.

[0870] Specific operation: The server retrieves the security policy for the relevant user from the database and sends it to the user's browser as an HTTP response in JSON format.

[0871] Step 4:

[0872] Download and apply security policies

[0873] Input: Security policy received by the terminal from the server.

[0874] Output: The security policy is loaded into the device's memory.

[0875] Specific operation: The browser analyzes the security policy it receives, loads the settings for real-time monitoring into memory according to its content, and applies them.

[0876] Step 5:

[0877] Real-time monitoring of user input

[0878] Input: The user enters text into the input field.

[0879] Output: Input content is checked in real time.

[0880] Specific operation: When a user starts typing text on the email sending screen, JavaScript is used to monitor the input in real time within the browser. A specific regular expression pattern is used based on policy.

[0881] Step 6:

[0882] Detection and blocking of sensitive information

[0883] Input: User input and a regular expression pattern defined by the policy.

[0884] Output: If sensitive information is detected, the input will be blocked and an alert will be displayed.

[0885] Specific operation: JavaScript checks the input content against a regular expression pattern defined in the policy, blocks user input if it contains sensitive information, and displays a warning message.

[0886] Step 7:

[0887] Sending operation logs

[0888] Input: Operation log when confidential information is detected.

[0889] Output: Operation logs are sent to the server.

[0890] Specific operation: A history of operations such as the detection of confidential information and the display of associated alerts is generated and sent to the server as an HTTP request.

[0891] Step 8:

[0892] Operation log collection and security incident management

[0893] Input: Operation log sent from the terminal.

[0894] Output: Operation logs are recorded in the database, and security incidents are managed.

[0895] Specific actions: The server saves the operation logs it receives to a database and lists them in a format viewable by the administrator. It also notifies the administrator of any security incidents that occur.

[0896] (Application Example 1)

[0897] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0898] In recent years, the risk of confidential information leaks within companies has increased, with unauthorized information sharing via the internet being a particular problem. There is a need for technological measures to prevent the leakage of confidential information by monitoring employee input in real time when using web browsers. Furthermore, the ability to quickly display alerts to users through different devices (such as head-mounted displays) is also essential.

[0899] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0900] In this invention, the server includes means for authenticating a user to the company's authentication system, means for obtaining a security policy from the server if authentication is successful, means for applying the obtained security policy to the browser, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert on a head-mounted display if confidential information is detected, means for sending an operation log to the server if input of confidential information is detected, and means for the server to collect the operation log and manage security incidents. This makes it possible to monitor user input in real time, prevent the leakage of confidential information, and immediately display warnings through different devices.

[0901] A "corporate authentication system" is a system that authenticates users in order to control access to a company's network and systems.

[0902] A "security policy" refers to a set of standards and rules that define the protection of data in information systems and the prevention of unintentional information leaks.

[0903] A "browser" is software used to connect to the internet and display and interact with web pages.

[0904] A "head-mounted display" is a wearable device that is worn on the user's head and provides information visually through a display.

[0905] User authentication is the process of verifying that a user accessing a system is a legitimate user.

[0906] "Real-time monitoring" refers to the immediate monitoring of systems and processes, and the instantaneous detection and response to changes in status or anomalies.

[0907] "Confidential information" refers to important internal company information that is not made public and which, if revealed to a third party, could cause damage to the company.

[0908] An "alert" is a means of notifying a user and issuing a warning when specific conditions occur.

[0909] An "operation log" refers to a record of a series of operations or events performed on a system, which is used later for analysis and auditing.

[0910] A "security incident" refers to an event in which a problem or risk related to information security occurs.

[0911] This invention is a system that authenticates users using a company's authentication system and retrieves a security policy from the server upon successful authentication. This system applies the retrieved security policy to the browser and monitors user input in real time to prevent the leakage of confidential information. Furthermore, a key feature is the ability to display an alert on a head-mounted display (HMD) when confidential information is detected, enabling immediate response.

[0912] First, the user logs into the company's authentication system using a browser or HMD. During this process, authentication information is sent to the server, which then authenticates the user. Upon successful authentication, the server retrieves the appropriate security policy and sends it to the user's device or HMD. The security policy includes a regular expression pattern for identifying sensitive information.

[0913] When a user enters text via a browser or HMD, the device monitors the input in real time. If the input matches a regular expression pattern defined in the security policy, it is determined to be sensitive information, and the input is immediately blocked. At this time, a warning message is displayed on the HMD, and the user is aware that the input of sensitive information has been blocked. In parallel, an operation log is sent to the server, which records it and manages security incidents.

[0914] The specific configuration of this system is as follows:

[0915] 1. Hardware to be used:

[0916] Head-mounted display (HMD)

[0917] User's device (PC or tablet)

[0918] 2. Software to use:

[0919] Authentication system: Performs user authentication.

[0920] Security policy management system: Distributes security policies that include regular expression patterns.

[0921] Real-time monitoring software: Monitors input content in real time and detects confidential information.

[0922] The following scenarios are possible as specific examples:

[0923] For example, consider a scenario where an employee using "Secure Browsing" within a company wears an HMD (Head-Mounted Display) while working. When the user attempts to type "technical details of a patent-pending application" into the email composition screen, a real-time alert is displayed on the HMD, blocking the input. The alert displays "Confidential information is contained. Input has been blocked," and this information is simultaneously sent to the server as a log.

[0924] Examples of prompts for a generative AI model include the following:

[0925] "A user is wearing an HMD (Head-Mounted Display) and browsing the web. I want to build a system that displays a real-time warning and blocks input if the user attempts to enter sensitive information. I will implement this using Python. The security policy will be provided by the server, and user input will be checked using regular expressions. Operation logs will be sent to the server, and alerts will be displayed on the HMD. Please tell me how to design the code."

[0926] In this way, the system of this invention can enhance corporate information security and prevent the leakage of confidential information.

[0927] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0928] Step 1:

[0929] The server receives the user's authentication information.

[0930] Input: User ID and password

[0931] Output: Authentication result (success / failure)

[0932] Specific operation: The user sends their entered ID and password to the server, which then uses its authentication system to perform authentication. If authentication is successful, the user proceeds to the next step.

[0933] Step 2:

[0934] The server retrieves the security policy and sends it to the terminal.

[0935] Input: Authentication result (success / failure)

[0936] Output: Security Policy

[0937] Specific operation: For a user who has successfully authenticated, the server searches for an appropriate security policy and sends it to the terminal. The security policy includes a regular expression pattern to identify sensitive information.

[0938] Step 3:

[0939] The device loads the security policy into memory.

[0940] Input: Security Policy

[0941] Output: Security policy loaded into memory

[0942] Specific operation: The terminal parses the security policy received from the server and loads it into memory. This prepares the terminal to monitor user input.

[0943] Step 4:

[0944] The user enters text via a browser or head-mounted display (HMD).

[0945] Input: User input text

[0946] Output: Monitored input text

[0947] Specific operation: The user enters text using a browser or HMD, and this input is monitored in real time.

[0948] Step 5:

[0949] The terminal monitors input text in real time and uses regular expression patterns to detect sensitive information.

[0950] Input: Input text, regular expression pattern

[0951] Output: Confidential information detection result (detected / not detected)

[0952] Specific operation: The terminal checks the input text against a regular expression pattern defined in the security policy to determine if it contains sensitive information.

[0953] Step 6:

[0954] If confidential information is detected, the terminal will block input and display an alert on the head-mounted display (HMD).

[0955] Input: Confidential information detection results

[0956] Output: Alert message, blocked input text

[0957] Specific operation: If the device detects sensitive information, it will block the input and display an alert message on the HMD stating, "Sensitive information is contained. Input has been blocked."

[0958] Step 7:

[0959] If the terminal detects the input of confidential information, it will send an operation log to the server.

[0960] Input: Confidential information detection results, input text

[0961] Output: Operation Log

[0962] Specific action: Send an operation log containing the detected confidential information and detection results to the server.

[0963] Step 8:

[0964] The server collects operation logs and manages them as security incidents.

[0965] Input: Operation Log

[0966] Output: Security incident log

[0967] Specific operation: The server saves the received operation logs to a database and manages them as security incidents.

[0968] Step 9:

[0969] The server notifies the administrator of the occurrence of a security incident.

[0970] Input: Security Incident Log

[0971] Output: Notification message

[0972] Specific operation: The server notifies the administrator of recorded security incidents to encourage a quick response.

[0973] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0974] This invention achieves more advanced information leakage prevention and security management by combining a browser system that applies a company's security policy and monitors user input in real time with an emotion engine that recognizes user emotions. This system mainly consists of a server, a terminal (the user's browser), and the user.

[0975] Server operation

[0976] The server manages security policies and distributes them to endpoints. It connects to the company's authentication system and verifies user credentials. Upon successful authentication, the server generates an appropriate security policy and sends it to the device. In the event of a security incident, the server collects the information and notifies the administrator. The server also manages data from the sentiment engine and uses it to improve or enforce security policies as needed.

[0977] Terminal (user's browser) operation

[0978] The device receives security policies sent from the server and monitors user input based on those policies. When a user launches a browser, authentication is performed first. If authentication is successful, the device downloads the latest security policy from the server and loads it into memory. Furthermore, the device has an emotion engine built in that recognizes emotions in real time from the user's facial expressions and vocalizations.

[0979] When a user enters text into a browser input field, that information is monitored in real time within the browser. The device inspects the input using regular expression patterns defined in the policy. If sensitive information is detected, the device blocks the input and displays an alert to the user. It also sends an operation log to the server.

[0980] Simultaneously, the emotion engine analyzes the user's facial expressions and vocalizations to recognize their emotional state in real time. This emotional data is then used to inform actions based on security policies. For example, if the system detects that a user is experiencing frustration, it might take steps such as providing appropriate psychological safety measures.

[0981] User actions

[0982] The user first launches a browser and logs into the company's authentication system. Upon successful authentication, the user begins using the browser as usual. If the user attempts to enter specific confidential information into the browser, the input is monitored in real time. If confidential information is detected, the input is blocked and an alert is displayed. Simultaneously, the user's emotions are monitored in real time, and appropriate measures are taken as needed.

[0983] Specific example

[0984] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user opens an email sending screen and attempts to enter "technical details of a patent application," the device's browser monitors this input in real time. Because a regular expression pattern for "technical details of a patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[0985] Furthermore, if the user experiences stress or displays anger during this process, the emotion engine recognizes this. The device sends this emotion data to a server, which then notifies the administrator based on that data. The administrator can then understand the circumstances under which the user experienced stress and provide appropriate support.

[0986] In this way, a sophisticated system that integrates security and user sentiment management is realized through each processing step.

[0987] The following describes the processing flow.

[0988] Step 1:

[0989] The device launches a browser. The user accesses the login screen and enters their authentication information into the company's authentication system.

[0990] Step 2:

[0991] The server receives the user's authentication information and uses it to verify the user's identity. If authentication is successful, the server generates an appropriate security policy and sends it to the terminal.

[0992] Step 3:

[0993] The device receives the security policy sent from the server and applies it within the browser. Specifically, it loads a regular expression pattern for detecting sensitive information into memory.

[0994] Step 4:

[0995] The device activates its emotion engine and prepares to analyze the user's facial expressions and vocalizations. The emotion engine uses the camera and microphone to collect the user's facial expressions and voice data.

[0996] Step 5:

[0997] The user begins typing text within the browser. As the user enters information into the input field, that information is monitored in real time within the browser.

[0998] Step 6:

[0999] The terminal analyzes the entered text based on security policies. It checks for content that matches regular expression patterns.

[1000] Step 7:

[1001] If the device detects sensitive information, it will block the input and display an alert message to the user. The message will read, "This contains sensitive information. Input has been blocked."

[1002] Step 8:

[1003] At the same time, the device uses an emotion engine to recognize the user's emotional state in real time. For example, if the user feels irritated or anxious, that emotion is detected in real time by the emotion engine.

[1004] Step 9:

[1005] The terminal logs the input of confidential information and sends that log to the server. The log includes the user's input, the time the alert occurred, and the user's emotional state.

[1006] Step 10:

[1007] The server saves the received logs to the database. The server sends an alert to notify the administrator that a security incident has occurred.

[1008] Step 11:

[1009] The server also notifies the administrator of the user's emotion data sent from the emotion engine. This allows the administrator to see what emotions the user displayed during a particular action.

[1010] Step 12:

[1011] The administrator receives an alert notification from the server, reviews the details of the security incident and the user's emotional state, and takes the necessary action.

[1012] In this way, by combining security policy-based functions with an emotion engine, a system is built that simultaneously prevents information leaks and manages user emotions.

[1013] (Example 2)

[1014] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1015] In recent years, information leaks and security incidents within companies have been increasing, and effective countermeasures are needed. Furthermore, preventing the leakage of confidential information due to user actions and responding quickly when users experience stress or anger are particularly important. However, conventional systems have not achieved comprehensive security management by combining user input monitoring with emotion recognition. Therefore, the present invention aims to provide advanced information leakage prevention and security management by monitoring both user input and emotions.

[1016] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for authenticating the user to the company's authentication system, means for obtaining a security policy from the server when authentication is successful, means for applying the obtained security policy to the terminal, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, means for sending an operation log to the server when input of confidential information is detected, means for the server to collect the operation log and manage security incidents, means for analyzing emotional data using an emotion engine that recognizes the user's emotions, and means for sending the emotional data to the server and notifying the administrator. This not only prevents the leakage of confidential information but also makes it possible to quickly address the stress and anger felt by the user.

[1017] A "corporate authentication system" refers to a system used within a company to verify the identity of users. Specific examples include LDAP and Active Directory.

[1018] A "server" is a computer that provides specific services or resources, and in this invention, it refers to a central device that manages security policies, collects operation logs, and analyzes sentiment data.

[1019] A "security policy" outlines the rules and guidelines for maintaining information security, including the rules and patterns used in monitoring input data and detecting confidential information.

[1020] A "device" refers to a device operated by a user, specifically a computer or smartphone, including a web browser.

[1021] An "emotion engine" refers to software or hardware used to analyze a user's emotional state, and includes technologies that determine emotions in real time based on the user's facial expressions and voice.

[1022] An "operation log" is data that records the history of operations performed by a user, and is used for tracking and analyzing security incidents.

[1023] A "security incident" refers to any event or action that could pose a threat to information security, including attempted data breaches and unauthorized access.

[1024] A "regular expression" is a set of string rules used to match text patterns, and is used to efficiently detect text that matches specific conditions.

[1025] "Emotional data" refers to digital information that indicates the user's emotional state as analyzed by the emotion engine, and includes data transmitted to the server in real time.

[1026] This invention integrates an emotion engine that recognizes user emotions into a browser system that applies a company's security policy to terminals and monitors user input in real time. This system mainly consists of a server, terminals (user's browser), and the user.

[1027] Server operation

[1028] The server plays a central role and performs the following tasks:

[1029] 1. Connect to the company's authentication system and verify the user's authentication information. Specifically, this involves using authentication systems such as LDAP or Active Directory.

[1030] 2. Upon successful authentication, the system generates and sends the company's security policy to the device. The security policy includes regular expression patterns for detecting sensitive information.

[1031] 3. In the event of a security incident, operation logs will be collected and managed. Furthermore, sentiment data from the sentiment engine will be managed, and a swift response will be initiated based on this data.

[1032] Terminal (user's browser) operation

[1033] A device is a device that users use on a daily basis and has the following functions:

[1034] 1. Receive the security policy sent from the server and load it into memory. The browser uses JavaScript to parse this data.

[1035] 2. Monitor user input in real time and detect sensitive information based on regular expression patterns defined in the policy. If sensitive information is detected, block the input and display an alert to the user.

[1036] 3. Send user input and operation logs to the server.

[1037] 4. The built-in emotion engine analyzes the user's facial expressions and voice to recognize their emotional state in real time. In particular, if the user shows signs of stress or anger, the data is sent to the server and notified to the administrator.

[1038] User operating procedures

[1039] Users access the system through the following steps:

[1040] 1. Launch your browser and log in to your company's authentication system. If authentication is successful, the security policy will be sent from the server to your device and applied.

[1041] 2. When a user enters information in their browser, that information is monitored in real time. For example, if a user attempts to enter "technical details of a patent application" on an email sending screen, the browser will detect this content, block the input, and display an alert.

[1042] 3. Simultaneously, the emotion engine analyzes the user's emotions, and if it detects stress or anger, that data is sent to the server and notified to the administrator.

[1043] Specific example

[1044] For example, suppose a user launches a browser, authenticates, and a security policy is applied. When the user attempts to enter "technical details of a patent application," the browser monitors this text in real time and blocks the input because it matches a regular expression pattern defined in the policy. An alert appears stating, "This contains sensitive information. Input blocked," and an operation log is sent to the server.

[1045] Furthermore, if a user experiences stress or displays anger during this process, the emotion engine recognizes this. Emotional data is sent to the server, which notifies the administrator, enabling them to provide appropriate support based on the situation. This results in an advanced data leakage prevention system that integrates security and user emotion management.

[1046] Example of a prompt

[1047] 1. "Describe a system in which, when a user attempts to enter patent information into a browser, the input is blocked and saved to the server as an operational log."

[1048] 2. "Please describe in detail a scenario in which emotions are recognized from the end user's facial expressions and reflected in security policies in real time."

[1049] Thus, the present invention provides a system that integrates corporate security and user sentiment management, enabling information leakage prevention and rapid response.

[1050] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1051] Step 1:

[1052] When a user launches a browser and attempts to log in to the authentication system, the server receives the authentication information (username and password) entered by the user. The server queries the company's authentication system, such as LDAP or Active Directory, with this authentication information. Upon receiving a response from the authentication system and confirming successful authentication, the server generates a session ID and sends that session ID to the terminal.

[1053] Input: Authentication information entered by the user

[1054] Data processing: Query authentication information with the authentication system.

[1055] Output: Authentication success / failure and session ID

[1056] Step 2:

[1057] The server retrieves the latest security policy information from the database for users who have successfully authenticated, in order to generate the company's security policy. It then encodes the retrieved policy information into JSON format and sends it to the terminal via an HTTP / S response.

[1058] Input: User information for successful authentication

[1059] Data processing: Retrieving security policies from the database and JSON encoding

[1060] Output: Security policy in JSON format

[1061] Step 3:

[1062] The terminal receives JSON data of the security policy from the server and parses it using JavaScript. It loads the parsed security policy into memory and begins monitoring user input under its influence.

[1063] Input: Security policy in JSON format received from the server

[1064] Data processing: Parsing and loading JSON data into memory.

[1065] Output: Loaded security policy

[1066] Step 4:

[1067] When a user enters text into a browser input field, JavaScript on the device monitors the input in real time. It checks whether the input matches a regular expression pattern defined in the security policy.

[1068] Input: User input

[1069] Data processing: Matching based on regular expression patterns

[1070] Output: Match result

[1071] Step 5:

[1072] If the input is detected as confidential information, the device will block the input and clear the input field using JavaScript. Then, it will display an alert to the user stating, "Confidential information is contained. Input has been blocked."

[1073] Input: Match result (confidential information)

[1074] Data processing: Clear input fields and display alerts.

[1075] Output: Alert displayed and input fields cleared

[1076] Step 6:

[1077] The terminal generates a user operation log and sends it to the server via an AJAX request. The operation log includes the user's input, the time, and the type of incident.

[1078] Input: User input and incident information

[1079] Data processing: Generating operation logs and sending AJAX requests.

[1080] Output: Operation log sent to the server

[1081] Step 7:

[1082] Simultaneously, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state in real time. In particular, if the user exhibits stress or anger, the emotion data is compiled in JSON format and sent from the device to the server.

[1083] Input: User's facial expressions and voice data

[1084] Data processing: Analysis of emotional states and conversion to JSON format.

[1085] Output: Sentiment data sent to the server

[1086] Step 8:

[1087] The server analyzes the received emotional data and, if it detects emotions such as stress or anger, notifies the administrator of this information. The administrator then takes appropriate action based on this notification.

[1088] Input: Sentiment data

[1089] Data processing: Analysis of emotional data and generation of notifications.

[1090] Output: Notification to administrator

[1091] In this way, by comprehensively monitoring and managing user input and emotional states, a system can be realized that prevents information leaks and appropriately responds to the user's psychological state.

[1092] (Application Example 2)

[1093] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1094] Existing security management systems have the capability to detect confidential information entered by users in real time, but they do not take into account the user's emotional state. Therefore, when a user experiences stress or anger, appropriate responses may not be taken quickly, resulting in increased security risks. To solve this problem, it is necessary to recognize the user's emotions in real time and take appropriate actions based on that recognition.

[1095] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for authenticating the user to the company's authentication system, means for obtaining a security policy when authentication is successful, means for applying the obtained security policy to the web browser, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, means for sending an operation log to the server when input of confidential information is detected, means for the server to collect the operation log and manage security incidents, means for recognizing the user's emotional state in real time, means for sending the recognized emotional data to the server, and means for automatically performing appropriate actions based on the emotional data. This makes it possible to detect security risks early based on the user's emotional state and take appropriate countermeasures quickly.

[1096] A "corporate authentication system" is a mechanism that identifies and verifies a user's access to authorized systems to which they have access rights.

[1097] A "server" is a computer system that provides services to other computers and terminals on a network.

[1098] A "security policy" is a set of rules and procedures established for the purpose of protecting information.

[1099] A "web browser" is software used to view web pages on the internet.

[1100] "User input" refers to information that a user provides to a computer system through a keyboard or other input devices.

[1101] "Confidential information" refers to information that, if disclosed, could potentially cause harm to a company, such as unreleased technical information or trade secrets.

[1102] An "alert" is a warning message that alerts users or administrators when certain conditions are met.

[1103] An "operation log" is a record of actions performed by a user and is used for security monitoring and problem solving.

[1104] A "security incident" refers to security-related problems or accidents, such as unauthorized access to a company's information systems or data breaches.

[1105] "Emotional state" refers to the user's psychological state or mood, and includes things like anger and stress.

[1106] "Real-time" means that a process or calculation produces results immediately, with virtually no delay.

[1107] "Appropriate action" refers to the best course of action to take in a given situation.

[1108] This invention is a system that authenticates users based on a company's authentication system, retrieves a security policy from a server upon successful authentication, and applies the retrieved security policy to the web browser. Furthermore, it monitors user input and emotional state in real time, blocks input and displays an alert if confidential information is detected. If confidential information is detected, it sends an operation log to the server, which collects the operation log and manages security incidents. In addition, it recognizes the user's emotional state in real time, sends the recognized emotional data to the server, and automatically performs appropriate actions based on the emotional data.

[1109] Server operation

[1110] The server connects to the company's authentication system and verifies the user's credentials. If authentication is successful, it generates a security policy and sends it to the terminal. It also collects operation logs and sentiment data to manage security incidents. The specific cloud services used are Amazon Web Services (AWS) and Microsoft Azure.

[1111] Terminal (user's browser) operation

[1112] The device has a function to monitor user input in real time and detect sensitive information. It incorporates an emotion recognition engine that recognizes emotions based on the user's facial expressions and tone of voice. Specifically, it uses the Microsoft Azure Emotion API as the emotion recognition engine. If a user attempts to enter sensitive information, the device will detect it, block the input, and display an alert.

[1113] User actions

[1114] The user first launches a browser and logs into the company's authentication system. Upon successful authentication, the security policy is applied. If the user attempts to enter certain sensitive information into the browser, the device monitors it in real time, immediately blocking the input and displaying an alert if sensitive information is detected. Simultaneously, the user's emotions are monitored in real time, and if stress or anger is detected, emotion data is sent to the server.

[1115] Specific example

[1116] For example, if a user attempts to enter technical details of a patent application, the system immediately detects this and displays an alert stating, "This contains confidential information. Input has been blocked." If the user also exhibits feelings of stress or anger, the emotion recognition engine identifies these emotions and sends the emotion data to the server. Administrators can then receive this data and take appropriate action.

[1117] Example of a prompt

[1118] "A user is attempting to enter technical details of a patent application into an external email. Please implement a function that detects and blocks this input, and simultaneously recognizes and reports the user's emotional state (e.g., anger)."

[1119] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1120] Step 1:

[1121] The server authenticates the user to the company's authentication system. The user enters their authentication information (username and password) to log in to the authentication system and sends this information to the server. The server verifies the authentication information and determines whether authentication was successful. If authentication is successful, the user's authentication is complete and a token is issued to proceed to the next step. The input is the user's authentication information, and the output is whether authentication was successful or not (a token if successful).

[1122] Step 2:

[1123] The server generates a security policy upon successful authentication and sends it to the user's device using a token. The security policy is a set of rules containing a regular expression pattern for sensitive information. The input is the authentication token, and the output is the security policy.

[1124] Step 3:

[1125] The device applies the acquired security policy to the web browser. The applied security policy monitors user input in real time and determines whether sensitive information is detected. The input is the security policy, and the output is the browser environment to which the policy is applied.

[1126] Step 4:

[1127] The terminal monitors user input in real time and uses regular expression patterns to detect whether it contains sensitive information. When a user begins typing in the web browser, the text is inspected based on security policies. The input is the user's text, and the output is the result of the determination of whether it contains sensitive information.

[1128] Step 5:

[1129] The terminal blocks user input and displays an alert if sensitive information is detected. When a user attempts to enter text containing sensitive information, the system blocks the input and displays a warning message stating, "This contains sensitive information. Input blocked." The input is the result of the sensitive information detection, and the output is the blocked input and the displayed alert.

[1130] Step 6:

[1131] The terminal sends an operation log to the server when it detects the input of confidential information. The operation log contains details of the confidential information the user attempted to input and the actions that occurred. Input is the event of entering confidential information, and output is the operation log.

[1132] Step 7:

[1133] The server collects operation logs and manages security incidents. It analyzes the collected operation logs to enable administrators to take appropriate action. The input is operation logs, and the output is security incident management information.

[1134] Step 8:

[1135] The device utilizes an emotion recognition engine to recognize the user's emotional state in real time. Based on the user's facial expressions and voice data acquired from the camera and microphone, the emotion recognition algorithm classifies the emotion. The input is the user's facial expressions and voice data, and the output is the recognized emotional state.

[1136] Step 9:

[1137] The device sends recognized emotion data to the server. This emotion data includes the user's stress and anger levels. The input is the emotional state, and the output is the emotion data sent to the server.

[1138] Step 10:

[1139] The server automatically takes appropriate actions based on sentiment data. Based on this sentiment data, it can, for example, notify administrators or provide psychological support to users. The input is sentiment data, and the output is the appropriate action taken.

[1140] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1141] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1142] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1143] [Fourth Embodiment]

[1144] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1145] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1146] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1147] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1148] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1149] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1150] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1151] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1152] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1153] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1154] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1155] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1156] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1157] This invention is a system that prevents the leakage of confidential information by applying security policies to corporate browsers and monitoring user input in real time. This system mainly consists of a server, a terminal (user's browser), and the user.

[1158] Server operation

[1159] The server manages security policies and distributes them to endpoints. The server connects to the company's authentication system and verifies user credentials. Upon successful authentication, the server sends the appropriate security policy to the device. In the event of a security incident, the server collects the information and notifies the administrator.

[1160] Terminal (user's browser) operation

[1161] The device receives security policies sent from the server and monitors user input based on those policies. When a user launches a browser, authentication is performed first. If authentication is successful, the device downloads the latest security policy from the server and loads it into memory.

[1162] When a user enters text into a browser input field, the terminal monitors the input in real time. Using regular expression patterns defined in the policy, the terminal inspects the input. If sensitive information is detected, the terminal blocks the input and displays an alert to the user. It also sends an operation log to the server.

[1163] User actions

[1164] The user first launches a browser and logs into the company's authentication system. Once authentication is successful, the user begins using the browser as usual. If the user attempts to enter certain sensitive information into the browser, the input is monitored in real time, and if sensitive information is detected, the input is blocked and an alert is displayed.

[1165] Specific example

[1166] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user opens an email sending screen and attempts to enter "technical details of a patent application," the device's browser monitors this input in real time. Because a regular expression pattern for "technical details of a patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[1167] In this way, the system can apply a company's security policies to the browser and prevent the leakage of confidential information in real time. Furthermore, security incidents that occur are recorded and managed on the server, facilitating a rapid response.

[1168] The following describes the processing flow.

[1169] Step 1:

[1170] The device launches a browser. The user accesses the login screen and enters their authentication information into the company's authentication system.

[1171] Step 2:

[1172] The server receives the user's authentication information and uses it to verify the user's identity. If authentication is successful, the server generates an appropriate security policy and sends it to the terminal.

[1173] Step 3:

[1174] The device receives the security policy sent from the server and applies it within the browser. Specifically, it loads a regular expression pattern for detecting sensitive information into memory.

[1175] Step 4:

[1176] The user begins typing text within the browser. As the user enters information into the input field, that information is monitored in real time within the browser.

[1177] Step 5:

[1178] The terminal analyzes the entered text based on security policies. It checks for content that matches regular expression patterns.

[1179] Step 6:

[1180] If the device detects sensitive information, it will block the input and display an alert message to the user. The message will read, "This contains sensitive information. Input has been blocked."

[1181] Step 7:

[1182] The terminal logs the input of confidential information and sends that log to the server. The log includes information such as the user's input and the time when the alert occurred.

[1183] Step 8:

[1184] The server saves the received logs to the database. The server sends an alert to notify the administrator that a security incident has occurred.

[1185] Step 9:

[1186] The administrator receives an alert notification from the server, reviews the details of the security incident, and takes the necessary action.

[1187] In this way, a system is realized that monitors user input in real time throughout each processing step, thereby preventing the leakage of confidential information.

[1188] (Example 1)

[1189] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1190] In today's business environment, the leakage of confidential information poses a significant risk, requiring advanced security measures. However, existing security systems struggle to monitor user input in real time and immediately detect and address confidential information breaches. As a result, the risk of confidential information leakage remains, and companies' information assets cannot be adequately protected. This invention aims to solve these problems and provide a system that effectively protects companies' confidential information.

[1191] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1192] In this invention, the server includes means for authenticating the user, means for obtaining a security policy from the server upon successful authentication, and means for applying the obtained security policy to the user interface. This enables the application of a security policy to the terminal after user authentication by coordinating with the company's authentication system. The terminal also includes means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, and means for sending an operation log to the server when input of confidential information is detected. This prevents the leakage of confidential information and enables a rapid response. Furthermore, the server includes means for collecting operation logs and managing security incidents, which notifies the administrator of the occurrence of a security incident and enables appropriate countermeasures to be taken. This makes it possible to comprehensively protect the company's confidential information.

[1193] "User authentication means" refers to a function that verifies a user's authentication information and confirms whether the user has legitimate access rights.

[1194] The "means for obtaining security policies" refer to a function that downloads security policies from the server to users who have successfully authenticated and applies them to their devices.

[1195] "Means of application to the user interface" refers to a function that reflects the acquired security policy in the user's browser or other user interfaces.

[1196] "Real-time monitoring of user input" refers to a function that monitors data entered by users in a browser or other interface in real time and performs appropriate checks.

[1197] A "means for detecting confidential information" is a function for determining whether or not the monitored input data contains confidential information.

[1198] The "means for blocking user input and displaying alerts" are functions that prevent user input and display a warning message when confidential information is detected.

[1199] The "means for sending operation logs to the server" refers to a function for sending operation history, such as the detection of confidential information and the display of associated alerts, to the server.

[1200] "A means for collecting operation logs and managing security incidents" refers to a function that stores transmitted operation logs on a server and centrally manages security incidents.

[1201] A "regular expression pattern" is a format for defining a pattern of strings, and is used to determine whether a specific string is found.

[1202] "Administrator notification method" refers to a function that notifies administrators via email, alerts, or other methods when a security incident occurs.

[1203] This invention is a system that prevents the leakage of confidential information by applying security policies to a company's user interface (e.g., a browser) and monitoring user input in real time. This system mainly consists of a server, a terminal (the user's browser), and the user.

[1204] Server configuration and operation

[1205] The server implements the following functions:

[1206] 1. User Authentication Function: Receives authentication requests from users and verifies user information using the company's authentication system (e.g., LDAP or OAuth). This function is a crucial step in verifying whether the user has legitimate access rights.

[1207] 2. Security Policy Retrieval and Distribution Function: Upon successful authentication, the server retrieves the appropriate security policy from the database and sends it to the user's device. The security policy is managed as data in JSON format.

[1208] 3. Operation log collection and management function: Operation logs sent from terminals are recorded in a database, and security incidents are managed centrally.

[1209] The server can use a Linux-based system, and the database can use a common database management system such as MySQL or PostgreSQL.

[1210] Terminal (user's browser) configuration and operation

[1211] The device has the following features:

[1212] 1. Authentication Function: When a user launches a browser, they enter authentication information and send an authentication request to the server. This is expected to work with common web browsers such as Chrome and Firefox.

[1213] 2. Security policy reception and application function: Upon successful authentication, the security policy is downloaded from the server, loaded into memory, and applied to the user interface.

[1214] 3. Real-time monitoring function: User input is monitored in real time, and whether it contains sensitive information is checked using regular expression patterns defined in the policy. For example, a script using JavaScript can be created and executed in the browser.

[1215] 4. Input Blocking and Alert Display Function: If confidential information is detected, input will be blocked and an alert will be displayed to the user.

[1216] 5. Operation Log Transmission Function: Detected security incidents are sent to the server as operation logs.

[1217] User actions

[1218] The user performs the following actions:

[1219] 1. Launching the browser and authenticating: Launch your browser and log in to the company's authentication system.

[1220] 2. Normal browser operation: After successful authentication, the user uses the browser as usual. For example, they enter text on the email sending screen.

[1221] 3. Alert response: If an alert is displayed when attempting to enter specific confidential information, review the content of the alert and take appropriate action.

[1222] Specific example

[1223] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user attempts to enter the text "Technical details of patent application" on the email sending screen, the device's browser monitors this input in real time. Because a regular expression pattern for "Technical details of patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[1224] Examples of prompt statements

[1225] Prompt example:

[1226] "Please describe a system that monitors in real time when a user attempts to enter sensitive information in their browser, blocks the input if sensitive information is detected, and displays an alert."

[1227] This invention allows a company's security policies to be applied to the user interface, preventing the leakage of confidential information in real time. Furthermore, security incidents that occur are recorded and managed on the server, facilitating a rapid response.

[1228] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1229] Step 1:

[1230] User authentication request submission

[1231] Input: The user enters their authentication information (username and password) into the browser's login form.

[1232] Output: The entered authentication information is sent to the server.

[1233] Specific action: The user launches a browser, enters their authentication information into the login form, and clicks the "Login" button. This request is sent to the server as an HTTP POST request.

[1234] Step 2:

[1235] Authentication information verification on the server

[1236] Input: The server receives authentication information sent by the user.

[1237] Output: The result of authentication success or failure is generated.

[1238] Specific operation: The server sends a query to the company's authentication system (LDAP or OAuth) to verify the received authentication information and receives a response. If authentication is successful, an authentication token is generated and sent to the user.

[1239] Step 3:

[1240] Sending security policies

[1241] Input: A database containing user information and security policies for users who have successfully authenticated.

[1242] Output: The latest security policy is sent to the device.

[1243] Specific operation: The server retrieves the security policy for the relevant user from the database and sends it to the user's browser as an HTTP response in JSON format.

[1244] Step 4:

[1245] Download and apply security policies

[1246] Input: Security policy received by the terminal from the server.

[1247] Output: The security policy is loaded into the device's memory.

[1248] Specific operation: The browser analyzes the security policy it receives, loads the settings for real-time monitoring into memory according to its content, and applies them.

[1249] Step 5:

[1250] Real-time monitoring of user input

[1251] Input: The user enters text into the input field.

[1252] Output: Input content is checked in real time.

[1253] Specific operation: When a user starts typing text on the email sending screen, JavaScript is used to monitor the input in real time within the browser. A specific regular expression pattern is used based on policy.

[1254] Step 6:

[1255] Detection and blocking of sensitive information

[1256] Input: User input and a regular expression pattern defined by the policy.

[1257] Output: If sensitive information is detected, the input will be blocked and an alert will be displayed.

[1258] Specific operation: JavaScript checks the input content against a regular expression pattern defined in the policy, blocks user input if it contains sensitive information, and displays a warning message.

[1259] Step 7:

[1260] Sending operation logs

[1261] Input: Operation log when confidential information is detected.

[1262] Output: Operation logs are sent to the server.

[1263] Specific operation: A history of operations such as the detection of confidential information and the display of associated alerts is generated and sent to the server as an HTTP request.

[1264] Step 8:

[1265] Operation log collection and security incident management

[1266] Input: Operation log sent from the terminal.

[1267] Output: Operation logs are recorded in the database, and security incidents are managed.

[1268] Specific actions: The server saves the operation logs it receives to a database and lists them in a format viewable by the administrator. It also notifies the administrator of any security incidents that occur.

[1269] (Application Example 1)

[1270] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1271] In recent years, the risk of confidential information leaks within companies has increased, with unauthorized information sharing via the internet being a particular problem. There is a need for technological measures to prevent the leakage of confidential information by monitoring employee input in real time when using web browsers. Furthermore, the ability to quickly display alerts to users through different devices (such as head-mounted displays) is also essential.

[1272] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1273] In this invention, the server includes means for authenticating a user to the company's authentication system, means for obtaining a security policy from the server if authentication is successful, means for applying the obtained security policy to the browser, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert on a head-mounted display if confidential information is detected, means for sending an operation log to the server if input of confidential information is detected, and means for the server to collect the operation log and manage security incidents. This makes it possible to monitor user input in real time, prevent the leakage of confidential information, and immediately display warnings through different devices.

[1274] A "corporate authentication system" is a system that authenticates users in order to control access to a company's network and systems.

[1275] A "security policy" refers to a set of standards and rules that define the protection of data in information systems and the prevention of unintentional information leaks.

[1276] A "browser" is software used to connect to the internet and display and interact with web pages.

[1277] A "head-mounted display" is a wearable device that is worn on the user's head and provides information visually through a display.

[1278] User authentication is the process of verifying that a user accessing a system is a legitimate user.

[1279] "Real-time monitoring" refers to the immediate monitoring of systems and processes, and the instantaneous detection and response to changes in status or anomalies.

[1280] "Confidential information" refers to important internal company information that is not made public and which, if revealed to a third party, could cause damage to the company.

[1281] An "alert" is a means of notifying a user and issuing a warning when specific conditions occur.

[1282] An "operation log" refers to a record of a series of operations or events performed on a system, which is used later for analysis and auditing.

[1283] A "security incident" refers to an event in which a problem or risk related to information security occurs.

[1284] This invention is a system that authenticates users using a company's authentication system and retrieves a security policy from the server upon successful authentication. This system applies the retrieved security policy to the browser and monitors user input in real time to prevent the leakage of confidential information. Furthermore, a key feature is the ability to display an alert on a head-mounted display (HMD) when confidential information is detected, enabling immediate response.

[1285] First, the user logs into the company's authentication system using a browser or HMD. During this process, authentication information is sent to the server, which then authenticates the user. Upon successful authentication, the server retrieves the appropriate security policy and sends it to the user's device or HMD. The security policy includes a regular expression pattern for identifying sensitive information.

[1286] When a user enters text via a browser or HMD, the device monitors the input in real time. If the input matches a regular expression pattern defined in the security policy, it is determined to be sensitive information, and the input is immediately blocked. At this time, a warning message is displayed on the HMD, and the user is aware that the input of sensitive information has been blocked. In parallel, an operation log is sent to the server, which records it and manages security incidents.

[1287] The specific configuration of this system is as follows:

[1288] 1. Hardware to be used:

[1289] Head-mounted display (HMD)

[1290] User's device (PC or tablet)

[1291] 2. Software to use:

[1292] Authentication system: Performs user authentication.

[1293] Security policy management system: Distributes security policies that include regular expression patterns.

[1294] Real-time monitoring software: Monitors input content in real time and detects confidential information.

[1295] The following scenarios are possible as specific examples:

[1296] For example, consider a scenario where an employee using "Secure Browsing" within a company wears an HMD (Head-Mounted Display) while working. When the user attempts to type "technical details of a patent-pending application" into the email composition screen, a real-time alert is displayed on the HMD, blocking the input. The alert displays "Confidential information is contained. Input has been blocked," and this information is simultaneously sent to the server as a log.

[1297] Examples of prompts for a generative AI model include the following:

[1298] "A user is wearing an HMD (Head-Mounted Display) and browsing the web. I want to build a system that displays a real-time warning and blocks input if the user attempts to enter sensitive information. I will implement this using Python. The security policy will be provided by the server, and user input will be checked using regular expressions. Operation logs will be sent to the server, and alerts will be displayed on the HMD. Please tell me how to design the code."

[1299] In this way, the system of this invention can enhance corporate information security and prevent the leakage of confidential information.

[1300] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1301] Step 1:

[1302] The server receives the user's authentication information.

[1303] Input: User ID and password

[1304] Output: Authentication result (success / failure)

[1305] Specific operation: The user sends their entered ID and password to the server, which then uses its authentication system to perform authentication. If authentication is successful, the user proceeds to the next step.

[1306] Step 2:

[1307] The server retrieves the security policy and sends it to the terminal.

[1308] Input: Authentication result (success / failure)

[1309] Output: Security Policy

[1310] Specific operation: For a user who has successfully authenticated, the server searches for an appropriate security policy and sends it to the terminal. The security policy includes a regular expression pattern to identify sensitive information.

[1311] Step 3:

[1312] The device loads the security policy into memory.

[1313] Input: Security Policy

[1314] Output: Security policy loaded into memory

[1315] Specific operation: The terminal parses the security policy received from the server and loads it into memory. This prepares the terminal to monitor user input.

[1316] Step 4:

[1317] The user enters text via a browser or head-mounted display (HMD).

[1318] Input: User input text

[1319] Output: Monitored input text

[1320] Specific operation: The user enters text using a browser or HMD, and this input is monitored in real time.

[1321] Step 5:

[1322] The terminal monitors input text in real time and uses regular expression patterns to detect sensitive information.

[1323] Input: Input text, regular expression pattern

[1324] Output: Confidential information detection result (detected / not detected)

[1325] Specific operation: The terminal checks the input text against a regular expression pattern defined in the security policy to determine if it contains sensitive information.

[1326] Step 6:

[1327] If confidential information is detected, the terminal will block input and display an alert on the head-mounted display (HMD).

[1328] Input: Confidential information detection results

[1329] Output: Alert message, blocked input text

[1330] Specific operation: If the device detects sensitive information, it will block the input and display an alert message on the HMD stating, "Sensitive information is contained. Input has been blocked."

[1331] Step 7:

[1332] If the terminal detects the input of confidential information, it will send an operation log to the server.

[1333] Input: Confidential information detection results, input text

[1334] Output: Operation Log

[1335] Specific action: Send an operation log containing the detected confidential information and detection results to the server.

[1336] Step 8:

[1337] The server collects operation logs and manages them as security incidents.

[1338] Input: Operation Log

[1339] Output: Security incident log

[1340] Specific operation: The server saves the received operation logs to a database and manages them as security incidents.

[1341] Step 9:

[1342] The server notifies the administrator of the occurrence of a security incident.

[1343] Input: Security Incident Log

[1344] Output: Notification message

[1345] Specific operation: The server notifies the administrator of recorded security incidents to encourage a quick response.

[1346] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1347] This invention achieves more advanced information leakage prevention and security management by combining a browser system that applies a company's security policy and monitors user input in real time with an emotion engine that recognizes user emotions. This system mainly consists of a server, a terminal (the user's browser), and the user.

[1348] Server operation

[1349] The server manages security policies and distributes them to endpoints. It connects to the company's authentication system and verifies user credentials. Upon successful authentication, the server generates an appropriate security policy and sends it to the device. In the event of a security incident, the server collects the information and notifies the administrator. The server also manages data from the sentiment engine and uses it to improve or enforce security policies as needed.

[1350] Terminal (user's browser) operation

[1351] The device receives security policies sent from the server and monitors user input based on those policies. When a user launches a browser, authentication is performed first. If authentication is successful, the device downloads the latest security policy from the server and loads it into memory. Furthermore, the device has an emotion engine built in that recognizes emotions in real time from the user's facial expressions and vocalizations.

[1352] When a user enters text into a browser input field, that information is monitored in real time within the browser. The device inspects the input using regular expression patterns defined in the policy. If sensitive information is detected, the device blocks the input and displays an alert to the user. It also sends an operation log to the server.

[1353] Simultaneously, the emotion engine analyzes the user's facial expressions and vocalizations to recognize their emotional state in real time. This emotional data is then used to inform actions based on security policies. For example, if the system detects that a user is experiencing frustration, it might take steps such as providing appropriate psychological safety measures.

[1354] User actions

[1355] The user first launches a browser and logs into the company's authentication system. Upon successful authentication, the user begins using the browser as usual. If the user attempts to enter specific confidential information into the browser, the input is monitored in real time. If confidential information is detected, the input is blocked and an alert is displayed. Simultaneously, the user's emotions are monitored in real time, and appropriate measures are taken as needed.

[1356] Specific example

[1357] For example, suppose a user launches a browser, authenticates, and the company's security policy is applied. If the user opens an email sending screen and attempts to enter "technical details of a patent application," the device's browser monitors this input in real time. Because a regular expression pattern for "technical details of a patent application" is defined based on the policy, the browser immediately detects this text and blocks the input. The user receives an alert stating, "This contains confidential information. Input has been blocked," and the same information is sent to the server and saved as an operation log.

[1358] Furthermore, if the user experiences stress or displays anger during this process, the emotion engine recognizes this. The device sends this emotion data to a server, which then notifies the administrator based on that data. The administrator can then understand the circumstances under which the user experienced stress and provide appropriate support.

[1359] In this way, a sophisticated system that integrates security and user sentiment management is realized through each processing step.

[1360] The following describes the processing flow.

[1361] Step 1:

[1362] The device launches a browser. The user accesses the login screen and enters their authentication information into the company's authentication system.

[1363] Step 2:

[1364] The server receives the user's authentication information and uses it to verify the user's identity. If authentication is successful, the server generates an appropriate security policy and sends it to the terminal.

[1365] Step 3:

[1366] The device receives the security policy sent from the server and applies it within the browser. Specifically, it loads a regular expression pattern for detecting sensitive information into memory.

[1367] Step 4:

[1368] The device activates its emotion engine and prepares to analyze the user's facial expressions and vocalizations. The emotion engine uses the camera and microphone to collect the user's facial expressions and voice data.

[1369] Step 5:

[1370] The user begins typing text within the browser. As the user enters information into the input field, that information is monitored in real time within the browser.

[1371] Step 6:

[1372] The terminal analyzes the entered text based on security policies. It checks for content that matches regular expression patterns.

[1373] Step 7:

[1374] If the device detects sensitive information, it will block the input and display an alert message to the user. The message will read, "This contains sensitive information. Input has been blocked."

[1375] Step 8:

[1376] At the same time, the device uses an emotion engine to recognize the user's emotional state in real time. For example, if the user feels irritated or anxious, that emotion is detected in real time by the emotion engine.

[1377] Step 9:

[1378] The terminal logs the input of confidential information and sends that log to the server. The log includes the user's input, the time the alert occurred, and the user's emotional state.

[1379] Step 10:

[1380] The server saves the received logs to the database. The server sends an alert to notify the administrator that a security incident has occurred.

[1381] Step 11:

[1382] The server also notifies the administrator of the user's emotion data sent from the emotion engine. This allows the administrator to see what emotions the user displayed during a particular action.

[1383] Step 12:

[1384] The administrator receives an alert notification from the server, reviews the details of the security incident and the user's emotional state, and takes the necessary action.

[1385] In this way, by combining security policy-based functions with an emotion engine, a system is built that simultaneously prevents information leaks and manages user emotions.

[1386] (Example 2)

[1387] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1388] In recent years, information leaks and security incidents within companies have been increasing, and effective countermeasures are needed. Furthermore, preventing the leakage of confidential information due to user actions and responding quickly when users experience stress or anger are particularly important. However, conventional systems have not achieved comprehensive security management by combining user input monitoring with emotion recognition. Therefore, the present invention aims to provide advanced information leakage prevention and security management by monitoring both user input and emotions.

[1389] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for authenticating the user to the company's authentication system, means for obtaining a security policy from the server when authentication is successful, means for applying the obtained security policy to the terminal, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, means for sending an operation log to the server when input of confidential information is detected, means for the server to collect the operation log and manage security incidents, means for analyzing emotional data using an emotion engine that recognizes the user's emotions, and means for sending the emotional data to the server and notifying the administrator. This not only prevents the leakage of confidential information but also makes it possible to quickly address the stress and anger felt by the user.

[1390] A "corporate authentication system" refers to a system used within a company to verify the identity of users. Specific examples include LDAP and Active Directory.

[1391] A "server" is a computer that provides specific services or resources, and in this invention, it refers to a central device that manages security policies, collects operation logs, and analyzes sentiment data.

[1392] A "security policy" outlines the rules and guidelines for maintaining information security, including the rules and patterns used in monitoring input data and detecting confidential information.

[1393] A "device" refers to a device operated by a user, specifically a computer or smartphone, including a web browser.

[1394] An "emotion engine" refers to software or hardware used to analyze a user's emotional state, and includes technologies that determine emotions in real time based on the user's facial expressions and voice.

[1395] An "operation log" is data that records the history of operations performed by a user, and is used for tracking and analyzing security incidents.

[1396] A "security incident" refers to any event or action that could pose a threat to information security, including attempted data breaches and unauthorized access.

[1397] A "regular expression" is a set of string rules used to match text patterns, and is used to efficiently detect text that matches specific conditions.

[1398] "Emotional data" refers to digital information that indicates the user's emotional state as analyzed by the emotion engine, and includes data transmitted to the server in real time.

[1399] This invention integrates an emotion engine that recognizes user emotions into a browser system that applies a company's security policy to terminals and monitors user input in real time. This system mainly consists of a server, terminals (user's browser), and the user.

[1400] Server operation

[1401] The server plays a central role and performs the following tasks:

[1402] 1. Connect to the company's authentication system and verify the user's authentication information. Specifically, this involves using authentication systems such as LDAP or Active Directory.

[1403] 2. Upon successful authentication, the system generates and sends the company's security policy to the device. The security policy includes regular expression patterns for detecting sensitive information.

[1404] 3. In the event of a security incident, operation logs will be collected and managed. Furthermore, sentiment data from the sentiment engine will be managed, and a swift response will be initiated based on this data.

[1405] Terminal (user's browser) operation

[1406] A device is a device that users use on a daily basis and has the following functions:

[1407] 1. Receive the security policy sent from the server and load it into memory. The browser uses JavaScript to parse this data.

[1408] 2. Monitor user input in real time and detect sensitive information based on regular expression patterns defined in the policy. If sensitive information is detected, block the input and display an alert to the user.

[1409] 3. Send user input and operation logs to the server.

[1410] 4. The built-in emotion engine analyzes the user's facial expressions and voice to recognize their emotional state in real time. In particular, if the user shows signs of stress or anger, the data is sent to the server and notified to the administrator.

[1411] User operating procedures

[1412] Users access the system through the following steps:

[1413] 1. Launch your browser and log in to your company's authentication system. If authentication is successful, the security policy will be sent from the server to your device and applied.

[1414] 2. When a user enters information in their browser, that information is monitored in real time. For example, if a user attempts to enter "technical details of a patent application" on an email sending screen, the browser will detect this content, block the input, and display an alert.

[1415] 3. Simultaneously, the emotion engine analyzes the user's emotions, and if it detects stress or anger, that data is sent to the server and notified to the administrator.

[1416] Specific example

[1417] For example, suppose a user launches a browser, authenticates, and a security policy is applied. When the user attempts to enter "technical details of a patent application," the browser monitors this text in real time and blocks the input because it matches a regular expression pattern defined in the policy. An alert appears stating, "This contains sensitive information. Input blocked," and an operation log is sent to the server.

[1418] Furthermore, if a user experiences stress or displays anger during this process, the emotion engine recognizes this. Emotional data is sent to the server, which notifies the administrator, enabling them to provide appropriate support based on the situation. This results in an advanced data leakage prevention system that integrates security and user emotion management.

[1419] Example of a prompt

[1420] 1. "Describe a system in which, when a user attempts to enter patent information into a browser, the input is blocked and saved to the server as an operational log."

[1421] 2. "Please describe in detail a scenario in which emotions are recognized from the end user's facial expressions and reflected in security policies in real time."

[1422] Thus, the present invention provides a system that integrates corporate security and user sentiment management, enabling information leakage prevention and rapid response.

[1423] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1424] Step 1:

[1425] When a user launches a browser and attempts to log in to the authentication system, the server receives the authentication information (username and password) entered by the user. The server queries the company's authentication system, such as LDAP or Active Directory, with this authentication information. Upon receiving a response from the authentication system and confirming successful authentication, the server generates a session ID and sends that session ID to the terminal.

[1426] Input: Authentication information entered by the user

[1427] Data processing: Query authentication information with the authentication system.

[1428] Output: Authentication success / failure and session ID

[1429] Step 2:

[1430] The server retrieves the latest security policy information from the database for users who have successfully authenticated, in order to generate the company's security policy. It then encodes the retrieved policy information into JSON format and sends it to the terminal via an HTTP / S response.

[1431] Input: User information for successful authentication

[1432] Data processing: Retrieving security policies from the database and JSON encoding

[1433] Output: Security policy in JSON format

[1434] Step 3:

[1435] The terminal receives JSON data of the security policy from the server and parses it using JavaScript. It loads the parsed security policy into memory and begins monitoring user input under its influence.

[1436] Input: Security policy in JSON format received from the server

[1437] Data processing: Parsing and loading JSON data into memory.

[1438] Output: Loaded security policy

[1439] Step 4:

[1440] When a user enters text into a browser input field, JavaScript on the device monitors the input in real time. It checks whether the input matches a regular expression pattern defined in the security policy.

[1441] Input: User input

[1442] Data processing: Matching based on regular expression patterns

[1443] Output: Match result

[1444] Step 5:

[1445] If the input is detected as confidential information, the device will block the input and clear the input field using JavaScript. Then, it will display an alert to the user stating, "Confidential information is contained. Input has been blocked."

[1446] Input: Match result (confidential information)

[1447] Data processing: Clear input fields and display alerts.

[1448] Output: Alert displayed and input fields cleared

[1449] Step 6:

[1450] The terminal generates a user operation log and sends it to the server via an AJAX request. The operation log includes the user's input, the time, and the type of incident.

[1451] Input: User input and incident information

[1452] Data processing: Generating operation logs and sending AJAX requests.

[1453] Output: Operation log sent to the server

[1454] Step 7:

[1455] Simultaneously, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state in real time. In particular, if the user exhibits stress or anger, the emotion data is compiled in JSON format and sent from the device to the server.

[1456] Input: User's facial expressions and voice data

[1457] Data processing: Analysis of emotional states and conversion to JSON format.

[1458] Output: Sentiment data sent to the server

[1459] Step 8:

[1460] The server analyzes the received emotional data and, if it detects emotions such as stress or anger, notifies the administrator of this information. The administrator then takes appropriate action based on this notification.

[1461] Input: Sentiment data

[1462] Data processing: Analysis of emotional data and generation of notifications.

[1463] Output: Notification to administrator

[1464] In this way, by comprehensively monitoring and managing user input and emotional states, a system can be realized that prevents information leaks and appropriately responds to the user's psychological state.

[1465] (Application Example 2)

[1466] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1467] Existing security management systems have the capability to detect confidential information entered by users in real time, but they do not take into account the user's emotional state. Therefore, when a user experiences stress or anger, appropriate responses may not be taken quickly, resulting in increased security risks. To solve this problem, it is necessary to recognize the user's emotions in real time and take appropriate actions based on that recognition.

[1468] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for authenticating the user to the company's authentication system, means for obtaining a security policy when authentication is successful, means for applying the obtained security policy to the web browser, means for monitoring user input in real time and detecting confidential information, means for blocking user input and displaying an alert when confidential information is detected, means for sending an operation log to the server when input of confidential information is detected, means for the server to collect the operation log and manage security incidents, means for recognizing the user's emotional state in real time, means for sending the recognized emotional data to the server, and means for automatically performing appropriate actions based on the emotional data. This makes it possible to detect security risks early based on the user's emotional state and take appropriate countermeasures quickly.

[1469] A "corporate authentication system" is a mechanism that identifies and verifies a user's access to authorized systems to which they have access rights.

[1470] A "server" is a computer system that provides services to other computers and terminals on a network.

[1471] A "security policy" is a set of rules and procedures established for the purpose of protecting information.

[1472] A "web browser" is software used to view web pages on the internet.

[1473] "User input" refers to information that a user provides to a computer system through a keyboard or other input devices.

[1474] "Confidential information" refers to information that, if disclosed, could potentially cause harm to a company, such as unreleased technical information or trade secrets.

[1475] An "alert" is a warning message that alerts users or administrators when certain conditions are met.

[1476] An "operation log" is a record of actions performed by a user and is used for security monitoring and problem solving.

[1477] A "security incident" refers to security-related problems or accidents, such as unauthorized access to a company's information systems or data breaches.

[1478] "Emotional state" refers to the user's psychological state or mood, and includes things like anger and stress.

[1479] "Real-time" means that a process or calculation produces results immediately, with virtually no delay.

[1480] "Appropriate action" refers to the best course of action to take in a given situation.

[1481] This invention is a system that authenticates users based on a company's authentication system, retrieves a security policy from a server upon successful authentication, and applies the retrieved security policy to the web browser. Furthermore, it monitors user input and emotional state in real time, blocks input and displays an alert if confidential information is detected. If confidential information is detected, it sends an operation log to the server, which collects the operation log and manages security incidents. In addition, it recognizes the user's emotional state in real time, sends the recognized emotional data to the server, and automatically performs appropriate actions based on the emotional data.

[1482] Server operation

[1483] The server connects to the company's authentication system and verifies the user's credentials. If authentication is successful, it generates a security policy and sends it to the terminal. It also collects operation logs and sentiment data to manage security incidents. The specific cloud services used are Amazon Web Services (AWS) and Microsoft Azure.

[1484] Terminal (user's browser) operation

[1485] The device has a function to monitor user input in real time and detect sensitive information. It incorporates an emotion recognition engine that recognizes emotions based on the user's facial expressions and tone of voice. Specifically, it uses the Microsoft Azure Emotion API as the emotion recognition engine. If a user attempts to enter sensitive information, the device will detect it, block the input, and display an alert.

[1486] User actions

[1487] The user first launches a browser and logs into the company's authentication system. Upon successful authentication, the security policy is applied. If the user attempts to enter certain sensitive information into the browser, the device monitors it in real time, immediately blocking the input and displaying an alert if sensitive information is detected. Simultaneously, the user's emotions are monitored in real time, and if stress or anger is detected, emotion data is sent to the server.

[1488] Specific example

[1489] For example, if a user attempts to enter technical details of a patent application, the system immediately detects this and displays an alert stating, "This contains confidential information. Input has been blocked." If the user also exhibits feelings of stress or anger, the emotion recognition engine identifies these emotions and sends the emotion data to the server. Administrators can then receive this data and take appropriate action.

[1490] Example of a prompt

[1491] "A user is attempting to enter technical details of a patent application into an external email. Please implement a function that detects and blocks this input, and simultaneously recognizes and reports the user's emotional state (e.g., anger)."

[1492] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1493] Step 1:

[1494] The server authenticates the user to the company's authentication system. The user enters their authentication information (username and password) to log in to the authentication system and sends this information to the server. The server verifies the authentication information and determines whether authentication was successful. If authentication is successful, the user's authentication is complete and a token is issued to proceed to the next step. The input is the user's authentication information, and the output is whether authentication was successful or not (a token if successful).

[1495] Step 2:

[1496] The server generates a security policy upon successful authentication and sends it to the user's device using a token. The security policy is a set of rules containing a regular expression pattern for sensitive information. The input is the authentication token, and the output is the security policy.

[1497] Step 3:

[1498] The device applies the acquired security policy to the web browser. The applied security policy monitors user input in real time and determines whether sensitive information is detected. The input is the security policy, and the output is the browser environment to which the policy is applied.

[1499] Step 4:

[1500] The terminal monitors user input in real time and uses regular expression patterns to detect whether it contains sensitive information. When a user begins typing in the web browser, the text is inspected based on security policies. The input is the user's text, and the output is the result of the determination of whether it contains sensitive information.

[1501] Step 5:

[1502] The terminal blocks user input and displays an alert if sensitive information is detected. When a user attempts to enter text containing sensitive information, the system blocks the input and displays a warning message stating, "This contains sensitive information. Input blocked." The input is the result of the sensitive information detection, and the output is the blocked input and the displayed alert.

[1503] Step 6:

[1504] The terminal sends an operation log to the server when it detects the input of confidential information. The operation log contains details of the confidential information the user attempted to input and the actions that occurred. Input is the event of entering confidential information, and output is the operation log.

[1505] Step 7:

[1506] The server collects operation logs and manages security incidents. It analyzes the collected operation logs to enable administrators to take appropriate action. The input is operation logs, and the output is security incident management information.

[1507] Step 8:

[1508] The device utilizes an emotion recognition engine to recognize the user's emotional state in real time. Based on the user's facial expressions and voice data acquired from the camera and microphone, the emotion recognition algorithm classifies the emotion. The input is the user's facial expressions and voice data, and the output is the recognized emotional state.

[1509] Step 9:

[1510] The device sends recognized emotion data to the server. This emotion data includes the user's stress and anger levels. The input is the emotional state, and the output is the emotion data sent to the server.

[1511] Step 10:

[1512] The server automatically takes appropriate actions based on sentiment data. Based on this sentiment data, it can, for example, notify administrators or provide psychological support to users. The input is sentiment data, and the output is the appropriate action taken.

[1513] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1514] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1515] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1516] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1517] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1518] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1519] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1520] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1521] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1522] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1523] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1524] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1525] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1526] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1527] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1528] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1529] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1530] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1531] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1532] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1533] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1534] The following is further disclosed regarding the embodiments described above.

[1535] (Claim 1)

[1536] A means of having a company's authentication system authenticate users,

[1537] A means of obtaining the security policy from the server if authentication is successful,

[1538] A means of applying the acquired security policy to the browser,

[1539] A means of monitoring user input in real time and detecting confidential information,

[1540] A means to block user input and display an alert if confidential information is detected,

[1541] A means of sending an operation log to the server when the input of confidential information is detected,

[1542] A means of collecting operation logs from the server and managing security incidents,

[1543] A system that includes this.

[1544] (Claim 2)

[1545] The system according to claim 1, characterized in that the detection of confidential information uses policy-based regular expression patterns.

[1546] (Claim 3)

[1547] The system according to claim 1, further comprising means for notifying an administrator of the occurrence of a security incident.

[1548] "Example 1"

[1549] (Claim 1)

[1550] The user is used as an authentication method,

[1551] If authentication is successful, a means of obtaining the security policy from the server,

[1552] The means of applying the acquired security policy to the user interface,

[1553] User input is monitored in real time and confidential information is detected as a means of monitoring,

[1554] If confidential information is detected, the system will block user input and display an alert.

[1555] A means for sending an operation log to a server when the input of confidential information is detected,

[1556] The server collects operation logs and uses them as a means of managing security incidents.

[1557] A system that includes this.

[1558] (Claim 2)

[1559] The system according to claim 1, characterized in that the detection of confidential information uses policy-based regular expression patterns.

[1560] (Claim 3)

[1561] The system according to claim 1, further comprising means for notifying an administrator of the occurrence of a security incident.

[1562] "Application Example 1"

[1563] (Claim 1)

[1564] A means of having a company's authentication system authenticate users,

[1565] A means of obtaining the security policy from the server if authentication is successful,

[1566] A means of applying the acquired security policy to the browser,

[1567] A means of monitoring user input in real time and detecting confidential information,

[1568] A means of blocking user input and displaying an alert on a head-mounted display when confidential information is detected,

[1569] A means of sending an operation log to the server when the input of confidential information is detected,

[1570] A means of collecting operation logs from the server and managing security incidents,

[1571] A system that includes this.

[1572] (Claim 2)

[1573] The system according to claim 1, characterized in that the detection of confidential information uses policy-based regular expression patterns.

[1574] (Claim 3)

[1575] The system according to claim 1, further comprising means for notifying an administrator of the occurrence of a security incident.

[1576] "Example 2 of combining an emotion engine"

[1577] (Claim 1)

[1578] A means of having a company's authentication system authenticate users,

[1579] A means of obtaining the security policy from the server if authentication is successful,

[1580] A means of applying the acquired security policy to the device,

[1581] A means of monitoring user input in real time and detecting confidential information,

[1582] A means to block user input and display an alert if confidential information is detected,

[1583] A means of sending an operation log to the server when the input of confidential information is detected,

[1584] A means of collecting operation logs from the server and managing security incidents,

[1585] A means of analyzing emotional data using an emotion engine that recognizes user emotions,

[1586] A means of sending emotional data to a server and notifying the administrator,

[1587] A system that includes this.

[1588] (Claim 2)

[1589] The system according to claim 1, characterized in that the detection of confidential information uses policy-based regular expression patterns.

[1590] (Claim 3)

[1591] The system according to claim 1, further comprising means for notifying an administrator of the occurrence of a security incident based on emotion recognition.

[1592] "Application example 2 when combining with an emotional engine"

[1593] (Claim 1)

[1594] A means of having a company's authentication system authenticate users,

[1595] A means of obtaining the security policy from the server if authentication is successful,

[1596] A means of applying the acquired security policy to a web browser,

[1597] A means of monitoring user input in real time and detecting confidential information,

[1598] A means to block user input and display an alert if confidential information is detected,

[1599] A means of sending an operation log to the server when the input of confidential information is detected,

[1600] A means of collecting operation logs from the server and managing security incidents,

[1601] A means of recognizing the user's emotional state in real time,

[1602] A means of sending recognized emotion data to a server,

[1603] A means of automatically taking appropriate actions based on emotional data,

[1604] A system that includes this.

[1605] (Claim 2)

[1606] The system according to claim 1, characterized in that the detection of confidential information uses policy-based regular expression patterns.

[1607] (Claim 3)

[1608] The system according to claim 1, further comprising means for notifying an administrator of the occurrence of a security incident. [Explanation of Symbols]

[1609] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of having a company's authentication system authenticate users, A means of obtaining the security policy from the server if authentication is successful, A means of applying the acquired security policy to the browser, A means of monitoring user input in real time and detecting confidential information, A means to block user input and display an alert if confidential information is detected, A means of sending an operation log to the server when the input of confidential information is detected, A means of collecting operation logs from the server and managing security incidents, A system that includes this.

2. The system according to claim 1, characterized in that the detection of confidential information uses a policy-based regular expression pattern.

3. The system according to claim 1, further comprising means for notifying an administrator of the occurrence of a security incident.

Images