system

Abstract

Provide a system. [Solution] natural language processing means for analyzing text retrieved from a website; means for extracting specific keywords from the text analyzed by the natural language processing means; means for checking the extracted keywords against a pre-defined allow list; The system includes a means to automatically take consent or non-consent action based on the match result.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem with consent pop-ups about cookies and privacy policies that appear when using websites is that it is difficult for users to properly understand the content and decide whether to consent or not for each one. In addition, it is troublesome for users to have to consent to pop-ups with similar content every time, which causes a poor user experience. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems with a system including a natural language processing means for analyzing text obtained from a website, a means for extracting specific keywords from the text analyzed by the natural language processing means, a means for comparing the extracted keywords with a preset permission list, and a means for automatically performing an action of consent or disagreement based on the comparison result. Furthermore, by having a means for returning the comparison result to a terminal and automatically clicking an consent or disagreement button on the terminal, user operation can be eliminated. Furthermore, the natural language processing means performs text tokenization, part-of-speech tagging, and dependency analysis, enabling highly accurate keyword extraction and more appropriate consent decisions.

[0006] "Natural language processing means" refers to technical means for analyzing text data and processing it as structured information. Specifically, it performs processes such as text tokenization, part-of-speech tagging, and dependency analysis.

[0007] The "extraction means" is a means for automatically extracting specific keywords or phrases from the text analyzed by the natural language processing means.

[0008] An "allow list" is a list of items that a user pre-defines to allow or deny based on specific purposes or conditions.

[0009] A "matching method" is a method for comparing extracted keywords or phrases with an allowed list to see if they match.

[0010] A "consent action" is a means of automatically determining whether a user consents based on the results of a check against a permission list, and then performing an operation according to the result.

[0011] A "terminal" is a user's device used to access a website, and includes, for example, a PC, a smartphone, a tablet, etc.

[0012] "Text" refers to the explanatory text about cookies and privacy policy that is displayed when you visit the website. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] The present invention relates to a system that enables users to appropriately understand the content of consent popups regarding cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. The system includes a natural language processing unit, an extraction unit, a matching unit, and a set of units for executing consent actions.

[0035] Program processing

[0036] Program Overview

[0037] 1. User:

[0038] When you first visit the website, a pop-up will appear explaining the cookies and privacy policy.

[0039] 2. Terminal:

[0040] Gets the text of the displayed popup and sends it to the server.

[0041] 3. Server:

[0042] The received text is passed to a natural language processing (NLP) module for analysis.

[0043] The NLP module performs text tokenization, part-of-speech tagging, and dependency analysis to extract specific keywords and phrases.

[0044] 4. Server:

[0045] Matches extracted keywords and phrases against a pre-configured allow list.

[0046] The allow list may include items such as "No use for advertising purposes" and "Measurement for site analytics is allowed."

[0047] 5. Server:

[0048] A matching result is generated and a determination is made as to whether or not there is a match. If there is a match, a status of "Agree" is generated; if there is no match, a status of "Disagree" is generated.

[0049] 6. Terminal:

[0050] Automatically clicks the accept or decline button based on the status received from the server.

[0051] Specific examples

[0052] Example 1:

[0053] 1. User:

[0054] You visit an e-commerce website for the first time and a pop-up appears saying:

[0055] "We use cookies for advertising purposes. Do you agree?"

[0056] 2. Terminal:

[0057] Get the popup text "We use cookies for advertising purposes. Do you agree?" and send it to the server.

[0058] 3. Server:

[0059] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "advertising," "purpose," and "cookies" through tokenization, part-of-speech tagging, and dependency analysis.

[0060] 4. Server:

[0061] The extracted keywords are compared with the allow list. Since the allow list includes a setting that says "Use for advertising purposes is prohibited," the result of the comparison is "Disapproval."

[0062] 5. Terminal:

[0063] Based on the server's matching result of "disagree," the "disagree" button on the pop-up will be automatically clicked.

[0064] Example 2:

[0065] 1. User:

[0066] You visit a news site for the first time and a pop-up appears saying:

[0067] "This site uses cookies to analyze and improve it. Do you agree?"

[0068] 2. Terminal:

[0069] The text of the popup "This site uses cookies for analysis and improvement purposes. Do you agree?" is obtained and sent to the server.

[0070] 3. Server:

[0071] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "analysis," "improvement," and "cookie" through tokenization, part-of-speech tagging, and dependency analysis.

[0072] 4. Server:

[0073] The extracted keywords are compared with the allow list. Since the allow list is set to "Allow measurement for site analysis," the result is "Agree."

[0074] 5. Terminal:

[0075] Based on the matching result "Agree" from the server, the "Agree" button on the pop-up will be automatically clicked.

[0076] In this way, the system of the present invention streamlines the cookie consent process when users visit websites and automatically takes appropriate action based on pre-defined policies, thereby improving the user experience.

[0077] The processing flow will be explained below.

[0078] Step 1:

[0079] User:

[0080] Visit a website for the first time. Upon arrival, the site displays a pop-up about cookies and privacy policies.

[0081] Step 2:

[0082] Device:

[0083] Gets the text of the popup that was displayed.

[0084] Specifically, elements of the popup text are extracted using document.querySelector in JavaScript (registered trademark).

[0085] Step 3:

[0086] Device:

[0087] The retrieved text is sent to the server.

[0088] Specifically, Ajax is used to send text to the server via a POST request.

[0089] Step 4:

[0090] server:

[0091] The received text is passed to a natural language processing (NLP) module.

[0092] Specifically, it calls libraries such as Python's spaCy and NLTK.

[0093] Step 5:

[0094] server:

[0095] The NLP module analyzes the text and extracts specific keywords using techniques such as tokenization and part-of-speech tagging.

[0096] Specifically, the text is divided into tokens, each token is tagged with a part-of-speech tag, and dependency analysis is performed.

[0097] Step 6:

[0098] server:

[0099] The extracted keywords are compared against a pre-defined allow list.

[0100] Specifically, the extracted keywords are compared with each item in the permission list to see if they match.

[0101] Step 7:

[0102] server:

[0103] A matching result is generated, and if there is a match, a status of "Agree" is generated, and if there is no match, a status of "Disagree."

[0104] Specifically, the status is determined based on the conditions in the permission list, and the result is returned as a response.

[0105] Step 8:

[0106] Device:

[0107] Based on the status received from the server, an action of consent or non-consent is automatically taken.

[0108] Specifically, the corresponding button (agree or disagree) is clicked using the JavaScript click method.

[0109] Step 9:

[0110] User:

[0111] Confirm that the pop-up was interacted with and continue using the website.

[0112] As a result, proper consent management is carried out without the user having to perform the consent operation themselves.

[0113] Example 1

[0114] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0115] When users visit a website, they may find it difficult to properly understand the cookie and privacy policy consent pop-up and quickly decide whether to consent or not. This situation not only impairs the user experience, but also may result in users not making appropriate privacy choices. Furthermore, manually performing this series of operations is time-consuming and cumbersome for users. The present invention aims to solve this problem.

[0116] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0117] In this invention, the server includes a natural language processing unit for analyzing text acquired from a website, a unit for transmitting the acquired text to the server, and a unit for the server to generate a consent status based on the text analysis result, thereby streamlining the cookie consent process when a user visits a website and automatically taking appropriate action based on a pre-defined policy.

[0118] "Natural language processing means" is a technology that analyzes text obtained from websites and understands their grammatical structure and meaning.

[0119] "Keyword extraction means" is a technique for selecting important words and phrases from text analyzed by natural language processing means.

[0120] The "allowed list matching means" is a technology that compares extracted keywords and phrases with a pre-set allowed list to determine whether they match.

[0121] The "means for automatically executing consent action" is a technology that automatically performs consent or non-consent operations on behalf of the user based on the collation results.

[0122] The "text transmission means" is a technique for transmitting the acquired pop-up text to the server.

[0123] The "agreement status generation means" is a technique in which the server generates a status of "agree" or "disagree" based on the collation result.

[0124] The "automatic button clicking means" is a technology that automatically clicks a corresponding button on the terminal based on the generated consent status.

[0125] A "server" is a central computer system that processes and analyzes data.

[0126] A "terminal" is a device operated by a user, which receives instructions from a server and executes operations.

[0127] The present invention relates to a system that enables users to properly understand the content of consent popups regarding cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. This system is designed to operate in cooperation with three main entities: a server, a terminal, and a user.

[0128] System configuration

[0129] Hardware and software used

[0130] 1. Server

[0131] The server acts as the center of data analysis and recognition. Specific software used includes a natural language processing (NLP) module, such as Tensorflow (registered trademark).

[0132] 2. Terminal

[0133] A device is a user-operated device, such as a smartphone, tablet, or computer, that has software installed on it to capture the text of the displayed popup and send it to a server for analysis.

[0134] 3. Users

[0135] A user visits a website for the first time, and the system automatically performs certain actions via the device operated by the user.

[0136] System Operation

[0137] 1. User Behavior

[0138] The first time a user visits a particular website, they will see a consent popup regarding cookies and privacy policies.

[0139] 2. Terminal Processing

[0140] The device detects the popup text, retrieves its contents, and sends it to the server. Specifically, the popup text is retrieved from the DOM using JavaScript and asynchronously sent to the server using AJAX.

[0141] 3. Server Processing

[0142] The server passes the received text to a natural language processing (NLP) module, which tokenizes the text, tags it with parts of speech, and performs dependency analysis to extract specific keywords and phrases.

[0143] The extracted keywords are checked against a pre-defined allow list, which includes items such as "not for advertising purposes" and "allowed for measurement for site analytics."

[0144] Based on the matching result, the server generates a status of "agree" or "disagree" and returns it to the terminal.

[0145] 4. Automatic actions on the device

[0146] The device will automatically click the accept or decline button in the pop-up based on the status received from the server, saving the user the trouble of manually clicking the button.

[0147] Specific examples

[0148] Example 1:

[0149] 1. User Behavior

[0150] A user visits an e-commerce site for the first time and is greeted with a pop-up message saying, "We use cookies for advertising purposes. Do you agree?"

[0151] 2. Terminal Processing

[0152] Gets the popup text and sends it to the server.

[0153] 3. Server Processing

[0154] The server passes the text to the NLP module for analysis. The extracted keywords "advertising", "purpose", and "cookies" are checked against the allow list. Since the allow list includes "not allowed for advertising purposes", the server generates a "disagree" status.

[0155] 4. Automatic actions on the device

[0156] The device will automatically click the "Decline" button in the pop-up.

[0157] Example 2:

[0158] 1. User Behavior

[0159] A user visits a news site for the first time and is greeted with a pop-up message saying, "We use cookies to help us analyze and improve our site. Do you agree?"

[0160] 2. Terminal Processing

[0161] Gets the popup text and sends it to the server.

[0162] 3. Server Processing

[0163] The server passes the text to the NLP module for analysis. The extracted keywords "analysis," "improvement," and "cookies" are checked against the allow list. Since the allow list includes "Allow measurement for site analytics," the server generates an "Agree" status.

[0164] 4. Automatic actions on the device

[0165] The device will automatically click the "Agree" button in the pop-up.

[0166] Example prompts for generative AI models

[0167] "Please parse the content of the following popup and generate an accept or decline status based on your allow list: 'We use cookies for advertising purposes. Do you agree?'"

[0168] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0169] Step 1:

[0170] A user visits a website

[0171] Specific behavior: A user enters a URL into the address bar of a web browser and presses the Enter key.

[0172] Input: The URL entered by the user.

[0173] Output: The web page loads and displays the cookie and privacy policy consent popup.

[0174] Step 2:

[0175] The device detects the popup and retrieves the text

[0176] What happens: The browser's JavaScript monitors the DOM and detects the presence of a popup.

[0177] Parse the text node to get the popup text.

[0178] Input: The DOM structure of the displayed web page.

[0179] Output: The text of the popup (e.g. "We use cookies for advertising purposes. Do you agree?").

[0180] Step 3:

[0181] The device sends the text to the server

[0182] Specific behavior: Encodes the popup text and sends it to the server as an HTTP request.

[0183] AJAX is used for asynchronous communication.

[0184] Input: The text of the popup.

[0185] Output: The HTTP request sent to the server.

[0186] Step 4:

[0187] The server passes the text to a natural language processing (NLP) module

[0188] Specific operation: The server passes the received text as input to the NLP module.

[0189] The NLP module uses frameworks such as TensorFlow.

[0190] Input: The text of the HTTP request body.

[0191] Output: The text data passed to the NLP module.

[0192] Step 5:

[0193] NLP module analyzes the text

[0194] What it does: The NLP module tokenizes the text, tags it with parts of speech, and performs dependency analysis.

[0195] Further analysis is carried out to extract keywords and phrases.

[0196] Input: Text data passed from the server.

[0197] Output: Extracted keywords and phrases (e.g. "advertising", "purpose", "cookies").

[0198] Step 6:

[0199] The server checks the analysis results against the permission list

[0200] What it does: The server compares the extracted keywords and phrases with a pre-defined allow list.

[0201] An allow list might consist of, for example, "No use for advertising purposes" or "Measurement for site analytics is allowed."

[0202] Input: Extracted keywords or phrases, allow lists.

[0203] Output: Matching result (e.g., matched "Not for advertising purposes").

[0204] Step 7:

[0205] The server generates the consent status

[0206] Specific operation: The server generates a consent status (e.g., "consent" or "disagree") based on the matching result.

[0207] If the matching result matches the conditions of the allow list, an "Agree" status is generated; if not, a "Disagree" status is generated.

[0208] Input: Matching result.

[0209] Output: The consent status (e.g. "Disagree").

[0210] Step 8:

[0211] Automatic button click based on the status received by the device from the server

[0212] Specific operation: The terminal analyzes the consent status received from the server and automatically clicks the corresponding button ("Agree" or "Disagree").

[0213] Use JavaScript to fire a click event.

[0214] Input: consent status.

[0215] Output: Automatic click action of the corresponding button.

[0216] (Application example 1)

[0217] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0218] In autonomous vehicles, it is important for passengers to respond promptly and appropriately to cookie and privacy policy consent popups. However, manually performing these consent operations during autonomous driving not only degrades the user experience but can also be a distraction. To solve this problem, a system is needed that automates cookie and privacy policy consent actions, ensuring a safe and comfortable ride for passengers.

[0219] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0220] In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords from the text analyzed by the natural language processing means, and means for comparing the extracted keywords with a preset permission list. This enables the user's consent operation in an autonomous vehicle to be safely and quickly performed by a system including means for automatically processing a consent pop-up displayed on a vehicle terminal, means for analyzing text acquired from the vehicle terminal and comparing it with permission items, and means for automatically clicking a virtual button based on the comparison result.

[0221] "Natural language processing means" refers to means for carrying out a series of processes to analyze acquired text data and extract meanings and keywords.

[0222] The "means for extracting specific keywords" refers to a means for identifying and extracting important words and phrases from text analyzed by the natural language processing means.

[0223] An "allow list" is a pre-defined list of specific keywords or phrases that are allowed.

[0224] A "matching method" is a method for comparing extracted keywords or phrases with an allowed list to determine whether they match or mismatch.

[0225] "Automatic execution means" refers to a means for taking an action of consent or disagreement based on the matching results without user intervention.

[0226] "Means for automatically clicking virtual buttons" means means for a system to simulate the act of pressing an "agree" or "disagree" button through a program without the user having to manually operate the system.

[0227] A "vehicle terminal" is a terminal such as a computer or display device located inside an autonomous driving vehicle.

[0228] A "consent popup" is a short notification screen that appears when a website or application asks users to consent to the use of cookies or privacy policy.

[0229] The present invention relates to a system for quickly and appropriately handling cookie and privacy policy consent popups in an autonomous vehicle, including natural language processing, keyword extraction, permission list matching, and automatic action execution.

[0230] This system uses the following hardware and software:

[0231] Hardware:

[0232] Vehicle terminals in autonomous vehicles (e.g., HMDs with displays and camera functions)

[0233] Server (using cloud services, typically AWS (registered trademark) etc.)

[0234] software:

[0235] Natural language processing libraries (e.g., SpaCy, NLTK)

[0236] Server application (implemented in Python)

[0237] Cloud storage (e.g. Amazon S3)

[0238] The outline of the processing procedure is as follows.

[0239] 1. When a user gets into an autonomous vehicle, a cookie and privacy policy consent pop-up will appear on the vehicle's device.

[0240] 2. The vehicle device captures the text of the displayed popup using its camera function and sends that text to the server.

[0241] 3. The server analyzes the received text using a natural language processing module, tokenizing it, tagging it as a part of speech, and performing dependency analysis, thereby extracting important keywords such as "cookie" and "privacy."

[0242] 4. The extracted keywords are compared against an allow list, which may include items such as "Analysis of driving data is allowed" or "Use for advertising purposes is prohibited."

[0243] 5. Based on the matching result, the server generates a status of "agree" or "disagree" and sends it to the vehicle terminal.

[0244] 6. The device will automatically click the virtual button "Agree" or "Disagree" based on the received status.

[0245] Specific examples are shown below.

[0246] Example 1: When a user gets into an autonomous vehicle and a pop-up message appears on the HMD saying, "This vehicle uses cookies to analyze communication data. Do you agree?", the following prompt sentence is input into the AI ​​model.

[0247] "Analyze the following text and extract keywords: 'This vehicle uses cookies to analyze communication data. Do you agree?'"

[0248] The server extracts the keywords "communication data analysis," "cookies," and "consent" from the analysis results and compares them with the permission list. In this case, the setting is "permit driving data analysis," so the comparison result is "consent." The HMD on the vehicle terminal automatically clicks the "consent" button.

[0249] In this way, the system of the present invention enables the user's consent operation to be performed safely and quickly in an autonomous vehicle.

[0250] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0251] Step 1:

[0252] When a user gets into an autonomous vehicle, a consent pop-up regarding cookies and privacy policies is displayed on the vehicle's head-mounted display (HMD). This pop-up contains specific text.

[0253] Input: Cookie and privacy policy consent popup text displayed by the user

[0254] Output: Image data for capturing popup text

[0255] Specific operation: The HMD's camera function is activated and the text of the displayed popup is captured as image data.

[0256] Step 2:

[0257] The vehicle terminal processes the captured image data and extracts text, which is then sent to the server.

[0258] Input: Image data of the captured popup text

[0259] Output: Text data (e.g. "This vehicle uses cookies to analyze communication data. Do you agree?")

[0260] What it does: Image processing software extracts text from the captured image and sends it to a server, such as Tesseract OCR.

[0261] Step 3:

[0262] The server passes the received text data to a natural language processing (NLP) module for analysis, which includes tokenization, part-of-speech tagging, and dependency analysis. Keywords such as "communication data analysis," "cookies," and "consent" are extracted.

[0263] Input: Received text data

[0264] Output: Extracted keywords (e.g., "communication data analysis," "cookies," "consent")

[0265] Specific operation: The NLP module analyzes the text. The NLP library used is, for example, SpaCy.

[0266] Step 4:

[0267] The server compares the extracted keywords with a pre-defined permission list, which may include items such as "driving data analysis is allowed" or "usage for advertising purposes is prohibited," and generates a consent or denial status based on the match.

[0268] Input: Extracted keywords (e.g., "communication data analysis," "cookies," "consent") and allow list

[0269] Output: Matching result (e.g., "Agree" or "Disagree")

[0270] What happens: The server checks the keywords against the allow list, which is stored in a database, and generates a status.

[0271] Step 5:

[0272] The server transmits the generated status of consent or disagreement to the vehicle terminal.

[0273] Input: The status of the match (e.g., "Agree" or "Disagree")

[0274] Output: Status sent to vehicle terminal

[0275] Specific operation: The server sends the status to the vehicle terminal.

[0276] Step 6:

[0277] The vehicle terminal automatically clicks the virtual button based on the received status, without any manual operation by the user.

[0278] Input: Status sent to the vehicle terminal (e.g. "Agree" or "Disagree")

[0279] Output: Virtual button clicked

[0280] Specific operation: The vehicle terminal software simulates the clicking of a virtual button based on the received status. The virtual button interface is integrated into the vehicle terminal UI.

[0281] Through the above processing steps, the system of the present invention enables users to quickly and appropriately consent to cookies and privacy policies in autonomous vehicles.

[0282] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0283] The present invention relates to a system that enables users to properly understand the content of consent popups for cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. The system includes a natural language processing means, an extraction means, a matching means, a means for executing consent actions, and an emotion engine that recognizes the user's emotions.

[0284] Program processing

[0285] Program Overview

[0286] 1. User:

[0287] When you first visit the website, a pop-up will appear explaining the cookies and privacy policy.

[0288] 2. Terminal:

[0289] Gets the text of the displayed popup and sends it to the server.

[0290] 3. Server:

[0291] The received text is passed to a natural language processing (NLP) module for analysis.

[0292] The NLP module performs text tokenization, part-of-speech tagging, and dependency analysis to extract specific keywords and phrases.

[0293] 4. Server:

[0294] Matches extracted keywords and phrases against a pre-configured allow list.

[0295] The allow list may include items such as "No use for advertising purposes" and "Measurement for site analytics is allowed."

[0296] 5. Server:

[0297] A matching result is generated and a determination is made as to whether or not there is a match. If there is a match, a status of "Agree" is generated; if there is no match, a status of "Disagree" is generated.

[0298] 6. Terminal:

[0299] The matching results are received from the server and the emotion engine is activated. The emotion engine analyzes the user's text input, voice input, and facial expressions to recognize the user's emotions.

[0300] 7. Server:

[0301] Tailor your consent or disagreement actions based on the perceived emotion, for example automatically clicking the disagree button or showing a detailed explanation if the user expresses anxiety or doubt.

[0302] 8. Terminal:

[0303] Based on the judgment results of the emotion engine, the agree or disagree button is automatically clicked.

[0304] Specific examples

[0305] Example 1:

[0306] 1. User:

[0307] You visit an e-commerce website for the first time and a pop-up appears saying:

[0308] "We use cookies for advertising purposes. Do you agree?"

[0309] 2. Terminal:

[0310] Get the popup text "We use cookies for advertising purposes. Do you agree?" and send it to the server.

[0311] 3. Server:

[0312] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "advertising," "purpose," and "cookies" through tokenization, part-of-speech tagging, and dependency analysis.

[0313] 4. Server:

[0314] The extracted keywords are compared with the allow list. Since the allow list includes a setting that says "Use for advertising purposes is prohibited," the result of the comparison is "Disapproval."

[0315] 5. Terminal:

[0316] The matching result "disagree" is received from the server and the emotion engine is activated.

[0317] 6. Terminal:

[0318] The emotion engine analyzes the user's text and voice input and recognizes that the user has doubts, such as "Why rejection?"

[0319] 7. Server:

[0320] Automatically clicking the "disagree" button in a pop-up based on perceived sentiment,

[0321] Example 2:

[0322] 1. User:

[0323] You visit a news site for the first time and a pop-up appears saying:

[0324] "This site uses cookies to analyze and improve it. Do you agree?"

[0325] 2. Terminal:

[0326] The text of the popup "This site uses cookies for analysis and improvement purposes. Do you agree?" is obtained and sent to the server.

[0327] 3. Server:

[0328] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "analysis," "improvement," and "cookie" through tokenization, part-of-speech tagging, and dependency analysis.

[0329] 4. Server:

[0330] The extracted keywords are compared with the allow list. Since the allow list is set to "Allow measurement for site analysis," the result is "Agree."

[0331] 5. Terminal:

[0332] The matching result "Agreement" is received from the server and the emotion engine is activated.

[0333] 6. Terminal:

[0334] The emotion engine analyzes the user's facial expressions and recognizes, for example, that the user is feeling at ease.

[0335] 7. Server:

[0336] Automatically click the "Agree" button on a pop-up based on perceived sentiment.

[0337] In this way, the system of the present invention streamlines the cookie consent process when users visit websites, and by taking into account user sentiment in addition to pre-defined policies, it achieves more appropriate and intuitive consent management.

[0338] The processing flow will be explained below.

[0339] Step 1:

[0340] User:

[0341] Visit a website for the first time. Upon arrival, the site displays a pop-up about cookies and privacy policies.

[0342] Step 2:

[0343] Device:

[0344] Gets the text of the popup that was displayed.

[0345] Specifically, JavaScript's document.querySelector is used to extract the popup text elements.

[0346] Step 3:

[0347] Device:

[0348] The retrieved text is sent to the server.

[0349] Specifically, Ajax is used to send text to the server via a POST request.

[0350] Step 4:

[0351] server:

[0352] The received text is passed to a natural language processing (NLP) module.

[0353] Specifically, it calls libraries such as Python's spaCy and NLTK.

[0354] Step 5:

[0355] server:

[0356] The NLP module analyzes the text and extracts specific keywords using techniques such as tokenization, part-of-speech tagging, and dependency analysis.

[0357] Specifically, the text is divided into tokens, each token is tagged with a part-of-speech tag, and dependency analysis is performed.

[0358] Step 6:

[0359] server:

[0360] The extracted keywords are compared against a pre-defined allow list.

[0361] Specifically, the extracted keywords are compared with each item in the permission list to see if they match.

[0362] Step 7:

[0363] server:

[0364] A matching result is generated, and if there is a match, a status of "Agree" is generated, and if there is no match, a status of "Disagree."

[0365] Specifically, the status is determined based on the conditions in the permission list, and the result is returned as a response.

[0366] Step 8:

[0367] Device:

[0368] Receives the status received from the server. When the status is received, it starts the emotion engine.

[0369] Step 9:

[0370] Device:

[0371] The emotion engine analyzes the user's text input, voice input, or facial expressions to recognize the user's emotions.

[0372] Specifically, when analyzing voice input, it uses voice recognition technology, and when analyzing text input, it analyzes the input string. For facial expression analysis, it analyzes the user's facial expressions from camera footage.

[0373] Step 10:

[0374] server:

[0375] Tailor actions to agree or disagree based on perceived sentiment.

[0376] For example, if a user expresses concerns or doubts, we can provide a detailed explanation or automatically click a no consent button.

[0377] Step 11:

[0378] Device:

[0379] Based on the judgment results of the emotion engine, an action of agreement or disagreement is taken.

[0380] Specifically, the corresponding button (agree or disagree) is clicked using the JavaScript click method.

[0381] Step 12:

[0382] User:

[0383] Confirm that the pop-up was interacted with and continue using the website.

[0384] As a result, proper consent management is carried out without the user having to perform the consent operation themselves.

[0385] Example 2

[0386] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0387] In recent years, users are required to understand the content of cookie and privacy policy consent popups that appear when visiting websites and make a quick decision. However, current systems often cause confusion and bewilderment when users choose to consent or not. In addition, consent management does not take user emotions into consideration, resulting in a poor user experience.

[0388] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords and phrases from the text analyzed by the natural language processing means, means for comparing the extracted keywords and phrases with a preset permission list, means for automatically performing an action of consent or disagreement based on the comparison result, and means for recognizing the user's emotions and adjusting the action of consent or disagreement based on the recognition result. This allows the user to properly understand the contents of cookies and privacy policies and make quick and appropriate decisions, improving the user experience.

[0389] "Natural language processing means" is a technology that analyzes text obtained from a website and extracts specific keywords and phrases by tokenizing words and phrases, tagging them by part of speech, and performing dependency analysis.

[0390] "Keyword and phrase extraction means" refers to a technique for identifying and collecting specific words and phrases from text analyzed by natural language processing means.

[0391] An "allow list" is a pre-defined list of specific keywords or phrases that define the conditions under which a decision to accept or reject them will be made.

[0392] "Matching methods" are techniques that compare extracted keywords and phrases with items on an allow list to determine whether they match.

[0393] A "match result" is a determination of whether keywords and phrases match the allow list, and is information used to generate a consent or disagreement status.

[0394] "Means for automatically performing an action of consent or denial" refers to technology that recreates the state in which a user clicks the consent or denial button on a website based on the results of the comparison.

[0395] "Means for recognizing user emotions" refers to technology that analyzes the user's facial expressions, voice input, text input, etc. to determine the user's emotional state.

[0396] The "means for adjusting actions based on emotions" is a technology that provides additional information or changes actions when automatically taking an action of agreement or disagreement based on the results of user emotion recognition.

[0397] MODE FOR CARRYING OUT THE INVENTION

[0398] The present invention relates to a system that helps users quickly and appropriately decide whether to agree or disagree to pop-ups about cookies and privacy policies that appear when visiting a website. The system includes a natural language processing means, an extraction means, a matching means, a means for executing an agreement action, and an emotion engine that recognizes the user's emotions.

[0399] Specifically, it is configured as follows:

[0400] 1. Website visits and pop-ups

[0401] Users: When they first visit a website, they will see a pop-up explaining cookies and privacy policies, such as "We use cookies for advertising purposes. Do you agree?"

[0402] 2. Getting and sending popup text

[0403] Terminal: The text of the displayed popup is obtained and sent to the server. The software used generally uses the browser's API.

[0404] 3. Text Analysis

[0405] Server: Passes the received text to a natural language processing (NLP) module for analysis, using a natural language processing library such as SpaCy or NLTK.

[0406] NLP module: Tokenizes text, tags it with parts of speech, and performs dependency analysis, for example, extracting important keywords and phrases such as "advertising," "purpose," and "cookies."

[0407] 4. Keyword Matching

[0408] Server: The extracted keywords and phrases are compared against a pre-defined allow list, which may include items such as "no advertising use" or "allowed for site analytics measurement."

[0409] 5. Judgment of matching results

[0410] Server: Based on the match, generate a "Agree" or "Disagree" status. For example, if the match matches "Not for advertising purposes", it will be determined as a disagreement.

[0411] 6. Activating the Emotional Engine

[0412] Device: Receives the matching results from the server and activates the emotion engine. The emotion engine uses Google® Cloud Vision or Microsoft® Azure® Emotion API to analyze the user's facial expressions, voice input, and text input.

[0413] 7. Adjusting actions based on emotions

[0414] Server: Automatically adjust the action of agreeing or disagreeing based on the user's emotion recognition results. For example, if the user expresses anxiety or doubt, display a detailed explanation or automatically click the "disagree" button.

[0415] 8. Automated consent / non-consent decisions

[0416] Device: Automatically clicks the "Agree" or "Disagree" button on the pop-up based on the emotion engine's judgment.

[0417] Specific examples

[0418] Below is a specific scenario using this system.

[0419] Example 1: E-commerce site

[0420] User: Visits an e-commerce website for the first time and sees a popup saying "We use cookies for advertising purposes. Do you agree?"

[0421] Terminal: Get this popup text and send it to the server.

[0422] Server: Passes the text to a natural language processing (NLP) module, which analyzes it for keywords such as "advertising," "purpose," and "cookies."

[0423] Server: Compares the analysis results with the permission list and issues a "deny" decision, stating that "use for advertising purposes is not permitted."

[0424] Device: Receives the matching results, activates the emotion engine, and analyzes the user's facial expressions and voice.

[0425] Server: As a result of the sentiment engine, it recognizes that the user is expressing doubt and automatically clicks the "disagree" button.

[0426] Example 2: News site

[0427] User: Visits a news site for the first time and sees a pop-up that says "We use cookies to help us analyze and improve our site. Do you agree?"

[0428] Terminal: Get this popup text and send it to the server.

[0429] Server: Passes the text to a natural language processing (NLP) module, which analyzes it for keywords like "analysis," "improvement," and "cookie."

[0430] Server: Compares the analysis results with the allow list and determines "Agree" by stating "Measurement for site analysis is permitted."

[0431] Terminal: Receives the matching results, activates the emotion engine, and analyzes the user's facial expressions.

[0432] Server: As a result of the emotion engine, it recognizes that the user feels safe and automatically clicks the "Agree" button.

[0433] In this way, the system of the present invention helps users properly understand the content of cookies and privacy policies and make quick decisions, which greatly improves the user experience.

[0434] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0435] Step 1:

[0436] User: When a user first visits a website, a popup explaining cookies and privacy policy is displayed. Input is a user visiting a website. Output is a popup being generated.

[0437] Step 2:

[0438] Terminal: Get the text of the displayed popup and send it to the server. This process includes the specific operation of extracting the popup text from the DOM (Document Object Model). The input is the displayed popup text, and the output is the text data to be sent to the server.

[0439] Step 3:

[0440] Server: Passes the received text to a natural language processing (NLP) module for analysis. Specifically, it uses natural language processing libraries such as SpaCy and NLTK. In processing by the NLP module, text data is provided as input, and the results of tokenization, part-of-speech tagging, and dependency analysis are generated as output.

[0441] Step 4:

[0442] Server: Extracts specific keywords and phrases from the text analyzed by the NLP module. This includes programs that identify and collect important words and phrases. The input is the analyzed text data, and the output is the extracted keywords and phrases.

[0443] Step 5:

[0444] Server: Compares extracted keywords and phrases against a pre-configured allow list, which may contain items such as "not for advertising purposes" or "allowed for site analytics measurement." The input is the extracted keywords and phrases and the allow list, and the output is the match result.

[0445] Step 6:

[0446] Server: Generates a status of "agree" or "disagree" based on the matching result. The input is the matching result, and the output is the status of agree or disagree.

[0447] Step 7:

[0448] Device: Receives the status obtained from the matching result and starts the emotion engine. The emotion engine uses Google Cloud Vision or Microsoft Azure's Emotion API to analyze the user's facial expressions, voice input, and text input. The input is the status of agreement or disagreement, and the output is the emotion recognition result.

[0449] Step 8:

[0450] Server: Automatically adjusts the action of agreeing or disagreeing based on the user's emotion recognition results. For example, if the user expresses anxiety or doubt, it displays a detailed explanation or automatically clicks the "disagree" button. The input is the emotion recognition result, and the output is the adjusted action.

[0451] Step 9:

[0452] Terminal: Automatically clicks the "Agree" or "Disagree" button in a pop-up based on the emotion engine's judgment. The input is the coordinated action, and the output is the actual button click.

[0453] (Application example 2)

[0454] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0455] Conventional consent popups for cookies and privacy policies on websites make it difficult for users to understand their content and quickly decide whether to consent or not. In particular, little consideration is given to the user's psychological burden and level of understanding, resulting in a non-intuitive consent process. Furthermore, users are often asked to consent while feeling anxious or suspicious, which reduces the quality of the user experience.

[0456] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords from the text analyzed by the natural language processing means, and means for comparing the extracted keywords with a preset allow list. This makes it possible to streamline the cookie consent process when a user visits a website and enables more appropriate and intuitive consent management by taking into account the user's feelings in addition to preset policies.

[0457] "Natural language processing means" refers to technical means that analyzes text data obtained from websites and performs processes such as tokenization, part-of-speech tagging, and dependency analysis.

[0458] The "means for extracting specific keywords" refers to a technical means for extracting predefined important words or phrases from text analyzed by natural language processing means.

[0459] "Means for checking against an allow list" refers to a technical means for comparing extracted keywords with a pre-established allow list to determine whether they match.

[0460] "Means for automatically performing consent or denial actions" refers to technical means for automatically performing consent or denial actions based on the results of a match against an allow list.

[0461] "Emotion analysis means" refers to a technical means for reading emotions from the user's input text, voice, facial expressions, etc., and recognizing their emotional state.

[0462] The "means for adjusting consent or disagreement actions" refers to technical means for optimizing consent or disagreement actions based on the recognized user sentiment.

[0463] In this invention, when a user visits a website for the first time, a pop-up message explaining cookies and privacy policies is displayed. The terminal obtains the text of the pop-up message and sends it to the server.

[0464] The server then analyzes the retrieved text using natural language processing tools (e.g., HuggingFace's Transformers library), which involve tokenizing the text, tagging it as a part of speech, and performing dependency analysis to extract specific keywords and phrases.

[0465] The extracted keywords are then checked against a pre-defined allow list, which may include items such as "no advertising use" or "allowed for site analytics measurement." Based on the result of the check, a "yes" or "no" status is generated for a particular policy.

[0466] The device then receives the matching results from the server and activates an emotion analyzer (e.g., an emotion analysis model in the HuggingFace Transformers library) that analyzes the user's text input, voice input, or facial expressions to recognize the user's emotions.

[0467] Furthermore, the server adjusts the action of agreeing or disagreeing based on the perceived emotion: for example, if the user expresses anxiety or doubt, it automatically clicks the disagree button or displays a detailed explanation, etc. Similarly, if the user feels reassured, it automatically clicks the agree button.

[0468] Specific examples

[0469] Example 1:

[0470] 1. A user visits a shopping site for the first time and is greeted with a pop-up that reads: "We use cookies for advertising purposes. Do you agree?"

[0471] 2. The device retrieves the popup text and sends it to the server.

[0472] 3. The server performs natural language processing to extract keywords from the text, such as "advertising," "purpose," and "cookie."

[0473] 4. The server checks the extracted keywords against the allow list. Since the allow list includes a setting that says "not for advertising purposes," the result is "disapproval."

[0474] 5. The device receives the matching results from the server and activates the emotion analysis function. It recognizes that the user has doubts, such as "Why was it rejected?"

[0475] 6. The "Disagree" button in the pop-up is automatically clicked based on the perceived sentiment.

[0476] Prompt Sentence Examples

[0477] "Please develop a system that uses natural language processing to analyze how users feel about the privacy policy pop-up that appears for the first time on a website they visit, and helps them decide whether to agree or disagree based on that information."

[0478] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0479] Step 1:

[0480] When a user visits a website for the first time, the device gets a popup with a description of cookies and privacy policy. The input is the display data of the web page, and the output is the text data of the popup. This text data is sent to the server for further processing.

[0481] Step 2:

[0482] The server analyzes the text sent from the terminal using a natural language processing means (NLP module). The input is the text data of the popup, and the output is the analysis data generated by tokenization, part-of-speech tagging, and dependency analysis. Specifically, the natural language processing means tokenizes the text, tags each token with a part-of-speech tag, and analyzes dependencies.

[0483] Step 3:

[0484] The server extracts specific keywords and phrases from the analysis data. The input is the analysis data from the natural language processing tool, and the output is a list of extracted keywords. This clarifies the main intent and purpose of the popup.

[0485] Step 4:

[0486] The server compares the extracted keywords with a pre-defined allow list. The input is the list of extracted keywords and the allow list, and the output is the match result (agreement or disagreement status). Specifically, it compares whether the keywords match the items in the allow list, and generates a consent status based on the result.

[0487] Step 5:

[0488] The terminal receives the matching results from the server and activates the emotion analysis means. The inputs are the matching results and the user's text input, voice input, or facial expression data, and the output is the recognized emotion. The emotion analysis means analyzes the user's emotion based on the input data and recognizes their state.

[0489] Step 6:

[0490] The server adjusts the action of agreeing or disagreeing based on the recognized emotion. The input is the recognized emotion and the matching result, and the output is the final action (agreeing or disagreeing). For example, if the user expresses anxiety or doubt, a detailed explanation is displayed, and if there is no anxiety, the agree button is automatically clicked.

[0491] Step 7:

[0492] The terminal automatically clicks the accept or decline button based on the final action received from the server. The input is the final action status from the server, and the output is the operation (clicking accept or decline) based on that status.

[0493] This process allows users to quickly and appropriately respond to pop-ups about cookies and privacy policies.

[0494] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0495] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0496] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0497] [Second embodiment]

[0498] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0499] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0500] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0501] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0502] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0503] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0504] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0505] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0506] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0507] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0508] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0509] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0510] The present invention relates to a system that enables users to appropriately understand the content of consent popups regarding cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. The system includes a natural language processing unit, an extraction unit, a matching unit, and a set of units for executing consent actions.

[0511] Program processing

[0512] Program Overview

[0513] 1. User:

[0514] When you first visit the website, a pop-up will appear explaining the cookies and privacy policy.

[0515] 2. Terminal:

[0516] Gets the text of the displayed popup and sends it to the server.

[0517] 3. Server:

[0518] The received text is passed to a natural language processing (NLP) module for analysis.

[0519] The NLP module performs text tokenization, part-of-speech tagging, and dependency analysis to extract specific keywords and phrases.

[0520] 4. Server:

[0521] Matches extracted keywords and phrases against a pre-configured allow list.

[0522] The allow list may include items such as "No use for advertising purposes" and "Measurement for site analytics is allowed."

[0523] 5. Server:

[0524] A matching result is generated and a determination is made as to whether or not there is a match. If there is a match, a status of "Agree" is generated; if there is no match, a status of "Disagree" is generated.

[0525] 6. Terminal:

[0526] Automatically clicks the accept or decline button based on the status received from the server.

[0527] Specific examples

[0528] Example 1:

[0529] 1. User:

[0530] You visit an e-commerce website for the first time and a pop-up appears saying:

[0531] "We use cookies for advertising purposes. Do you agree?"

[0532] 2. Terminal:

[0533] Get the popup text "We use cookies for advertising purposes. Do you agree?" and send it to the server.

[0534] 3. Server:

[0535] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "advertising," "purpose," and "cookies" through tokenization, part-of-speech tagging, and dependency analysis.

[0536] 4. Server:

[0537] The extracted keywords are compared with the allow list. Since the allow list includes a setting that says "Use for advertising purposes is prohibited," the result of the comparison is "Disapproval."

[0538] 5. Terminal:

[0539] Based on the server's matching result of "disagree," the "disagree" button on the pop-up will be automatically clicked.

[0540] Example 2:

[0541] 1. User:

[0542] You visit a news site for the first time and a pop-up appears saying:

[0543] "This site uses cookies to analyze and improve it. Do you agree?"

[0544] 2. Terminal:

[0545] The text of the popup "This site uses cookies for analysis and improvement purposes. Do you agree?" is obtained and sent to the server.

[0546] 3. Server:

[0547] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "analysis," "improvement," and "cookie" through tokenization, part-of-speech tagging, and dependency analysis.

[0548] 4. Server:

[0549] The extracted keywords are compared with the allow list. Since the allow list is set to "Allow measurement for site analysis," the result is "Agree."

[0550] 5. Terminal:

[0551] Based on the matching result "Agree" from the server, the "Agree" button on the pop-up will be automatically clicked.

[0552] In this way, the system of the present invention streamlines the cookie consent process when users visit websites and automatically takes appropriate action based on pre-defined policies, thereby improving the user experience.

[0553] The processing flow will be explained below.

[0554] Step 1:

[0555] User:

[0556] Visit a website for the first time. Upon arrival, the site displays a pop-up about cookies and privacy policies.

[0557] Step 2:

[0558] Device:

[0559] Gets the text of the popup that was displayed.

[0560] Specifically, JavaScript's document.querySelector is used to extract the popup text elements.

[0561] Step 3:

[0562] Device:

[0563] The retrieved text is sent to the server.

[0564] Specifically, Ajax is used to send text to the server via a POST request.

[0565] Step 4:

[0566] server:

[0567] The received text is passed to a natural language processing (NLP) module.

[0568] Specifically, it calls libraries such as Python's spaCy and NLTK.

[0569] Step 5:

[0570] server:

[0571] The NLP module analyzes the text and extracts specific keywords using techniques such as tokenization and part-of-speech tagging.

[0572] Specifically, the text is divided into tokens, each token is tagged with a part-of-speech tag, and dependency analysis is performed.

[0573] Step 6:

[0574] server:

[0575] The extracted keywords are compared against a pre-defined allow list.

[0576] Specifically, the extracted keywords are compared with each item in the permission list to see if they match.

[0577] Step 7:

[0578] server:

[0579] A matching result is generated, and if there is a match, a status of "Agree" is generated, and if there is no match, a status of "Disagree."

[0580] Specifically, the status is determined based on the conditions in the permission list, and the result is returned as a response.

[0581] Step 8:

[0582] Device:

[0583] Based on the status received from the server, an action of consent or non-consent is automatically taken.

[0584] Specifically, the corresponding button (agree or disagree) is clicked using the JavaScript click method.

[0585] Step 9:

[0586] User:

[0587] Confirm that the pop-up was interacted with and continue using the website.

[0588] As a result, proper consent management is carried out without the user having to perform the consent operation themselves.

[0589] Example 1

[0590] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0591] When users visit a website, they may find it difficult to properly understand the cookie and privacy policy consent pop-up and quickly decide whether to consent or not. This situation not only impairs the user experience, but also may result in users not making appropriate privacy choices. Furthermore, manually performing this series of operations is time-consuming and cumbersome for users. The present invention aims to solve this problem.

[0592] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0593] In this invention, the server includes a natural language processing unit for analyzing text acquired from a website, a unit for transmitting the acquired text to the server, and a unit for the server to generate a consent status based on the text analysis result, thereby streamlining the cookie consent process when a user visits a website and automatically taking appropriate action based on a pre-defined policy.

[0594] "Natural language processing means" is a technology that analyzes text obtained from websites and understands their grammatical structure and meaning.

[0595] "Keyword extraction means" is a technique for selecting important words and phrases from text analyzed by natural language processing means.

[0596] The "allowed list matching means" is a technology that compares extracted keywords and phrases with a pre-set allowed list to determine whether they match.

[0597] The "means for automatically executing consent action" is a technology that automatically performs consent or non-consent operations on behalf of the user based on the collation results.

[0598] The "text transmission means" is a technique for transmitting the acquired pop-up text to the server.

[0599] The "agreement status generation means" is a technique in which the server generates a status of "agree" or "disagree" based on the collation result.

[0600] The "automatic button clicking means" is a technology that automatically clicks a corresponding button on the terminal based on the generated consent status.

[0601] A "server" is a central computer system that processes and analyzes data.

[0602] A "terminal" is a device operated by a user, which receives instructions from a server and executes operations.

[0603] The present invention relates to a system that enables users to properly understand the content of consent popups regarding cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. This system is designed to operate in cooperation with three main entities: a server, a terminal, and a user.

[0604] System configuration

[0605] Hardware and software used

[0606] 1. Server

[0607] The server acts as the center of data analysis and recognition. The specific software used includes a natural language processing (NLP) module, such as TensorFlow.

[0608] 2. Terminal

[0609] A device is a user-operated device, such as a smartphone, tablet, or computer, that has software installed on it to capture the text of the displayed popup and send it to a server for analysis.

[0610] 3. Users

[0611] A user visits a website for the first time, and the system automatically performs certain actions via the device operated by the user.

[0612] System Operation

[0613] 1. User Behavior

[0614] The first time a user visits a particular website, they will see a consent popup regarding cookies and privacy policies.

[0615] 2. Terminal Processing

[0616] The device detects the popup text, retrieves its contents, and sends it to the server. Specifically, the popup text is retrieved from the DOM using JavaScript and asynchronously sent to the server using AJAX.

[0617] 3. Server Processing

[0618] The server passes the received text to a natural language processing (NLP) module, which tokenizes the text, tags it with parts of speech, and performs dependency analysis to extract specific keywords and phrases.

[0619] The extracted keywords are checked against a pre-defined allow list, which includes items such as "not for advertising purposes" and "allowed for measurement for site analytics."

[0620] Based on the matching result, the server generates a status of "agree" or "disagree" and returns it to the terminal.

[0621] 4. Automatic actions on the device

[0622] The device will automatically click the accept or decline button in the pop-up based on the status received from the server, saving the user the trouble of manually clicking the button.

[0623] Specific examples

[0624] Example 1:

[0625] 1. User Behavior

[0626] A user visits an e-commerce site for the first time and is greeted with a pop-up message saying, "We use cookies for advertising purposes. Do you agree?"

[0627] 2. Terminal Processing

[0628] Gets the popup text and sends it to the server.

[0629] 3. Server Processing

[0630] The server passes the text to the NLP module for analysis. The extracted keywords "advertising", "purpose", and "cookies" are checked against the allow list. Since the allow list includes "not allowed for advertising purposes", the server generates a "disagree" status.

[0631] 4. Automatic actions on the device

[0632] The device will automatically click the "Decline" button in the pop-up.

[0633] Example 2:

[0634] 1. User Behavior

[0635] A user visits a news site for the first time and is greeted with a pop-up message saying, "We use cookies to help us analyze and improve our site. Do you agree?"

[0636] 2. Terminal Processing

[0637] Gets the popup text and sends it to the server.

[0638] 3. Server Processing

[0639] The server passes the text to the NLP module for analysis. The extracted keywords "analysis," "improvement," and "cookies" are checked against the allow list. Since the allow list includes "Allow measurement for site analytics," the server generates an "Agree" status.

[0640] 4. Automatic actions on the device

[0641] The device will automatically click the "Agree" button in the pop-up.

[0642] Example prompts for generative AI models

[0643] "Please parse the content of the following popup and generate an accept or decline status based on your allow list: 'We use cookies for advertising purposes. Do you agree?'"

[0644] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0645] Step 1:

[0646] A user visits a website

[0647] Specific behavior: A user enters a URL into the address bar of a web browser and presses the Enter key.

[0648] Input: The URL entered by the user.

[0649] Output: The web page loads and displays the cookie and privacy policy consent popup.

[0650] Step 2:

[0651] The device detects the popup and retrieves the text

[0652] What happens: The browser's JavaScript monitors the DOM and detects the presence of a popup.

[0653] Parse the text node to get the popup text.

[0654] Input: The DOM structure of the displayed web page.

[0655] Output: The text of the popup (e.g. "We use cookies for advertising purposes. Do you agree?").

[0656] Step 3:

[0657] The device sends the text to the server

[0658] Specific behavior: Encodes the popup text and sends it to the server as an HTTP request.

[0659] AJAX is used for asynchronous communication.

[0660] Input: The text of the popup.

[0661] Output: The HTTP request sent to the server.

[0662] Step 4:

[0663] The server passes the text to a natural language processing (NLP) module

[0664] Specific operation: The server passes the received text as input to the NLP module.

[0665] The NLP module uses frameworks such as TensorFlow.

[0666] Input: The text of the HTTP request body.

[0667] Output: The text data passed to the NLP module.

[0668] Step 5:

[0669] NLP module analyzes the text

[0670] What it does: The NLP module tokenizes the text, tags it with parts of speech, and performs dependency analysis.

[0671] Further analysis is carried out to extract keywords and phrases.

[0672] Input: Text data passed from the server.

[0673] Output: Extracted keywords and phrases (e.g. "advertising", "purpose", "cookies").

[0674] Step 6:

[0675] The server checks the analysis results against the permission list

[0676] What it does: The server compares the extracted keywords and phrases with a pre-defined allow list.

[0677] An allow list might consist of, for example, "No use for advertising purposes" or "Measurement for site analytics is allowed."

[0678] Input: Extracted keywords or phrases, allow lists.

[0679] Output: Matching result (e.g., matched "Not for advertising purposes").

[0680] Step 7:

[0681] The server generates the consent status

[0682] Specific operation: The server generates a consent status (e.g., "consent" or "disagree") based on the matching result.

[0683] If the matching result matches the conditions of the allow list, an "Agree" status is generated; if not, a "Disagree" status is generated.

[0684] Input: Matching result.

[0685] Output: The consent status (e.g. "Disagree").

[0686] Step 8:

[0687] Automatic button click based on the status received by the device from the server

[0688] Specific operation: The terminal analyzes the consent status received from the server and automatically clicks the corresponding button ("Agree" or "Disagree").

[0689] Use JavaScript to fire a click event.

[0690] Input: consent status.

[0691] Output: Automatic click action of the corresponding button.

[0692] (Application example 1)

[0693] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0694] In autonomous vehicles, it is important for passengers to respond promptly and appropriately to cookie and privacy policy consent popups. However, manually performing these consent operations during autonomous driving not only degrades the user experience but can also be a distraction. To solve this problem, a system is needed that automates cookie and privacy policy consent actions, ensuring a safe and comfortable ride for passengers.

[0695] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0696] In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords from the text analyzed by the natural language processing means, and means for comparing the extracted keywords with a preset permission list. This enables the user's consent operation in an autonomous vehicle to be safely and quickly performed by a system including means for automatically processing a consent pop-up displayed on a vehicle terminal, means for analyzing text acquired from the vehicle terminal and comparing it with permission items, and means for automatically clicking a virtual button based on the comparison result.

[0697] "Natural language processing means" refers to means for carrying out a series of processes to analyze acquired text data and extract meanings and keywords.

[0698] The "means for extracting specific keywords" refers to a means for identifying and extracting important words and phrases from text analyzed by the natural language processing means.

[0699] An "allow list" is a pre-defined list of specific keywords or phrases that are allowed.

[0700] A "matching method" is a method for comparing extracted keywords or phrases with an allowed list to determine whether they match or mismatch.

[0701] "Automatic execution means" refers to a means for taking an action of consent or disagreement based on the matching results without user intervention.

[0702] "Means for automatically clicking virtual buttons" means means for a system to simulate the act of pressing an "agree" or "disagree" button through a program without the user having to manually operate the system.

[0703] A "vehicle terminal" is a terminal such as a computer or display device located inside an autonomous driving vehicle.

[0704] A "consent popup" is a short notification screen that appears when a website or application asks users to consent to the use of cookies or privacy policy.

[0705] The present invention relates to a system for quickly and appropriately handling cookie and privacy policy consent popups in an autonomous vehicle, including natural language processing, keyword extraction, permission list matching, and automatic action execution.

[0706] This system uses the following hardware and software:

[0707] Hardware:

[0708] Vehicle terminals in autonomous vehicles (e.g., HMDs with displays and camera functions)

[0709] Server (using cloud services, typically AWS, etc.)

[0710] software:

[0711] Natural language processing libraries (e.g., SpaCy, NLTK)

[0712] Server application (implemented in Python)

[0713] Cloud storage (e.g. Amazon S3)

[0714] The outline of the processing procedure is as follows.

[0715] 1. When a user gets into an autonomous vehicle, a cookie and privacy policy consent pop-up will appear on the vehicle's device.

[0716] 2. The vehicle device captures the text of the displayed popup using its camera function and sends that text to the server.

[0717] 3. The server analyzes the received text using a natural language processing module, tokenizing it, tagging it as a part of speech, and performing dependency analysis, thereby extracting important keywords such as "cookie" and "privacy."

[0718] 4. The extracted keywords are compared against an allow list, which may include items such as "Analysis of driving data is allowed" or "Use for advertising purposes is prohibited."

[0719] 5. Based on the matching result, the server generates a status of "agree" or "disagree" and sends it to the vehicle terminal.

[0720] 6. The device will automatically click the virtual button "Agree" or "Disagree" based on the received status.

[0721] Specific examples are shown below.

[0722] Example 1: When a user gets into an autonomous vehicle and a pop-up message appears on the HMD saying, "This vehicle uses cookies to analyze communication data. Do you agree?", the following prompt sentence is input into the AI ​​model.

[0723] "Analyze the following text and extract keywords: 'This vehicle uses cookies to analyze communication data. Do you agree?'"

[0724] The server extracts the keywords "communication data analysis," "cookies," and "consent" from the analysis results and compares them with the permission list. In this case, the setting is "permit driving data analysis," so the comparison result is "consent." The HMD on the vehicle terminal automatically clicks the "consent" button.

[0725] In this way, the system of the present invention enables the user's consent operation to be performed safely and quickly in an autonomous vehicle.

[0726] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0727] Step 1:

[0728] When a user gets into an autonomous vehicle, a consent pop-up regarding cookies and privacy policies is displayed on the vehicle's head-mounted display (HMD). This pop-up contains specific text.

[0729] Input: Cookie and privacy policy consent popup text displayed by the user

[0730] Output: Image data for capturing popup text

[0731] Specific operation: The HMD's camera function is activated and the text of the displayed popup is captured as image data.

[0732] Step 2:

[0733] The vehicle terminal processes the captured image data and extracts text, which is then sent to the server.

[0734] Input: Image data of the captured popup text

[0735] Output: Text data (e.g. "This vehicle uses cookies to analyze communication data. Do you agree?")

[0736] What it does: Image processing software extracts text from the captured image and sends it to a server, such as Tesseract OCR.

[0737] Step 3:

[0738] The server passes the received text data to a natural language processing (NLP) module for analysis, which includes tokenization, part-of-speech tagging, and dependency analysis. Keywords such as "communication data analysis," "cookies," and "consent" are extracted.

[0739] Input: Received text data

[0740] Output: Extracted keywords (e.g., "communication data analysis," "cookies," "consent")

[0741] Specific operation: The NLP module analyzes the text. The NLP library used is, for example, SpaCy.

[0742] Step 4:

[0743] The server compares the extracted keywords with a pre-defined permission list, which may include items such as "driving data analysis is allowed" or "usage for advertising purposes is prohibited," and generates a consent or denial status based on the match.

[0744] Input: Extracted keywords (e.g., "communication data analysis," "cookies," "consent") and allow list

[0745] Output: Matching result (e.g., "Agree" or "Disagree")

[0746] What happens: The server checks the keywords against the allow list, which is stored in a database, and generates a status.

[0747] Step 5:

[0748] The server transmits the generated status of consent or disagreement to the vehicle terminal.

[0749] Input: The status of the match (e.g., "Agree" or "Disagree")

[0750] Output: Status sent to vehicle terminal

[0751] Specific operation: The server sends the status to the vehicle terminal.

[0752] Step 6:

[0753] The vehicle terminal automatically clicks the virtual button based on the received status, without any manual operation by the user.

[0754] Input: Status sent to the vehicle terminal (e.g. "Agree" or "Disagree")

[0755] Output: Virtual button clicked

[0756] Specific operation: The vehicle terminal software simulates the clicking of a virtual button based on the received status. The virtual button interface is integrated into the vehicle terminal UI.

[0757] Through the above processing steps, the system of the present invention enables users to quickly and appropriately consent to cookies and privacy policies in autonomous vehicles.

[0758] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0759] The present invention relates to a system that enables users to properly understand the content of consent popups for cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. The system includes a natural language processing means, an extraction means, a matching means, a means for executing consent actions, and an emotion engine that recognizes the user's emotions.

[0760] Program processing

[0761] Program Overview

[0762] 1. User:

[0763] When you first visit the website, a pop-up will appear explaining the cookies and privacy policy.

[0764] 2. Terminal:

[0765] Gets the text of the displayed popup and sends it to the server.

[0766] 3. Server:

[0767] The received text is passed to a natural language processing (NLP) module for analysis.

[0768] The NLP module performs text tokenization, part-of-speech tagging, and dependency analysis to extract specific keywords and phrases.

[0769] 4. Server:

[0770] Matches extracted keywords and phrases against a pre-configured allow list.

[0771] The allow list may include items such as "No use for advertising purposes" and "Measurement for site analytics is allowed."

[0772] 5. Server:

[0773] A matching result is generated and a determination is made as to whether or not there is a match. If there is a match, a status of "Agree" is generated; if there is no match, a status of "Disagree" is generated.

[0774] 6. Terminal:

[0775] The matching results are received from the server and the emotion engine is activated. The emotion engine analyzes the user's text input, voice input, and facial expressions to recognize the user's emotions.

[0776] 7. Server:

[0777] Tailor your consent or disagreement actions based on the perceived emotion, for example automatically clicking the disagree button or showing a detailed explanation if the user expresses anxiety or doubt.

[0778] 8. Terminal:

[0779] Based on the judgment results of the emotion engine, the agree or disagree button is automatically clicked.

[0780] Specific examples

[0781] Example 1:

[0782] 1. User:

[0783] You visit an e-commerce website for the first time and a pop-up appears saying:

[0784] "We use cookies for advertising purposes. Do you agree?"

[0785] 2. Terminal:

[0786] Get the popup text "We use cookies for advertising purposes. Do you agree?" and send it to the server.

[0787] 3. Server:

[0788] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "advertising," "purpose," and "cookies" through tokenization, part-of-speech tagging, and dependency analysis.

[0789] 4. Server:

[0790] The extracted keywords are compared with the allow list. Since the allow list includes a setting that says "Use for advertising purposes is prohibited," the result of the comparison is "Disapproval."

[0791] 5. Terminal:

[0792] The matching result "disagree" is received from the server and the emotion engine is activated.

[0793] 6. Terminal:

[0794] The emotion engine analyzes the user's text and voice input and recognizes that the user has doubts, such as "Why rejection?"

[0795] 7. Server:

[0796] Automatically clicking the "disagree" button in a pop-up based on perceived sentiment,

[0797] Example 2:

[0798] 1. User:

[0799] You visit a news site for the first time and a pop-up appears saying:

[0800] "This site uses cookies to analyze and improve it. Do you agree?"

[0801] 2. Terminal:

[0802] The text of the popup "This site uses cookies for analysis and improvement purposes. Do you agree?" is obtained and sent to the server.

[0803] 3. Server:

[0804] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "analysis," "improvement," and "cookie" through tokenization, part-of-speech tagging, and dependency analysis.

[0805] 4. Server:

[0806] The extracted keywords are compared with the allow list. Since the allow list is set to "Allow measurement for site analysis," the result is "Agree."

[0807] 5. Terminal:

[0808] The matching result "Agreement" is received from the server and the emotion engine is activated.

[0809] 6. Terminal:

[0810] The emotion engine analyzes the user's facial expressions and recognizes, for example, that the user is feeling at ease.

[0811] 7. Server:

[0812] Automatically click the "Agree" button on a pop-up based on perceived sentiment.

[0813] In this way, the system of the present invention streamlines the cookie consent process when users visit websites, and by taking into account user sentiment in addition to pre-defined policies, it achieves more appropriate and intuitive consent management.

[0814] The processing flow will be explained below.

[0815] Step 1:

[0816] User:

[0817] Visit a website for the first time. Upon arrival, the site displays a pop-up about cookies and privacy policies.

[0818] Step 2:

[0819] Device:

[0820] Gets the text of the popup that was displayed.

[0821] Specifically, JavaScript's document.querySelector is used to extract the popup text elements.

[0822] Step 3:

[0823] Device:

[0824] The retrieved text is sent to the server.

[0825] Specifically, Ajax is used to send text to the server via a POST request.

[0826] Step 4:

[0827] server:

[0828] The received text is passed to a natural language processing (NLP) module.

[0829] Specifically, it calls libraries such as Python's spaCy and NLTK.

[0830] Step 5:

[0831] server:

[0832] The NLP module analyzes the text and extracts specific keywords using techniques such as tokenization, part-of-speech tagging, and dependency analysis.

[0833] Specifically, the text is divided into tokens, each token is tagged with a part-of-speech tag, and dependency analysis is performed.

[0834] Step 6:

[0835] server:

[0836] The extracted keywords are compared against a pre-defined allow list.

[0837] Specifically, the extracted keywords are compared with each item in the permission list to see if they match.

[0838] Step 7:

[0839] server:

[0840] A matching result is generated, and if there is a match, a status of "Agree" is generated, and if there is no match, a status of "Disagree."

[0841] Specifically, the status is determined based on the conditions in the permission list, and the result is returned as a response.

[0842] Step 8:

[0843] Device:

[0844] Receives the status received from the server. When the status is received, it starts the emotion engine.

[0845] Step 9:

[0846] Device:

[0847] The emotion engine analyzes the user's text input, voice input, or facial expressions to recognize the user's emotions.

[0848] Specifically, when analyzing voice input, it uses voice recognition technology, and when analyzing text input, it analyzes the input string. For facial expression analysis, it analyzes the user's facial expressions from camera footage.

[0849] Step 10:

[0850] server:

[0851] Tailor actions to agree or disagree based on perceived sentiment.

[0852] For example, if a user expresses concerns or doubts, we can provide a detailed explanation or automatically click a no consent button.

[0853] Step 11:

[0854] Device:

[0855] Based on the judgment results of the emotion engine, an action of agreement or disagreement is taken.

[0856] Specifically, the corresponding button (agree or disagree) is clicked using the JavaScript click method.

[0857] Step 12:

[0858] User:

[0859] Confirm that the pop-up was interacted with and continue using the website.

[0860] As a result, proper consent management is carried out without the user having to perform the consent operation themselves.

[0861] Example 2

[0862] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0863] In recent years, users are required to understand the content of cookie and privacy policy consent popups that appear when visiting websites and make a quick decision. However, current systems often cause confusion and bewilderment when users choose to consent or not. In addition, consent management does not take user emotions into consideration, resulting in a poor user experience.

[0864] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords and phrases from the text analyzed by the natural language processing means, means for comparing the extracted keywords and phrases with a preset permission list, means for automatically performing an action of consent or disagreement based on the comparison result, and means for recognizing the user's emotions and adjusting the action of consent or disagreement based on the recognition result. This allows the user to properly understand the contents of cookies and privacy policies and make quick and appropriate decisions, improving the user experience.

[0865] "Natural language processing means" is a technology that analyzes text obtained from a website and extracts specific keywords and phrases by tokenizing words and phrases, tagging them by part of speech, and performing dependency analysis.

[0866] "Keyword and phrase extraction means" refers to a technique for identifying and collecting specific words and phrases from text analyzed by natural language processing means.

[0867] An "allow list" is a pre-defined list of specific keywords or phrases that define the conditions under which a decision to accept or reject them will be made.

[0868] "Matching methods" are techniques that compare extracted keywords and phrases with items on an allow list to determine whether they match.

[0869] A "match result" is a determination of whether keywords and phrases match the allow list, and is information used to generate a consent or disagreement status.

[0870] "Means for automatically performing an action of consent or denial" refers to technology that recreates the state in which a user clicks the consent or denial button on a website based on the results of the comparison.

[0871] "Means for recognizing user emotions" refers to technology that analyzes the user's facial expressions, voice input, text input, etc. to determine the user's emotional state.

[0872] The "means for adjusting actions based on emotions" is a technology that provides additional information or changes actions when automatically taking an action of agreement or disagreement based on the results of user emotion recognition.

[0873] MODE FOR CARRYING OUT THE INVENTION

[0874] The present invention relates to a system that helps users quickly and appropriately decide whether to agree or disagree to pop-ups about cookies and privacy policies that appear when visiting a website. The system includes a natural language processing means, an extraction means, a matching means, a means for executing an agreement action, and an emotion engine that recognizes the user's emotions.

[0875] Specifically, it is configured as follows:

[0876] 1. Website visits and pop-ups

[0877] Users: When they first visit a website, they will see a pop-up explaining cookies and privacy policies, such as "We use cookies for advertising purposes. Do you agree?"

[0878] 2. Getting and sending popup text

[0879] Terminal: The text of the displayed popup is obtained and sent to the server. The software used generally uses the browser's API.

[0880] 3. Text Analysis

[0881] Server: Passes the received text to a natural language processing (NLP) module for analysis, using a natural language processing library such as SpaCy or NLTK.

[0882] NLP module: Tokenizes text, tags it with parts of speech, and performs dependency analysis, for example, extracting important keywords and phrases such as "advertising," "purpose," and "cookies."

[0883] 4. Keyword Matching

[0884] Server: The extracted keywords and phrases are compared against a pre-defined allow list, which may include items such as "no advertising use" or "allowed for site analytics measurement."

[0885] 5. Judgment of matching results

[0886] Server: Based on the match, generate a "Agree" or "Disagree" status. For example, if the match matches "Not for advertising purposes", it will be determined as a disagreement.

[0887] 6. Activating the Emotional Engine

[0888] Device: Receives the matching results from the server and activates the emotion engine, which uses Google Cloud Vision and Microsoft Azure's Emotion API to analyze the user's facial expressions, voice input, and text input.

[0889] 7. Adjusting actions based on emotions

[0890] Server: Automatically adjust the action of agreeing or disagreeing based on the user's emotion recognition results. For example, if the user expresses anxiety or doubt, display a detailed explanation or automatically click the "disagree" button.

[0891] 8. Automated consent / non-consent decisions

[0892] Device: Automatically clicks the "Agree" or "Disagree" button on the pop-up based on the emotion engine's judgment.

[0893] Specific examples

[0894] Below is a specific scenario using this system.

[0895] Example 1: E-commerce site

[0896] User: Visits an e-commerce website for the first time and sees a popup saying "We use cookies for advertising purposes. Do you agree?"

[0897] Terminal: Get this popup text and send it to the server.

[0898] Server: Passes the text to a natural language processing (NLP) module, which analyzes it for keywords such as "advertising," "purpose," and "cookies."

[0899] Server: Compares the analysis results with the permission list and issues a "deny" decision, stating that "use for advertising purposes is not permitted."

[0900] Device: Receives the matching results, activates the emotion engine, and analyzes the user's facial expressions and voice.

[0901] Server: As a result of the sentiment engine, it recognizes that the user is expressing doubt and automatically clicks the "disagree" button.

[0902] Example 2: News site

[0903] User: Visits a news site for the first time and sees a pop-up that says "We use cookies to help us analyze and improve our site. Do you agree?"

[0904] Terminal: Get this popup text and send it to the server.

[0905] Server: Passes the text to a natural language processing (NLP) module, which analyzes it for keywords like "analysis," "improvement," and "cookie."

[0906] Server: Compares the analysis results with the allow list and determines "Agree" by stating "Measurement for site analysis is permitted."

[0907] Terminal: Receives the matching results, activates the emotion engine, and analyzes the user's facial expressions.

[0908] Server: As a result of the emotion engine, it recognizes that the user feels safe and automatically clicks the "Agree" button.

[0909] In this way, the system of the present invention helps users properly understand the content of cookies and privacy policies and make quick decisions, which greatly improves the user experience.

[0910] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0911] Step 1:

[0912] User: When a user first visits a website, a popup explaining cookies and privacy policy is displayed. Input is a user visiting a website. Output is a popup being generated.

[0913] Step 2:

[0914] Terminal: Get the text of the displayed popup and send it to the server. This process includes the specific operation of extracting the popup text from the DOM (Document Object Model). The input is the displayed popup text, and the output is the text data to be sent to the server.

[0915] Step 3:

[0916] Server: Passes the received text to a natural language processing (NLP) module for analysis. Specifically, it uses natural language processing libraries such as SpaCy and NLTK. In processing by the NLP module, text data is provided as input, and the results of tokenization, part-of-speech tagging, and dependency analysis are generated as output.

[0917] Step 4:

[0918] Server: Extracts specific keywords and phrases from the text analyzed by the NLP module. This includes programs that identify and collect important words and phrases. The input is the analyzed text data, and the output is the extracted keywords and phrases.

[0919] Step 5:

[0920] Server: Compares extracted keywords and phrases against a pre-configured allow list, which may contain items such as "not for advertising purposes" or "allowed for site analytics measurement." The input is the extracted keywords and phrases and the allow list, and the output is the match result.

[0921] Step 6:

[0922] Server: Generates a status of "agree" or "disagree" based on the matching result. The input is the matching result, and the output is the status of agree or disagree.

[0923] Step 7:

[0924] Device: Receives the status obtained from the matching result and starts the emotion engine. The emotion engine uses Google Cloud Vision or Microsoft Azure's Emotion API to analyze the user's facial expressions, voice input, and text input. The input is the status of agreement or disagreement, and the output is the emotion recognition result.

[0925] Step 8:

[0926] Server: Automatically adjusts the action of agreeing or disagreeing based on the user's emotion recognition results. For example, if the user expresses anxiety or doubt, it displays a detailed explanation or automatically clicks the "disagree" button. The input is the emotion recognition result, and the output is the adjusted action.

[0927] Step 9:

[0928] Terminal: Automatically clicks the "Agree" or "Disagree" button in a pop-up based on the emotion engine's judgment. The input is the coordinated action, and the output is the actual button click.

[0929] (Application example 2)

[0930] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0931] Conventional consent popups for cookies and privacy policies on websites make it difficult for users to understand their content and quickly decide whether to consent or not. In particular, little consideration is given to the user's psychological burden and level of understanding, resulting in a non-intuitive consent process. Furthermore, users are often asked to consent while feeling anxious or suspicious, which reduces the quality of the user experience.

[0932] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords from the text analyzed by the natural language processing means, and means for comparing the extracted keywords with a preset allow list. This makes it possible to streamline the cookie consent process when a user visits a website and enables more appropriate and intuitive consent management by taking into account the user's feelings in addition to preset policies.

[0933] "Natural language processing means" refers to technical means that analyzes text data obtained from websites and performs processes such as tokenization, part-of-speech tagging, and dependency analysis.

[0934] The "means for extracting specific keywords" refers to a technical means for extracting predefined important words or phrases from text analyzed by natural language processing means.

[0935] "Means for checking against an allow list" refers to a technical means for comparing extracted keywords with a pre-established allow list to determine whether they match.

[0936] "Means for automatically performing consent or denial actions" refers to technical means for automatically performing consent or denial actions based on the results of a match against an allow list.

[0937] "Emotion analysis means" refers to a technical means for reading emotions from the user's input text, voice, facial expressions, etc., and recognizing their emotional state.

[0938] The "means for adjusting consent or disagreement actions" refers to technical means for optimizing consent or disagreement actions based on the recognized user sentiment.

[0939] In this invention, when a user visits a website for the first time, a pop-up message explaining cookies and privacy policies is displayed. The terminal obtains the text of the pop-up message and sends it to the server.

[0940] The server then analyzes the retrieved text using natural language processing tools (e.g., HuggingFace's Transformers library), which involve tokenizing the text, tagging it as a part of speech, and performing dependency analysis to extract specific keywords and phrases.

[0941] The extracted keywords are then checked against a pre-defined allow list, which may include items such as "no advertising use" or "allowed for site analytics measurement." Based on the result of the check, a "yes" or "no" status is generated for a particular policy.

[0942] The device then receives the matching results from the server and activates an emotion analyzer (e.g., an emotion analysis model in the HuggingFace Transformers library) that analyzes the user's text input, voice input, or facial expressions to recognize the user's emotions.

[0943] Furthermore, the server adjusts the action of agreeing or disagreeing based on the perceived emotion: for example, if the user expresses anxiety or doubt, it automatically clicks the disagree button or displays a detailed explanation, etc. Similarly, if the user feels reassured, it automatically clicks the agree button.

[0944] Specific examples

[0945] Example 1:

[0946] 1. A user visits a shopping site for the first time and is greeted with a pop-up that reads: "We use cookies for advertising purposes. Do you agree?"

[0947] 2. The device retrieves the popup text and sends it to the server.

[0948] 3. The server performs natural language processing to extract keywords from the text, such as "advertising," "purpose," and "cookie."

[0949] 4. The server checks the extracted keywords against the allow list. Since the allow list includes a setting that says "not for advertising purposes," the result is "disapproval."

[0950] 5. The device receives the matching results from the server and activates the emotion analysis function. It recognizes that the user has doubts, such as "Why was it rejected?"

[0951] 6. The "Disagree" button in the pop-up is automatically clicked based on the perceived sentiment.

[0952] Prompt Sentence Examples

[0953] "Please develop a system that uses natural language processing to analyze how users feel about the privacy policy pop-up that appears for the first time on a website they visit, and helps them decide whether to agree or disagree based on that information."

[0954] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0955] Step 1:

[0956] When a user visits a website for the first time, the device gets a popup with a description of cookies and privacy policy. The input is the display data of the web page, and the output is the text data of the popup. This text data is sent to the server for further processing.

[0957] Step 2:

[0958] The server analyzes the text sent from the terminal using a natural language processing means (NLP module). The input is the text data of the popup, and the output is the analysis data generated by tokenization, part-of-speech tagging, and dependency analysis. Specifically, the natural language processing means tokenizes the text, tags each token with a part-of-speech tag, and analyzes dependencies.

[0959] Step 3:

[0960] The server extracts specific keywords and phrases from the analysis data. The input is the analysis data from the natural language processing tool, and the output is a list of extracted keywords. This clarifies the main intent and purpose of the popup.

[0961] Step 4:

[0962] The server compares the extracted keywords with a pre-defined allow list. The input is the list of extracted keywords and the allow list, and the output is the match result (agreement or disagreement status). Specifically, it compares whether the keywords match the items in the allow list, and generates a consent status based on the result.

[0963] Step 5:

[0964] The terminal receives the matching results from the server and activates the emotion analysis means. The inputs are the matching results and the user's text input, voice input, or facial expression data, and the output is the recognized emotion. The emotion analysis means analyzes the user's emotion based on the input data and recognizes their state.

[0965] Step 6:

[0966] The server adjusts the action of agreeing or disagreeing based on the recognized emotion. The input is the recognized emotion and the matching result, and the output is the final action (agreeing or disagreeing). For example, if the user expresses anxiety or doubt, a detailed explanation is displayed, and if there is no anxiety, the agree button is automatically clicked.

[0967] Step 7:

[0968] The terminal automatically clicks the accept or decline button based on the final action received from the server. The input is the final action status from the server, and the output is the operation (clicking accept or decline) based on that status.

[0969] This process allows users to quickly and appropriately respond to pop-ups about cookies and privacy policies.

[0970] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0971] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0972] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0973] [Third embodiment]

[0974] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0975] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0976] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0977] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0978] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0979] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0980] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0981] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0982] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0983] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0984] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0985] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0986] The present invention relates to a system that enables users to appropriately understand the content of consent popups regarding cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. The system includes a natural language processing unit, an extraction unit, a matching unit, and a set of units for executing consent actions.

[0987] Program processing

[0988] Program Overview

[0989] 1. User:

[0990] When you first visit the website, a pop-up will appear explaining the cookies and privacy policy.

[0991] 2. Terminal:

[0992] Gets the text of the displayed popup and sends it to the server.

[0993] 3. Server:

[0994] The received text is passed to a natural language processing (NLP) module for analysis.

[0995] The NLP module performs text tokenization, part-of-speech tagging, and dependency analysis to extract specific keywords and phrases.

[0996] 4. Server:

[0997] Matches extracted keywords and phrases against a pre-configured allow list.

[0998] The allow list may include items such as "No use for advertising purposes" and "Measurement for site analytics is allowed."

[0999] 5. Server:

[1000] A matching result is generated and a determination is made as to whether or not there is a match. If there is a match, a status of "Agree" is generated; if there is no match, a status of "Disagree" is generated.

[1001] 6. Terminal:

[1002] Automatically clicks the accept or decline button based on the status received from the server.

[1003] Specific examples

[1004] Example 1:

[1005] 1. User:

[1006] You visit an e-commerce website for the first time and a pop-up appears saying:

[1007] "We use cookies for advertising purposes. Do you agree?"

[1008] 2. Terminal:

[1009] Get the popup text "We use cookies for advertising purposes. Do you agree?" and send it to the server.

[1010] 3. Server:

[1011] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "advertising," "purpose," and "cookies" through tokenization, part-of-speech tagging, and dependency analysis.

[1012] 4. Server:

[1013] The extracted keywords are compared with the allow list. Since the allow list includes a setting that says "Use for advertising purposes is prohibited," the result of the comparison is "Disapproval."

[1014] 5. Terminal:

[1015] Based on the server's matching result of "disagree," the "disagree" button on the pop-up will be automatically clicked.

[1016] Example 2:

[1017] 1. User:

[1018] You visit a news site for the first time and a pop-up appears saying:

[1019] "This site uses cookies to analyze and improve it. Do you agree?"

[1020] 2. Terminal:

[1021] The text of the popup "This site uses cookies for analysis and improvement purposes. Do you agree?" is obtained and sent to the server.

[1022] 3. Server:

[1023] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "analysis," "improvement," and "cookie" through tokenization, part-of-speech tagging, and dependency analysis.

[1024] 4. Server:

[1025] The extracted keywords are compared with the allow list. Since the allow list is set to "Allow measurement for site analysis," the result is "Agree."

[1026] 5. Terminal:

[1027] Based on the matching result "Agree" from the server, the "Agree" button on the pop-up will be automatically clicked.

[1028] In this way, the system of the present invention streamlines the cookie consent process when users visit websites and automatically takes appropriate action based on pre-defined policies, thereby improving the user experience.

[1029] The processing flow will be explained below.

[1030] Step 1:

[1031] User:

[1032] Visit a website for the first time. Upon arrival, the site displays a pop-up about cookies and privacy policies.

[1033] Step 2:

[1034] Device:

[1035] Gets the text of the popup that was displayed.

[1036] Specifically, JavaScript's document.querySelector is used to extract the popup text elements.

[1037] Step 3:

[1038] Device:

[1039] The retrieved text is sent to the server.

[1040] Specifically, Ajax is used to send text to the server via a POST request.

[1041] Step 4:

[1042] server:

[1043] The received text is passed to a natural language processing (NLP) module.

[1044] Specifically, it calls libraries such as Python's spaCy and NLTK.

[1045] Step 5:

[1046] server:

[1047] The NLP module analyzes the text and extracts specific keywords using techniques such as tokenization and part-of-speech tagging.

[1048] Specifically, the text is divided into tokens, each token is tagged with a part-of-speech tag, and dependency analysis is performed.

[1049] Step 6:

[1050] server:

[1051] The extracted keywords are compared against a pre-defined allow list.

[1052] Specifically, the extracted keywords are compared with each item in the permission list to see if they match.

[1053] Step 7:

[1054] server:

[1055] A matching result is generated, and if there is a match, a status of "Agree" is generated, and if there is no match, a status of "Disagree."

[1056] Specifically, the status is determined based on the conditions in the permission list, and the result is returned as a response.

[1057] Step 8:

[1058] Device:

[1059] Based on the status received from the server, an action of consent or non-consent is automatically taken.

[1060] Specifically, the corresponding button (agree or disagree) is clicked using the JavaScript click method.

[1061] Step 9:

[1062] User:

[1063] Confirm that the pop-up was interacted with and continue using the website.

[1064] As a result, proper consent management is carried out without the user having to perform the consent operation themselves.

[1065] Example 1

[1066] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1067] When users visit a website, they may find it difficult to properly understand the cookie and privacy policy consent pop-up and quickly decide whether to consent or not. This situation not only impairs the user experience, but also may result in users not making appropriate privacy choices. Furthermore, manually performing this series of operations is time-consuming and cumbersome for users. The present invention aims to solve this problem.

[1068] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1069] In this invention, the server includes a natural language processing unit for analyzing text acquired from a website, a unit for transmitting the acquired text to the server, and a unit for the server to generate a consent status based on the text analysis result, thereby streamlining the cookie consent process when a user visits a website and automatically taking appropriate action based on a pre-defined policy.

[1070] "Natural language processing means" is a technology that analyzes text obtained from websites and understands their grammatical structure and meaning.

[1071] "Keyword extraction means" is a technique for selecting important words and phrases from text analyzed by natural language processing means.

[1072] The "allowed list matching means" is a technology that compares extracted keywords and phrases with a pre-set allowed list to determine whether they match.

[1073] The "means for automatically executing consent action" is a technology that automatically performs consent or non-consent operations on behalf of the user based on the collation results.

[1074] The "text transmission means" is a technique for transmitting the acquired pop-up text to the server.

[1075] The "agreement status generation means" is a technique in which the server generates a status of "agree" or "disagree" based on the collation result.

[1076] The "automatic button clicking means" is a technology that automatically clicks a corresponding button on the terminal based on the generated consent status.

[1077] A "server" is a central computer system that processes and analyzes data.

[1078] A "terminal" is a device operated by a user, which receives instructions from a server and executes operations.

[1079] The present invention relates to a system that enables users to properly understand the content of consent popups regarding cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. This system is designed to operate in cooperation with three main entities: a server, a terminal, and a user.

[1080] System configuration

[1081] Hardware and software used

[1082] 1. Server

[1083] The server acts as the center of data analysis and recognition. The specific software used includes a natural language processing (NLP) module, such as TensorFlow.

[1084] 2. Terminal

[1085] A device is a user-operated device, such as a smartphone, tablet, or computer, that has software installed on it to capture the text of the displayed popup and send it to a server for analysis.

[1086] 3. Users

[1087] A user visits a website for the first time, and the system automatically performs certain actions via the device operated by the user.

[1088] System Operation

[1089] 1. User Behavior

[1090] The first time a user visits a particular website, they will see a consent popup regarding cookies and privacy policies.

[1091] 2. Terminal Processing

[1092] The device detects the popup text, retrieves its contents, and sends it to the server. Specifically, the popup text is retrieved from the DOM using JavaScript and asynchronously sent to the server using AJAX.

[1093] 3. Server Processing

[1094] The server passes the received text to a natural language processing (NLP) module, which tokenizes the text, tags it with parts of speech, and performs dependency analysis to extract specific keywords and phrases.

[1095] The extracted keywords are checked against a pre-defined allow list, which includes items such as "not for advertising purposes" and "allowed for measurement for site analytics."

[1096] Based on the matching result, the server generates a status of "agree" or "disagree" and returns it to the terminal.

[1097] 4. Automatic actions on the device

[1098] The device will automatically click the accept or decline button in the pop-up based on the status received from the server, saving the user the trouble of manually clicking the button.

[1099] Specific examples

[1100] Example 1:

[1101] 1. User Behavior

[1102] A user visits an e-commerce site for the first time and is greeted with a pop-up message saying, "We use cookies for advertising purposes. Do you agree?"

[1103] 2. Terminal Processing

[1104] Gets the popup text and sends it to the server.

[1105] 3. Server Processing

[1106] The server passes the text to the NLP module for analysis. The extracted keywords "advertising", "purpose", and "cookies" are checked against the allow list. Since the allow list includes "not allowed for advertising purposes", the server generates a "disagree" status.

[1107] 4. Automatic actions on the device

[1108] The device will automatically click the "Decline" button in the pop-up.

[1109] Example 2:

[1110] 1. User Behavior

[1111] A user visits a news site for the first time and is greeted with a pop-up message saying, "We use cookies to help us analyze and improve our site. Do you agree?"

[1112] 2. Terminal Processing

[1113] Gets the popup text and sends it to the server.

[1114] 3. Server Processing

[1115] The server passes the text to the NLP module for analysis. The extracted keywords "analysis," "improvement," and "cookies" are checked against the allow list. Since the allow list includes "Allow measurement for site analytics," the server generates an "Agree" status.

[1116] 4. Automatic actions on the device

[1117] The device will automatically click the "Agree" button in the pop-up.

[1118] Example prompts for generative AI models

[1119] "Please parse the content of the following popup and generate an accept or decline status based on your allow list: 'We use cookies for advertising purposes. Do you agree?'"

[1120] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1121] Step 1:

[1122] A user visits a website

[1123] Specific behavior: A user enters a URL into the address bar of a web browser and presses the Enter key.

[1124] Input: The URL entered by the user.

[1125] Output: The web page loads and displays the cookie and privacy policy consent popup.

[1126] Step 2:

[1127] The device detects the popup and retrieves the text

[1128] What happens: The browser's JavaScript monitors the DOM and detects the presence of a popup.

[1129] Parse the text node to get the popup text.

[1130] Input: The DOM structure of the displayed web page.

[1131] Output: The text of the popup (e.g. "We use cookies for advertising purposes. Do you agree?").

[1132] Step 3:

[1133] The device sends the text to the server

[1134] Specific behavior: Encodes the popup text and sends it to the server as an HTTP request.

[1135] AJAX is used for asynchronous communication.

[1136] Input: The text of the popup.

[1137] Output: The HTTP request sent to the server.

[1138] Step 4:

[1139] The server passes the text to a natural language processing (NLP) module

[1140] Specific operation: The server passes the received text as input to the NLP module.

[1141] The NLP module uses frameworks such as TensorFlow.

[1142] Input: The text of the HTTP request body.

[1143] Output: The text data passed to the NLP module.

[1144] Step 5:

[1145] NLP module analyzes the text

[1146] What it does: The NLP module tokenizes the text, tags it with parts of speech, and performs dependency analysis.

[1147] Further analysis is carried out to extract keywords and phrases.

[1148] Input: Text data passed from the server.

[1149] Output: Extracted keywords and phrases (e.g. "advertising", "purpose", "cookies").

[1150] Step 6:

[1151] The server checks the analysis results against the permission list

[1152] What it does: The server compares the extracted keywords and phrases with a pre-defined allow list.

[1153] An allow list might consist of, for example, "No use for advertising purposes" or "Measurement for site analytics is allowed."

[1154] Input: Extracted keywords or phrases, allow lists.

[1155] Output: Matching result (e.g., matched "Not for advertising purposes").

[1156] Step 7:

[1157] The server generates the consent status

[1158] Specific operation: The server generates a consent status (e.g., "consent" or "disagree") based on the matching result.

[1159] If the matching result matches the conditions of the allow list, an "Agree" status is generated; if not, a "Disagree" status is generated.

[1160] Input: Matching result.

[1161] Output: The consent status (e.g. "Disagree").

[1162] Step 8:

[1163] Automatic button click based on the status received by the device from the server

[1164] Specific operation: The terminal analyzes the consent status received from the server and automatically clicks the corresponding button ("Agree" or "Disagree").

[1165] Use JavaScript to fire a click event.

[1166] Input: consent status.

[1167] Output: Automatic click action of the corresponding button.

[1168] (Application example 1)

[1169] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1170] In autonomous vehicles, it is important for passengers to respond promptly and appropriately to cookie and privacy policy consent popups. However, manually performing these consent operations during autonomous driving not only degrades the user experience but can also be a distraction. To solve this problem, a system is needed that automates cookie and privacy policy consent actions, ensuring a safe and comfortable ride for passengers.

[1171] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1172] In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords from the text analyzed by the natural language processing means, and means for comparing the extracted keywords with a preset permission list. This enables the user's consent operation in an autonomous vehicle to be safely and quickly performed by a system including means for automatically processing a consent pop-up displayed on a vehicle terminal, means for analyzing text acquired from the vehicle terminal and comparing it with permission items, and means for automatically clicking a virtual button based on the comparison result.

[1173] "Natural language processing means" refers to means for carrying out a series of processes to analyze acquired text data and extract meanings and keywords.

[1174] The "means for extracting specific keywords" refers to a means for identifying and extracting important words and phrases from text analyzed by the natural language processing means.

[1175] An "allow list" is a pre-defined list of specific keywords or phrases that are allowed.

[1176] A "matching method" is a method for comparing extracted keywords or phrases with an allowed list to determine whether they match or mismatch.

[1177] "Automatic execution means" refers to a means for taking an action of consent or disagreement based on the matching results without user intervention.

[1178] "Means for automatically clicking virtual buttons" means means for a system to simulate the act of pressing an "agree" or "disagree" button through a program without the user having to manually operate the system.

[1179] A "vehicle terminal" is a terminal such as a computer or display device located inside an autonomous driving vehicle.

[1180] A "consent popup" is a short notification screen that appears when a website or application asks users to consent to the use of cookies or privacy policy.

[1181] The present invention relates to a system for quickly and appropriately handling cookie and privacy policy consent popups in an autonomous vehicle, including natural language processing, keyword extraction, permission list matching, and automatic action execution.

[1182] This system uses the following hardware and software:

[1183] Hardware:

[1184] Vehicle terminals in autonomous vehicles (e.g., HMDs with displays and camera functions)

[1185] Server (using cloud services, typically AWS, etc.)

[1186] software:

[1187] Natural language processing libraries (e.g., SpaCy, NLTK)

[1188] Server application (implemented in Python)

[1189] Cloud storage (e.g. Amazon S3)

[1190] The outline of the processing procedure is as follows.

[1191] 1. When a user gets into an autonomous vehicle, a cookie and privacy policy consent pop-up will appear on the vehicle's device.

[1192] 2. The vehicle device captures the text of the displayed popup using its camera function and sends that text to the server.

[1193] 3. The server analyzes the received text using a natural language processing module, tokenizing it, tagging it as a part of speech, and performing dependency analysis, thereby extracting important keywords such as "cookie" and "privacy."

[1194] 4. The extracted keywords are compared against an allow list, which may include items such as "Analysis of driving data is allowed" or "Use for advertising purposes is prohibited."

[1195] 5. Based on the matching result, the server generates a status of "agree" or "disagree" and sends it to the vehicle terminal.

[1196] 6. The device will automatically click the virtual button "Agree" or "Disagree" based on the received status.

[1197] Specific examples are shown below.

[1198] Example 1: When a user gets into an autonomous vehicle and a pop-up message appears on the HMD saying, "This vehicle uses cookies to analyze communication data. Do you agree?", the following prompt sentence is input into the AI ​​model.

[1199] "Analyze the following text and extract keywords: 'This vehicle uses cookies to analyze communication data. Do you agree?'"

[1200] The server extracts the keywords "communication data analysis," "cookies," and "consent" from the analysis results and compares them with the permission list. In this case, the setting is "permit driving data analysis," so the comparison result is "consent." The HMD on the vehicle terminal automatically clicks the "consent" button.

[1201] In this way, the system of the present invention enables the user's consent operation to be performed safely and quickly in an autonomous vehicle.

[1202] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1203] Step 1:

[1204] When a user gets into an autonomous vehicle, a consent pop-up regarding cookies and privacy policies is displayed on the vehicle's head-mounted display (HMD). This pop-up contains specific text.

[1205] Input: Cookie and privacy policy consent popup text displayed by the user

[1206] Output: Image data for capturing popup text

[1207] Specific operation: The HMD's camera function is activated and the text of the displayed popup is captured as image data.

[1208] Step 2:

[1209] The vehicle terminal processes the captured image data and extracts text, which is then sent to the server.

[1210] Input: Image data of the captured popup text

[1211] Output: Text data (e.g. "This vehicle uses cookies to analyze communication data. Do you agree?")

[1212] What it does: Image processing software extracts text from the captured image and sends it to a server, such as Tesseract OCR.

[1213] Step 3:

[1214] The server passes the received text data to a natural language processing (NLP) module for analysis, which includes tokenization, part-of-speech tagging, and dependency analysis. Keywords such as "communication data analysis," "cookies," and "consent" are extracted.

[1215] Input: Received text data

[1216] Output: Extracted keywords (e.g., "communication data analysis," "cookies," "consent")

[1217] Specific operation: The NLP module analyzes the text. The NLP library used is, for example, SpaCy.

[1218] Step 4:

[1219] The server compares the extracted keywords with a pre-defined permission list, which may include items such as "driving data analysis is allowed" or "usage for advertising purposes is prohibited," and generates a consent or denial status based on the match.

[1220] Input: Extracted keywords (e.g., "communication data analysis," "cookies," "consent") and allow list

[1221] Output: Matching result (e.g., "Agree" or "Disagree")

[1222] What happens: The server checks the keywords against the allow list, which is stored in a database, and generates a status.

[1223] Step 5:

[1224] The server transmits the generated status of consent or disagreement to the vehicle terminal.

[1225] Input: The status of the match (e.g., "Agree" or "Disagree")

[1226] Output: Status sent to vehicle terminal

[1227] Specific operation: The server sends the status to the vehicle terminal.

[1228] Step 6:

[1229] The vehicle terminal automatically clicks the virtual button based on the received status, without any manual operation by the user.

[1230] Input: Status sent to the vehicle terminal (e.g. "Agree" or "Disagree")

[1231] Output: Virtual button clicked

[1232] Specific operation: The vehicle terminal software simulates the clicking of a virtual button based on the received status. The virtual button interface is integrated into the vehicle terminal UI.

[1233] Through the above processing steps, the system of the present invention enables users to quickly and appropriately consent to cookies and privacy policies in autonomous vehicles.

[1234] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1235] The present invention relates to a system that enables users to properly understand the content of consent popups for cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. The system includes a natural language processing means, an extraction means, a matching means, a means for executing consent actions, and an emotion engine that recognizes the user's emotions.

[1236] Program processing

[1237] Program Overview

[1238] 1. User:

[1239] When you first visit the website, a pop-up will appear explaining the cookies and privacy policy.

[1240] 2. Terminal:

[1241] Gets the text of the displayed popup and sends it to the server.

[1242] 3. Server:

[1243] The received text is passed to a natural language processing (NLP) module for analysis.

[1244] The NLP module performs text tokenization, part-of-speech tagging, and dependency analysis to extract specific keywords and phrases.

[1245] 4. Server:

[1246] Matches extracted keywords and phrases against a pre-configured allow list.

[1247] The allow list may include items such as "No use for advertising purposes" and "Measurement for site analytics is allowed."

[1248] 5. Server:

[1249] A matching result is generated and a determination is made as to whether or not there is a match. If there is a match, a status of "Agree" is generated; if there is no match, a status of "Disagree" is generated.

[1250] 6. Terminal:

[1251] The matching results are received from the server and the emotion engine is activated. The emotion engine analyzes the user's text input, voice input, and facial expressions to recognize the user's emotions.

[1252] 7. Server:

[1253] Tailor your consent or disagreement actions based on the perceived emotion, for example automatically clicking the disagree button or showing a detailed explanation if the user expresses anxiety or doubt.

[1254] 8. Terminal:

[1255] Based on the judgment results of the emotion engine, the agree or disagree button is automatically clicked.

[1256] Specific examples

[1257] Example 1:

[1258] 1. User:

[1259] You visit an e-commerce website for the first time and a pop-up appears saying:

[1260] "We use cookies for advertising purposes. Do you agree?"

[1261] 2. Terminal:

[1262] Get the popup text "We use cookies for advertising purposes. Do you agree?" and send it to the server.

[1263] 3. Server:

[1264] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "advertising," "purpose," and "cookies" through tokenization, part-of-speech tagging, and dependency analysis.

[1265] 4. Server:

[1266] The extracted keywords are compared with the allow list. Since the allow list includes a setting that says "Use for advertising purposes is prohibited," the result of the comparison is "Disapproval."

[1267] 5. Terminal:

[1268] The matching result "disagree" is received from the server and the emotion engine is activated.

[1269] 6. Terminal:

[1270] The emotion engine analyzes the user's text and voice input and recognizes that the user has doubts, such as "Why rejection?"

[1271] 7. Server:

[1272] Automatically clicking the "disagree" button in a pop-up based on perceived sentiment,

[1273] Example 2:

[1274] 1. User:

[1275] You visit a news site for the first time and a pop-up appears saying:

[1276] "This site uses cookies to analyze and improve it. Do you agree?"

[1277] 2. Terminal:

[1278] The text of the popup "This site uses cookies for analysis and improvement purposes. Do you agree?" is obtained and sent to the server.

[1279] 3. Server:

[1280] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "analysis," "improvement," and "cookie" through tokenization, part-of-speech tagging, and dependency analysis.

[1281] 4. Server:

[1282] The extracted keywords are compared with the allow list. Since the allow list is set to "Allow measurement for site analysis," the result is "Agree."

[1283] 5. Terminal:

[1284] The matching result "Agreement" is received from the server and the emotion engine is activated.

[1285] 6. Terminal:

[1286] The emotion engine analyzes the user's facial expressions and recognizes, for example, that the user is feeling at ease.

[1287] 7. Server:

[1288] Automatically click the "Agree" button on a pop-up based on perceived sentiment.

[1289] In this way, the system of the present invention streamlines the cookie consent process when users visit websites, and by taking into account user sentiment in addition to pre-defined policies, it achieves more appropriate and intuitive consent management.

[1290] The processing flow will be explained below.

[1291] Step 1:

[1292] User:

[1293] Visit a website for the first time. Upon arrival, the site displays a pop-up about cookies and privacy policies.

[1294] Step 2:

[1295] Device:

[1296] Gets the text of the popup that was displayed.

[1297] Specifically, JavaScript's document.querySelector is used to extract the popup text elements.

[1298] Step 3:

[1299] Device:

[1300] The retrieved text is sent to the server.

[1301] Specifically, Ajax is used to send text to the server via a POST request.

[1302] Step 4:

[1303] server:

[1304] The received text is passed to a natural language processing (NLP) module.

[1305] Specifically, it calls libraries such as Python's spaCy and NLTK.

[1306] Step 5:

[1307] server:

[1308] The NLP module analyzes the text and extracts specific keywords using techniques such as tokenization, part-of-speech tagging, and dependency analysis.

[1309] Specifically, the text is divided into tokens, each token is tagged with a part-of-speech tag, and dependency analysis is performed.

[1310] Step 6:

[1311] server:

[1312] The extracted keywords are compared against a pre-defined allow list.

[1313] Specifically, the extracted keywords are compared with each item in the permission list to see if they match.

[1314] Step 7:

[1315] server:

[1316] A matching result is generated, and if there is a match, a status of "Agree" is generated, and if there is no match, a status of "Disagree."

[1317] Specifically, the status is determined based on the conditions in the permission list, and the result is returned as a response.

[1318] Step 8:

[1319] Device:

[1320] Receives the status received from the server. When the status is received, it starts the emotion engine.

[1321] Step 9:

[1322] Device:

[1323] The emotion engine analyzes the user's text input, voice input, or facial expressions to recognize the user's emotions.

[1324] Specifically, when analyzing voice input, it uses voice recognition technology, and when analyzing text input, it analyzes the input string. For facial expression analysis, it analyzes the user's facial expressions from camera footage.

[1325] Step 10:

[1326] server:

[1327] Tailor actions to agree or disagree based on perceived sentiment.

[1328] For example, if a user expresses concerns or doubts, we can provide a detailed explanation or automatically click a no consent button.

[1329] Step 11:

[1330] Device:

[1331] Based on the judgment results of the emotion engine, an action of agreement or disagreement is taken.

[1332] Specifically, the corresponding button (agree or disagree) is clicked using the JavaScript click method.

[1333] Step 12:

[1334] User:

[1335] Confirm that the pop-up was interacted with and continue using the website.

[1336] As a result, proper consent management is carried out without the user having to perform the consent operation themselves.

[1337] Example 2

[1338] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1339] In recent years, users are required to understand the content of cookie and privacy policy consent popups that appear when visiting websites and make a quick decision. However, current systems often cause confusion and bewilderment when users choose to consent or not. In addition, consent management does not take user emotions into consideration, resulting in a poor user experience.

[1340] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords and phrases from the text analyzed by the natural language processing means, means for comparing the extracted keywords and phrases with a preset permission list, means for automatically performing an action of consent or disagreement based on the comparison result, and means for recognizing the user's emotions and adjusting the action of consent or disagreement based on the recognition result. This allows the user to properly understand the contents of cookies and privacy policies and make quick and appropriate decisions, improving the user experience.

[1341] "Natural language processing means" is a technology that analyzes text obtained from a website and extracts specific keywords and phrases by tokenizing words and phrases, tagging them by part of speech, and performing dependency analysis.

[1342] "Keyword and phrase extraction means" refers to a technique for identifying and collecting specific words and phrases from text analyzed by natural language processing means.

[1343] An "allow list" is a pre-defined list of specific keywords or phrases that define the conditions under which a decision to accept or reject them will be made.

[1344] "Matching methods" are techniques that compare extracted keywords and phrases with items on an allow list to determine whether they match.

[1345] A "match result" is a determination of whether keywords and phrases match the allow list, and is information used to generate a consent or disagreement status.

[1346] "Means for automatically performing an action of consent or denial" refers to technology that recreates the state in which a user clicks the consent or denial button on a website based on the results of the comparison.

[1347] "Means for recognizing user emotions" refers to technology that analyzes the user's facial expressions, voice input, text input, etc. to determine the user's emotional state.

[1348] The "means for adjusting actions based on emotions" is a technology that provides additional information or changes actions when automatically taking an action of agreement or disagreement based on the results of user emotion recognition.

[1349] MODE FOR CARRYING OUT THE INVENTION

[1350] The present invention relates to a system that helps users quickly and appropriately decide whether to agree or disagree to pop-ups about cookies and privacy policies that appear when visiting a website. The system includes a natural language processing means, an extraction means, a matching means, a means for executing an agreement action, and an emotion engine that recognizes the user's emotions.

[1351] Specifically, it is configured as follows:

[1352] 1. Website visits and pop-ups

[1353] Users: When they first visit a website, they will see a pop-up explaining cookies and privacy policies, such as "We use cookies for advertising purposes. Do you agree?"

[1354] 2. Getting and sending popup text

[1355] Terminal: The text of the displayed popup is obtained and sent to the server. The software used generally uses the browser's API.

[1356] 3. Text Analysis

[1357] Server: Passes the received text to a natural language processing (NLP) module for analysis, using a natural language processing library such as SpaCy or NLTK.

[1358] NLP module: Tokenizes text, tags it with parts of speech, and performs dependency analysis, for example, extracting important keywords and phrases such as "advertising," "purpose," and "cookies."

[1359] 4. Keyword Matching

[1360] Server: The extracted keywords and phrases are compared against a pre-defined allow list, which may include items such as "no advertising use" or "allowed for site analytics measurement."

[1361] 5. Judgment of matching results

[1362] Server: Based on the match, generate a "Agree" or "Disagree" status. For example, if the match matches "Not for advertising purposes", it will be determined as a disagreement.

[1363] 6. Activating the Emotional Engine

[1364] Device: Receives the matching results from the server and activates the emotion engine, which uses Google Cloud Vision and Microsoft Azure's Emotion API to analyze the user's facial expressions, voice input, and text input.

[1365] 7. Adjusting actions based on emotions

[1366] Server: Automatically adjust the action of agreeing or disagreeing based on the user's emotion recognition results. For example, if the user expresses anxiety or doubt, display a detailed explanation or automatically click the "disagree" button.

[1367] 8. Automated consent / non-consent decisions

[1368] Device: Automatically clicks the "Agree" or "Disagree" button on the pop-up based on the emotion engine's judgment.

[1369] Specific examples

[1370] Below is a specific scenario using this system.

[1371] Example 1: E-commerce site

[1372] User: Visits an e-commerce website for the first time and sees a popup saying "We use cookies for advertising purposes. Do you agree?"

[1373] Terminal: Get this popup text and send it to the server.

[1374] Server: Passes the text to a natural language processing (NLP) module, which analyzes it for keywords such as "advertising," "purpose," and "cookies."

[1375] Server: Compares the analysis results with the permission list and issues a "deny" decision, stating that "use for advertising purposes is not permitted."

[1376] Device: Receives the matching results, activates the emotion engine, and analyzes the user's facial expressions and voice.

[1377] Server: As a result of the sentiment engine, it recognizes that the user is expressing doubt and automatically clicks the "disagree" button.

[1378] Example 2: News site

[1379] User: Visits a news site for the first time and sees a pop-up that says "We use cookies to help us analyze and improve our site. Do you agree?"

[1380] Terminal: Get this popup text and send it to the server.

[1381] Server: Passes the text to a natural language processing (NLP) module, which analyzes it for keywords like "analysis," "improvement," and "cookie."

[1382] Server: Compares the analysis results with the allow list and determines "Agree" by stating "Measurement for site analysis is permitted."

[1383] Terminal: Receives the matching results, activates the emotion engine, and analyzes the user's facial expressions.

[1384] Server: As a result of the emotion engine, it recognizes that the user feels safe and automatically clicks the "Agree" button.

[1385] In this way, the system of the present invention helps users properly understand the content of cookies and privacy policies and make quick decisions, which greatly improves the user experience.

[1386] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1387] Step 1:

[1388] User: When a user first visits a website, a popup explaining cookies and privacy policy is displayed. Input is a user visiting a website. Output is a popup being generated.

[1389] Step 2:

[1390] Terminal: Get the text of the displayed popup and send it to the server. This process includes the specific operation of extracting the popup text from the DOM (Document Object Model). The input is the displayed popup text, and the output is the text data to be sent to the server.

[1391] Step 3:

[1392] Server: Passes the received text to a natural language processing (NLP) module for analysis. Specifically, it uses natural language processing libraries such as SpaCy and NLTK. In processing by the NLP module, text data is provided as input, and the results of tokenization, part-of-speech tagging, and dependency analysis are generated as output.

[1393] Step 4:

[1394] Server: Extracts specific keywords and phrases from the text analyzed by the NLP module. This includes programs that identify and collect important words and phrases. The input is the analyzed text data, and the output is the extracted keywords and phrases.

[1395] Step 5:

[1396] Server: Compares extracted keywords and phrases against a pre-configured allow list, which may contain items such as "not for advertising purposes" or "allowed for site analytics measurement." The input is the extracted keywords and phrases and the allow list, and the output is the match result.

[1397] Step 6:

[1398] Server: Generates a status of "agree" or "disagree" based on the matching result. The input is the matching result, and the output is the status of agree or disagree.

[1399] Step 7:

[1400] Device: Receives the status obtained from the matching result and starts the emotion engine. The emotion engine uses Google Cloud Vision or Microsoft Azure's Emotion API to analyze the user's facial expressions, voice input, and text input. The input is the status of agreement or disagreement, and the output is the emotion recognition result.

[1401] Step 8:

[1402] Server: Automatically adjusts the action of agreeing or disagreeing based on the user's emotion recognition results. For example, if the user expresses anxiety or doubt, it displays a detailed explanation or automatically clicks the "disagree" button. The input is the emotion recognition result, and the output is the adjusted action.

[1403] Step 9:

[1404] Terminal: Automatically clicks the "Agree" or "Disagree" button in a pop-up based on the emotion engine's judgment. The input is the coordinated action, and the output is the actual button click.

[1405] (Application example 2)

[1406] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1407] Conventional consent popups for cookies and privacy policies on websites make it difficult for users to understand their content and quickly decide whether to consent or not. In particular, little consideration is given to the user's psychological burden and level of understanding, resulting in a non-intuitive consent process. Furthermore, users are often asked to consent while feeling anxious or suspicious, which reduces the quality of the user experience.

[1408] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords from the text analyzed by the natural language processing means, and means for comparing the extracted keywords with a preset allow list. This makes it possible to streamline the cookie consent process when a user visits a website and enables more appropriate and intuitive consent management by taking into account the user's feelings in addition to preset policies.

[1409] "Natural language processing means" refers to technical means that analyzes text data obtained from websites and performs processes such as tokenization, part-of-speech tagging, and dependency analysis.

[1410] The "means for extracting specific keywords" refers to a technical means for extracting predefined important words or phrases from text analyzed by natural language processing means.

[1411] "Means for checking against an allow list" refers to a technical means for comparing extracted keywords with a pre-established allow list to determine whether they match.

[1412] "Means for automatically performing consent or denial actions" refers to technical means for automatically performing consent or denial actions based on the results of a match against an allow list.

[1413] "Emotion analysis means" refers to a technical means for reading emotions from the user's input text, voice, facial expressions, etc., and recognizing their emotional state.

[1414] The "means for adjusting consent or disagreement actions" refers to technical means for optimizing consent or disagreement actions based on the recognized user sentiment.

[1415] In this invention, when a user visits a website for the first time, a pop-up message explaining cookies and privacy policies is displayed. The terminal obtains the text of the pop-up message and sends it to the server.

[1416] The server then analyzes the retrieved text using natural language processing tools (e.g., HuggingFace's Transformers library), which involve tokenizing the text, tagging it as a part of speech, and performing dependency analysis to extract specific keywords and phrases.

[1417] The extracted keywords are then checked against a pre-defined allow list, which may include items such as "no advertising use" or "allowed for site analytics measurement." Based on the result of the check, a "yes" or "no" status is generated for a particular policy.

[1418] The device then receives the matching results from the server and activates an emotion analyzer (e.g., an emotion analysis model in the HuggingFace Transformers library) that analyzes the user's text input, voice input, or facial expressions to recognize the user's emotions.

[1419] Furthermore, the server adjusts the action of agreeing or disagreeing based on the perceived emotion: for example, if the user expresses anxiety or doubt, it automatically clicks the disagree button or displays a detailed explanation, etc. Similarly, if the user feels reassured, it automatically clicks the agree button.

[1420] Specific examples

[1421] Example 1:

[1422] 1. A user visits a shopping site for the first time and is greeted with a pop-up that reads: "We use cookies for advertising purposes. Do you agree?"

[1423] 2. The device retrieves the popup text and sends it to the server.

[1424] 3. The server performs natural language processing to extract keywords from the text, such as "advertising," "purpose," and "cookie."

[1425] 4. The server checks the extracted keywords against the allow list. Since the allow list includes a setting that says "not for advertising purposes," the result is "disapproval."

[1426] 5. The device receives the matching results from the server and activates the emotion analysis function. It recognizes that the user has doubts, such as "Why was it rejected?"

[1427] 6. The "Disagree" button in the pop-up is automatically clicked based on the perceived sentiment.

[1428] Prompt Sentence Examples

[1429] "Please develop a system that uses natural language processing to analyze how users feel about the privacy policy pop-up that appears for the first time on a website they visit, and helps them decide whether to agree or disagree based on that information."

[1430] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1431] Step 1:

[1432] When a user visits a website for the first time, the device gets a popup with a description of cookies and privacy policy. The input is the display data of the web page, and the output is the text data of the popup. This text data is sent to the server for further processing.

[1433] Step 2:

[1434] The server analyzes the text sent from the terminal using a natural language processing means (NLP module). The input is the text data of the popup, and the output is the analysis data generated by tokenization, part-of-speech tagging, and dependency analysis. Specifically, the natural language processing means tokenizes the text, tags each token with a part-of-speech tag, and analyzes dependencies.

[1435] Step 3:

[1436] The server extracts specific keywords and phrases from the analysis data. The input is the analysis data from the natural language processing tool, and the output is a list of extracted keywords. This clarifies the main intent and purpose of the popup.

[1437] Step 4:

[1438] The server compares the extracted keywords with a pre-defined allow list. The input is the list of extracted keywords and the allow list, and the output is the match result (agreement or disagreement status). Specifically, it compares whether the keywords match the items in the allow list, and generates a consent status based on the result.

[1439] Step 5:

[1440] The terminal receives the matching results from the server and activates the emotion analysis means. The inputs are the matching results and the user's text input, voice input, or facial expression data, and the output is the recognized emotion. The emotion analysis means analyzes the user's emotion based on the input data and recognizes their state.

[1441] Step 6:

[1442] The server adjusts the action of agreeing or disagreeing based on the recognized emotion. The input is the recognized emotion and the matching result, and the output is the final action (agreeing or disagreeing). For example, if the user expresses anxiety or doubt, a detailed explanation is displayed, and if there is no anxiety, the agree button is automatically clicked.

[1443] Step 7:

[1444] The terminal automatically clicks the accept or decline button based on the final action received from the server. The input is the final action status from the server, and the output is the operation (clicking accept or decline) based on that status.

[1445] This process allows users to quickly and appropriately respond to pop-ups about cookies and privacy policies.

[1446] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1447] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1448] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1449] [Fourth embodiment]

[1450] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1451] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1452] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1453] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1454] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1455] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1456] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1457] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1458] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1459] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1460] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1461] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1462] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1463] The present invention relates to a system that enables users to appropriately understand the content of consent popups regarding cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. The system includes a natural language processing unit, an extraction unit, a matching unit, and a set of units for executing consent actions.

[1464] Program processing

[1465] Program Overview

[1466] 1. User:

[1467] When you first visit the website, a pop-up will appear explaining the cookies and privacy policy.

[1468] 2. Terminal:

[1469] Gets the text of the displayed popup and sends it to the server.

[1470] 3. Server:

[1471] The received text is passed to a natural language processing (NLP) module for analysis.

[1472] The NLP module performs text tokenization, part-of-speech tagging, and dependency analysis to extract specific keywords and phrases.

[1473] 4. Server:

[1474] Matches extracted keywords and phrases against a pre-configured allow list.

[1475] The allow list may include items such as "No use for advertising purposes" and "Measurement for site analytics is allowed."

[1476] 5. Server:

[1477] A matching result is generated and a determination is made as to whether or not there is a match. If there is a match, a status of "Agree" is generated; if there is no match, a status of "Disagree" is generated.

[1478] 6. Terminal:

[1479] Automatically clicks the accept or decline button based on the status received from the server.

[1480] Specific examples

[1481] Example 1:

[1482] 1. User:

[1483] You visit an e-commerce website for the first time and a pop-up appears saying:

[1484] "We use cookies for advertising purposes. Do you agree?"

[1485] 2. Terminal:

[1486] Get the popup text "We use cookies for advertising purposes. Do you agree?" and send it to the server.

[1487] 3. Server:

[1488] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "advertising," "purpose," and "cookies" through tokenization, part-of-speech tagging, and dependency analysis.

[1489] 4. Server:

[1490] The extracted keywords are compared with the allow list. Since the allow list includes a setting that says "Use for advertising purposes is prohibited," the result of the comparison is "Disapproval."

[1491] 5. Terminal:

[1492] Based on the server's matching result of "disagree," the "disagree" button on the pop-up will be automatically clicked.

[1493] Example 2:

[1494] 1. User:

[1495] You visit a news site for the first time and a pop-up appears saying:

[1496] "This site uses cookies to analyze and improve it. Do you agree?"

[1497] 2. Terminal:

[1498] The text of the popup "This site uses cookies for analysis and improvement purposes. Do you agree?" is obtained and sent to the server.

[1499] 3. Server:

[1500] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "analysis," "improvement," and "cookie" through tokenization, part-of-speech tagging, and dependency analysis.

[1501] 4. Server:

[1502] The extracted keywords are compared with the allow list. Since the allow list is set to "Allow measurement for site analysis," the result is "Agree."

[1503] 5. Terminal:

[1504] Based on the matching result "Agree" from the server, the "Agree" button on the pop-up will be automatically clicked.

[1505] In this way, the system of the present invention streamlines the cookie consent process when users visit websites and automatically takes appropriate action based on pre-defined policies, thereby improving the user experience.

[1506] The processing flow will be explained below.

[1507] Step 1:

[1508] User:

[1509] Visit a website for the first time. Upon arrival, the site displays a pop-up about cookies and privacy policies.

[1510] Step 2:

[1511] Device:

[1512] Gets the text of the popup that was displayed.

[1513] Specifically, JavaScript's document.querySelector is used to extract the popup text elements.

[1514] Step 3:

[1515] Device:

[1516] The retrieved text is sent to the server.

[1517] Specifically, Ajax is used to send text to the server via a POST request.

[1518] Step 4:

[1519] server:

[1520] The received text is passed to a natural language processing (NLP) module.

[1521] Specifically, it calls libraries such as Python's spaCy and NLTK.

[1522] Step 5:

[1523] server:

[1524] The NLP module analyzes the text and extracts specific keywords using techniques such as tokenization and part-of-speech tagging.

[1525] Specifically, the text is divided into tokens, each token is tagged with a part-of-speech tag, and dependency analysis is performed.

[1526] Step 6:

[1527] server:

[1528] The extracted keywords are compared against a pre-defined allow list.

[1529] Specifically, the extracted keywords are compared with each item in the permission list to see if they match.

[1530] Step 7:

[1531] server:

[1532] A matching result is generated, and if there is a match, a status of "Agree" is generated, and if there is no match, a status of "Disagree."

[1533] Specifically, the status is determined based on the conditions in the permission list, and the result is returned as a response.

[1534] Step 8:

[1535] Device:

[1536] Based on the status received from the server, an action of consent or non-consent is automatically taken.

[1537] Specifically, the corresponding button (agree or disagree) is clicked using the JavaScript click method.

[1538] Step 9:

[1539] User:

[1540] Confirm that the pop-up was interacted with and continue using the website.

[1541] As a result, proper consent management is carried out without the user having to perform the consent operation themselves.

[1542] Example 1

[1543] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1544] When users visit a website, they may find it difficult to properly understand the cookie and privacy policy consent pop-up and quickly decide whether to consent or not. This situation not only impairs the user experience, but also may result in users not making appropriate privacy choices. Furthermore, manually performing this series of operations is time-consuming and cumbersome for users. The present invention aims to solve this problem.

[1545] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1546] In this invention, the server includes a natural language processing unit for analyzing text acquired from a website, a unit for transmitting the acquired text to the server, and a unit for the server to generate a consent status based on the text analysis result, thereby streamlining the cookie consent process when a user visits a website and automatically taking appropriate action based on a pre-defined policy.

[1547] "Natural language processing means" is a technology that analyzes text obtained from websites and understands their grammatical structure and meaning.

[1548] "Keyword extraction means" is a technique for selecting important words and phrases from text analyzed by natural language processing means.

[1549] The "allowed list matching means" is a technology that compares extracted keywords and phrases with a pre-set allowed list to determine whether they match.

[1550] The "means for automatically executing consent action" is a technology that automatically performs consent or non-consent operations on behalf of the user based on the collation results.

[1551] The "text transmission means" is a technique for transmitting the acquired pop-up text to the server.

[1552] The "agreement status generation means" is a technique in which the server generates a status of "agree" or "disagree" based on the collation result.

[1553] The "automatic button clicking means" is a technology that automatically clicks a corresponding button on the terminal based on the generated consent status.

[1554] A "server" is a central computer system that processes and analyzes data.

[1555] A "terminal" is a device operated by a user, which receives instructions from a server and executes operations.

[1556] The present invention relates to a system that enables users to properly understand the content of consent popups regarding cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. This system is designed to operate in cooperation with three main entities: a server, a terminal, and a user.

[1557] System configuration

[1558] Hardware and software used

[1559] 1. Server

[1560] The server acts as the center of data analysis and recognition. The specific software used includes a natural language processing (NLP) module, such as TensorFlow.

[1561] 2. Terminal

[1562] A device is a user-operated device, such as a smartphone, tablet, or computer, that has software installed on it to capture the text of the displayed popup and send it to a server for analysis.

[1563] 3. Users

[1564] A user visits a website for the first time, and the system automatically performs certain actions via the device operated by the user.

[1565] System Operation

[1566] 1. User Behavior

[1567] The first time a user visits a particular website, they will see a consent popup regarding cookies and privacy policies.

[1568] 2. Terminal Processing

[1569] The device detects the popup text, retrieves its contents, and sends it to the server. Specifically, the popup text is retrieved from the DOM using JavaScript and asynchronously sent to the server using AJAX.

[1570] 3. Server Processing

[1571] The server passes the received text to a natural language processing (NLP) module, which tokenizes the text, tags it with parts of speech, and performs dependency analysis to extract specific keywords and phrases.

[1572] The extracted keywords are checked against a pre-defined allow list, which includes items such as "not for advertising purposes" and "allowed for measurement for site analytics."

[1573] Based on the matching result, the server generates a status of "agree" or "disagree" and returns it to the terminal.

[1574] 4. Automatic actions on the device

[1575] The device will automatically click the accept or decline button in the pop-up based on the status received from the server, saving the user the trouble of manually clicking the button.

[1576] Specific examples

[1577] Example 1:

[1578] 1. User Behavior

[1579] A user visits an e-commerce site for the first time and is greeted with a pop-up message saying, "We use cookies for advertising purposes. Do you agree?"

[1580] 2. Terminal Processing

[1581] Gets the popup text and sends it to the server.

[1582] 3. Server Processing

[1583] The server passes the text to the NLP module for analysis. The extracted keywords "advertising", "purpose", and "cookies" are checked against the allow list. Since the allow list includes "not allowed for advertising purposes", the server generates a "disagree" status.

[1584] 4. Automatic actions on the device

[1585] The device will automatically click the "Decline" button in the pop-up.

[1586] Example 2:

[1587] 1. User Behavior

[1588] A user visits a news site for the first time and is greeted with a pop-up message saying, "We use cookies to help us analyze and improve our site. Do you agree?"

[1589] 2. Terminal Processing

[1590] Gets the popup text and sends it to the server.

[1591] 3. Server Processing

[1592] The server passes the text to the NLP module for analysis. The extracted keywords "analysis," "improvement," and "cookies" are checked against the allow list. Since the allow list includes "Allow measurement for site analytics," the server generates an "Agree" status.

[1593] 4. Automatic actions on the device

[1594] The device will automatically click the "Agree" button in the pop-up.

[1595] Example prompts for generative AI models

[1596] "Please parse the content of the following popup and generate an accept or decline status based on your allow list: 'We use cookies for advertising purposes. Do you agree?'"

[1597] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1598] Step 1:

[1599] A user visits a website

[1600] Specific behavior: A user enters a URL into the address bar of a web browser and presses the Enter key.

[1601] Input: The URL entered by the user.

[1602] Output: The web page loads and displays the cookie and privacy policy consent popup.

[1603] Step 2:

[1604] The device detects the popup and retrieves the text

[1605] What happens: The browser's JavaScript monitors the DOM and detects the presence of a popup.

[1606] Parse the text node to get the popup text.

[1607] Input: The DOM structure of the displayed web page.

[1608] Output: The text of the popup (e.g. "We use cookies for advertising purposes. Do you agree?").

[1609] Step 3:

[1610] The device sends the text to the server

[1611] Specific behavior: Encodes the popup text and sends it to the server as an HTTP request.

[1612] AJAX is used for asynchronous communication.

[1613] Input: The text of the popup.

[1614] Output: The HTTP request sent to the server.

[1615] Step 4:

[1616] The server passes the text to a natural language processing (NLP) module

[1617] Specific operation: The server passes the received text as input to the NLP module.

[1618] The NLP module uses frameworks such as TensorFlow.

[1619] Input: The text of the HTTP request body.

[1620] Output: The text data passed to the NLP module.

[1621] Step 5:

[1622] NLP module analyzes the text

[1623] What it does: The NLP module tokenizes the text, tags it with parts of speech, and performs dependency analysis.

[1624] Further analysis is carried out to extract keywords and phrases.

[1625] Input: Text data passed from the server.

[1626] Output: Extracted keywords and phrases (e.g. "advertising", "purpose", "cookies").

[1627] Step 6:

[1628] The server checks the analysis results against the permission list

[1629] What it does: The server compares the extracted keywords and phrases with a pre-defined allow list.

[1630] An allow list might consist of, for example, "No use for advertising purposes" or "Measurement for site analytics is allowed."

[1631] Input: Extracted keywords or phrases, allow lists.

[1632] Output: Matching result (e.g., matched "Not for advertising purposes").

[1633] Step 7:

[1634] The server generates the consent status

[1635] Specific operation: The server generates a consent status (e.g., "consent" or "disagree") based on the matching result.

[1636] If the matching result matches the conditions of the allow list, an "Agree" status is generated; if not, a "Disagree" status is generated.

[1637] Input: Matching result.

[1638] Output: The consent status (e.g. "Disagree").

[1639] Step 8:

[1640] Automatic button click based on the status received by the device from the server

[1641] Specific operation: The terminal analyzes the consent status received from the server and automatically clicks the corresponding button ("Agree" or "Disagree").

[1642] Use JavaScript to fire a click event.

[1643] Input: consent status.

[1644] Output: Automatic click action of the corresponding button.

[1645] (Application example 1)

[1646] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1647] In autonomous vehicles, it is important for passengers to respond promptly and appropriately to cookie and privacy policy consent popups. However, manually performing these consent operations during autonomous driving not only degrades the user experience but can also be a distraction. To solve this problem, a system is needed that automates cookie and privacy policy consent actions, ensuring a safe and comfortable ride for passengers.

[1648] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1649] In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords from the text analyzed by the natural language processing means, and means for comparing the extracted keywords with a preset permission list. This enables the user's consent operation in an autonomous vehicle to be safely and quickly performed by a system including means for automatically processing a consent pop-up displayed on a vehicle terminal, means for analyzing text acquired from the vehicle terminal and comparing it with permission items, and means for automatically clicking a virtual button based on the comparison result.

[1650] "Natural language processing means" refers to means for carrying out a series of processes to analyze acquired text data and extract meanings and keywords.

[1651] The "means for extracting specific keywords" refers to a means for identifying and extracting important words and phrases from text analyzed by the natural language processing means.

[1652] An "allow list" is a pre-defined list of specific keywords or phrases that are allowed.

[1653] A "matching method" is a method for comparing extracted keywords or phrases with an allowed list to determine whether they match or mismatch.

[1654] "Automatic execution means" refers to a means for taking an action of consent or disagreement based on the matching results without user intervention.

[1655] "Means for automatically clicking virtual buttons" means means for a system to simulate the act of pressing an "agree" or "disagree" button through a program without the user having to manually operate the system.

[1656] A "vehicle terminal" is a terminal such as a computer or display device located inside an autonomous driving vehicle.

[1657] A "consent popup" is a short notification screen that appears when a website or application asks users to consent to the use of cookies or privacy policy.

[1658] The present invention relates to a system for quickly and appropriately handling cookie and privacy policy consent popups in an autonomous vehicle, including natural language processing, keyword extraction, permission list matching, and automatic action execution.

[1659] This system uses the following hardware and software:

[1660] Hardware:

[1661] Vehicle terminals in autonomous vehicles (e.g., HMDs with displays and camera functions)

[1662] Server (using cloud services, typically AWS, etc.)

[1663] software:

[1664] Natural language processing libraries (e.g., SpaCy, NLTK)

[1665] Server application (implemented in Python)

[1666] Cloud storage (e.g. Amazon S3)

[1667] The outline of the processing procedure is as follows.

[1668] 1. When a user gets into an autonomous vehicle, a cookie and privacy policy consent pop-up will appear on the vehicle's device.

[1669] 2. The vehicle device captures the text of the displayed popup using its camera function and sends that text to the server.

[1670] 3. The server analyzes the received text using a natural language processing module, tokenizing it, tagging it as a part of speech, and performing dependency analysis, thereby extracting important keywords such as "cookie" and "privacy."

[1671] 4. The extracted keywords are compared against an allow list, which may include items such as "Analysis of driving data is allowed" or "Use for advertising purposes is prohibited."

[1672] 5. Based on the matching result, the server generates a status of "agree" or "disagree" and sends it to the vehicle terminal.

[1673] 6. The device will automatically click the virtual button "Agree" or "Disagree" based on the received status.

[1674] Specific examples are shown below.

[1675] Example 1: When a user gets into an autonomous vehicle and a pop-up message appears on the HMD saying, "This vehicle uses cookies to analyze communication data. Do you agree?", the following prompt sentence is input into the AI ​​model.

[1676] "Analyze the following text and extract keywords: 'This vehicle uses cookies to analyze communication data. Do you agree?'"

[1677] The server extracts the keywords "communication data analysis," "cookies," and "consent" from the analysis results and compares them with the permission list. In this case, the setting is "permit driving data analysis," so the comparison result is "consent." The HMD on the vehicle terminal automatically clicks the "consent" button.

[1678] In this way, the system of the present invention enables the user's consent operation to be performed safely and quickly in an autonomous vehicle.

[1679] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1680] Step 1:

[1681] When a user gets into an autonomous vehicle, a consent pop-up regarding cookies and privacy policies is displayed on the vehicle's head-mounted display (HMD). This pop-up contains specific text.

[1682] Input: Cookie and privacy policy consent popup text displayed by the user

[1683] Output: Image data for capturing popup text

[1684] Specific operation: The HMD's camera function is activated and the text of the displayed popup is captured as image data.

[1685] Step 2:

[1686] The vehicle terminal processes the captured image data and extracts text, which is then sent to the server.

[1687] Input: Image data of the captured popup text

[1688] Output: Text data (e.g. "This vehicle uses cookies to analyze communication data. Do you agree?")

[1689] What it does: Image processing software extracts text from the captured image and sends it to a server, such as Tesseract OCR.

[1690] Step 3:

[1691] The server passes the received text data to a natural language processing (NLP) module for analysis, which includes tokenization, part-of-speech tagging, and dependency analysis. Keywords such as "communication data analysis," "cookies," and "consent" are extracted.

[1692] Input: Received text data

[1693] Output: Extracted keywords (e.g., "communication data analysis," "cookies," "consent")

[1694] Specific operation: The NLP module analyzes the text. The NLP library used is, for example, SpaCy.

[1695] Step 4:

[1696] The server compares the extracted keywords with a pre-defined permission list, which may include items such as "driving data analysis is allowed" or "usage for advertising purposes is prohibited," and generates a consent or denial status based on the match.

[1697] Input: Extracted keywords (e.g., "communication data analysis," "cookies," "consent") and allow list

[1698] Output: Matching result (e.g., "Agree" or "Disagree")

[1699] What happens: The server checks the keywords against the allow list, which is stored in a database, and generates a status.

[1700] Step 5:

[1701] The server transmits the generated status of consent or disagreement to the vehicle terminal.

[1702] Input: The status of the match (e.g., "Agree" or "Disagree")

[1703] Output: Status sent to vehicle terminal

[1704] Specific operation: The server sends the status to the vehicle terminal.

[1705] Step 6:

[1706] The vehicle terminal automatically clicks the virtual button based on the received status, without any manual operation by the user.

[1707] Input: Status sent to the vehicle terminal (e.g. "Agree" or "Disagree")

[1708] Output: Virtual button clicked

[1709] Specific operation: The vehicle terminal software simulates the clicking of a virtual button based on the received status. The virtual button interface is integrated into the vehicle terminal UI.

[1710] Through the above processing steps, the system of the present invention enables users to quickly and appropriately consent to cookies and privacy policies in autonomous vehicles.

[1711] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1712] The present invention relates to a system that enables users to properly understand the content of consent popups for cookies and privacy policies that are displayed when visiting a website and quickly decide whether to consent or not. The system includes a natural language processing means, an extraction means, a matching means, a means for executing consent actions, and an emotion engine that recognizes the user's emotions.

[1713] Program processing

[1714] Program Overview

[1715] 1. User:

[1716] When you first visit the website, a pop-up will appear explaining the cookies and privacy policy.

[1717] 2. Terminal:

[1718] Gets the text of the displayed popup and sends it to the server.

[1719] 3. Server:

[1720] The received text is passed to a natural language processing (NLP) module for analysis.

[1721] The NLP module performs text tokenization, part-of-speech tagging, and dependency analysis to extract specific keywords and phrases.

[1722] 4. Server:

[1723] Matches extracted keywords and phrases against a pre-configured allow list.

[1724] The allow list may include items such as "No use for advertising purposes" and "Measurement for site analytics is allowed."

[1725] 5. Server:

[1726] A matching result is generated and a determination is made as to whether or not there is a match. If there is a match, a status of "Agree" is generated; if there is no match, a status of "Disagree" is generated.

[1727] 6. Terminal:

[1728] The matching results are received from the server and the emotion engine is activated. The emotion engine analyzes the user's text input, voice input, and facial expressions to recognize the user's emotions.

[1729] 7. Server:

[1730] Tailor your consent or disagreement actions based on the perceived emotion, for example automatically clicking the disagree button or showing a detailed explanation if the user expresses anxiety or doubt.

[1731] 8. Terminal:

[1732] Based on the judgment results of the emotion engine, the agree or disagree button is automatically clicked.

[1733] Specific examples

[1734] Example 1:

[1735] 1. User:

[1736] You visit an e-commerce website for the first time and a pop-up appears saying:

[1737] "We use cookies for advertising purposes. Do you agree?"

[1738] 2. Terminal:

[1739] Get the popup text "We use cookies for advertising purposes. Do you agree?" and send it to the server.

[1740] 3. Server:

[1741] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "advertising," "purpose," and "cookies" through tokenization, part-of-speech tagging, and dependency analysis.

[1742] 4. Server:

[1743] The extracted keywords are compared with the allow list. Since the allow list includes a setting that says "Use for advertising purposes is prohibited," the result of the comparison is "Disapproval."

[1744] 5. Terminal:

[1745] The matching result "disagree" is received from the server and the emotion engine is activated.

[1746] 6. Terminal:

[1747] The emotion engine analyzes the user's text and voice input and recognizes that the user has doubts, such as "Why rejection?"

[1748] 7. Server:

[1749] Automatically clicking the "disagree" button in a pop-up based on perceived sentiment,

[1750] Example 2:

[1751] 1. User:

[1752] You visit a news site for the first time and a pop-up appears saying:

[1753] "This site uses cookies to analyze and improve it. Do you agree?"

[1754] 2. Terminal:

[1755] The text of the popup "This site uses cookies for analysis and improvement purposes. Do you agree?" is obtained and sent to the server.

[1756] 3. Server:

[1757] The text is passed to a natural language processing (NLP) module, which extracts keywords such as "analysis," "improvement," and "cookie" through tokenization, part-of-speech tagging, and dependency analysis.

[1758] 4. Server:

[1759] The extracted keywords are compared with the allow list. Since the allow list is set to "Allow measurement for site analysis," the result is "Agree."

[1760] 5. Terminal:

[1761] The matching result "Agreement" is received from the server and the emotion engine is activated.

[1762] 6. Terminal:

[1763] The emotion engine analyzes the user's facial expressions and recognizes, for example, that the user is feeling at ease.

[1764] 7. Server:

[1765] Automatically click the "Agree" button on a pop-up based on perceived sentiment.

[1766] In this way, the system of the present invention streamlines the cookie consent process when users visit websites, and by taking into account user sentiment in addition to pre-defined policies, it achieves more appropriate and intuitive consent management.

[1767] The processing flow will be explained below.

[1768] Step 1:

[1769] User:

[1770] Visit a website for the first time. Upon arrival, the site displays a pop-up about cookies and privacy policies.

[1771] Step 2:

[1772] Device:

[1773] Gets the text of the popup that was displayed.

[1774] Specifically, JavaScript's document.querySelector is used to extract the popup text elements.

[1775] Step 3:

[1776] Device:

[1777] The retrieved text is sent to the server.

[1778] Specifically, Ajax is used to send text to the server via a POST request.

[1779] Step 4:

[1780] server:

[1781] The received text is passed to a natural language processing (NLP) module.

[1782] Specifically, it calls libraries such as Python's spaCy and NLTK.

[1783] Step 5:

[1784] server:

[1785] The NLP module analyzes the text and extracts specific keywords using techniques such as tokenization, part-of-speech tagging, and dependency analysis.

[1786] Specifically, the text is divided into tokens, each token is tagged with a part-of-speech tag, and dependency analysis is performed.

[1787] Step 6:

[1788] server:

[1789] The extracted keywords are compared against a pre-defined allow list.

[1790] Specifically, the extracted keywords are compared with each item in the permission list to see if they match.

[1791] Step 7:

[1792] server:

[1793] A matching result is generated, and if there is a match, a status of "Agree" is generated, and if there is no match, a status of "Disagree."

[1794] Specifically, the status is determined based on the conditions in the permission list, and the result is returned as a response.

[1795] Step 8:

[1796] Device:

[1797] Receives the status received from the server. When the status is received, it starts the emotion engine.

[1798] Step 9:

[1799] Device:

[1800] The emotion engine analyzes the user's text input, voice input, or facial expressions to recognize the user's emotions.

[1801] Specifically, when analyzing voice input, it uses voice recognition technology, and when analyzing text input, it analyzes the input string. For facial expression analysis, it analyzes the user's facial expressions from camera footage.

[1802] Step 10:

[1803] server:

[1804] Tailor actions to agree or disagree based on perceived sentiment.

[1805] For example, if a user expresses concerns or doubts, we can provide a detailed explanation or automatically click a no consent button.

[1806] Step 11:

[1807] Device:

[1808] Based on the judgment results of the emotion engine, an action of agreement or disagreement is taken.

[1809] Specifically, the corresponding button (agree or disagree) is clicked using the JavaScript click method.

[1810] Step 12:

[1811] User:

[1812] Confirm that the pop-up was interacted with and continue using the website.

[1813] As a result, proper consent management is carried out without the user having to perform the consent operation themselves.

[1814] Example 2

[1815] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1816] In recent years, users are required to understand the content of cookie and privacy policy consent popups that appear when visiting websites and make a quick decision. However, current systems often cause confusion and bewilderment when users choose to consent or not. In addition, consent management does not take user emotions into consideration, resulting in a poor user experience.

[1817] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords and phrases from the text analyzed by the natural language processing means, means for comparing the extracted keywords and phrases with a preset permission list, means for automatically performing an action of consent or disagreement based on the comparison result, and means for recognizing the user's emotions and adjusting the action of consent or disagreement based on the recognition result. This allows the user to properly understand the contents of cookies and privacy policies and make quick and appropriate decisions, improving the user experience.

[1818] "Natural language processing means" is a technology that analyzes text obtained from a website and extracts specific keywords and phrases by tokenizing words and phrases, tagging them by part of speech, and performing dependency analysis.

[1819] "Keyword and phrase extraction means" refers to a technique for identifying and collecting specific words and phrases from text analyzed by natural language processing means.

[1820] An "allow list" is a pre-defined list of specific keywords or phrases that define the conditions under which a decision to accept or reject them will be made.

[1821] "Matching methods" are techniques that compare extracted keywords and phrases with items on an allow list to determine whether they match.

[1822] A "match result" is a determination of whether keywords and phrases match the allow list, and is information used to generate a consent or disagreement status.

[1823] "Means for automatically performing an action of consent or denial" refers to technology that recreates the state in which a user clicks the consent or denial button on a website based on the results of the comparison.

[1824] "Means for recognizing user emotions" refers to technology that analyzes the user's facial expressions, voice input, text input, etc. to determine the user's emotional state.

[1825] The "means for adjusting actions based on emotions" is a technology that provides additional information or changes actions when automatically taking an action of agreement or disagreement based on the results of user emotion recognition.

[1826] MODE FOR CARRYING OUT THE INVENTION

[1827] The present invention relates to a system that helps users quickly and appropriately decide whether to agree or disagree to pop-ups about cookies and privacy policies that appear when visiting a website. The system includes a natural language processing means, an extraction means, a matching means, a means for executing an agreement action, and an emotion engine that recognizes the user's emotions.

[1828] Specifically, it is configured as follows:

[1829] 1. Website visits and pop-ups

[1830] Users: When they first visit a website, they will see a pop-up explaining cookies and privacy policies, such as "We use cookies for advertising purposes. Do you agree?"

[1831] 2. Getting and sending popup text

[1832] Terminal: The text of the displayed popup is obtained and sent to the server. The software used generally uses the browser's API.

[1833] 3. Text Analysis

[1834] Server: Passes the received text to a natural language processing (NLP) module for analysis, using a natural language processing library such as SpaCy or NLTK.

[1835] NLP module: Tokenizes text, tags it with parts of speech, and performs dependency analysis, for example, extracting important keywords and phrases such as "advertising," "purpose," and "cookies."

[1836] 4. Keyword Matching

[1837] Server: The extracted keywords and phrases are compared against a pre-defined allow list, which may include items such as "no advertising use" or "allowed for site analytics measurement."

[1838] 5. Judgment of matching results

[1839] Server: Based on the match, generate a "Agree" or "Disagree" status. For example, if the match matches "Not for advertising purposes", it will be determined as a disagreement.

[1840] 6. Activating the Emotional Engine

[1841] Device: Receives the matching results from the server and activates the emotion engine, which uses Google Cloud Vision and Microsoft Azure's Emotion API to analyze the user's facial expressions, voice input, and text input.

[1842] 7. Adjusting actions based on emotions

[1843] Server: Automatically adjust the action of agreeing or disagreeing based on the user's emotion recognition results. For example, if the user expresses anxiety or doubt, display a detailed explanation or automatically click the "disagree" button.

[1844] 8. Automated consent / non-consent decisions

[1845] Device: Automatically clicks the "Agree" or "Disagree" button on the pop-up based on the emotion engine's judgment.

[1846] Specific examples

[1847] Below is a specific scenario using this system.

[1848] Example 1: E-commerce site

[1849] User: Visits an e-commerce website for the first time and sees a popup saying "We use cookies for advertising purposes. Do you agree?"

[1850] Terminal: Get this popup text and send it to the server.

[1851] Server: Passes the text to a natural language processing (NLP) module, which analyzes it for keywords such as "advertising," "purpose," and "cookies."

[1852] Server: Compares the analysis results with the permission list and issues a "deny" decision, stating that "use for advertising purposes is not permitted."

[1853] Device: Receives the matching results, activates the emotion engine, and analyzes the user's facial expressions and voice.

[1854] Server: As a result of the sentiment engine, it recognizes that the user is expressing doubt and automatically clicks the "disagree" button.

[1855] Example 2: News site

[1856] User: Visits a news site for the first time and sees a pop-up that says "We use cookies to help us analyze and improve our site. Do you agree?"

[1857] Terminal: Get this popup text and send it to the server.

[1858] Server: Passes the text to a natural language processing (NLP) module, which analyzes it for keywords like "analysis," "improvement," and "cookie."

[1859] Server: Compares the analysis results with the allow list and determines "Agree" by stating "Measurement for site analysis is permitted."

[1860] Terminal: Receives the matching results, activates the emotion engine, and analyzes the user's facial expressions.

[1861] Server: As a result of the emotion engine, it recognizes that the user feels safe and automatically clicks the "Agree" button.

[1862] In this way, the system of the present invention helps users properly understand the content of cookies and privacy policies and make quick decisions, which greatly improves the user experience.

[1863] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1864] Step 1:

[1865] User: When a user first visits a website, a popup explaining cookies and privacy policy is displayed. Input is a user visiting a website. Output is a popup being generated.

[1866] Step 2:

[1867] Terminal: Get the text of the displayed popup and send it to the server. This process includes the specific operation of extracting the popup text from the DOM (Document Object Model). The input is the displayed popup text, and the output is the text data to be sent to the server.

[1868] Step 3:

[1869] Server: Passes the received text to a natural language processing (NLP) module for analysis. Specifically, it uses natural language processing libraries such as SpaCy and NLTK. In processing by the NLP module, text data is provided as input, and the results of tokenization, part-of-speech tagging, and dependency analysis are generated as output.

[1870] Step 4:

[1871] Server: Extracts specific keywords and phrases from the text analyzed by the NLP module. This includes programs that identify and collect important words and phrases. The input is the analyzed text data, and the output is the extracted keywords and phrases.

[1872] Step 5:

[1873] Server: Compares extracted keywords and phrases against a pre-configured allow list, which may contain items such as "not for advertising purposes" or "allowed for site analytics measurement." The input is the extracted keywords and phrases and the allow list, and the output is the match result.

[1874] Step 6:

[1875] Server: Generates a status of "agree" or "disagree" based on the matching result. The input is the matching result, and the output is the status of agree or disagree.

[1876] Step 7:

[1877] Device: Receives the status obtained from the matching result and starts the emotion engine. The emotion engine uses Google Cloud Vision or Microsoft Azure's Emotion API to analyze the user's facial expressions, voice input, and text input. The input is the status of agreement or disagreement, and the output is the emotion recognition result.

[1878] Step 8:

[1879] Server: Automatically adjusts the action of agreeing or disagreeing based on the user's emotion recognition results. For example, if the user expresses anxiety or doubt, it displays a detailed explanation or automatically clicks the "disagree" button. The input is the emotion recognition result, and the output is the adjusted action.

[1880] Step 9:

[1881] Terminal: Automatically clicks the "Agree" or "Disagree" button in a pop-up based on the emotion engine's judgment. The input is the coordinated action, and the output is the actual button click.

[1882] (Application example 2)

[1883] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1884] Conventional consent popups for cookies and privacy policies on websites make it difficult for users to understand their content and quickly decide whether to consent or not. In particular, little consideration is given to the user's psychological burden and level of understanding, resulting in a non-intuitive consent process. Furthermore, users are often asked to consent while feeling anxious or suspicious, which reduces the quality of the user experience.

[1885] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes natural language processing means for analyzing text acquired from a website, means for extracting specific keywords from the text analyzed by the natural language processing means, and means for comparing the extracted keywords with a preset allow list. This makes it possible to streamline the cookie consent process when a user visits a website and enables more appropriate and intuitive consent management by taking into account the user's feelings in addition to preset policies.

[1886] "Natural language processing means" refers to technical means that analyzes text data obtained from websites and performs processes such as tokenization, part-of-speech tagging, and dependency analysis.

[1887] The "means for extracting specific keywords" refers to a technical means for extracting predefined important words or phrases from text analyzed by natural language processing means.

[1888] "Means for checking against an allow list" refers to a technical means for comparing extracted keywords with a pre-established allow list to determine whether they match.

[1889] "Means for automatically performing consent or denial actions" refers to technical means for automatically performing consent or denial actions based on the results of a match against an allow list.

[1890] "Emotion analysis means" refers to a technical means for reading emotions from the user's input text, voice, facial expressions, etc., and recognizing their emotional state.

[1891] The "means for adjusting consent or disagreement actions" refers to technical means for optimizing consent or disagreement actions based on the recognized user sentiment.

[1892] In this invention, when a user visits a website for the first time, a pop-up message explaining cookies and privacy policies is displayed. The terminal obtains the text of the pop-up message and sends it to the server.

[1893] The server then analyzes the retrieved text using natural language processing tools (e.g., HuggingFace's Transformers library), which involve tokenizing the text, tagging it as a part of speech, and performing dependency analysis to extract specific keywords and phrases.

[1894] The extracted keywords are then checked against a pre-defined allow list, which may include items such as "no advertising use" or "allowed for site analytics measurement." Based on the result of the check, a "yes" or "no" status is generated for a particular policy.

[1895] The device then receives the matching results from the server and activates an emotion analyzer (e.g., an emotion analysis model in the HuggingFace Transformers library) that analyzes the user's text input, voice input, or facial expressions to recognize the user's emotions.

[1896] Furthermore, the server adjusts the action of agreeing or disagreeing based on the perceived emotion: for example, if the user expresses anxiety or doubt, it automatically clicks the disagree button or displays a detailed explanation, etc. Similarly, if the user feels reassured, it automatically clicks the agree button.

[1897] Specific examples

[1898] Example 1:

[1899] 1. A user visits a shopping site for the first time and is greeted with a pop-up that reads: "We use cookies for advertising purposes. Do you agree?"

[1900] 2. The device retrieves the popup text and sends it to the server.

[1901] 3. The server performs natural language processing to extract keywords from the text, such as "advertising," "purpose," and "cookie."

[1902] 4. The server checks the extracted keywords against the allow list. Since the allow list includes a setting that says "not for advertising purposes," the result is "disapproval."

[1903] 5. The device receives the matching results from the server and activates the emotion analysis function. It recognizes that the user has doubts, such as "Why was it rejected?"

[1904] 6. The "Disagree" button in the pop-up is automatically clicked based on the perceived sentiment.

[1905] Prompt Sentence Examples

[1906] "Please develop a system that uses natural language processing to analyze how users feel about the privacy policy pop-up that appears for the first time on a website they visit, and helps them decide whether to agree or disagree based on that information."

[1907] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1908] Step 1:

[1909] When a user visits a website for the first time, the device gets a popup with a description of cookies and privacy policy. The input is the display data of the web page, and the output is the text data of the popup. This text data is sent to the server for further processing.

[1910] Step 2:

[1911] The server analyzes the text sent from the terminal using a natural language processing means (NLP module). The input is the text data of the popup, and the output is the analysis data generated by tokenization, part-of-speech tagging, and dependency analysis. Specifically, the natural language processing means tokenizes the text, tags each token with a part-of-speech tag, and analyzes dependencies.

[1912] Step 3:

[1913] The server extracts specific keywords and phrases from the analysis data. The input is the analysis data from the natural language processing tool, and the output is a list of extracted keywords. This clarifies the main intent and purpose of the popup.

[1914] Step 4:

[1915] The server compares the extracted keywords with a pre-defined allow list. The input is the list of extracted keywords and the allow list, and the output is the match result (agreement or disagreement status). Specifically, it compares whether the keywords match the items in the allow list, and generates a consent status based on the result.

[1916] Step 5:

[1917] The terminal receives the matching results from the server and activates the emotion analysis means. The inputs are the matching results and the user's text input, voice input, or facial expression data, and the output is the recognized emotion. The emotion analysis means analyzes the user's emotion based on the input data and recognizes their state.

[1918] Step 6:

[1919] The server adjusts the action of agreeing or disagreeing based on the recognized emotion. The input is the recognized emotion and the matching result, and the output is the final action (agreeing or disagreeing). For example, if the user expresses anxiety or doubt, a detailed explanation is displayed, and if there is no anxiety, the agree button is automatically clicked.

[1920] Step 7:

[1921] The terminal automatically clicks the accept or decline button based on the final action received from the server. The input is the final action status from the server, and the output is the operation (clicking accept or decline) based on that status.

[1922] This process allows users to quickly and appropriately respond to pop-ups about cookies and privacy policies.

[1923] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1924] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1925] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1926] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1927] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1928] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1929] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1930] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1931] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1932] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1933] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1934] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1935] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1936] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1937] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1938] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1939] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1940] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1941] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1942] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1943] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1944] The following is further disclosed regarding the above embodiment.

[1945] (Claim 1)

[1946] natural language processing means for analyzing text retrieved from a website;

[1947] means for extracting specific keywords from the text analyzed by the natural language processing means;

[1948] means for checking the extracted keywords against a pre-defined allow list;

[1949] The system includes a means to automatically take consent or non-consent action based on the match result.

[1950] (Claim 2)

[1951] 2. The system according to claim 1, further comprising means for returning the collation result to the terminal and automatically clicking an agree or disagree button on the terminal.

[1952] (Claim 3)

[1953] 2. The system of claim 1, wherein the natural language processing means includes means for performing text tokenization, part-of-speech tagging, and dependency analysis.

[1954] "Example 1"

[1955] (Claim 1)

[1956] natural language processing means for analyzing text retrieved from a website;

[1957] means for extracting specific keywords from the text analyzed by the natural language processing means;

[1958] means for checking the extracted keywords against a pre-defined allow list;

[1959] A means for automatically taking consent or non-consent actions based on the matching results;

[1960] means for transmitting the retrieved text to a server;

[1961] a means for the server to generate a consent status based on the text analysis result;

[1962] A means to automatically click a button based on the generated status;

[1963] A system including:

[1964] (Claim 2)

[1965] 2. The system according to claim 1, further comprising means for returning the collation result to the terminal and automatically clicking an agree or disagree button on the terminal.

[1966] (Claim 3)

[1967] 2. The system of claim 1, wherein the natural language processing means includes means for performing text tokenization, part-of-speech tagging, and dependency analysis.

[1968] "Application Example 1"

[1969] (Claim 1)

[1970] natural language processing means for analyzing text retrieved from a website;

[1971] means for extracting specific keywords from the text analyzed by the natural language processing means;

[1972] means for checking the extracted keywords against a pre-defined allow list;

[1973] A means for automatically taking consent or non-consent actions based on the matching results;

[1974] means for automatically handling consent pop-ups displayed on the vehicle terminal;

[1975] means for analyzing the text acquired from the vehicle terminal and comparing it with permission items;

[1976] A means for automatically clicking a virtual button based on the matching results;

[1977] A system including:

[1978] (Claim 2)

[1979] 2. The system according to claim 1, further comprising means for returning the collation result to the vehicle terminal and automatically clicking a virtual button indicating whether to agree or disagree on the vehicle terminal.

[1980] (Claim 3)

[1981] 2. The system of claim 1, wherein the natural language processing means includes means for performing text tokenization, part-of-speech tagging, and dependency analysis.

[1982] "Example 2: Combining Emotion Engines"

[1983] (Claim 1)

[1984] natural language processing means for analyzing text retrieved from a website;

[1985] means for extracting specific keywords and phrases from the text analyzed by the natural language processing means;

[1986] means for matching the extracted keywords and phrases with a pre-defined allow list;

[1987] A means for automatically taking consent or non-consent actions based on the matching results;

[1988] A system that includes a means for recognizing a user's emotions and adjusting actions of agreement or disagreement based on the recognition results.

[1989] (Claim 2)

[1990] 2. The system according to claim 1, further comprising means for returning the collation result to the terminal and automatically clicking an agree or disagree button on the terminal.

[1991] (Claim 3)

[1992] 2. The system of claim 1, wherein the natural language processing means includes means for performing text tokenization, part-of-speech tagging, and dependency analysis.

[1993] "Application example 2 when combining emotion engines"

[1994] (Claim 1)

[1995] natural language processing means for analyzing text retrieved from a website;

[1996] means for extracting specific keywords from the text analyzed by the natural language processing means;

[1997] means for checking the extracted keywords against a pre-defined allow list;

[1998] A means for automatically taking consent or non-consent actions based on the matching results;

[1999] emotion analysis means for recognizing the emotion of a user;

[2000] The system includes means for adjusting an action of agreement or disagreement based on the emotion recognized by the emotion analysis means.

[2001] (Claim 2)

[2002] 2. The system according to claim 1, further comprising means for returning the collation result to the terminal and automatically clicking an agree or disagree button on the terminal.

[2003] (Claim 3)

[2004] 2. The system of claim 1, wherein the natural language processing means includes means for performing text tokenization, part-of-speech tagging, and dependency analysis. [Explanation of symbols]

[2005] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. natural language processing means for analyzing text retrieved from a website; means for extracting specific keywords from the text analyzed by the natural language processing means; means for checking the extracted keywords against a pre-defined allow list; The system includes a means to automatically take consent or non-consent action based on the match result.

2. 2. The system according to claim 1, further comprising means for returning the collation result to the terminal and automatically clicking an "agree" or "disagree" button on the terminal.

3. 2. The system of claim 1, wherein the natural language processing means includes means for performing text tokenization, part-of-speech tagging, and dependency analysis.

Images