system

Abstract

We provide the system. [Solution] Means for acquiring images, A means for extracting feature information from acquired images, A means for obtaining information from the internet and identifying similar information by comparing it with the obtained characteristic information, A means of notifying the administrator of the relevant information, Means for processing the relevant information, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Due to the spread of social media platforms on the Internet, there is an increasing possibility that photos taken unintentionally by individuals are made public and privacy is violated. For such problems, there is a need for a method to quickly and efficiently identify relevant photos and protect privacy.

Means for Solving the Problems

[0005] This invention provides a system for extracting feature information based on images provided by users and identifying similar information by comparing it with photographs publicly available on the internet. This system protects user privacy by notifying administrators of the identified information and processing it as necessary. It also includes a function to continuously monitor information on the internet and prevent unauthorized disclosure of user information.

[0006] "Means for acquiring images" refers to devices or programs that have the function of receiving digital images transmitted by a user.

[0007] "Means for extracting feature information from acquired images" refers to techniques for analyzing and generating unique features from received image data.

[0008] "Means for acquiring information from the internet and identifying similar information by comparing it with the obtained characteristic information" refers to a system that collects publicly available information from internet databases and websites and matches it with existing characteristic information.

[0009] "Means for notifying the administrator of relevant information" refers to devices or programs that have the function of electronically reporting identified information or problems to human administrators.

[0010] "Means for processing the relevant information" refers to technical means that address privacy protection and other issues by making changes to the identified information.

[0011] "Means for continuously monitoring and updating information" refers to technologies that perform a process of checking information periodically or in real time and updating the system as soon as new data is detected.

[0012] "Means for standardizing images" refers to techniques used to bring image data of different formats and resolutions into a consistent standard. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] An example of an embodiment of the system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is a system in which users, servers, and terminals cooperate to efficiently protect the privacy of personal photographs on online platforms. Specifically, the invention will be implemented in the following forms.

[0035] First, the user saves a photo of their face to a digital device and launches a dedicated privacy blocker application. Through this application, the user can upload the image to the system, which serves as the initial input data.

[0036] Next, the server receives the image sent by the user and performs a standardization process on it. From the standardized image data, the server extracts feature information using facial recognition technology. This feature information digitally represents the unique elements of the image and is used for comparison with existing photographic data available on the internet.

[0037] The server, while connected to the internet, collects photo data published from designated social media and other sites. This data is compared with extracted feature information, and an AI-powered process is executed to identify similar photos.

[0038] If an image that may have violated a user's privacy is detected, the server will activate a function to notify the administrator of this information. Furthermore, the server can execute protocols to take privacy-protecting measures for the detected image, such as sending a request to automatically apply mosaic blurring.

[0039] Ultimately, the server generates a report detailing the processing results and any detected issues, and sends a notification to the user. This notification includes information about the platform where the identified photos reside, allowing the user to understand their own privacy situation. The server can also continuously monitor the internet and repeat the same process whenever it detects the publication of new photos.

[0040] In this way, the invention aims to proactively protect users' digital privacy and minimize the risk of unintentional image disclosure.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The user saves a photo of their face to their smartphone or PC. They then launch a dedicated privacy blocker application on their device and select an image using its interface. The user then uploads the selected image to the system.

[0044] Step 2:

[0045] The server receives images sent by users. After receiving the images, it performs image standardization processing and converts them into a format suitable for processing. Image standardization refers to changing the format and adjusting the resolution.

[0046] Step 3:

[0047] The server performs facial recognition analysis on standardized images. It extracts feature information from the images and uses this information to generate a unique digital "fingerprint." This step analyzes the unique features of the images.

[0048] Step 4:

[0049] The server connects to the internet and accesses the database of the specified social media platform. The server uses the extracted feature information to search the internet for photos of the same or similar faces.

[0050] Step 5:

[0051] The server analyzes identified photos on social media to determine if they may be infringing on users' privacy. If similar photos are found, the server automatically sends a notification to the administrator.

[0052] Step 6:

[0053] The server will perform privacy-protecting processing on the discovered photos. Specifically, it will send requests to the social media administrator to blur the photos or restrict their public access.

[0054] Step 7:

[0055] The server aggregates the processing results and sends a report to the user. The user can view this report on their device and learn about the platforms on which their facial image was found and their detailed information.

[0056] Step 8:

[0057] The server will continue to monitor new images on the internet, maintaining a continuous process to protect user privacy. This continuous monitoring ensures that users' privacy is protected over the long term.

[0058] (Example 1)

[0059] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0060] The unauthorized publication of personal photographs online constitutes a violation of privacy. This situation is particularly prevalent on social media platforms where photos are shared widely, highlighting the need for effective privacy protection. Traditional methods have been insufficient for monitoring and addressing this issue.

[0061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0062] In this invention, the server includes means for acquiring images, means for standardizing the acquired images and extracting feature information, and means for collecting information from the Internet and identifying similar information by comparing it with the extracted feature information. This makes it possible to efficiently identify images that may infringe on an individual's privacy and to quickly take protective measures against them.

[0063] "Means for acquiring images" refers to the process and tools that allow a user to acquire their own digital images or photographs using a device and upload them to the system.

[0064] "Means for standardizing acquired images and extracting feature information" refers to the process of extracting specific facial information or unique visual features using image processing techniques that convert received image data into an appropriate format and enable consistent analysis.

[0065] "Means for collecting information from the internet and identifying similar information by comparing it with extracted feature information" refers to algorithms and processes for identifying photographs similar to a user's image by collecting image data accessible online and matching it with feature information extracted by the system.

[0066] "Means for sending notifications to administrators when identified information may constitute a privacy violation" refers to a communication system and protocol that transmits appropriate warnings to administrators when similar images related to a user's privacy are found.

[0067] "Means of processing identified information to protect privacy" refers to technologies and methods for applying mosaic effects or concealing information in images to reduce the risk of discovery.

[0068] "Means for detecting new image publications by continuously collecting and updating information from the internet" refers to an automated system that constantly monitors newly published image data on the internet and detects privacy risks based on regularly updated information.

[0069] This invention is an online surveillance system that protects individual privacy, and its functionality is realized through the cooperation of the user, terminal, and server. The user uploads their facial photograph to a privacy blocker application using the terminal. The terminal is responsible for receiving the image in this step and transferring it to the server in an appropriate format.

[0070] The server standardizes the received images and extracts feature information using specific facial recognition technology. This process utilizes advanced image processing software and AI technology. Specifically, a generative AI model used for model training is employed, and this technology effectively supports feature information extraction and similar image identification.

[0071] Next, the server connects to the internet and collects image data from specified websites and social media platforms. Using this collected data and extracted feature information, the AI ​​identifies similar images and assesses the risks related to user privacy.

[0072] If similar images are found, the server sends a notification to the administrator, who can then apply privacy-protecting measures such as blurring to the detected image information as needed. The resulting report is sent to the user, allowing them to understand their privacy situation and take necessary actions.

[0073] As a concrete example, the following prompt can be used for a generative AI model: "Please upload a photo of your face to the system. If an image that may violate privacy is detected, what measures will you take?"

[0074] This invention aims to effectively protect users' digital privacy and prevent unintended public disclosure of images.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] The user launches a privacy blocker application on their device, selects a photo of their face, and uploads it to the system. The input data is the face photo selected by the user, and the output is the transmission of image data from the device to the server. The application checks the file format to ensure the image is in the correct format and performs any necessary conversions.

[0078] Step 2:

[0079] The server receives image data transmitted from the terminal. The input is image data from the terminal, and the output is standardized image data. The server adjusts the image resolution and performs standardization processes such as unifying the color space. This allows for more efficient subsequent feature information extraction processes.

[0080] Step 3:

[0081] The server extracts feature information from standardized images using facial recognition technology. The input is standardized image data, and the output is a feature information vector. The server utilizes a generative AI model to identify unique facial features within the image and represents them as numerical data. This data is used in subsequent comparison processing.

[0082] Step 4:

[0083] The server collects photo data from specified websites and social media on the internet. The input is an internet connection and a list of target sites, and the output is a dataset of collected images. The server efficiently collects data using automated crawling technology.

[0084] Step 5:

[0085] The server compares the collected image data with extracted feature information. The input is the collected image data and user feature information, and the output is the identification result of similar images. The AI ​​algorithm calculates the similarity of the images and assesses the potential risks related to user privacy.

[0086] Step 6:

[0087] The server automatically notifies the administrator and takes measures to protect privacy when similar images are found. The input is the identified similar images and their metadata, and the output is a notification report and the execution of corrective actions. The server mitigates privacy risks by requesting mosaic processing on detected images.

[0088] Step 7:

[0089] The server generates a report of the results of all processing and sends a notification to the user. The input is the processing result information, and the output is a detailed report. Through this notification, the user can understand their privacy status in real time and take further action if necessary.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] In our digital society, the risk of personal image data being unintentionally published on the internet and resulting in privacy violations is increasing. However, because users lack the means to proactively manage their own information and take immediate countermeasures, this problem needs to be solved quickly and efficiently.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] In this invention, the server includes a device for acquiring images, a device for extracting attribute information from the acquired images, and a device for acquiring data on a communication network and identifying similar data by comparing it with the obtained attribute information. This makes it possible for users to monitor the online publication status of their own image data in real time and to proactively protect their personal privacy.

[0095] A "device for acquiring images" is a system that uses electronic devices to collect image data from users.

[0096] A "device for extracting attribute information from acquired images" is a mechanism that uses image processing technology to analyze and extract characteristic information from images.

[0097] A "device for acquiring data on a communication network and identifying similar data by comparing it with the obtained attribute information" is a mechanism that searches a database on the internet and determines similarity by comparing it with previously extracted attribute information.

[0098] A "device for notifying the administrator of the relevant data" is a means of communication for promptly informing the system administrator of data in which similarity has been detected.

[0099] "A device for processing the relevant data" refers to a mechanism that automatically performs appropriate processing on identified data to protect privacy.

[0100] A "device for sending notifications to user terminals" is a system that sends information to an individual's terminal in order to directly inform the user of the detection results.

[0101] A "device for providing interactive prompt notifications using mobile devices" is an interactive notification system that presents users with options and confirmations via portable electronic devices such as smartphones and tablets.

[0102] A "device for continuously monitoring data and reflecting changes" is a device that constantly monitors new data and changes on a communication network and quickly reflects the results in the system.

[0103] A "device for standardizing images" is a device that converts image data of different formats into a unified format to improve processing efficiency.

[0104] This invention is a system for efficiently protecting the privacy of users' personal images online. First, the user uses a mobile device, including a smartphone, to launch a dedicated application. This application functions as an "image acquisition device" and is responsible for acquiring the user's facial image and uploading it to a server.

[0105] The server processes the received image data using the AWS® Rekognition API. It acts as a "device for extracting attribute information from acquired images," extracting facial feature information in digital format. This feature information is then used with the Google® Cloud Vision API to scan the internet, functioning as a "device for acquiring data on communication networks and identifying similar data by comparing it with the obtained attribute information."

[0106] If similar images are detected, the server uses Firebase to function as a "device for sending notifications to the user's device" in real time, providing relevant information. The notification is then sent to the user's smartphone via an interactive prompt, acting as a "device for providing interactive prompt notifications using mobile devices," presenting the user with confirmations and options.

[0107] For example, if a photo of an event attended by a user is published without permission, the system will immediately notify the user as soon as the image is detected and suggest automatic blurring, thereby ensuring rapid privacy protection.

[0108] Example of a prompt:

[0109] "A newly released photo matches your information. Would you like to have it blurred?"

[0110] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0111] Step 1:

[0112] The user launches a dedicated application using their mobile device and uploads a facial image by taking or selecting one. The input at this stage is the user's facial image data, and the output is the transfer of that image data to the server. Through this process, the system obtains image data that forms the basis for privacy protection.

[0113] Step 2:

[0114] The server analyzes the received facial image data using the AWS Rekognition API and extracts feature information. The input here is the user's facial image data, and the output is digital data represented as feature information. This specific operation yields attribute information that enables image identification.

[0115] Step 3:

[0116] The server uses the Google Cloud Vision API to scan databases on the internet and search for images that match the feature information. The input for this step is the feature information obtained in the previous step, and the output is whether or not similar images exist. This operation checks whether the user's images have been published without permission.

[0117] Step 4:

[0118] When the server detects the presence of similar images, it sends a notification to the user's mobile device via Firebase. The input is information about the similar image detection, and the output is a warning message delivered to the user. This allows the user to stay informed in real time.

[0119] Step 5:

[0120] Based on notifications received on their smartphones, users select an action in response to displayed prompts. For example, a confirmation prompt might be presented to perform a mosaic effect. The input for this step is a notification message from the server, and the output is the user's chosen action. This action then executes the necessary privacy protection processes.

[0121] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0122] This invention combines a system that monitors publicly available photos on the internet to protect individual privacy with an emotion engine that recognizes user emotions. Specific embodiments are described below.

[0123] First, the user launches a system-specific application using a device such as a smartphone or computer. Here, the user selects and uploads a photo of their face. At this point, an emotion engine recognizes the user's current emotional state from the image. This emotional information is used to more accurately reflect the user's intentions.

[0124] The server receives images sent by users and performs image standardization. Feature information is extracted from these standardized images, and the server begins comparing them with images publicly available on the internet. It accesses databases of specific social media and websites and identifies similar photos based on the extracted feature information.

[0125] The server integrates identified photos with user sentiment information and determines how to process the images based on this. For example, if a user indicates discomfort, stricter privacy protections are applied. The server can also customize notifications to administrators based on sentiment information and take action according to priority and urgency.

[0126] Finally, the server generates a report based on the processing results and notifies the user. The user can review this through their terminal and choose further action if necessary. The server also performs continuous internet monitoring and applies similar protective measures to newly published photos.

[0127] For example, if the emotion engine recognizes a user's anger based on a photo uploaded by the user, the server will immediately blur or request the deletion of the relevant image and prioritize contacting the administrator. In this way, incorporating an emotion engine enables more user-centric privacy protection.

[0128] The following describes the processing flow.

[0129] Step 1:

[0130] The user launches a privacy blocker application on their device and selects a photo of their face. The user then uploads the photo and sends it to the system.

[0131] Step 2:

[0132] The device sends a photo of the user's face to an emotion engine to analyze the user's current emotional state. This emotional information is used to adjust the level of protection in subsequent processes.

[0133] Step 3:

[0134] The server receives the uploaded facial photographs and performs image standardization. This process adjusts the image format and resolution to a format suitable for the recognition model.

[0135] Step 4:

[0136] The server extracts facial feature information from standardized images and generates a digital fingerprint. This fingerprint serves as a standard for detecting photos on the internet.

[0137] Step 5:

[0138] The server accesses the internet and collects photo data from specific social media and websites. This data is then used to search for similar photos by comparing it with extracted facial feature information.

[0139] Step 6:

[0140] When similar photos are detected, the server considers emotional information to determine the appropriate image processing method. For example, if a user indicates discomfort, the server may choose to apply a mosaic effect to the image.

[0141] Step 7:

[0142] The server generates a notification for the administrator based on the user's emotions and identified images. The notification includes the location of the image and recommended actions.

[0143] Step 8:

[0144] The server generates a report of the processing results and sends it to the user. The user can review the report on their terminal and take any further necessary actions.

[0145] Step 9:

[0146] The server continuously monitors the internet and repeats the same process each time a new photo is uploaded, protecting user privacy.

[0147] (Example 2)

[0148] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0149] There is a need to more effectively protect individual privacy regarding images published on the internet and to respond flexibly while considering the emotional state of users. Conventional systems have had difficulty reflecting privacy settings based on users' emotions, resulting in insufficient individual responses.

[0150] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0151] In this invention, the server includes means for acquiring images, means for extracting feature information from acquired images, and means for analyzing emotional states. This makes it possible to determine processing methods based on the user's emotions and to apply appropriate privacy protection measures to images on the internet.

[0152] "Means for acquiring images" refers to elements that have the function of collecting image data using a digital device and converting it into a format that can be processed within the system.

[0153] "Means for extracting feature information from acquired images" refers to elements that perform the process of identifying specific patterns or attributes from image data and extracting information necessary for analysis.

[0154] "Means for acquiring information from the internet and identifying similar information by comparing it with the obtained characteristic information" refers to elements for accessing publicly available online databases and websites, and evaluating similarity by comparing it with the collected characteristic information.

[0155] "Means for notifying the administrator of relevant information" refers to elements that have the function of communicating identified important information or alerts to the administrator or user in an appropriate format.

[0156] "Means for processing the relevant information" refers to elements that have the function of editing or modifying images or data of identified information, in accordance with purposes such as privacy protection.

[0157] "Means for analyzing emotional states" refers to elements that perform a process of analyzing images and data obtained from users and identifying the emotions expressed therein.

[0158] "Means for determining processing methods based on emotional information" refers to an element that has the function of considering the analyzed emotional state and selecting and implementing the most appropriate data or image processing method.

[0159] This invention is a system that utilizes user emotional information to achieve effective privacy protection for images published on the internet.

[0160] Users launch a dedicated application for the system using a device such as a smartphone or PC. Through this application, users can upload a photo of their face. The application uses a built-in emotion engine to analyze the user's current emotional state from this image data. The emotion engine combines an artificial intelligence model and image processing algorithms to analyze the user's facial features and expressions to recognize emotions.

[0161] The server receives the uploaded images and performs image standardization. This standardization process converts the images into a consistent format. The server then uses the feature information extracted from the images to access databases of specific social media and various websites to identify similar images.

[0162] The server integrates identified similar images with the user's analyzed emotional information. This process determines appropriate image processing methods based on the emotional information and implements privacy protection measures for specific images. For example, if the user's emotion is identified as "anxiety," the server may apply a mosaic effect to the image or request its removal from the relevant parties.

[0163] Finally, the server generates a report of these processing results and notifies the user's terminal. The user can review the report and select further actions as needed. The server also continuously monitors the internet and automatically applies protective measures to any newly discovered related images.

[0164] For example, if the emotion of "anger" is analyzed from a photo uploaded by a user, the relevant image will be quickly blurred, and administrators will be notified preferentially. This makes it possible to implement privacy protection that is tailored to the user's emotions.

[0165] An example of a prompt message might be, "Please explain in detail how to analyze the emotions from a user's photo and apply privacy protections to similar publicly available images."

[0166] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0167] Step 1:

[0168] The user launches a system-specific application using a terminal. The user selects a photo of their face and uploads it to the application. At this point, the input is the user's facial image data. The application receives this input data and passes it to the emotion engine. The emotion engine uses image processing technology to analyze the facial expression and classifies the emotional state into categories such as "joy," "sadness," and "anger." At this point, the output is the analyzed emotional information.

[0169] Step 2:

[0170] The server receives facial images and emotion information sent from the user's terminal. The input consists of unstandardized image data and emotion information. The server performs image standardization processing, converting the data into a consistent format. This conversion makes the image data suitable for subsequent processing. The output is standardized image data.

[0171] Step 3:

[0172] The server extracts feature information from standardized image data. This feature information includes facial shape, color, and texture. The input is standardized image data, and the output is the extracted feature information. Based on this feature information, the server accesses a specific online database to search for similar images.

[0173] Step 4:

[0174] The server uses the extracted feature information to compare it with image data publicly available on the internet. The input is the extracted feature information, which is used to identify similar images. This identification process is performed by an image similarity search algorithm. The output is a list of similar images and associated metadata.

[0175] Step 5:

[0176] The server integrates and analyzes identified similar images with the user's emotion information. The input consists of a list of similar images and emotion information. This integration determines how to process the images. For example, if the user's emotion is classified as "anger," the server immediately decides to apply a mosaic effect to the associated images. The output is the determined processing method.

[0177] Step 6:

[0178] The server processes the image based on the determined processing method. The input is a similar image and the processing method. Processing includes applying mosaics and executing deletion requests. The output is the processed image data or a deletion request notification.

[0179] Step 7:

[0180] Finally, the server generates a report based on the processing results and notifies the user. The inputs are the processing results and processing status. The user can review this report via their terminal and select further actions. The output is a notification report to the user, which includes processing details and whether the processing was successful or not.

[0181] (Application Example 2)

[0182] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0183] When images posted online may infringe on a user's privacy against their will, it becomes difficult for users to properly manage that information and protect their privacy. Furthermore, a uniform approach that disregards user feelings will not enable appropriate privacy protection tailored to individual circumstances.

[0184] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0185] In this invention, the server includes a device for acquiring images, a device for analyzing emotions from the acquired images and applying privacy protection measures based on the emotional information, and a device for processing the relevant information. This enables situation-appropriate privacy protection that takes into account the user's emotions.

[0186] A "device for acquiring images" is a device that receives image data from a user, converts it into a format that can be processed within the system, and saves it.

[0187] A "device for extracting feature information from acquired images" is a device that has the function of analyzing and extracting features and patterns necessary for identification and classification from received image data.

[0188] The "function for acquiring information from the internet and identifying similar information by comparing it with the obtained feature information" refers to a function that searches for information on the internet based on the feature information of an image, evaluates its similarity, and identifies the target information.

[0189] A "device for notifying the administrator of relevant information" is a device equipped with communication functions for notifying administrators or users of identified information.

[0190] A "device for processing relevant information" is a device that has the function of processing and modifying image data based on specific conditions.

[0191] A "device for analyzing emotions from acquired images and applying privacy protection measures based on emotional information" is a device that analyzes the emotions of the subjects depicted in an image and determines how to handle and process the data based on the results.

[0192] A "device for continuously monitoring and updating information" is a device that constantly collects and analyzes information on the internet and has the function of updating the relevant information within the system to the latest state.

[0193] A "device for image standardization" is a device that converts images of different formats and sizes into a unified format, thereby facilitating subsequent processing.

[0194] To implement this invention, it is necessary to construct a system that utilizes advanced image processing technology. This system mainly consists of a server and user terminals, with each device performing a specific function. Specifically, it operates as follows:

[0195] The server, acting as an image acquisition device, receives images from the user's terminal. The received image data is converted into a unified format by an image standardization device, and then a device for extracting feature information from the acquired images analyzes and extracts specific patterns and features.

[0196] Next, a function is used to identify similar information by acquiring information from the internet and comparing it with the obtained feature information. This process utilizes TENSORFLOW®, an open-source machine learning library, and is particularly applied to sentiment analysis. This sentiment analysis is performed by a specially trained generative AI model, and sentiment information is acquired simultaneously.

[0197] Furthermore, the device used to process the relevant information will perform actions such as blurring the image based on the privacy settings identified by the system. In addition, if the user indicates discomfort based on emotional information, stricter privacy protection measures will be applied.

[0198] Through the device, users can check the status and processing results of their images in real time. In particular, the results of the emotion engine's analysis are fed back to the user, enabling appropriate responses based on their emotions.

[0199] A concrete example is when a user publishes a particular photo; if the emotion engine detects potential offense before publication, the image is automatically blurred, preventing unprotected publication on social media, etc.

[0200] An example of a prompt message to a generative AI model would be, "I am about to post this image on social media, but I feel uncomfortable with it. Please take this into consideration and appropriately restrict the visibility." Through this prompt message, the generative AI model can analyze the user's intent and make optimal privacy settings.

[0201] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0202] Step 1:

[0203] The user uses their device to launch an application for acquiring images, selects an image, and uploads it. The input is the image data selected by the user, and this image data is sent to the server. The output is the image data transferred to the server.

[0204] Step 2:

[0205] The server performs a process to standardize the received images. The input is the received image data, which is standardized using an image processing library such as OpenCV. The output is the image data converted to a unified format.

[0206] Step 3:

[0207] The server performs a function to extract feature information from standardized images. The input is standardized image data, and a machine learning algorithm is used to extract specific features. The output is the extracted feature data.

[0208] Step 4:

[0209] The server receives image data acquired using a generative AI model and performs sentiment analysis. The input is standardized image data, and the generative AI model analyzes the user's emotions. The output is the user's sentiment information.

[0210] Step 5:

[0211] The server retrieves information from the internet and performs a process to identify similar information by comparing it with the obtained feature information. The input is extracted feature data, and related images on the internet are searched using a similar image search engine. The output is information about the identified similar images.

[0212] Step 6:

[0213] Based on emotional information and similar image information, the server employs means to process the relevant information. The input consists of the user's emotional information and information on identified similar images, and image processing (e.g., mosaic processing) and notification measures are performed to protect privacy. The output consists of the processed image and notification information regarding the processing results.

[0214] Step 7:

[0215] Ultimately, the server notifies the user of the processing results in report format. The input consists of the processed image and the processing results, which are fed back to the user via the terminal. The output is the notification data received by the user.

[0216] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0217] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0218] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0219] [Second Embodiment]

[0220] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0221] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0222] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0223] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0224] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0225] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0226] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0227] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0228] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0229] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0230] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0231] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0232] This invention is a system in which users, servers, and terminals cooperate to efficiently protect the privacy of personal photographs on online platforms. Specifically, the invention will be implemented in the following forms.

[0233] First, the user saves a photo of their face to a digital device and launches a dedicated privacy blocker application. Through this application, the user can upload the image to the system, which serves as the initial input data.

[0234] Next, the server receives the image sent by the user and performs a standardization process on it. From the standardized image data, the server extracts feature information using facial recognition technology. This feature information digitally represents the unique elements of the image and is used for comparison with existing photographic data available on the internet.

[0235] The server, while connected to the internet, collects photo data published from designated social media and other sites. This data is compared with extracted feature information, and an AI-powered process is executed to identify similar photos.

[0236] If an image that may have violated a user's privacy is detected, the server will activate a function to notify the administrator of this information. Furthermore, the server can execute protocols to take privacy-protecting measures for the detected image, such as sending a request to automatically apply mosaic blurring.

[0237] Ultimately, the server generates a report detailing the processing results and any detected issues, and sends a notification to the user. This notification includes information about the platform where the identified photos reside, allowing the user to understand their own privacy situation. The server can also continuously monitor the internet and repeat the same process whenever it detects the publication of new photos.

[0238] In this way, the invention aims to proactively protect users' digital privacy and minimize the risk of unintentional image disclosure.

[0239] The following describes the processing flow.

[0240] Step 1:

[0241] The user saves a photo of their face to their smartphone or PC. They then launch a dedicated privacy blocker application on their device and select an image using its interface. The user then uploads the selected image to the system.

[0242] Step 2:

[0243] The server receives images sent by users. After receiving the images, it performs image standardization processing and converts them into a format suitable for processing. Image standardization refers to changing the format and adjusting the resolution.

[0244] Step 3:

[0245] The server performs facial recognition analysis on standardized images. It extracts feature information from the images and uses this information to generate a unique digital "fingerprint." This step analyzes the unique features of the images.

[0246] Step 4:

[0247] The server connects to the internet and accesses the database of the specified social media platform. The server uses the extracted feature information to search the internet for photos of the same or similar faces.

[0248] Step 5:

[0249] The server analyzes identified photos on social media to determine if they may be infringing on users' privacy. If similar photos are found, the server automatically sends a notification to the administrator.

[0250] Step 6:

[0251] The server will perform privacy-protecting processing on the discovered photos. Specifically, it will send requests to the social media administrator to blur the photos or restrict their public access.

[0252] Step 7:

[0253] The server aggregates the processing results and sends a report to the user. The user can view this report on their device and learn about the platforms on which their facial image was found and their detailed information.

[0254] Step 8:

[0255] The server will continue to monitor new images on the internet, maintaining a continuous process to protect user privacy. This continuous monitoring ensures that users' privacy is protected over the long term.

[0256] (Example 1)

[0257] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0258] The unauthorized publication of personal photographs online constitutes a violation of privacy. This situation is particularly prevalent on social media platforms where photos are shared widely, highlighting the need for effective privacy protection. Traditional methods have been insufficient for monitoring and addressing this issue.

[0259] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0260] In this invention, the server includes means for acquiring images, means for standardizing the acquired images and extracting feature information, and means for collecting information from the Internet and identifying similar information by comparing it with the extracted feature information. This makes it possible to efficiently identify images that may infringe on an individual's privacy and to quickly take protective measures against them.

[0261] "Means for acquiring images" refers to the process and tools that allow a user to acquire their own digital images or photographs using a device and upload them to the system.

[0262] "Means for standardizing acquired images and extracting feature information" refers to the process of extracting specific facial information or unique visual features using image processing techniques that convert received image data into an appropriate format and enable consistent analysis.

[0263] "Means for collecting information from the internet and identifying similar information by comparing it with extracted feature information" refers to algorithms and processes for identifying photographs similar to a user's image by collecting image data accessible online and matching it with feature information extracted by the system.

[0264] "Means for sending notifications to administrators when identified information may constitute a privacy violation" refers to a communication system and protocol that transmits appropriate warnings to administrators when similar images related to a user's privacy are found.

[0265] "Means of processing identified information to protect privacy" refers to technologies and methods for applying mosaic effects or concealing information in images to reduce the risk of discovery.

[0266] "Means for detecting new image publications by continuously collecting and updating information from the internet" refers to an automated system that constantly monitors newly published image data on the internet and detects privacy risks based on regularly updated information.

[0267] This invention is an online surveillance system that protects individual privacy, and its functionality is realized through the cooperation of the user, terminal, and server. The user uploads their facial photograph to a privacy blocker application using the terminal. The terminal is responsible for receiving the image in this step and transferring it to the server in an appropriate format.

[0268] The server standardizes the received images and extracts feature information using specific facial recognition technology. This process utilizes advanced image processing software and AI technology. Specifically, a generative AI model used for model training is employed, and this technology effectively supports feature information extraction and similar image identification.

[0269] Next, the server connects to the internet and collects image data from specified websites and social media platforms. Using this collected data and extracted feature information, the AI ​​identifies similar images and assesses the risks related to user privacy.

[0270] If similar images are found, the server sends a notification to the administrator, who can then apply privacy-protecting measures such as blurring to the detected image information as needed. The resulting report is sent to the user, allowing them to understand their privacy situation and take necessary actions.

[0271] As a concrete example, the following prompt can be used for a generative AI model: "Please upload a photo of your face to the system. If an image that may violate privacy is detected, what measures will you take?"

[0272] This invention aims to effectively protect users' digital privacy and prevent unintended public disclosure of images.

[0273] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0274] Step 1:

[0275] The user launches a privacy blocker application on their device, selects a photo of their face, and uploads it to the system. The input data is the face photo selected by the user, and the output is the transmission of image data from the device to the server. The application checks the file format to ensure the image is in the correct format and performs any necessary conversions.

[0276] Step 2:

[0277] The server receives image data transmitted from the terminal. The input is image data from the terminal, and the output is standardized image data. The server adjusts the image resolution and performs standardization processes such as unifying the color space. This allows for more efficient subsequent feature information extraction processes.

[0278] Step 3:

[0279] The server extracts feature information from standardized images using facial recognition technology. The input is standardized image data, and the output is a feature information vector. The server utilizes a generative AI model to identify unique facial features within the image and represents them as numerical data. This data is used in subsequent comparison processing.

[0280] Step 4:

[0281] The server collects photo data from specified websites and social media on the internet. The input is an internet connection and a list of target sites, and the output is a dataset of collected images. The server efficiently collects data using automated crawling technology.

[0282] Step 5:

[0283] The server compares the collected image data with the extracted feature information. The input is the collected image data and the user's feature information, and the output is the identification result of similar images. The AI algorithm calculates the similarity of the images and evaluates potential risks related to user privacy.

[0284] Step 6:

[0285] If the server finds similar images, it notifies the administrator and simultaneously automatically executes measures to protect privacy. The input is the identified similar images and their metadata, and the output is the notification report and the execution of corrective measures. The server issues a request to mosaic the detected images to mitigate privacy risks.

[0286] Step 7:

[0287] The server generates the results of all processes as a report and sends a notification to the user. The input is the result information of the process, and the output is a report with details described. Through this notification, the user can grasp their privacy situation in real time and take further actions if necessary.

[0288] (Application Example 1)

[0289] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0290] In the digital society, there is an increasing risk that personal image data is inadvertently made public on the Internet and privacy is violated. However, since users lack means to actively manage their own information and take immediate countermeasures, it is necessary to solve this problem quickly and efficiently.

[0291] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0292] In this invention, the server includes a device for acquiring images, a device for extracting attribute information from the acquired images, and a device for acquiring data on a communication network and identifying similar data by comparing it with the obtained attribute information. This makes it possible for users to monitor the online publication status of their own image data in real time and to proactively protect their personal privacy.

[0293] A "device for acquiring images" is a system that uses electronic devices to collect image data from users.

[0294] A "device for extracting attribute information from acquired images" is a mechanism that uses image processing technology to analyze and extract characteristic information from images.

[0295] A "device for acquiring data on a communication network and identifying similar data by comparing it with the obtained attribute information" is a mechanism that searches a database on the internet and determines similarity by comparing it with previously extracted attribute information.

[0296] A "device for notifying the administrator of the relevant data" is a means of communication for promptly informing the system administrator of data in which similarity has been detected.

[0297] "A device for processing the relevant data" refers to a mechanism that automatically performs appropriate processing on identified data to protect privacy.

[0298] A "device for sending notifications to user terminals" is a system that sends information to an individual's terminal in order to directly inform the user of the detection results.

[0299] A "device for providing interactive prompt notifications using mobile devices" is an interactive notification system that presents users with options and confirmations via portable electronic devices such as smartphones and tablets.

[0300] A "device for continuously monitoring data and reflecting changes" is a device that constantly monitors new data and changes on a communication network and quickly reflects the results in the system.

[0301] A "device for standardizing images" is a device that converts image data of different formats into a unified format to improve processing efficiency.

[0302] This invention is a system for efficiently protecting the privacy of users' personal images online. First, the user uses a mobile device, including a smartphone, to launch a dedicated application. This application functions as an "image acquisition device" and is responsible for acquiring the user's facial image and uploading it to a server.

[0303] The server processes the received image data using the AWS Rekognition API. It acts as a "device for extracting attribute information from acquired images," digitally extracting facial feature information. This feature information is then used with the Google Cloud Vision API to scan the internet, functioning as a "device for acquiring data on communication networks and identifying similar data by comparing it with the obtained attribute information."

[0304] If similar images are detected, the server uses Firebase to function as a "device for sending notifications to the user's device" in real time, providing relevant information. The notification is then sent to the user's smartphone via an interactive prompt, acting as a "device for providing interactive prompt notifications using mobile devices," presenting the user with confirmations and options.

[0305] For example, if a photo of an event attended by a user is published without permission, the system will immediately notify the user as soon as the image is detected and suggest automatic blurring, thereby ensuring rapid privacy protection.

[0306] Example of a prompt sentence:

[0307] "A newly published photo matches your information. Do you want to perform mosaic processing?"

[0308] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0309] Step 1:

[0310] The user uses a mobile terminal to launch a dedicated application, takes or selects a face image, and uploads it. The input at this stage is the user's face image data, and the output is that the image data is transferred to the server. By this operation, the system obtains the image data that forms the basis of privacy protection.

[0311] Step 2:

[0312] The server analyzes the received face image data using the AWS Rekognition API and extracts feature information. The input here is the user's face image data, and the output is digital data represented as feature information. By this specific operation, attribute information that enables the identification of the image is obtained.

[0313] Step 3:

[0314] The server scans the database on the Internet using the Google Cloud Vision API and searches for images that match the feature information. The input for this step is the feature information obtained in the previous step, and the output is the result of whether similar images exist. By this operation, it is checked whether the user's image has been publicly disclosed without permission.

[0315] Step 4:

[0316] When the server detects the presence of similar images, it sends a notification to the user's mobile device via Firebase. The input is information about the similar image detection, and the output is a warning message delivered to the user. This allows the user to stay informed in real time.

[0317] Step 5:

[0318] Based on notifications received on their smartphones, users select an action in response to displayed prompts. For example, a confirmation prompt might be presented to perform a mosaic effect. The input for this step is a notification message from the server, and the output is the user's chosen action. This action then executes the necessary privacy protection processes.

[0319] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0320] This invention combines a system that monitors publicly available photos on the internet to protect individual privacy with an emotion engine that recognizes user emotions. Specific embodiments are described below.

[0321] First, the user launches a system-specific application using a device such as a smartphone or computer. Here, the user selects and uploads a photo of their face. At this point, an emotion engine recognizes the user's current emotional state from the image. This emotional information is used to more accurately reflect the user's intentions.

[0322] The server receives images sent by users and performs image standardization. Feature information is extracted from these standardized images, and the server begins comparing them with images publicly available on the internet. It accesses databases of specific social media and websites and identifies similar photos based on the extracted feature information.

[0323] The server integrates identified photos with user sentiment information and determines how to process the images based on this. For example, if a user indicates discomfort, stricter privacy protections are applied. The server can also customize notifications to administrators based on sentiment information and take action according to priority and urgency.

[0324] Finally, the server generates a report based on the processing results and notifies the user. The user can review this through their terminal and choose further action if necessary. The server also performs continuous internet monitoring and applies similar protective measures to newly published photos.

[0325] For example, if the emotion engine recognizes a user's anger based on a photo uploaded by the user, the server will immediately blur or request the deletion of the relevant image and prioritize contacting the administrator. In this way, incorporating an emotion engine enables more user-centric privacy protection.

[0326] The following describes the processing flow.

[0327] Step 1:

[0328] The user launches a privacy blocker application on their device and selects a photo of their face. The user then uploads the photo and sends it to the system.

[0329] Step 2:

[0330] The device sends a photo of the user's face to an emotion engine to analyze the user's current emotional state. This emotional information is used to adjust the level of protection in subsequent processes.

[0331] Step 3:

[0332] The server receives the uploaded facial photographs and performs image standardization. This process adjusts the image format and resolution to a format suitable for the recognition model.

[0333] Step 4:

[0334] The server extracts facial feature information from standardized images and generates a digital fingerprint. This fingerprint serves as a standard for detecting photos on the internet.

[0335] Step 5:

[0336] The server accesses the internet and collects photo data from specific social media and websites. This data is then used to search for similar photos by comparing it with extracted facial feature information.

[0337] Step 6:

[0338] When similar photos are detected, the server considers emotional information to determine the appropriate image processing method. For example, if a user indicates discomfort, the server may choose to apply a mosaic effect to the image.

[0339] Step 7:

[0340] The server generates a notification for the administrator based on the user's emotions and identified images. The notification includes the location of the image and recommended actions.

[0341] Step 8:

[0342] The server generates a report of the processing results and sends it to the user. The user can review the report on their terminal and take any further necessary actions.

[0343] Step 9:

[0344] The server continuously monitors the internet and repeats the same process each time a new photo is uploaded, protecting user privacy.

[0345] (Example 2)

[0346] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0347] There is a need to more effectively protect individual privacy regarding images published on the internet and to respond flexibly while considering the emotional state of users. Conventional systems have had difficulty reflecting privacy settings based on users' emotions, resulting in insufficient individual responses.

[0348] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0349] In this invention, the server includes means for acquiring images, means for extracting feature information from acquired images, and means for analyzing emotional states. This makes it possible to determine processing methods based on the user's emotions and to apply appropriate privacy protection measures to images on the internet.

[0350] "Means for acquiring images" refers to elements that have the function of collecting image data using a digital device and converting it into a format that can be processed within the system.

[0351] "Means for extracting feature information from acquired images" refers to elements that perform the process of identifying specific patterns or attributes from image data and extracting information necessary for analysis.

[0352] "Means for acquiring information from the internet and identifying similar information by comparing it with the obtained characteristic information" refers to elements for accessing publicly available online databases and websites, and evaluating similarity by comparing it with the collected characteristic information.

[0353] "Means for notifying the administrator of relevant information" refers to elements that have the function of communicating identified important information or alerts to the administrator or user in an appropriate format.

[0354] "Means for processing the relevant information" refers to elements that have the function of editing or modifying images or data of identified information, in accordance with purposes such as privacy protection.

[0355] "Means for analyzing emotional states" refers to elements that perform a process of analyzing images and data obtained from users and identifying the emotions expressed therein.

[0356] "Means for determining processing methods based on emotional information" refers to an element that has the function of considering the analyzed emotional state and selecting and implementing the most appropriate data or image processing method.

[0357] This invention is a system that utilizes user emotional information to achieve effective privacy protection for images published on the internet.

[0358] Users launch a dedicated application for the system using a device such as a smartphone or PC. Through this application, users can upload a photo of their face. The application uses a built-in emotion engine to analyze the user's current emotional state from this image data. The emotion engine combines an artificial intelligence model and image processing algorithms to analyze the user's facial features and expressions to recognize emotions.

[0359] The server receives the uploaded images and performs image standardization. This standardization process converts the images into a consistent format. The server then uses the feature information extracted from the images to access databases of specific social media and various websites to identify similar images.

[0360] The server integrates identified similar images with the user's analyzed emotional information. This process determines appropriate image processing methods based on the emotional information and implements privacy protection measures for specific images. For example, if the user's emotion is identified as "anxiety," the server may apply a mosaic effect to the image or request its removal from the relevant parties.

[0361] Finally, the server generates a report of these processing results and notifies the user's terminal. The user can review the report and select further actions as needed. The server also continuously monitors the internet and automatically applies protective measures to any newly discovered related images.

[0362] For example, if the emotion of "anger" is analyzed from a photo uploaded by a user, the relevant image will be quickly blurred, and administrators will be notified preferentially. This makes it possible to implement privacy protection that is tailored to the user's emotions.

[0363] An example of a prompt message might be, "Please explain in detail how to analyze the emotions from a user's photo and apply privacy protections to similar publicly available images."

[0364] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0365] Step 1:

[0366] The user launches a system-specific application using a terminal. The user selects a photo of their face and uploads it to the application. At this point, the input is the user's facial image data. The application receives this input data and passes it to the emotion engine. The emotion engine uses image processing technology to analyze the facial expression and classifies the emotional state into categories such as "joy," "sadness," and "anger." At this point, the output is the analyzed emotional information.

[0367] Step 2:

[0368] The server receives facial images and emotion information sent from the user's terminal. The input consists of unstandardized image data and emotion information. The server performs image standardization processing, converting the data into a consistent format. This conversion makes the image data suitable for subsequent processing. The output is standardized image data.

[0369] Step 3:

[0370] The server extracts feature information from standardized image data. This feature information includes facial shape, color, and texture. The input is standardized image data, and the output is the extracted feature information. Based on this feature information, the server accesses a specific online database to search for similar images.

[0371] Step 4:

[0372] The server uses the extracted feature information to compare it with image data publicly available on the internet. The input is the extracted feature information, which is used to identify similar images. This identification process is performed by an image similarity search algorithm. The output is a list of similar images and associated metadata.

[0373] Step 5:

[0374] The server integrates and analyzes identified similar images with the user's emotion information. The input consists of a list of similar images and emotion information. This integration determines how to process the images. For example, if the user's emotion is classified as "anger," the server immediately decides to apply a mosaic effect to the associated images. The output is the determined processing method.

[0375] Step 6:

[0376] The server processes the image based on the determined processing method. The input is a similar image and the processing method. Processing includes applying mosaics and executing deletion requests. The output is the processed image data or a deletion request notification.

[0377] Step 7:

[0378] Finally, the server generates a report based on the processing results and notifies the user. The inputs are the processing results and processing status. The user can review this report via their terminal and select further actions. The output is a notification report to the user, which includes processing details and whether the processing was successful or not.

[0379] (Application Example 2)

[0380] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0381] When images posted online may infringe on a user's privacy against their will, it becomes difficult for users to properly manage that information and protect their privacy. Furthermore, a uniform approach that disregards user feelings will not enable appropriate privacy protection tailored to individual circumstances.

[0382] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0383] In this invention, the server includes a device for acquiring images, a device for analyzing emotions from the acquired images and applying privacy protection measures based on the emotional information, and a device for processing the relevant information. This enables situation-appropriate privacy protection that takes into account the user's emotions.

[0384] A "device for acquiring images" is a device that receives image data from a user, converts it into a format that can be processed within the system, and saves it.

[0385] A "device for extracting feature information from acquired images" is a device that has the function of analyzing and extracting features and patterns necessary for identification and classification from received image data.

[0386] The "function for acquiring information from the internet and identifying similar information by comparing it with the obtained feature information" refers to a function that searches for information on the internet based on the feature information of an image, evaluates its similarity, and identifies the target information.

[0387] A "device for notifying the administrator of relevant information" is a device equipped with communication functions for notifying administrators or users of identified information.

[0388] A "device for processing relevant information" is a device that has the function of processing and modifying image data based on specific conditions.

[0389] A "device for analyzing emotions from acquired images and applying privacy protection measures based on emotional information" is a device that analyzes the emotions of the subjects depicted in an image and determines how to handle and process the data based on the results.

[0390] A "device for continuously monitoring and updating information" is a device that constantly collects and analyzes information on the internet and has the function of updating the relevant information within the system to the latest state.

[0391] A "device for image standardization" is a device that converts images of different formats and sizes into a unified format, thereby facilitating subsequent processing.

[0392] To implement this invention, it is necessary to construct a system that utilizes advanced image processing technology. This system mainly consists of a server and user terminals, with each device performing a specific function. Specifically, it operates as follows:

[0393] The server, acting as an image acquisition device, receives images from the user's terminal. The received image data is converted into a unified format by an image standardization device, and then a device for extracting feature information from the acquired images analyzes and extracts specific patterns and features.

[0394] Next, a function is used to identify similar information by acquiring information from the internet and comparing it with the obtained feature information. This process utilizes TensorFlow, an open-source machine learning library, and is particularly applied to sentiment analysis. This sentiment analysis is performed by a specially trained generative AI model, and sentiment information is acquired simultaneously.

[0395] Furthermore, the device used to process the relevant information will perform actions such as blurring the image based on the privacy settings identified by the system. In addition, if the user indicates discomfort based on emotional information, stricter privacy protection measures will be applied.

[0396] Through the device, users can check the status and processing results of their images in real time. In particular, the results of the emotion engine's analysis are fed back to the user, enabling appropriate responses based on their emotions.

[0397] A concrete example is when a user publishes a particular photo; if the emotion engine detects potential offense before publication, the image is automatically blurred, preventing unprotected publication on social media, etc.

[0398] An example of a prompt message to a generative AI model would be, "I am about to post this image on social media, but I feel uncomfortable with it. Please take this into consideration and appropriately restrict the visibility." Through this prompt message, the generative AI model can analyze the user's intent and make optimal privacy settings.

[0399] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0400] Step 1:

[0401] The user uses their device to launch an application for acquiring images, selects an image, and uploads it. The input is the image data selected by the user, and this image data is sent to the server. The output is the image data transferred to the server.

[0402] Step 2:

[0403] The server performs a process to standardize the received images. The input is the received image data, which is standardized using an image processing library such as OpenCV. The output is the image data converted to a unified format.

[0404] Step 3:

[0405] The server performs a function to extract feature information from standardized images. The input is standardized image data, and a machine learning algorithm is used to extract specific features. The output is the extracted feature data.

[0406] Step 4:

[0407] The server receives image data acquired using a generative AI model and performs sentiment analysis. The input is standardized image data, and the generative AI model analyzes the user's emotions. The output is the user's sentiment information.

[0408] Step 5:

[0409] The server retrieves information from the internet and performs a process to identify similar information by comparing it with the obtained feature information. The input is extracted feature data, and related images on the internet are searched using a similar image search engine. The output is information about the identified similar images.

[0410] Step 6:

[0411] Based on emotional information and similar image information, the server employs means to process the relevant information. The input consists of the user's emotional information and information on identified similar images, and image processing (e.g., mosaic processing) and notification measures are performed to protect privacy. The output consists of the processed image and notification information regarding the processing results.

[0412] Step 7:

[0413] Ultimately, the server notifies the user of the processing results in report format. The input consists of the processed image and the processing results, which are fed back to the user via the terminal. The output is the notification data received by the user.

[0414] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0415] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0416] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0417] [Third Embodiment]

[0418] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0419] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0420] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0421] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0422] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0423] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0424] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0425] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0426] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0427] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0428] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0429] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0430] This invention is a system in which users, servers, and terminals cooperate to efficiently protect the privacy of personal photographs on online platforms. Specifically, the invention will be implemented in the following forms.

[0431] First, the user saves a photo of their face to a digital device and launches a dedicated privacy blocker application. Through this application, the user can upload the image to the system, which serves as the initial input data.

[0432] Next, the server receives the image sent by the user and performs a standardization process on it. From the standardized image data, the server extracts feature information using facial recognition technology. This feature information digitally represents the unique elements of the image and is used for comparison with existing photographic data available on the internet.

[0433] The server, while connected to the internet, collects photo data published from designated social media and other sites. This data is compared with extracted feature information, and an AI-powered process is executed to identify similar photos.

[0434] If an image that may have violated a user's privacy is detected, the server will activate a function to notify the administrator of this information. Furthermore, the server can execute protocols to take privacy-protecting measures for the detected image, such as sending a request to automatically apply mosaic blurring.

[0435] Ultimately, the server generates a report detailing the processing results and any detected issues, and sends a notification to the user. This notification includes information about the platform where the identified photos reside, allowing the user to understand their own privacy situation. The server can also continuously monitor the internet and repeat the same process whenever it detects the publication of new photos.

[0436] In this way, the invention aims to proactively protect users' digital privacy and minimize the risk of unintentional image disclosure.

[0437] The following describes the processing flow.

[0438] Step 1:

[0439] The user saves a photo of their face to their smartphone or PC. They then launch a dedicated privacy blocker application on their device and select an image using its interface. The user then uploads the selected image to the system.

[0440] Step 2:

[0441] The server receives images sent by users. After receiving the images, it performs image standardization processing and converts them into a format suitable for processing. Image standardization refers to changing the format and adjusting the resolution.

[0442] Step 3:

[0443] The server performs facial recognition analysis on standardized images. It extracts feature information from the images and uses this information to generate a unique digital "fingerprint." This step analyzes the unique features of the images.

[0444] Step 4:

[0445] The server connects to the internet and accesses the database of the specified social media platform. The server uses the extracted feature information to search the internet for photos of the same or similar faces.

[0446] Step 5:

[0447] The server analyzes identified photos on social media to determine if they may be infringing on users' privacy. If similar photos are found, the server automatically sends a notification to the administrator.

[0448] Step 6:

[0449] The server will perform privacy-protecting processing on the discovered photos. Specifically, it will send requests to the social media administrator to blur the photos or restrict their public access.

[0450] Step 7:

[0451] The server aggregates the processing results and sends a report to the user. The user can view this report on their device and learn about the platforms on which their facial image was found and their detailed information.

[0452] Step 8:

[0453] The server will continue to monitor new images on the internet, maintaining a continuous process to protect user privacy. This continuous monitoring ensures that users' privacy is protected over the long term.

[0454] (Example 1)

[0455] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0456] The unauthorized publication of personal photographs online constitutes a violation of privacy. This situation is particularly prevalent on social media platforms where photos are shared widely, highlighting the need for effective privacy protection. Traditional methods have been insufficient for monitoring and addressing this issue.

[0457] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0458] In this invention, the server includes means for acquiring images, means for standardizing the acquired images and extracting feature information, and means for collecting information from the Internet and identifying similar information by comparing it with the extracted feature information. This makes it possible to efficiently identify images that may infringe on an individual's privacy and to quickly take protective measures against them.

[0459] "Means for acquiring images" refers to the process and tools that allow a user to acquire their own digital images or photographs using a device and upload them to the system.

[0460] "Means for standardizing acquired images and extracting feature information" refers to the process of extracting specific facial information or unique visual features using image processing techniques that convert received image data into an appropriate format and enable consistent analysis.

[0461] "Means for collecting information from the internet and identifying similar information by comparing it with extracted feature information" refers to algorithms and processes for identifying photographs similar to a user's image by collecting image data accessible online and matching it with feature information extracted by the system.

[0462] "Means for sending notifications to administrators when identified information may constitute a privacy violation" refers to a communication system and protocol that transmits appropriate warnings to administrators when similar images related to a user's privacy are found.

[0463] "Means of processing identified information to protect privacy" refers to technologies and methods for applying mosaic effects or concealing information in images to reduce the risk of discovery.

[0464] "Means for detecting new image publications by continuously collecting and updating information from the internet" refers to an automated system that constantly monitors newly published image data on the internet and detects privacy risks based on regularly updated information.

[0465] This invention is an online surveillance system that protects individual privacy, and its functionality is realized through the cooperation of the user, terminal, and server. The user uploads their facial photograph to a privacy blocker application using the terminal. The terminal is responsible for receiving the image in this step and transferring it to the server in an appropriate format.

[0466] The server standardizes the received images and extracts feature information using specific facial recognition technology. This process utilizes advanced image processing software and AI technology. Specifically, a generative AI model used for model training is employed, and this technology effectively supports feature information extraction and similar image identification.

[0467] Next, the server connects to the internet and collects image data from specified websites and social media platforms. Using this collected data and extracted feature information, the AI ​​identifies similar images and assesses the risks related to user privacy.

[0468] If similar images are found, the server sends a notification to the administrator, who can then apply privacy-protecting measures such as blurring to the detected image information as needed. The resulting report is sent to the user, allowing them to understand their privacy situation and take necessary actions.

[0469] As a concrete example, the following prompt can be used for a generative AI model: "Please upload a photo of your face to the system. If an image that may violate privacy is detected, what measures will you take?"

[0470] This invention aims to effectively protect users' digital privacy and prevent unintended public disclosure of images.

[0471] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0472] Step 1:

[0473] The user launches a privacy blocker application on their device, selects a photo of their face, and uploads it to the system. The input data is the face photo selected by the user, and the output is the transmission of image data from the device to the server. The application checks the file format to ensure the image is in the correct format and performs any necessary conversions.

[0474] Step 2:

[0475] The server receives image data transmitted from the terminal. The input is image data from the terminal, and the output is standardized image data. The server adjusts the image resolution and performs standardization processes such as unifying the color space. This allows for more efficient subsequent feature information extraction processes.

[0476] Step 3:

[0477] The server extracts feature information from standardized images using facial recognition technology. The input is standardized image data, and the output is a feature information vector. The server utilizes a generative AI model to identify unique facial features within the image and represents them as numerical data. This data is used in subsequent comparison processing.

[0478] Step 4:

[0479] The server collects photo data from specified websites and social media on the internet. The input is an internet connection and a list of target sites, and the output is a dataset of collected images. The server efficiently collects data using automated crawling technology.

[0480] Step 5:

[0481] The server compares the collected image data with extracted feature information. The input is the collected image data and user feature information, and the output is the identification result of similar images. The AI ​​algorithm calculates the similarity of the images and assesses the potential risks related to user privacy.

[0482] Step 6:

[0483] The server automatically notifies the administrator and takes measures to protect privacy when similar images are found. The input is the identified similar images and their metadata, and the output is a notification report and the execution of corrective actions. The server mitigates privacy risks by requesting mosaic processing on detected images.

[0484] Step 7:

[0485] The server generates a report of the results of all processing and sends a notification to the user. The input is the processing result information, and the output is a detailed report. Through this notification, the user can understand their privacy status in real time and take further action if necessary.

[0486] (Application Example 1)

[0487] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0488] In our digital society, the risk of personal image data being unintentionally published on the internet and resulting in privacy violations is increasing. However, because users lack the means to proactively manage their own information and take immediate countermeasures, this problem needs to be solved quickly and efficiently.

[0489] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0490] In this invention, the server includes a device for acquiring images, a device for extracting attribute information from the acquired images, and a device for acquiring data on a communication network and identifying similar data by comparing it with the obtained attribute information. This makes it possible for users to monitor the online publication status of their own image data in real time and to proactively protect their personal privacy.

[0491] A "device for acquiring images" is a system that uses electronic devices to collect image data from users.

[0492] A "device for extracting attribute information from acquired images" is a mechanism that uses image processing technology to analyze and extract characteristic information from images.

[0493] A "device for acquiring data on a communication network and identifying similar data by comparing it with the obtained attribute information" is a mechanism that searches a database on the internet and determines similarity by comparing it with previously extracted attribute information.

[0494] A "device for notifying the administrator of the relevant data" is a means of communication for promptly informing the system administrator of data in which similarity has been detected.

[0495] "A device for processing the relevant data" refers to a mechanism that automatically performs appropriate processing on identified data to protect privacy.

[0496] A "device for sending notifications to user terminals" is a system that sends information to an individual's terminal in order to directly inform the user of the detection results.

[0497] A "device for providing interactive prompt notifications using mobile devices" is an interactive notification system that presents users with options and confirmations via portable electronic devices such as smartphones and tablets.

[0498] A "device for continuously monitoring data and reflecting changes" is a device that constantly monitors new data and changes on a communication network and quickly reflects the results in the system.

[0499] A "device for standardizing images" is a device that converts image data of different formats into a unified format to improve processing efficiency.

[0500] This invention is a system for efficiently protecting the privacy of users' personal images online. First, the user uses a mobile device, including a smartphone, to launch a dedicated application. This application functions as an "image acquisition device" and is responsible for acquiring the user's facial image and uploading it to a server.

[0501] The server processes the received image data using the AWS Rekognition API. It acts as a "device for extracting attribute information from acquired images," digitally extracting facial feature information. This feature information is then used with the Google Cloud Vision API to scan the internet, functioning as a "device for acquiring data on communication networks and identifying similar data by comparing it with the obtained attribute information."

[0502] If similar images are detected, the server uses Firebase to function as a "device for sending notifications to the user's device" in real time, providing relevant information. The notification is then sent to the user's smartphone via an interactive prompt, acting as a "device for providing interactive prompt notifications using mobile devices," presenting the user with confirmations and options.

[0503] For example, if a photo of an event attended by a user is published without permission, the system will immediately notify the user as soon as the image is detected and suggest automatic blurring, thereby ensuring rapid privacy protection.

[0504] Example of a prompt:

[0505] "A newly released photo matches your information. Would you like to have it blurred?"

[0506] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0507] Step 1:

[0508] The user launches a dedicated application using their mobile device and uploads a facial image by taking or selecting one. The input at this stage is the user's facial image data, and the output is the transfer of that image data to the server. Through this process, the system obtains image data that forms the basis for privacy protection.

[0509] Step 2:

[0510] The server analyzes the received facial image data using the AWS Rekognition API and extracts feature information. The input here is the user's facial image data, and the output is digital data represented as feature information. This specific operation yields attribute information that enables image identification.

[0511] Step 3:

[0512] The server uses the Google Cloud Vision API to scan databases on the internet and search for images that match the feature information. The input for this step is the feature information obtained in the previous step, and the output is whether or not similar images exist. This operation checks whether the user's images have been published without permission.

[0513] Step 4:

[0514] When the server detects the presence of similar images, it sends a notification to the user's mobile device via Firebase. The input is information about the similar image detection, and the output is a warning message delivered to the user. This allows the user to stay informed in real time.

[0515] Step 5:

[0516] Based on notifications received on their smartphones, users select an action in response to displayed prompts. For example, a confirmation prompt might be presented to perform a mosaic effect. The input for this step is a notification message from the server, and the output is the user's chosen action. This action then executes the necessary privacy protection processes.

[0517] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0518] This invention combines a system that monitors publicly available photos on the internet to protect individual privacy with an emotion engine that recognizes user emotions. Specific embodiments are described below.

[0519] First, the user launches a system-specific application using a device such as a smartphone or computer. Here, the user selects and uploads a photo of their face. At this point, an emotion engine recognizes the user's current emotional state from the image. This emotional information is used to more accurately reflect the user's intentions.

[0520] The server receives images sent by users and performs image standardization. Feature information is extracted from these standardized images, and the server begins comparing them with images publicly available on the internet. It accesses databases of specific social media and websites and identifies similar photos based on the extracted feature information.

[0521] The server integrates identified photos with user sentiment information and determines how to process the images based on this. For example, if a user indicates discomfort, stricter privacy protections are applied. The server can also customize notifications to administrators based on sentiment information and take action according to priority and urgency.

[0522] Finally, the server generates a report based on the processing results and notifies the user. The user can review this through their terminal and choose further action if necessary. The server also performs continuous internet monitoring and applies similar protective measures to newly published photos.

[0523] For example, if the emotion engine recognizes a user's anger based on a photo uploaded by the user, the server will immediately blur or request the deletion of the relevant image and prioritize contacting the administrator. In this way, incorporating an emotion engine enables more user-centric privacy protection.

[0524] The following describes the processing flow.

[0525] Step 1:

[0526] The user launches a privacy blocker application on their device and selects a photo of their face. The user then uploads the photo and sends it to the system.

[0527] Step 2:

[0528] The device sends a photo of the user's face to an emotion engine to analyze the user's current emotional state. This emotional information is used to adjust the level of protection in subsequent processes.

[0529] Step 3:

[0530] The server receives the uploaded facial photographs and performs image standardization. This process adjusts the image format and resolution to a format suitable for the recognition model.

[0531] Step 4:

[0532] The server extracts facial feature information from standardized images and generates a digital fingerprint. This fingerprint serves as a standard for detecting photos on the internet.

[0533] Step 5:

[0534] The server accesses the internet and collects photo data from specific social media and websites. This data is then used to search for similar photos by comparing it with extracted facial feature information.

[0535] Step 6:

[0536] When similar photos are detected, the server considers emotional information to determine the appropriate image processing method. For example, if a user indicates discomfort, the server may choose to apply a mosaic effect to the image.

[0537] Step 7:

[0538] The server generates a notification for the administrator based on the user's emotions and identified images. The notification includes the location of the image and recommended actions.

[0539] Step 8:

[0540] The server generates a report of the processing results and sends it to the user. The user can review the report on their terminal and take any further necessary actions.

[0541] Step 9:

[0542] The server continuously monitors the internet and repeats the same process each time a new photo is uploaded, protecting user privacy.

[0543] (Example 2)

[0544] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0545] There is a need to more effectively protect individual privacy regarding images published on the internet and to respond flexibly while considering the emotional state of users. Conventional systems have had difficulty reflecting privacy settings based on users' emotions, resulting in insufficient individual responses.

[0546] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0547] In this invention, the server includes means for acquiring images, means for extracting feature information from acquired images, and means for analyzing emotional states. This makes it possible to determine processing methods based on the user's emotions and to apply appropriate privacy protection measures to images on the internet.

[0548] "Means for acquiring images" refers to elements that have the function of collecting image data using a digital device and converting it into a format that can be processed within the system.

[0549] "Means for extracting feature information from acquired images" refers to elements that perform the process of identifying specific patterns or attributes from image data and extracting information necessary for analysis.

[0550] "Means for acquiring information from the internet and identifying similar information by comparing it with the obtained characteristic information" refers to elements for accessing publicly available online databases and websites, and evaluating similarity by comparing it with the collected characteristic information.

[0551] "Means for notifying the administrator of relevant information" refers to elements that have the function of communicating identified important information or alerts to the administrator or user in an appropriate format.

[0552] "Means for processing the relevant information" refers to elements that have the function of editing or modifying images or data of identified information, in accordance with purposes such as privacy protection.

[0553] "Means for analyzing emotional states" refers to elements that perform a process of analyzing images and data obtained from users and identifying the emotions expressed therein.

[0554] "Means for determining processing methods based on emotional information" refers to an element that has the function of considering the analyzed emotional state and selecting and implementing the most appropriate data or image processing method.

[0555] This invention is a system that utilizes user emotional information to achieve effective privacy protection for images published on the internet.

[0556] Users launch a dedicated application for the system using a device such as a smartphone or PC. Through this application, users can upload a photo of their face. The application uses a built-in emotion engine to analyze the user's current emotional state from this image data. The emotion engine combines an artificial intelligence model and image processing algorithms to analyze the user's facial features and expressions to recognize emotions.

[0557] The server receives the uploaded images and performs image standardization. This standardization process converts the images into a consistent format. The server then uses the feature information extracted from the images to access databases of specific social media and various websites to identify similar images.

[0558] The server integrates identified similar images with the user's analyzed emotional information. This process determines appropriate image processing methods based on the emotional information and implements privacy protection measures for specific images. For example, if the user's emotion is identified as "anxiety," the server may apply a mosaic effect to the image or request its removal from the relevant parties.

[0559] Finally, the server generates a report of these processing results and notifies the user's terminal. The user can review the report and select further actions as needed. The server also continuously monitors the internet and automatically applies protective measures to any newly discovered related images.

[0560] For example, if the emotion of "anger" is analyzed from a photo uploaded by a user, the relevant image will be quickly blurred, and administrators will be notified preferentially. This makes it possible to implement privacy protection that is tailored to the user's emotions.

[0561] An example of a prompt message might be, "Please explain in detail how to analyze the emotions from a user's photo and apply privacy protections to similar publicly available images."

[0562] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0563] Step 1:

[0564] The user launches a system-specific application using a terminal. The user selects a photo of their face and uploads it to the application. At this point, the input is the user's facial image data. The application receives this input data and passes it to the emotion engine. The emotion engine uses image processing technology to analyze the facial expression and classifies the emotional state into categories such as "joy," "sadness," and "anger." At this point, the output is the analyzed emotional information.

[0565] Step 2:

[0566] The server receives facial images and emotion information sent from the user's terminal. The input consists of unstandardized image data and emotion information. The server performs image standardization processing, converting the data into a consistent format. This conversion makes the image data suitable for subsequent processing. The output is standardized image data.

[0567] Step 3:

[0568] The server extracts feature information from standardized image data. This feature information includes facial shape, color, and texture. The input is standardized image data, and the output is the extracted feature information. Based on this feature information, the server accesses a specific online database to search for similar images.

[0569] Step 4:

[0570] The server uses the extracted feature information to compare it with image data publicly available on the internet. The input is the extracted feature information, which is used to identify similar images. This identification process is performed by an image similarity search algorithm. The output is a list of similar images and associated metadata.

[0571] Step 5:

[0572] The server integrates and analyzes identified similar images with the user's emotion information. The input consists of a list of similar images and emotion information. This integration determines how to process the images. For example, if the user's emotion is classified as "anger," the server immediately decides to apply a mosaic effect to the associated images. The output is the determined processing method.

[0573] Step 6:

[0574] The server processes the image based on the determined processing method. The input is a similar image and the processing method. Processing includes applying mosaics and executing deletion requests. The output is the processed image data or a deletion request notification.

[0575] Step 7:

[0576] Finally, the server generates a report based on the processing results and notifies the user. The inputs are the processing results and processing status. The user can review this report via their terminal and select further actions. The output is a notification report to the user, which includes processing details and whether the processing was successful or not.

[0577] (Application Example 2)

[0578] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0579] When images posted online may infringe on a user's privacy against their will, it becomes difficult for users to properly manage that information and protect their privacy. Furthermore, a uniform approach that disregards user feelings will not enable appropriate privacy protection tailored to individual circumstances.

[0580] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0581] In this invention, the server includes a device for acquiring images, a device for analyzing emotions from the acquired images and applying privacy protection measures based on the emotional information, and a device for processing the relevant information. This enables situation-appropriate privacy protection that takes into account the user's emotions.

[0582] A "device for acquiring images" is a device that receives image data from a user, converts it into a format that can be processed within the system, and saves it.

[0583] A "device for extracting feature information from acquired images" is a device that has the function of analyzing and extracting features and patterns necessary for identification and classification from received image data.

[0584] The "function for acquiring information from the internet and identifying similar information by comparing it with the obtained feature information" refers to a function that searches for information on the internet based on the feature information of an image, evaluates its similarity, and identifies the target information.

[0585] A "device for notifying the administrator of relevant information" is a device equipped with communication functions for notifying administrators or users of identified information.

[0586] A "device for processing relevant information" is a device that has the function of processing and modifying image data based on specific conditions.

[0587] A "device for analyzing emotions from acquired images and applying privacy protection measures based on emotional information" is a device that analyzes the emotions of the subjects depicted in an image and determines how to handle and process the data based on the results.

[0588] A "device for continuously monitoring and updating information" is a device that constantly collects and analyzes information on the internet and has the function of updating the relevant information within the system to the latest state.

[0589] A "device for image standardization" is a device that converts images of different formats and sizes into a unified format, thereby facilitating subsequent processing.

[0590] To implement this invention, it is necessary to construct a system that utilizes advanced image processing technology. This system mainly consists of a server and user terminals, with each device performing a specific function. Specifically, it operates as follows:

[0591] The server, acting as an image acquisition device, receives images from the user's terminal. The received image data is converted into a unified format by an image standardization device, and then a device for extracting feature information from the acquired images analyzes and extracts specific patterns and features.

[0592] Next, a function is used to identify similar information by acquiring information from the internet and comparing it with the obtained feature information. This process utilizes TensorFlow, an open-source machine learning library, and is particularly applied to sentiment analysis. This sentiment analysis is performed by a specially trained generative AI model, and sentiment information is acquired simultaneously.

[0593] Furthermore, the device used to process the relevant information will perform actions such as blurring the image based on the privacy settings identified by the system. In addition, if the user indicates discomfort based on emotional information, stricter privacy protection measures will be applied.

[0594] Through the device, users can check the status and processing results of their images in real time. In particular, the results of the emotion engine's analysis are fed back to the user, enabling appropriate responses based on their emotions.

[0595] A concrete example is when a user publishes a particular photo; if the emotion engine detects potential offense before publication, the image is automatically blurred, preventing unprotected publication on social media, etc.

[0596] An example of a prompt message to a generative AI model would be, "I am about to post this image on social media, but I feel uncomfortable with it. Please take this into consideration and appropriately restrict the visibility." Through this prompt message, the generative AI model can analyze the user's intent and make optimal privacy settings.

[0597] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0598] Step 1:

[0599] The user uses their device to launch an application for acquiring images, selects an image, and uploads it. The input is the image data selected by the user, and this image data is sent to the server. The output is the image data transferred to the server.

[0600] Step 2:

[0601] The server performs a process to standardize the received images. The input is the received image data, which is standardized using an image processing library such as OpenCV. The output is the image data converted to a unified format.

[0602] Step 3:

[0603] The server performs a function to extract feature information from standardized images. The input is standardized image data, and a machine learning algorithm is used to extract specific features. The output is the extracted feature data.

[0604] Step 4:

[0605] The server receives image data acquired using a generative AI model and performs sentiment analysis. The input is standardized image data, and the generative AI model analyzes the user's emotions. The output is the user's sentiment information.

[0606] Step 5:

[0607] The server retrieves information from the internet and performs a process to identify similar information by comparing it with the obtained feature information. The input is extracted feature data, and related images on the internet are searched using a similar image search engine. The output is information about the identified similar images.

[0608] Step 6:

[0609] Based on emotional information and similar image information, the server employs means to process the relevant information. The input consists of the user's emotional information and information on identified similar images, and image processing (e.g., mosaic processing) and notification measures are performed to protect privacy. The output consists of the processed image and notification information regarding the processing results.

[0610] Step 7:

[0611] Ultimately, the server notifies the user of the processing results in report format. The input consists of the processed image and the processing results, which are fed back to the user via the terminal. The output is the notification data received by the user.

[0612] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0613] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0614] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0615] [Fourth Embodiment]

[0616] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0617] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0618] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0619] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0620] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0621] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0622] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0623] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0624] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0625] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0626] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0627] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0628] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0629] This invention is a system in which users, servers, and terminals cooperate to efficiently protect the privacy of personal photographs on online platforms. Specifically, the invention will be implemented in the following forms.

[0630] First, the user saves a photo of their face to a digital device and launches a dedicated privacy blocker application. Through this application, the user can upload the image to the system, which serves as the initial input data.

[0631] Next, the server receives the image sent by the user and performs a standardization process on it. From the standardized image data, the server extracts feature information using facial recognition technology. This feature information digitally represents the unique elements of the image and is used for comparison with existing photographic data available on the internet.

[0632] The server, while connected to the internet, collects photo data published from designated social media and other sites. This data is compared with extracted feature information, and an AI-powered process is executed to identify similar photos.

[0633] If an image that may have violated a user's privacy is detected, the server will activate a function to notify the administrator of this information. Furthermore, the server can execute protocols to take privacy-protecting measures for the detected image, such as sending a request to automatically apply mosaic blurring.

[0634] Ultimately, the server generates a report detailing the processing results and any detected issues, and sends a notification to the user. This notification includes information about the platform where the identified photos reside, allowing the user to understand their own privacy situation. The server can also continuously monitor the internet and repeat the same process whenever it detects the publication of new photos.

[0635] In this way, the invention aims to proactively protect users' digital privacy and minimize the risk of unintentional image disclosure.

[0636] The following describes the processing flow.

[0637] Step 1:

[0638] The user saves a photo of their face to their smartphone or PC. They then launch a dedicated privacy blocker application on their device and select an image using its interface. The user then uploads the selected image to the system.

[0639] Step 2:

[0640] The server receives images sent by users. After receiving the images, it performs image standardization processing and converts them into a format suitable for processing. Image standardization refers to changing the format and adjusting the resolution.

[0641] Step 3:

[0642] The server performs facial recognition analysis on standardized images. It extracts feature information from the images and uses this information to generate a unique digital "fingerprint." This step analyzes the unique features of the images.

[0643] Step 4:

[0644] The server connects to the internet and accesses the database of the specified social media platform. The server uses the extracted feature information to search the internet for photos of the same or similar faces.

[0645] Step 5:

[0646] The server analyzes identified photos on social media to determine if they may be infringing on users' privacy. If similar photos are found, the server automatically sends a notification to the administrator.

[0647] Step 6:

[0648] The server will perform privacy-protecting processing on the discovered photos. Specifically, it will send requests to the social media administrator to blur the photos or restrict their public access.

[0649] Step 7:

[0650] The server aggregates the processing results and sends a report to the user. The user can view this report on their device and learn about the platforms on which their facial image was found and their detailed information.

[0651] Step 8:

[0652] The server will continue to monitor new images on the internet, maintaining a continuous process to protect user privacy. This continuous monitoring ensures that users' privacy is protected over the long term.

[0653] (Example 1)

[0654] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0655] The unauthorized publication of personal photographs online constitutes a violation of privacy. This situation is particularly prevalent on social media platforms where photos are shared widely, highlighting the need for effective privacy protection. Traditional methods have been insufficient for monitoring and addressing this issue.

[0656] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0657] In this invention, the server includes means for acquiring images, means for standardizing the acquired images and extracting feature information, and means for collecting information from the Internet and identifying similar information by comparing it with the extracted feature information. This makes it possible to efficiently identify images that may infringe on an individual's privacy and to quickly take protective measures against them.

[0658] "Means for acquiring images" refers to the process and tools that allow a user to acquire their own digital images or photographs using a device and upload them to the system.

[0659] "Means for standardizing acquired images and extracting feature information" refers to the process of extracting specific facial information or unique visual features using image processing techniques that convert received image data into an appropriate format and enable consistent analysis.

[0660] "Means for collecting information from the internet and identifying similar information by comparing it with extracted feature information" refers to algorithms and processes for identifying photographs similar to a user's image by collecting image data accessible online and matching it with feature information extracted by the system.

[0661] "Means for sending notifications to administrators when identified information may constitute a privacy violation" refers to a communication system and protocol that transmits appropriate warnings to administrators when similar images related to a user's privacy are found.

[0662] "Means of processing identified information to protect privacy" refers to technologies and methods for applying mosaic effects or concealing information in images to reduce the risk of discovery.

[0663] "Means for detecting new image publications by continuously collecting and updating information from the internet" refers to an automated system that constantly monitors newly published image data on the internet and detects privacy risks based on regularly updated information.

[0664] This invention is an online surveillance system that protects individual privacy, and its functionality is realized through the cooperation of the user, terminal, and server. The user uploads their facial photograph to a privacy blocker application using the terminal. The terminal is responsible for receiving the image in this step and transferring it to the server in an appropriate format.

[0665] The server standardizes the received images and extracts feature information using specific facial recognition technology. This process utilizes advanced image processing software and AI technology. Specifically, a generative AI model used for model training is employed, and this technology effectively supports feature information extraction and similar image identification.

[0666] Next, the server connects to the internet and collects image data from specified websites and social media platforms. Using this collected data and extracted feature information, the AI ​​identifies similar images and assesses the risks related to user privacy.

[0667] If similar images are found, the server sends a notification to the administrator, who can then apply privacy-protecting measures such as blurring to the detected image information as needed. The resulting report is sent to the user, allowing them to understand their privacy situation and take necessary actions.

[0668] As a concrete example, the following prompt can be used for a generative AI model: "Please upload a photo of your face to the system. If an image that may violate privacy is detected, what measures will you take?"

[0669] This invention aims to effectively protect users' digital privacy and prevent unintended public disclosure of images.

[0670] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0671] Step 1:

[0672] The user launches a privacy blocker application on their device, selects a photo of their face, and uploads it to the system. The input data is the face photo selected by the user, and the output is the transmission of image data from the device to the server. The application checks the file format to ensure the image is in the correct format and performs any necessary conversions.

[0673] Step 2:

[0674] The server receives image data transmitted from the terminal. The input is image data from the terminal, and the output is standardized image data. The server adjusts the image resolution and performs standardization processes such as unifying the color space. This allows for more efficient subsequent feature information extraction processes.

[0675] Step 3:

[0676] The server extracts feature information from standardized images using facial recognition technology. The input is standardized image data, and the output is a feature information vector. The server utilizes a generative AI model to identify unique facial features within the image and represents them as numerical data. This data is used in subsequent comparison processing.

[0677] Step 4:

[0678] The server collects photo data from specified websites and social media on the internet. The input is an internet connection and a list of target sites, and the output is a dataset of collected images. The server efficiently collects data using automated crawling technology.

[0679] Step 5:

[0680] The server compares the collected image data with extracted feature information. The input is the collected image data and user feature information, and the output is the identification result of similar images. The AI ​​algorithm calculates the similarity of the images and assesses the potential risks related to user privacy.

[0681] Step 6:

[0682] The server automatically notifies the administrator and takes measures to protect privacy when similar images are found. The input is the identified similar images and their metadata, and the output is a notification report and the execution of corrective actions. The server mitigates privacy risks by requesting mosaic processing on detected images.

[0683] Step 7:

[0684] The server generates a report of the results of all processing and sends a notification to the user. The input is the processing result information, and the output is a detailed report. Through this notification, the user can understand their privacy status in real time and take further action if necessary.

[0685] (Application Example 1)

[0686] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0687] In our digital society, the risk of personal image data being unintentionally published on the internet and resulting in privacy violations is increasing. However, because users lack the means to proactively manage their own information and take immediate countermeasures, this problem needs to be solved quickly and efficiently.

[0688] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0689] In this invention, the server includes a device for acquiring images, a device for extracting attribute information from the acquired images, and a device for acquiring data on a communication network and identifying similar data by comparing it with the obtained attribute information. This makes it possible for users to monitor the online publication status of their own image data in real time and to proactively protect their personal privacy.

[0690] A "device for acquiring images" is a system that uses electronic devices to collect image data from users.

[0691] A "device for extracting attribute information from acquired images" is a mechanism that uses image processing technology to analyze and extract characteristic information from images.

[0692] A "device for acquiring data on a communication network and identifying similar data by comparing it with the obtained attribute information" is a mechanism that searches a database on the internet and determines similarity by comparing it with previously extracted attribute information.

[0693] A "device for notifying the administrator of the relevant data" is a means of communication for promptly informing the system administrator of data in which similarity has been detected.

[0694] "A device for processing the relevant data" refers to a mechanism that automatically performs appropriate processing on identified data to protect privacy.

[0695] A "device for sending notifications to user terminals" is a system that sends information to an individual's terminal in order to directly inform the user of the detection results.

[0696] A "device for providing interactive prompt notifications using mobile devices" is an interactive notification system that presents users with options and confirmations via portable electronic devices such as smartphones and tablets.

[0697] A "device for continuously monitoring data and reflecting changes" is a device that constantly monitors new data and changes on a communication network and quickly reflects the results in the system.

[0698] A "device for standardizing images" is a device that converts image data of different formats into a unified format to improve processing efficiency.

[0699] This invention is a system for efficiently protecting the privacy of users' personal images online. First, the user uses a mobile device, including a smartphone, to launch a dedicated application. This application functions as an "image acquisition device" and is responsible for acquiring the user's facial image and uploading it to a server.

[0700] The server processes the received image data using the AWS Rekognition API. It acts as a "device for extracting attribute information from acquired images," digitally extracting facial feature information. This feature information is then used with the Google Cloud Vision API to scan the internet, functioning as a "device for acquiring data on communication networks and identifying similar data by comparing it with the obtained attribute information."

[0701] If similar images are detected, the server uses Firebase to function as a "device for sending notifications to the user's device" in real time, providing relevant information. The notification is then sent to the user's smartphone via an interactive prompt, acting as a "device for providing interactive prompt notifications using mobile devices," presenting the user with confirmations and options.

[0702] For example, if a photo of an event attended by a user is published without permission, the system will immediately notify the user as soon as the image is detected and suggest automatic blurring, thereby ensuring rapid privacy protection.

[0703] Example of a prompt:

[0704] "A newly released photo matches your information. Would you like to have it blurred?"

[0705] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0706] Step 1:

[0707] The user launches a dedicated application using their mobile device and uploads a facial image by taking or selecting one. The input at this stage is the user's facial image data, and the output is the transfer of that image data to the server. Through this process, the system obtains image data that forms the basis for privacy protection.

[0708] Step 2:

[0709] The server analyzes the received facial image data using the AWS Rekognition API and extracts feature information. The input here is the user's facial image data, and the output is digital data represented as feature information. This specific operation yields attribute information that enables image identification.

[0710] Step 3:

[0711] The server uses the Google Cloud Vision API to scan databases on the internet and search for images that match the feature information. The input for this step is the feature information obtained in the previous step, and the output is whether or not similar images exist. This operation checks whether the user's images have been published without permission.

[0712] Step 4:

[0713] When the server detects the presence of similar images, it sends a notification to the user's mobile device via Firebase. The input is information about the similar image detection, and the output is a warning message delivered to the user. This allows the user to stay informed in real time.

[0714] Step 5:

[0715] Based on notifications received on their smartphones, users select an action in response to displayed prompts. For example, a confirmation prompt might be presented to perform a mosaic effect. The input for this step is a notification message from the server, and the output is the user's chosen action. This action then executes the necessary privacy protection processes.

[0716] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0717] This invention combines a system that monitors publicly available photos on the internet to protect individual privacy with an emotion engine that recognizes user emotions. Specific embodiments are described below.

[0718] First, the user launches a system-specific application using a device such as a smartphone or computer. Here, the user selects and uploads a photo of their face. At this point, an emotion engine recognizes the user's current emotional state from the image. This emotional information is used to more accurately reflect the user's intentions.

[0719] The server receives images sent by users and performs image standardization. Feature information is extracted from these standardized images, and the server begins comparing them with images publicly available on the internet. It accesses databases of specific social media and websites and identifies similar photos based on the extracted feature information.

[0720] The server integrates identified photos with user sentiment information and determines how to process the images based on this. For example, if a user indicates discomfort, stricter privacy protections are applied. The server can also customize notifications to administrators based on sentiment information and take action according to priority and urgency.

[0721] Finally, the server generates a report based on the processing results and notifies the user. The user can review this through their terminal and choose further action if necessary. The server also performs continuous internet monitoring and applies similar protective measures to newly published photos.

[0722] For example, if the emotion engine recognizes a user's anger based on a photo uploaded by the user, the server will immediately blur or request the deletion of the relevant image and prioritize contacting the administrator. In this way, incorporating an emotion engine enables more user-centric privacy protection.

[0723] The following describes the processing flow.

[0724] Step 1:

[0725] The user launches a privacy blocker application on their device and selects a photo of their face. The user then uploads the photo and sends it to the system.

[0726] Step 2:

[0727] The device sends a photo of the user's face to an emotion engine to analyze the user's current emotional state. This emotional information is used to adjust the level of protection in subsequent processes.

[0728] Step 3:

[0729] The server receives the uploaded facial photographs and performs image standardization. This process adjusts the image format and resolution to a format suitable for the recognition model.

[0730] Step 4:

[0731] The server extracts facial feature information from standardized images and generates a digital fingerprint. This fingerprint serves as a standard for detecting photos on the internet.

[0732] Step 5:

[0733] The server accesses the internet and collects photo data from specific social media and websites. This data is then used to search for similar photos by comparing it with extracted facial feature information.

[0734] Step 6:

[0735] When similar photos are detected, the server considers emotional information to determine the appropriate image processing method. For example, if a user indicates discomfort, the server may choose to apply a mosaic effect to the image.

[0736] Step 7:

[0737] The server generates a notification for the administrator based on the user's emotions and identified images. The notification includes the location of the image and recommended actions.

[0738] Step 8:

[0739] The server generates a report of the processing results and sends it to the user. The user can review the report on their terminal and take any further necessary actions.

[0740] Step 9:

[0741] The server continuously monitors the internet and repeats the same process each time a new photo is uploaded, protecting user privacy.

[0742] (Example 2)

[0743] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0744] There is a need to more effectively protect individual privacy regarding images published on the internet and to respond flexibly while considering the emotional state of users. Conventional systems have had difficulty reflecting privacy settings based on users' emotions, resulting in insufficient individual responses.

[0745] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0746] In this invention, the server includes means for acquiring images, means for extracting feature information from acquired images, and means for analyzing emotional states. This makes it possible to determine processing methods based on the user's emotions and to apply appropriate privacy protection measures to images on the internet.

[0747] "Means for acquiring images" refers to elements that have the function of collecting image data using a digital device and converting it into a format that can be processed within the system.

[0748] "Means for extracting feature information from acquired images" refers to elements that perform the process of identifying specific patterns or attributes from image data and extracting information necessary for analysis.

[0749] "Means for acquiring information from the internet and identifying similar information by comparing it with the obtained characteristic information" refers to elements for accessing publicly available online databases and websites, and evaluating similarity by comparing it with the collected characteristic information.

[0750] "Means for notifying the administrator of relevant information" refers to elements that have the function of communicating identified important information or alerts to the administrator or user in an appropriate format.

[0751] "Means for processing the relevant information" refers to elements that have the function of editing or modifying images or data of identified information, in accordance with purposes such as privacy protection.

[0752] "Means for analyzing emotional states" refers to elements that perform a process of analyzing images and data obtained from users and identifying the emotions expressed therein.

[0753] "Means for determining processing methods based on emotional information" refers to an element that has the function of considering the analyzed emotional state and selecting and implementing the most appropriate data or image processing method.

[0754] This invention is a system that utilizes user emotional information to achieve effective privacy protection for images published on the internet.

[0755] Users launch a dedicated application for the system using a device such as a smartphone or PC. Through this application, users can upload a photo of their face. The application uses a built-in emotion engine to analyze the user's current emotional state from this image data. The emotion engine combines an artificial intelligence model and image processing algorithms to analyze the user's facial features and expressions to recognize emotions.

[0756] The server receives the uploaded images and performs image standardization. This standardization process converts the images into a consistent format. The server then uses the feature information extracted from the images to access databases of specific social media and various websites to identify similar images.

[0757] The server integrates identified similar images with the user's analyzed emotional information. This process determines appropriate image processing methods based on the emotional information and implements privacy protection measures for specific images. For example, if the user's emotion is identified as "anxiety," the server may apply a mosaic effect to the image or request its removal from the relevant parties.

[0758] Finally, the server generates a report of these processing results and notifies the user's terminal. The user can review the report and select further actions as needed. The server also continuously monitors the internet and automatically applies protective measures to any newly discovered related images.

[0759] For example, if the emotion of "anger" is analyzed from a photo uploaded by a user, the relevant image will be quickly blurred, and administrators will be notified preferentially. This makes it possible to implement privacy protection that is tailored to the user's emotions.

[0760] An example of a prompt message might be, "Please explain in detail how to analyze the emotions from a user's photo and apply privacy protections to similar publicly available images."

[0761] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0762] Step 1:

[0763] The user launches a system-specific application using a terminal. The user selects a photo of their face and uploads it to the application. At this point, the input is the user's facial image data. The application receives this input data and passes it to the emotion engine. The emotion engine uses image processing technology to analyze the facial expression and classifies the emotional state into categories such as "joy," "sadness," and "anger." At this point, the output is the analyzed emotional information.

[0764] Step 2:

[0765] The server receives facial images and emotion information sent from the user's terminal. The input consists of unstandardized image data and emotion information. The server performs image standardization processing, converting the data into a consistent format. This conversion makes the image data suitable for subsequent processing. The output is standardized image data.

[0766] Step 3:

[0767] The server extracts feature information from standardized image data. This feature information includes facial shape, color, and texture. The input is standardized image data, and the output is the extracted feature information. Based on this feature information, the server accesses a specific online database to search for similar images.

[0768] Step 4:

[0769] The server uses the extracted feature information to compare it with image data publicly available on the internet. The input is the extracted feature information, which is used to identify similar images. This identification process is performed by an image similarity search algorithm. The output is a list of similar images and associated metadata.

[0770] Step 5:

[0771] The server integrates and analyzes identified similar images with the user's emotion information. The input consists of a list of similar images and emotion information. This integration determines how to process the images. For example, if the user's emotion is classified as "anger," the server immediately decides to apply a mosaic effect to the associated images. The output is the determined processing method.

[0772] Step 6:

[0773] The server processes the image based on the determined processing method. The input is a similar image and the processing method. Processing includes applying mosaics and executing deletion requests. The output is the processed image data or a deletion request notification.

[0774] Step 7:

[0775] Finally, the server generates a report based on the processing results and notifies the user. The inputs are the processing results and processing status. The user can review this report via their terminal and select further actions. The output is a notification report to the user, which includes processing details and whether the processing was successful or not.

[0776] (Application Example 2)

[0777] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0778] When images posted online may infringe on a user's privacy against their will, it becomes difficult for users to properly manage that information and protect their privacy. Furthermore, a uniform approach that disregards user feelings will not enable appropriate privacy protection tailored to individual circumstances.

[0779] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0780] In this invention, the server includes a device for acquiring images, a device for analyzing emotions from the acquired images and applying privacy protection measures based on the emotional information, and a device for processing the relevant information. This enables situation-appropriate privacy protection that takes into account the user's emotions.

[0781] A "device for acquiring images" is a device that receives image data from a user, converts it into a format that can be processed within the system, and saves it.

[0782] A "device for extracting feature information from acquired images" is a device that has the function of analyzing and extracting features and patterns necessary for identification and classification from received image data.

[0783] The "function for acquiring information from the internet and identifying similar information by comparing it with the obtained feature information" refers to a function that searches for information on the internet based on the feature information of an image, evaluates its similarity, and identifies the target information.

[0784] A "device for notifying the administrator of relevant information" is a device equipped with communication functions for notifying administrators or users of identified information.

[0785] A "device for processing relevant information" is a device that has the function of processing and modifying image data based on specific conditions.

[0786] A "device for analyzing emotions from acquired images and applying privacy protection measures based on emotional information" is a device that analyzes the emotions of the subjects depicted in an image and determines how to handle and process the data based on the results.

[0787] A "device for continuously monitoring and updating information" is a device that constantly collects and analyzes information on the internet and has the function of updating the relevant information within the system to the latest state.

[0788] A "device for image standardization" is a device that converts images of different formats and sizes into a unified format, thereby facilitating subsequent processing.

[0789] To implement this invention, it is necessary to construct a system that utilizes advanced image processing technology. This system mainly consists of a server and user terminals, with each device performing a specific function. Specifically, it operates as follows:

[0790] The server, acting as an image acquisition device, receives images from the user's terminal. The received image data is converted into a unified format by an image standardization device, and then a device for extracting feature information from the acquired images analyzes and extracts specific patterns and features.

[0791] Next, a function is used to identify similar information by acquiring information from the internet and comparing it with the obtained feature information. This process utilizes TensorFlow, an open-source machine learning library, and is particularly applied to sentiment analysis. This sentiment analysis is performed by a specially trained generative AI model, and sentiment information is acquired simultaneously.

[0792] Furthermore, the device used to process the relevant information will perform actions such as blurring the image based on the privacy settings identified by the system. In addition, if the user indicates discomfort based on emotional information, stricter privacy protection measures will be applied.

[0793] Through the device, users can check the status and processing results of their images in real time. In particular, the results of the emotion engine's analysis are fed back to the user, enabling appropriate responses based on their emotions.

[0794] A concrete example is when a user publishes a particular photo; if the emotion engine detects potential offense before publication, the image is automatically blurred, preventing unprotected publication on social media, etc.

[0795] An example of a prompt message to a generative AI model would be, "I am about to post this image on social media, but I feel uncomfortable with it. Please take this into consideration and appropriately restrict the visibility." Through this prompt message, the generative AI model can analyze the user's intent and make optimal privacy settings.

[0796] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0797] Step 1:

[0798] The user uses their device to launch an application for acquiring images, selects an image, and uploads it. The input is the image data selected by the user, and this image data is sent to the server. The output is the image data transferred to the server.

[0799] Step 2:

[0800] The server performs a process to standardize the received images. The input is the received image data, which is standardized using an image processing library such as OpenCV. The output is the image data converted to a unified format.

[0801] Step 3:

[0802] The server performs a function to extract feature information from standardized images. The input is standardized image data, and a machine learning algorithm is used to extract specific features. The output is the extracted feature data.

[0803] Step 4:

[0804] The server receives image data acquired using a generative AI model and performs sentiment analysis. The input is standardized image data, and the generative AI model analyzes the user's emotions. The output is the user's sentiment information.

[0805] Step 5:

[0806] The server retrieves information from the internet and performs a process to identify similar information by comparing it with the obtained feature information. The input is extracted feature data, and related images on the internet are searched using a similar image search engine. The output is information about the identified similar images.

[0807] Step 6:

[0808] Based on emotional information and similar image information, the server employs means to process the relevant information. The input consists of the user's emotional information and information on identified similar images, and image processing (e.g., mosaic processing) and notification measures are performed to protect privacy. The output consists of the processed image and notification information regarding the processing results.

[0809] Step 7:

[0810] Ultimately, the server notifies the user of the processing results in report format. The input consists of the processed image and the processing results, which are fed back to the user via the terminal. The output is the notification data received by the user.

[0811] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0812] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0813] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0814] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0815] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0816] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0817] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0818] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0819] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0820] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0821] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0822] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0823] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0824] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0825] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0826] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0827] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0828] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0829] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0830] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0831] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0832] The following is further disclosed regarding the embodiments described above.

[0833] (Claim 1)

[0834] Means for acquiring images,

[0835] A means for extracting feature information from acquired images,

[0836] A means for obtaining information from the internet and identifying similar information by comparing it with the obtained characteristic information,

[0837] A means of notifying the administrator of the relevant information,

[0838] Means for processing the relevant information,

[0839] A system that includes this.

[0840] (Claim 2)

[0841] The system according to claim 1, comprising means for continuously monitoring and updating information.

[0842] (Claim 3)

[0843] The system according to claim 1, comprising means for standardizing images.

[0844] "Example 1"

[0845] (Claim 1)

[0846] Means for acquiring images,

[0847] A means for standardizing acquired images and extracting feature information,

[0848] A means for collecting information from the internet and identifying similar information by comparing it with extracted characteristic information,

[0849] A means of sending a notification to the administrator if the identified information may constitute a privacy violation,

[0850] The means of processing identified information and implementing privacy protections,

[0851] A means to generate a final processing result as a report and notify the user,

[0852] A system that includes this.

[0853] (Claim 2)

[0854] The system according to claim 1, a means for detecting new image publications by continuously collecting and updating information from the internet.

[0855] (Claim 3)

[0856] The system according to claim 1, which is a means for recognizing facial information with high accuracy and extracting feature information by standardizing images.

[0857] "Application Example 1"

[0858] (Claim 1)

[0859] A device and means for acquiring an image,

[0860] A device for extracting attribute information from acquired images,

[0861] A device means for acquiring data on a communication network and identifying similar data by comparing it with the obtained attribute information,

[0862] A device for notifying the administrator of the relevant data,

[0863] A device and means for processing the relevant data,

[0864] A device means for sending notifications to a user terminal,

[0865] A device and means for providing interactive prompt notifications using a mobile device,

[0866] A system that includes this.

[0867] (Claim 2)

[0868] A device for sequentially monitoring data and reflecting fluctuations, as described in claim 1.

[0869] (Claim 3)

[0870] A system for standardizing images, as described in claim 1.

[0871] "Example 2 of combining an emotion engine"

[0872] (Claim 1)

[0873] Means for acquiring images,

[0874] A means for extracting feature information from acquired images,

[0875] A means for obtaining information from the internet and identifying similar information by comparing it with the obtained characteristic information,

[0876] A means of notifying the administrator of the relevant information,

[0877] Means for processing the relevant information,

[0878] A means of analyzing emotional states,

[0879] A means for determining a processing method based on emotional information,

[0880] A system that includes this.

[0881] (Claim 2)

[0882] The system according to claim 1, comprising means for continuously monitoring and updating information.

[0883] (Claim 3)

[0884] The system according to claim 1, comprising means for standardizing images.

[0885] "Application example 2 when combining with an emotional engine"

[0886] (Claim 1)

[0887] A device and means for acquiring an image,

[0888] A device for extracting feature information from acquired images,

[0889] A functional means for acquiring information from the internet and identifying similar information by comparing it with the obtained characteristic information,

[0890] A device for notifying the administrator of the relevant information,

[0891] A device and means for processing the relevant information,

[0892] A device and means for analyzing emotions from acquired images and applying privacy protection measures based on emotional information,

[0893] A system that includes this.

[0894] (Claim 2)

[0895] The system according to claim 1, comprising means for a device for continuously monitoring and updating information.

[0896] (Claim 3)

[0897] The system according to claim 1, comprising apparatus means for standardizing images. [Explanation of Symbols]

[0898] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for acquiring images, A means for extracting feature information from acquired images, A means for obtaining information from the internet and identifying similar information by comparing it with the obtained characteristic information, A means of notifying the administrator of the relevant information, Means for processing the relevant information, A system that includes this.

2. The system according to claim 1, comprising means for continuously monitoring and updating information.

3. The system according to claim 1, comprising means for standardizing images.

Images