system

Abstract

The system according to the embodiment aims to protect individuals from fraudulent images created by generative AI. [Solution] A system according to an embodiment includes a registration unit, a storage unit, a matching unit, and a control unit. The registration unit registers a facial image of a user. The storage unit stores the facial image registered by the registration unit. The matching unit compares the generated image with the facial image stored in the storage unit when the generation AI generates an image. If the matching unit detects a matching facial image, the control unit prohibits the use of that facial image.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Existing technologies do not adequately protect individuals from fraudulent images created by generative AI, and there is room for improvement.

[0005] The system according to the embodiment aims to protect individuals from fraudulent images created by generative AI. [Means for solving the problem]

[0006] The system according to the embodiment includes a registration unit, a storage unit, a matching unit, and a control unit. The registration unit registers a facial image of a user. The storage unit stores the facial image registered by the registration unit. The matching unit matches the generated image with the facial image stored in the storage unit when the generation AI generates an image. If the matching unit detects a matching facial image, the control unit prohibits the use of the facial image. [Effects of the Invention]

[0007] The system according to the embodiment can protect individuals from fraudulent images created by generative AI. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A facial image protection system according to an embodiment of the present invention is a system for solving the problem of the widespread use of generative AI to create non-existent obscene images for blackmail. In this facial image protection system, a user registers their own facial image in the system and stores the registered facial image in a database. When the generative AI generates an image, it compares it with facial images registered in the database. If there is a match, the facial image is not used. This prevents the user's facial image from being used fraudulently by the generative AI. For example, a user registers their own facial image in the system. The registered facial image is then stored in a database. When the generative AI generates an image, it compares it with facial images registered in the database. If there is a match, the facial image is not used. This system prevents the user's facial image from being used fraudulently by the generative AI, thereby preventing blackmail and other harm. In this way, the facial image protection system prevents the user's facial image from being used fraudulently by the generative AI, thereby preventing blackmail and other harm.

[0029] A facial image protection system according to an embodiment includes a registration unit, a storage unit, a matching unit, and a control unit. The registration unit allows a user to register their own facial image. For example, a user can take a facial image using a dedicated application and upload it to the system. The storage unit stores the facial image registered by the registration unit. The storage unit encrypts and stores the facial image so that only those with access privileges can access it. For example, the storage unit can anonymize the facial image so that it cannot be linked to personal information. The matching unit compares the generated image with the facial image stored in the storage unit when the generation AI generates an image. The matching unit compares the generated image with facial images registered in a database using facial recognition technology that employs feature point extraction and deep learning. For example, the matching unit extracts feature points of the facial image using a feature point extraction algorithm and performs matching using a deep learning model. If the matching unit detects a matching facial image, the control unit prevents the facial image from being used. For example, the control unit controls the generation AI to prevent the matching facial image from being used. This enables the facial image protection system according to an embodiment to prevent the generation AI from misusing a user's facial image.

[0030] The registration unit allows a user to take a facial image using a dedicated application and upload it to the system. The dedicated application is designed to enable a user to easily register a facial image. For example, the application has a camera function that enables a user to take a facial image and provides an interface for uploading the taken facial image to the system. Furthermore, the application displays appropriate guidelines when taking a facial image to help the user take a high-quality facial image. For example, the application provides hints for adjusting the position of the face and lighting conditions. The application also simplifies the process of uploading a facial image, allowing a user to register a facial image to the system in just a few steps. This allows a user to easily register a facial image.

[0031] The storage unit can encrypt and store facial images so that only authorized persons can access them. The storage unit uses encryption technology to ensure the security of the facial images. For example, the storage unit encrypts facial images using encryption algorithms such as AES (Advanced Encryption Standard) or RSA (Rivest-Shamir-Adleman). The encrypted facial images can be decrypted and accessed only by authorized persons. For example, the storage unit provides an encryption key to authorized persons, providing a means for decrypting the encrypted facial images. The storage unit can also anonymize facial images so that they cannot be linked to personal information. For example, to anonymize facial images, the storage unit deletes information that can identify individuals from the facial images and stores the anonymized facial images. This improves the security of facial images and strengthens the protection of personal information.

[0032] The matching unit can use facial recognition technology using feature point extraction or deep learning to match an image generated by the generative AI with facial images registered in a database. The matching unit uses facial recognition technology using feature point extraction or deep learning to improve the accuracy of matching facial images. For example, the matching unit uses algorithms such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded-Up Robust Features) to extract facial feature points. These algorithms extract facial feature points with high accuracy and can be used as a basis for matching. The matching unit can also match facial images using facial recognition technology using deep learning. For example, the matching unit uses a facial recognition model using a CNN (Convolutional Neural Network) to learn the features of facial images and improve matching accuracy. This allows the matching unit to match an image generated by the generative AI with facial images registered in a database with high accuracy.

[0033] If a matching facial image is detected, the control unit can prohibit the use of that facial image. The control unit controls the generation AI to prevent it from using the matching facial image. For example, if a matching facial image is detected by the matching unit, the control unit instructs the generation AI not to use that facial image. The control unit can monitor the image generation process of the generation AI and intervene as necessary to prevent the generation AI from using the matching facial image. For example, when the generation AI generates an image, the control unit compares it with facial images registered in a database, and if there is a match, prevents the generation AI from using the matching facial image. The control unit can also set rules and policies to prevent the generation AI from using the matching facial image. For example, the control unit sets conditions and standards to prevent the generation AI from using the matching facial image and controls based on those conditions and standards. This allows the control unit to prevent the use of unauthorized facial images.

[0034] The storage unit can anonymize the facial image so that it cannot be linked to personal information. To anonymize the facial image, the storage unit deletes information that can identify an individual. For example, the storage unit deletes personal information such as a name and an address from the facial image and stores the anonymized facial image. The storage unit can also mask a portion of the facial image to anonymize the facial image. For example, the storage unit masks distinctive features such as the eyes and mouth of the facial image and stores the anonymized facial image. Furthermore, the storage unit can use technology to maintain the quality of the facial image when anonymizing the facial image. For example, the storage unit maintains the resolution of the facial image when anonymizing the facial image, ensuring that the anonymized facial image is of high quality. In this way, the storage unit can anonymize the facial image so that it cannot be linked to personal information.

[0035] The storage unit periodically updates the facial images registered in the database and can also add newly registered facial images. The storage unit periodically updates the facial image database to keep it up to date. For example, the storage unit periodically checks the facial images registered in the database, deletes old facial images, and adds newly registered facial images. When updating the facial images, the storage unit can also evaluate the quality of the facial images and delete low-quality facial images. For example, the storage unit evaluates the resolution and clarity of the facial images, deletes low-quality facial images, and stores high-quality facial images in the database. Furthermore, when updating the facial images, the storage unit can delete facial images in response to a user request. For example, if a user wishes to delete a facial image, the storage unit can contact a system administrator and request deletion. This allows the storage unit to keep the facial image database up to date and manage facial images in response to a user request.

[0036] The storage unit may allow a user to contact a system administrator and request deletion of a facial image when the user desires to delete the facial image. The storage unit may provide a procedure for a user to contact a system administrator and request deletion when the user desires to delete the facial image. For example, the storage unit may provide an online form for a user to request deletion of a facial image, and the user may enter necessary information in the form and submit it. The storage unit may also provide customer support for a user to request deletion of a facial image. For example, the storage unit may allow a user to contact a system administrator by telephone or email and request deletion of a facial image. Furthermore, the storage unit may also perform a deletion confirmation procedure when deleting a facial image. For example, the storage unit may send a deletion confirmation email after a user requests deletion of a facial image, and delete the facial image after the user confirms the deletion. In this way, the storage unit allows a user to easily request deletion of a facial image, and performs appropriate management of facial images.

[0037] The registration unit can analyze the user's past facial image registration history and select an appropriate registration method. The registration unit uses data analysis technology to analyze the user's past facial image registration history. For example, the registration unit can preferentially suggest a registration method (such as selection from a camera or gallery) that the user has used in the past. The registration unit can also analyze the time period during which the user previously registered images and send a notification prompting registration during that time period. Furthermore, the registration unit can analyze the quality of facial images previously registered by the user and suggest optimal shooting conditions. For example, the registration unit evaluates the resolution and clarity of the facial images and suggests optimal shooting conditions to the user. This allows the registration unit to register facial images in a method optimal for the user.

[0038] The registration unit can perform filtering based on the user's current living situation and areas of interest when registering a facial image. The registration unit uses a user profile to register a facial image taking into account the user's current living situation and areas of interest. For example, if the user is traveling, the registration unit can send a notification encouraging the user to register a facial image at the travel destination. Also, if the user is at work, the registration unit can send a notification encouraging the user to register a facial image in between work. Furthermore, if the user is immersed in a hobby, the registration unit can suggest registering a facial image at a place or time related to the hobby. This allows the registration unit to register a facial image according to the user's living situation and areas of interest.

[0039] When registering a facial image, the registration unit can preferentially register related facial images taking into account the user's geographical location information. The registration unit uses a location information service to register facial images taking into account the user's geographical location information. For example, when the user is in a specific location, the registration unit can preferentially register facial images taken at that location. Also, when the user is traveling, the registration unit can preferentially register facial images taken at the travel destination. Furthermore, when the user is at home, the registration unit can preferentially register facial images taken at home. This allows the registration unit to register the optimal facial image based on the user's geographical location information.

[0040] When registering a facial image, the registration unit can analyze the user's social media activity and register related facial images. The registration unit uses social media analysis technology to analyze the user's social media activity. For example, the registration unit can automatically register facial images posted by the user on social media. It can also preferentially register facial images tagged by the user on social media. It can also analyze facial images shared by the user on social media and register related facial images. This allows the registration unit to register the optimal facial image based on the user's social media activity.

[0041] The storage unit can adjust the details of storage based on the importance of the facial image when storing the facial image. The storage unit uses an importance evaluation algorithm to evaluate the importance of the facial image. For example, the storage unit can evaluate the importance of the facial image and store the facial image with high importance at high resolution and add detailed metadata. Alternatively, the storage unit can store the facial image with low importance at low resolution and minimize the metadata. Furthermore, the storage unit can adjust the storage period of the facial image according to the importance. For example, the storage unit can store the facial image with high importance for a long period of time and the facial image with low importance for a short period of time. This allows the storage unit to store the facial image in an optimal manner according to the importance of the facial image.

[0042] The storage unit can apply different storage algorithms depending on the category of the facial image when storing the facial image. The storage unit uses a category classification algorithm to classify the category of the facial image. For example, the storage unit can encrypt and store private facial images. Also, facial images taken in public places can be compressed and stored. Furthermore, work-related facial images can be stored in high resolution and detailed metadata can be added. This allows the storage unit to store the facial images in an optimal manner depending on the category.

[0043] The storage unit can set a storage priority based on the time of submission of the facial image when storing the facial images. The storage unit uses a submission time evaluation algorithm to evaluate the time of submission of the facial image. For example, the storage unit can prioritize and store recently submitted facial images. Also, the storage unit can store older submitted facial images with lower priority. Furthermore, the storage unit can adjust the storage period of the facial images depending on the time of submission. For example, the storage unit can store recently submitted facial images for a long period of time and older submitted facial images for a short period of time. This allows the storage unit to store the facial images in an optimal manner depending on the time of submission.

[0044] The storage unit can set the order of storage based on the relevance of the facial images when storing them. The storage unit uses a relevance evaluation algorithm to evaluate the relevance of the facial images. For example, the storage unit can prioritize storing highly relevant facial images. Also, the storage unit can store less relevant facial images later. Furthermore, the storage unit can adjust the storage period of the facial images according to their relevance. For example, the storage unit can store highly relevant facial images for a long period of time and less relevant facial images for a short period of time. This allows the storage unit to store the facial images in an optimal manner according to their relevance.

[0045] The matching unit can improve the accuracy of matching by taking into account the interrelationships between facial images during matching. The matching unit uses a interrelationship evaluation algorithm to evaluate the interrelationships between facial images. For example, the matching unit can analyze the interrelationships between facial images and prioritize matching of highly related facial images. The matching unit can also adjust the matching criteria based on the interrelationships between facial images. Furthermore, the matching results can be filtered by taking into account the interrelationships between facial images. For example, the matching unit evaluates the interrelationships between facial images and prioritizes matching of highly related facial images. Next, the matching unit adjusts the matching criteria based on the interrelationships between facial images. Finally, the matching unit filters the matching results by taking into account the interrelationships between facial images. In this way, the matching unit can improve the accuracy of matching by taking into account the interrelationships between facial images.

[0046] During matching, the matching unit can perform matching by taking into account attribute information of the person who submitted the facial image. The matching unit uses an attribute information evaluation algorithm to evaluate the attribute information of the person who submitted the facial image. For example, the matching unit can adjust the matching criteria by taking into account the age and gender of the person who submitted the facial image. The matching unit can also adjust the matching criteria by taking into account the occupation and hobbies of the person who submitted the facial image. Furthermore, the matching unit can adjust the matching criteria by taking into account the place of residence and place of origin of the person who submitted the facial image. For example, the matching unit adjusts the matching criteria by taking into account the age and gender of the person who submitted the facial image. Next, the matching unit adjusts the matching criteria by taking into account the occupation and hobbies of the person who submitted the facial image. Finally, the matching unit adjusts the matching criteria by taking into account the place of residence and place of origin of the person who submitted the facial image. In this way, the matching unit can improve the accuracy of matching by taking into account the attribute information of the person who submitted the facial image.

[0047] During matching, the matching unit can perform matching based on the geographic distribution of face images. The matching unit uses a geographic distribution evaluation algorithm to evaluate the geographic distribution of face images. For example, the matching unit can analyze the geographic distribution of face images and prioritize matching of highly relevant face images. The matching criteria can also be adjusted based on the geographic distribution of face images. Furthermore, the matching results can be filtered taking the geographic distribution of face images into consideration. For example, the matching unit analyzes the geographic distribution of face images and prioritizes matching of highly relevant face images. Next, the matching unit adjusts the matching criteria based on the geographic distribution of face images. Finally, the matching unit filters the matching results taking the geographic distribution of face images into consideration. In this way, the matching unit can improve the accuracy of matching by taking the geographic distribution of face images into consideration.

[0048] The matching unit can improve the accuracy of matching by referring to related literature of the facial image during matching. The matching unit uses a related literature evaluation algorithm to evaluate related literature of the facial image. For example, the matching unit can adjust the matching criteria by referring to related literature of the facial image. The matching unit can also analyze related literature of the facial image and filter the matching results. Furthermore, the matching accuracy can be improved based on the related literature of the facial image. For example, the matching unit adjusts the matching criteria by referring to related literature of the facial image. Next, the matching unit analyzes related literature of the facial image and filters the matching results. Finally, the matching unit improves the accuracy of matching based on the related literature of the facial image. In this way, the matching unit can improve the accuracy of matching by referring to related literature of the facial image.

[0049] The control unit can improve the accuracy of control by taking into account the interrelationships between facial images during control. The control unit uses a interrelationship evaluation algorithm to evaluate the interrelationships between facial images. For example, the control unit can analyze the interrelationships between facial images and prioritize control of highly related facial images. The control unit can also adjust the control criteria based on the interrelationships between facial images. Furthermore, the control results can be filtered by taking into account the interrelationships between facial images. For example, the control unit analyzes the interrelationships between facial images and prioritize control of highly related facial images. Next, the control unit adjusts the control criteria based on the interrelationships between facial images. Finally, the control unit filters the control results by taking into account the interrelationships between facial images. In this way, the control unit can improve the accuracy of control by taking into account the interrelationships between facial images.

[0050] During control, the control unit can perform control based on attribute information of the submitter of the facial image. The control unit uses an attribute information evaluation algorithm to evaluate the attribute information of the submitter of the facial image. For example, the control unit can adjust the control criteria taking into account the submitter's age and gender. The control criteria can also be adjusted taking into account the submitter's occupation and hobbies. The control criteria can also be adjusted taking into account the submitter's place of residence and hometown. For example, the control unit adjusts the control criteria taking into account the submitter's age and gender. Next, the control unit adjusts the control criteria taking into account the submitter's occupation and hobbies. Finally, the control unit adjusts the control criteria taking into account the submitter's place of residence and hometown. In this way, the control unit can improve the accuracy of control by taking into account the submitter's attribute information.

[0051] During control, the control unit can perform control based on the geographic distribution of facial images. The control unit uses a geographic distribution evaluation algorithm to evaluate the geographic distribution of facial images. For example, the control unit can analyze the geographic distribution of facial images and prioritize control of highly relevant facial images. The control unit can also adjust the control criteria based on the geographic distribution of facial images. Furthermore, the control results can be filtered taking the geographic distribution of facial images into consideration. For example, the control unit analyzes the geographic distribution of facial images and prioritize control of highly relevant facial images. Next, the control unit adjusts the control criteria based on the geographic distribution of facial images. Finally, the control unit filters the control results taking the geographic distribution of facial images into consideration. In this way, the control unit can improve the accuracy of control by taking the geographic distribution of facial images into consideration.

[0052] During control, the control unit can improve the accuracy of the control by referring to the literature related to the facial image. The control unit uses a related literature evaluation algorithm to evaluate the literature related to the facial image. For example, the control unit can adjust the criteria for the control by referring to the literature related to the facial image. The control unit can also analyze the literature related to the facial image and filter the control results. The control accuracy can also be improved based on the literature related to the facial image. For example, the control unit adjusts the criteria for the control by referring to the literature related to the facial image. Next, the control unit analyzes the literature related to the facial image and filters the control results. Finally, the control unit improves the accuracy of the control based on the literature related to the facial image. In this way, the control unit can improve the accuracy of the control by referring to the literature related to the facial image.

[0053] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0054] The registration unit can evaluate the quality of the facial image in real time and provide appropriate feedback when the user registers the facial image. For example, the registration unit can evaluate the resolution and lighting conditions of the facial image and provide the user with advice for improvement. The registration unit can also evaluate the clarity of the facial image and the position of the face and prompt the user to take a new photograph. Furthermore, the registration unit can evaluate the background of the facial image and instruct the user to select an appropriate background. In this way, the registration unit can support the user in registering a high-quality facial image.

[0055] The storage unit can automatically generate and store metadata for a facial image when storing the facial image. For example, the storage unit stores the date and time the facial image was photographed and the location where it was photographed as metadata. The storage unit can also store the resolution and file format of the facial image as metadata. Furthermore, the storage unit can store information about the device used to photograph the facial image as metadata. This allows the storage unit to easily manage detailed information about the facial image.

[0056] When matching a facial image, the matching unit can improve the accuracy of the matching by taking into account changes in the facial image. For example, the matching unit performs matching by taking into account changes in the user's face due to age. The matching unit can also perform matching by taking into account changes in the user's facial expression. Furthermore, the matching unit can perform matching by taking into account changes in the user's hairstyle and makeup. This allows the matching unit to perform highly accurate matching by taking into account changes in the facial image.

[0057] The control unit can control the use of the facial image by taking into consideration the user's privacy settings. For example, the control unit can limit the use of the facial image based on the privacy level set by the user. The control unit can also control by taking into consideration the user's setting to allow or prohibit the use of the facial image in a specific application or service. Furthermore, the control unit can control by taking into consideration the expiration date for use of the facial image set by the user. This allows the control unit to perform flexible control according to the user's privacy settings.

[0058] When storing a facial image, the storage unit can adjust the details of storage based on the importance of the facial image. For example, the storage unit can evaluate the importance of a facial image, store a facial image with high importance at high resolution, and add detailed metadata. Alternatively, the storage unit can store a facial image with low importance at low resolution, minimizing the metadata. Furthermore, the storage unit can adjust the storage period of the facial image according to the importance. For example, the storage unit can store a facial image with high importance for a long period of time, and store a facial image with low importance for a short period of time. This allows the storage unit to store the facial image in an optimal manner according to the importance of the facial image.

[0059] The processing flow of the first embodiment will be briefly explained below.

[0060] Step 1: The registration unit allows the user to register their own facial image. For example, the user can take a facial image using a dedicated application and upload it to the system. Step 2: The storage unit stores the facial images registered by the registration unit. The storage unit encrypts and stores the facial images so that only those with access rights can access them. For example, the storage unit can anonymize the facial images so that they cannot be linked to personal information. Step 3: When the generation AI generates an image, the matching unit compares it with the facial images stored in the storage unit. The matching unit uses facial recognition technology that employs feature point extraction and deep learning to match the image with facial images registered in the database. For example, the matching unit uses a feature point extraction algorithm to extract the feature points of the facial image, and then performs matching using a deep learning model. Step 4: If the matching facial image is detected by the matching unit, the control unit prevents the use of the facial image. For example, the control unit controls the generation AI to prevent the generation AI from using the matching facial image. In this way, the facial image protection system according to the embodiment can prevent the user's facial image from being used fraudulently by the generation AI.

[0061] (Example 2) A facial image protection system according to an embodiment of the present invention is a system for solving the problem of the widespread use of generative AI to create non-existent obscene images for blackmail. In this facial image protection system, a user registers their own facial image in the system and stores the registered facial image in a database. When the generative AI generates an image, it compares it with facial images registered in the database. If there is a match, the facial image is not used. This prevents the user's facial image from being used fraudulently by the generative AI. For example, a user registers their own facial image in the system. The registered facial image is then stored in a database. When the generative AI generates an image, it compares it with facial images registered in the database. If there is a match, the facial image is not used. This system prevents the user's facial image from being used fraudulently by the generative AI, thereby preventing blackmail and other harm. In this way, the facial image protection system prevents the user's facial image from being used fraudulently by the generative AI, thereby preventing blackmail and other harm.

[0062] A facial image protection system according to an embodiment includes a registration unit, a storage unit, a matching unit, and a control unit. The registration unit allows a user to register their own facial image. For example, a user can take a facial image using a dedicated application and upload it to the system. The storage unit stores the facial image registered by the registration unit. The storage unit encrypts and stores the facial image so that only those with access privileges can access it. For example, the storage unit can anonymize the facial image so that it cannot be linked to personal information. The matching unit compares the generated image with the facial image stored in the storage unit when the generation AI generates an image. The matching unit compares the generated image with facial images registered in a database using facial recognition technology that employs feature point extraction and deep learning. For example, the matching unit extracts feature points of the facial image using a feature point extraction algorithm and performs matching using a deep learning model. If the matching unit detects a matching facial image, the control unit prevents the facial image from being used. For example, the control unit controls the generation AI to prevent the matching facial image from being used. This enables the facial image protection system according to an embodiment to prevent the generation AI from misusing a user's facial image.

[0063] The registration unit allows a user to take a facial image using a dedicated application and upload it to the system. The dedicated application is designed to enable a user to easily register a facial image. For example, the application has a camera function that enables a user to take a facial image and provides an interface for uploading the taken facial image to the system. Furthermore, the application displays appropriate guidelines when taking a facial image to help the user take a high-quality facial image. For example, the application provides hints for adjusting the position of the face and lighting conditions. The application also simplifies the process of uploading a facial image, allowing a user to register a facial image to the system in just a few steps. This allows a user to easily register a facial image.

[0064] The storage unit can encrypt and store facial images so that only authorized persons can access them. The storage unit uses encryption technology to ensure the security of the facial images. For example, the storage unit encrypts facial images using encryption algorithms such as AES (Advanced Encryption Standard) or RSA (Rivest-Shamir-Adleman). The encrypted facial images can be decrypted and accessed only by authorized persons. For example, the storage unit provides an encryption key to authorized persons, providing a means for decrypting the encrypted facial images. The storage unit can also anonymize facial images so that they cannot be linked to personal information. For example, to anonymize facial images, the storage unit deletes information that can identify individuals from the facial images and stores the anonymized facial images. This improves the security of facial images and strengthens the protection of personal information.

[0065] The matching unit can use facial recognition technology using feature point extraction or deep learning to match an image generated by the generative AI with facial images registered in a database. The matching unit uses facial recognition technology using feature point extraction or deep learning to improve the accuracy of matching facial images. For example, the matching unit uses algorithms such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded-Up Robust Features) to extract facial feature points. These algorithms extract facial feature points with high accuracy and can be used as a basis for matching. The matching unit can also match facial images using facial recognition technology using deep learning. For example, the matching unit uses a facial recognition model using a CNN (Convolutional Neural Network) to learn the features of facial images and improve matching accuracy. This allows the matching unit to match an image generated by the generative AI with facial images registered in a database with high accuracy.

[0066] If a matching facial image is detected, the control unit can prohibit the use of that facial image. The control unit controls the generation AI to prevent it from using the matching facial image. For example, if a matching facial image is detected by the matching unit, the control unit instructs the generation AI not to use that facial image. The control unit can monitor the image generation process of the generation AI and intervene as necessary to prevent the generation AI from using the matching facial image. For example, when the generation AI generates an image, the control unit compares it with facial images registered in a database, and if there is a match, prevents the generation AI from using the matching facial image. The control unit can also set rules and policies to prevent the generation AI from using the matching facial image. For example, the control unit sets conditions and standards to prevent the generation AI from using the matching facial image and controls based on those conditions and standards. This allows the control unit to prevent the use of unauthorized facial images.

[0067] The storage unit can anonymize the facial image so that it cannot be linked to personal information. To anonymize the facial image, the storage unit deletes information that can identify an individual. For example, the storage unit deletes personal information such as a name and an address from the facial image and stores the anonymized facial image. The storage unit can also mask a portion of the facial image to anonymize the facial image. For example, the storage unit masks distinctive features such as the eyes and mouth of the facial image and stores the anonymized facial image. Furthermore, the storage unit can use technology to maintain the quality of the facial image when anonymizing the facial image. For example, the storage unit maintains the resolution of the facial image when anonymizing the facial image, ensuring that the anonymized facial image is of high quality. In this way, the storage unit can anonymize the facial image so that it cannot be linked to personal information.

[0068] The storage unit periodically updates the facial images registered in the database and can also add newly registered facial images. The storage unit periodically updates the facial image database to keep it up to date. For example, the storage unit periodically checks the facial images registered in the database, deletes old facial images, and adds newly registered facial images. When updating the facial images, the storage unit can also evaluate the quality of the facial images and delete low-quality facial images. For example, the storage unit evaluates the resolution and clarity of the facial images, deletes low-quality facial images, and stores high-quality facial images in the database. Furthermore, when updating the facial images, the storage unit can delete facial images in response to a user request. For example, if a user wishes to delete a facial image, the storage unit can contact a system administrator and request deletion. This allows the storage unit to keep the facial image database up to date and manage facial images in response to a user request.

[0069] The storage unit may allow a user to contact a system administrator and request deletion of a facial image when the user desires to delete the facial image. The storage unit may provide a procedure for a user to contact a system administrator and request deletion when the user desires to delete the facial image. For example, the storage unit may provide an online form for a user to request deletion of a facial image, and the user may enter necessary information in the form and submit it. The storage unit may also provide customer support for a user to request deletion of a facial image. For example, the storage unit may allow a user to contact a system administrator by telephone or email and request deletion of a facial image. Furthermore, the storage unit may also perform a deletion confirmation procedure when deleting a facial image. For example, the storage unit may send a deletion confirmation email after a user requests deletion of a facial image, and delete the facial image after the user confirms the deletion. In this way, the storage unit allows a user to easily request deletion of a facial image, and performs appropriate management of facial images.

[0070] The registration unit can estimate the user's emotion and appropriately adjust the timing of registering a facial image based on the estimated user's emotion. The registration unit uses an emotion estimation algorithm to estimate the user's emotion. For example, the registration unit can analyze the user's facial expressions and voice data to estimate the user's emotion. The emotion estimation algorithm uses a model using deep learning to estimate the user's emotion with high accuracy. For example, the registration unit can send a notification urging the user to register a facial image if the user is relaxed. Also, if the user is feeling stressed, the registration unit can suggest postponing the registration of a facial image. Furthermore, if the user is excited, the registration of a facial image can be temporarily suspended and a notification can be sent again when the user has calmed down. This allows the registration unit to register a facial image at the optimal timing according to the user's emotion.

[0071] The registration unit can analyze the user's past facial image registration history and select an appropriate registration method. The registration unit uses data analysis technology to analyze the user's past facial image registration history. For example, the registration unit can preferentially suggest a registration method (such as selection from a camera or gallery) that the user has used in the past. The registration unit can also analyze the time period during which the user previously registered images and send a notification prompting registration during that time period. Furthermore, the registration unit can analyze the quality of facial images previously registered by the user and suggest optimal shooting conditions. For example, the registration unit evaluates the resolution and clarity of the facial images and suggests optimal shooting conditions to the user. This allows the registration unit to register facial images in a method optimal for the user.

[0072] The registration unit can perform filtering based on the user's current living situation and areas of interest when registering a facial image. The registration unit uses a user profile to register a facial image taking into account the user's current living situation and areas of interest. For example, if the user is traveling, the registration unit can send a notification encouraging the user to register a facial image at the travel destination. Also, if the user is at work, the registration unit can send a notification encouraging the user to register a facial image in between work. Furthermore, if the user is immersed in a hobby, the registration unit can suggest registering a facial image at a place or time related to the hobby. This allows the registration unit to register a facial image according to the user's living situation and areas of interest.

[0073] The registration unit can estimate the user's emotion and appropriately determine the priority of facial images to be registered based on the estimated user's emotion. The registration unit uses an emotion estimation algorithm to estimate the user's emotion. For example, the registration unit can analyze the user's facial expressions and voice data to estimate the user's emotion. The emotion estimation algorithm uses a model using deep learning to estimate the user's emotion with high accuracy. For example, if the user is relaxed, the most recent facial image can be registered with priority. Also, if the user is stressed, facial images taken in the past can be registered with priority. Furthermore, if the user is excited, the registration of the facial image can be temporarily suspended and a notification can be sent again when the user has calmed down. This allows the registration unit to prioritize the registration of the optimal facial image according to the user's emotion.

[0074] When registering a facial image, the registration unit can preferentially register related facial images taking into account the user's geographical location information. The registration unit uses a location information service to register facial images taking into account the user's geographical location information. For example, when the user is in a specific location, the registration unit can preferentially register facial images taken at that location. Also, when the user is traveling, the registration unit can preferentially register facial images taken at the travel destination. Furthermore, when the user is at home, the registration unit can preferentially register facial images taken at home. This allows the registration unit to register the optimal facial image based on the user's geographical location information.

[0075] When registering a facial image, the registration unit can analyze the user's social media activity and register related facial images. The registration unit uses social media analysis technology to analyze the user's social media activity. For example, the registration unit can automatically register facial images posted by the user on social media. It can also preferentially register facial images tagged by the user on social media. It can also analyze facial images shared by the user on social media and register related facial images. This allows the registration unit to register the optimal facial image based on the user's social media activity.

[0076] The storage unit can estimate the user's emotion and appropriately adjust the method of storing the facial image based on the estimated user's emotion. The storage unit uses an emotion estimation algorithm to estimate the user's emotion. For example, the storage unit can analyze the user's facial expression and voice data to estimate the user's emotion. The emotion estimation algorithm uses a model using deep learning to estimate the user's emotion with high accuracy. For example, if the user is relaxed, the facial image can be stored at high resolution. Also, if the user is stressed, the facial image can be stored at low resolution. Furthermore, if the user is excited, the storage of the facial image can be temporarily suspended and the storage can be attempted again when the user has calmed down. This allows the storage unit to store the facial image in an optimal manner according to the user's emotion.

[0077] The storage unit can adjust the details of storage based on the importance of the facial image when storing the facial image. The storage unit uses an importance evaluation algorithm to evaluate the importance of the facial image. For example, the storage unit can evaluate the importance of the facial image and store the facial image with high importance at high resolution and add detailed metadata. Alternatively, the storage unit can store the facial image with low importance at low resolution and minimize the metadata. Furthermore, the storage unit can adjust the storage period of the facial image according to the importance. For example, the storage unit can store the facial image with high importance for a long period of time and the facial image with low importance for a short period of time. This allows the storage unit to store the facial image in an optimal manner according to the importance of the facial image.

[0078] The storage unit can apply different storage algorithms depending on the category of the facial image when storing the facial image. The storage unit uses a category classification algorithm to classify the category of the facial image. For example, the storage unit can encrypt and store private facial images. Also, facial images taken in public places can be compressed and stored. Furthermore, work-related facial images can be stored in high resolution and detailed metadata can be added. This allows the storage unit to store the facial images in an optimal manner depending on the category.

[0079] The storage unit can estimate the user's emotion and appropriately determine the priority of facial images to be saved based on the estimated user's emotion. The storage unit uses an emotion estimation algorithm to estimate the user's emotion. For example, the storage unit can analyze the user's facial expressions and voice data to estimate the user's emotion. The emotion estimation algorithm uses a model using deep learning to estimate the user's emotion with high accuracy. For example, if the user is relaxed, the latest facial image can be saved preferentially. Also, if the user is stressed, facial images taken in the past can be saved preferentially. Furthermore, if the user is excited, the storage of facial images can be temporarily suspended and the user can try saving them again when they have calmed down. This allows the storage unit to save the optimal facial image preferentially according to the user's emotion.

[0080] The storage unit can set a storage priority based on the time of submission of the facial image when storing the facial images. The storage unit uses a submission time evaluation algorithm to evaluate the time of submission of the facial image. For example, the storage unit can prioritize and store recently submitted facial images. Also, the storage unit can store older submitted facial images with lower priority. Furthermore, the storage unit can adjust the storage period of the facial images depending on the time of submission. For example, the storage unit can store recently submitted facial images for a long period of time and older submitted facial images for a short period of time. This allows the storage unit to store the facial images in an optimal manner depending on the time of submission.

[0081] The storage unit can set the order of storage based on the relevance of the facial images when storing them. The storage unit uses a relevance evaluation algorithm to evaluate the relevance of the facial images. For example, the storage unit can prioritize storing highly relevant facial images. Also, the storage unit can store less relevant facial images later. Furthermore, the storage unit can adjust the storage period of the facial images according to their relevance. For example, the storage unit can store highly relevant facial images for a long period of time and less relevant facial images for a short period of time. This allows the storage unit to store the facial images in an optimal manner according to their relevance.

[0082] The matching unit can estimate the user's emotion and appropriately adjust the matching criteria based on the estimated user's emotion. The matching unit uses an emotion estimation algorithm to estimate the user's emotion. For example, the matching unit can analyze the user's facial expressions and voice data to estimate the user's emotion. The emotion estimation algorithm uses a model using deep learning to estimate the user's emotion with high accuracy. For example, if the user is relaxed, the matching criteria can be made stricter. Also, if the user is feeling stressed, the matching criteria can be made looser. Furthermore, if the user is excited, the matching criteria can be temporarily suspended and matching can be attempted again when the user has calmed down. This allows the matching unit to match face images using optimal criteria according to the user's emotion.

[0083] The matching unit can improve the accuracy of matching by taking into account the interrelationships between facial images during matching. The matching unit uses a interrelationship evaluation algorithm to evaluate the interrelationships between facial images. For example, the matching unit can analyze the interrelationships between facial images and prioritize matching of highly related facial images. The matching unit can also adjust the matching criteria based on the interrelationships between facial images. Furthermore, the matching results can be filtered by taking into account the interrelationships between facial images. For example, the matching unit evaluates the interrelationships between facial images and prioritizes matching of highly related facial images. Next, the matching unit adjusts the matching criteria based on the interrelationships between facial images. Finally, the matching unit filters the matching results by taking into account the interrelationships between facial images. In this way, the matching unit can improve the accuracy of matching by taking into account the interrelationships between facial images.

[0084] During matching, the matching unit can perform matching by taking into account attribute information of the person who submitted the facial image. The matching unit uses an attribute information evaluation algorithm to evaluate the attribute information of the person who submitted the facial image. For example, the matching unit can adjust the matching criteria by taking into account the age and gender of the person who submitted the facial image. The matching unit can also adjust the matching criteria by taking into account the occupation and hobbies of the person who submitted the facial image. Furthermore, the matching unit can adjust the matching criteria by taking into account the place of residence and place of origin of the person who submitted the facial image. For example, the matching unit adjusts the matching criteria by taking into account the age and gender of the person who submitted the facial image. Next, the matching unit adjusts the matching criteria by taking into account the occupation and hobbies of the person who submitted the facial image. Finally, the matching unit adjusts the matching criteria by taking into account the place of residence and place of origin of the person who submitted the facial image. In this way, the matching unit can improve the accuracy of matching by taking into account the attribute information of the person who submitted the facial image.

[0085] The matching unit can estimate the user's emotions and appropriately set the order in which to display the matching results based on the estimated user's emotions. The matching unit uses an emotion estimation algorithm to estimate the user's emotions. For example, the matching unit can analyze the user's facial expressions and voice data to estimate the user's emotions. The emotion estimation algorithm uses a model based on deep learning to estimate the user's emotions with high accuracy. For example, if the user is relaxed, detailed matching results can be displayed preferentially. Also, if the user is stressed, concise matching results can be displayed preferentially. Furthermore, if the user is excited, the display of the matching results can be temporarily suspended and re-displayed when the user has calmed down. This allows the matching unit to display the matching results in an optimal order according to the user's emotions.

[0086] During matching, the matching unit can perform matching based on the geographic distribution of face images. The matching unit uses a geographic distribution evaluation algorithm to evaluate the geographic distribution of face images. For example, the matching unit can analyze the geographic distribution of face images and prioritize matching of highly relevant face images. The matching criteria can also be adjusted based on the geographic distribution of face images. Furthermore, the matching results can be filtered taking the geographic distribution of face images into consideration. For example, the matching unit analyzes the geographic distribution of face images and prioritizes matching of highly relevant face images. Next, the matching unit adjusts the matching criteria based on the geographic distribution of face images. Finally, the matching unit filters the matching results taking the geographic distribution of face images into consideration. In this way, the matching unit can improve the accuracy of matching by taking the geographic distribution of face images into consideration.

[0087] The matching unit can improve the accuracy of matching by referring to related literature of the facial image during matching. The matching unit uses a related literature evaluation algorithm to evaluate related literature of the facial image. For example, the matching unit can adjust the matching criteria by referring to related literature of the facial image. The matching unit can also analyze related literature of the facial image and filter the matching results. Furthermore, the matching accuracy can be improved based on the related literature of the facial image. For example, the matching unit adjusts the matching criteria by referring to related literature of the facial image. Next, the matching unit analyzes related literature of the facial image and filters the matching results. Finally, the matching unit improves the accuracy of matching based on the related literature of the facial image. In this way, the matching unit can improve the accuracy of matching by referring to related literature of the facial image.

[0088] The control unit can estimate the user's emotion and appropriately adjust the control method based on the estimated user's emotion. The control unit uses an emotion estimation algorithm to estimate the user's emotion. For example, the control unit can analyze the user's facial expression and voice data to estimate the user's emotion. The emotion estimation algorithm uses a model using deep learning to estimate the user's emotion with high accuracy. For example, if the user is relaxed, the control standard can be tightened. Also, if the user is stressed, the control standard can be loosened. Furthermore, if the user is excited, the control standard can be temporarily suspended and control can be attempted again when the user has calmed down. This allows the control unit to control the use of facial images in an optimal manner according to the user's emotion.

[0089] The control unit can improve the accuracy of control by taking into account the interrelationships between facial images during control. The control unit uses a interrelationship evaluation algorithm to evaluate the interrelationships between facial images. For example, the control unit can analyze the interrelationships between facial images and prioritize control of highly related facial images. The control unit can also adjust the control criteria based on the interrelationships between facial images. Furthermore, the control results can be filtered by taking into account the interrelationships between facial images. For example, the control unit analyzes the interrelationships between facial images and prioritize control of highly related facial images. Next, the control unit adjusts the control criteria based on the interrelationships between facial images. Finally, the control unit filters the control results by taking into account the interrelationships between facial images. In this way, the control unit can improve the accuracy of control by taking into account the interrelationships between facial images.

[0090] During control, the control unit can perform control based on attribute information of the submitter of the facial image. The control unit uses an attribute information evaluation algorithm to evaluate the attribute information of the submitter of the facial image. For example, the control unit can adjust the control criteria taking into account the submitter's age and gender. The control criteria can also be adjusted taking into account the submitter's occupation and hobbies. The control criteria can also be adjusted taking into account the submitter's place of residence and hometown. For example, the control unit adjusts the control criteria taking into account the submitter's age and gender. Next, the control unit adjusts the control criteria taking into account the submitter's occupation and hobbies. Finally, the control unit adjusts the control criteria taking into account the submitter's place of residence and hometown. In this way, the control unit can improve the accuracy of control by taking into account the submitter's attribute information.

[0091] The control unit can estimate the user's emotions and appropriately determine control priorities based on the estimated user emotions. The control unit uses an emotion estimation algorithm to estimate the user's emotions. For example, the control unit can analyze the user's facial expressions and voice data to estimate the user's emotions. The emotion estimation algorithm uses a model based on deep learning to estimate the user's emotions with high accuracy. For example, if the user is relaxed, the most recent facial image can be preferentially controlled. Also, if the user is stressed, the control unit can preferentially control a facial image taken in the past. Furthermore, if the user is excited, the control unit can temporarily suspend control of the facial image and attempt control again when the user has calmed down. This allows the control unit to preferentially control the optimal facial image according to the user's emotions.

[0092] During control, the control unit can perform control based on the geographic distribution of facial images. The control unit uses a geographic distribution evaluation algorithm to evaluate the geographic distribution of facial images. For example, the control unit can analyze the geographic distribution of facial images and prioritize control of highly relevant facial images. The control unit can also adjust the control criteria based on the geographic distribution of facial images. Furthermore, the control results can be filtered taking the geographic distribution of facial images into consideration. For example, the control unit analyzes the geographic distribution of facial images and prioritize control of highly relevant facial images. Next, the control unit adjusts the control criteria based on the geographic distribution of facial images. Finally, the control unit filters the control results taking the geographic distribution of facial images into consideration. In this way, the control unit can improve the accuracy of control by taking the geographic distribution of facial images into consideration.

[0093] During control, the control unit can improve the accuracy of the control by referring to the literature related to the facial image. The control unit uses a related literature evaluation algorithm to evaluate the literature related to the facial image. For example, the control unit can adjust the criteria for the control by referring to the literature related to the facial image. The control unit can also analyze the literature related to the facial image and filter the control results. The control accuracy can also be improved based on the literature related to the facial image. For example, the control unit adjusts the criteria for the control by referring to the literature related to the facial image. Next, the control unit analyzes the literature related to the facial image and filters the control results. Finally, the control unit improves the accuracy of the control based on the literature related to the facial image. In this way, the control unit can improve the accuracy of the control by referring to the literature related to the facial image. === Hard Collateral 1-1 === Each of the multiple elements, including the registration unit, storage unit, matching unit, and control unit, described above, is implemented, for example, by at least one of the smart device 14 and the data processing device 12. For example, the registration unit is implemented by the control unit 46A of the smart device 14, allowing a user to take a facial image using a dedicated application and upload it to the system. The storage unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, encrypting and storing the registered facial image so that only those with access rights can access it. The matching unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, and uses facial recognition technology using feature point extraction and deep learning to match the registered facial image with a facial image registered in a database. The control unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, and when a matching facial image is detected, controls the generation AI not to use that facial image. === Hard Collateral 1-2 === Each of the multiple elements, including the registration unit, storage unit, matching unit, and control unit, described above, is implemented, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the registration unit is implemented by the control unit 46A of the smart glasses 214, allowing a user to take a facial image using a dedicated application and upload it to the system. The storage unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, encrypting and storing the registered facial image so that only those with access rights can access it. The matching unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, and uses facial recognition technology using feature point extraction and deep learning to match the registered facial image with a facial image registered in a database. The control unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, and when a matching facial image is detected, controls the generation AI not to use that facial image. === Hard Collateral 1-3 === Each of the multiple elements, including the registration unit, storage unit, matching unit, and control unit, described above, is implemented, for example, by at least one of the headset-type terminal 314 and the data processing device 12. For example, the registration unit is implemented by the control unit 46A of the headset-type terminal 314, allowing a user to take a facial image using a dedicated application and upload it to the system. The storage unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, encrypting and storing the registered facial image so that only those with access rights can access it. The matching unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, and uses facial recognition technology such as feature point extraction and deep learning to match the facial image with a facial image registered in a database. The control unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, and when a matching facial image is detected, controls the generation AI not to use that facial image. === Hard Collateral 1-4 === Each of the multiple elements, including the registration unit, storage unit, matching unit, and control unit, described above, is implemented, for example, by at least one of the robot 414 and the data processing device 12. For example, the registration unit is implemented by the control unit 46A of the robot 414, allowing a user to take a facial image using a dedicated application and upload it to the system. The storage unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, encrypting and storing the registered facial image so that only those with access rights can access it. The matching unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, and uses facial recognition technology using feature point extraction and deep learning to match the registered facial image with a facial image registered in a database. The control unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, and when a matching facial image is detected, controls the generation AI not to use that facial image.

[0094] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0095] The registration unit can evaluate the quality of the facial image in real time and provide appropriate feedback when the user registers the facial image. For example, the registration unit can evaluate the resolution and lighting conditions of the facial image and provide the user with advice for improvement. The registration unit can also evaluate the clarity of the facial image and the position of the face and prompt the user to take a new photograph. Furthermore, the registration unit can evaluate the background of the facial image and instruct the user to select an appropriate background. In this way, the registration unit can support the user in registering a high-quality facial image.

[0096] The storage unit can automatically generate and store metadata for a facial image when storing the facial image. For example, the storage unit stores the date and time the facial image was photographed and the location where it was photographed as metadata. The storage unit can also store the resolution and file format of the facial image as metadata. Furthermore, the storage unit can store information about the device used to photograph the facial image as metadata. This allows the storage unit to easily manage detailed information about the facial image.

[0097] When matching a facial image, the matching unit can improve the accuracy of the matching by taking into account changes in the facial image. For example, the matching unit performs matching by taking into account changes in the user's face due to age. The matching unit can also perform matching by taking into account changes in the user's facial expression. Furthermore, the matching unit can perform matching by taking into account changes in the user's hairstyle and makeup. This allows the matching unit to perform highly accurate matching by taking into account changes in the facial image.

[0098] The control unit can control the use of the facial image by taking into consideration the user's privacy settings. For example, the control unit can limit the use of the facial image based on the privacy level set by the user. The control unit can also control by taking into consideration the user's setting to allow or prohibit the use of the facial image in a specific application or service. Furthermore, the control unit can control by taking into consideration the expiration date for use of the facial image set by the user. This allows the control unit to perform flexible control according to the user's privacy settings.

[0099] The registration unit can estimate the user's emotions and customize the facial image registration process based on the estimated user's emotions. For example, if the user is nervous, the registration unit can provide a guide to relax. Also, if the user is happy, the registration unit can provide a simplified procedure to quickly register the facial image. Furthermore, if the user is sad, the registration unit can temporarily suspend the registration of the facial image and send a notification again later. This allows the registration unit to provide a flexible registration process according to the user's emotions.

[0100] The storage unit can estimate the user's emotions when saving face images and set a priority for saving based on the estimated emotions. For example, if the user is relaxed, the latest face image can be saved with priority. Also, if the user is stressed, face images taken in the past can be saved with priority. Furthermore, if the user is excited, the storage unit can temporarily suspend saving of face images and try saving them again when the user has calmed down. This allows the storage unit to save the most appropriate face image with priority according to the user's emotions.

[0101] The matching unit can estimate the user's emotions and adjust the matching criteria based on the estimated emotions. For example, if the user is relaxed, the matching criteria can be made stricter. Also, if the user is feeling stressed, the matching criteria can be made looser. Furthermore, if the user is excited, the matching criteria can be temporarily suspended and matching can be attempted again when the user has calmed down. This allows the matching unit to match face images using optimal criteria according to the user's emotions.

[0102] The control unit can estimate the user's emotions and adjust the control method based on the estimated emotions. For example, if the user is relaxed, the control standards can be tightened. Also, if the user is feeling stressed, the control standards can be loosened. Furthermore, if the user is excited, the control standards can be temporarily suspended and the control can be attempted again when the user has calmed down. In this way, the control unit can control the use of facial images in an optimal manner according to the user's emotions.

[0103] The registration unit can estimate the user's emotions and determine the priority of facial images to be registered based on the estimated emotions. For example, if the user is relaxed, the most recent facial image can be registered with priority. Also, if the user is stressed, facial images taken in the past can be registered with priority. Furthermore, if the user is excited, the registration of a facial image can be temporarily suspended and a notification can be sent again when the user has calmed down. This allows the registration unit to register with priority the most appropriate facial image according to the user's emotions.

[0104] When storing a facial image, the storage unit can adjust the details of storage based on the importance of the facial image. For example, the storage unit can evaluate the importance of a facial image, store a facial image with high importance at high resolution, and add detailed metadata. Alternatively, the storage unit can store a facial image with low importance at low resolution, minimizing the metadata. Furthermore, the storage unit can adjust the storage period of the facial image according to the importance. For example, the storage unit can store a facial image with high importance for a long period of time, and store a facial image with low importance for a short period of time. This allows the storage unit to store the facial image in an optimal manner according to the importance of the facial image.

[0105] The processing flow of the second embodiment will be briefly explained below.

[0106] Step 1: The registration unit allows the user to register their own facial image. For example, the user can take a facial image using a dedicated application and upload it to the system. Step 2: The storage unit stores the facial images registered by the registration unit. The storage unit encrypts and stores the facial images so that only those with access rights can access them. For example, the storage unit can anonymize the facial images so that they cannot be linked to personal information. Step 3: When the generation AI generates an image, the matching unit compares it with the facial images stored in the storage unit. The matching unit uses facial recognition technology that employs feature point extraction and deep learning to match the image with facial images registered in the database. For example, the matching unit uses a feature point extraction algorithm to extract the feature points of the facial image, and then performs matching using a deep learning model. Step 4: If the matching facial image is detected by the matching unit, the control unit prevents the use of the facial image. For example, the control unit controls the generation AI to prevent the generation AI from using the matching facial image. In this way, the facial image protection system according to the embodiment can prevent the user's facial image from being used fraudulently by the generation AI.

[0107] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0108] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of the generative AI include a neural network (NN) and a neural network (NN). The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, image data, etc. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and may perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-mentioned parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. The processing performed by an AI including the generative AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI including the generative AI.

[0109] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0110] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0111] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0112] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0113] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0114] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0115] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0116] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0117] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0118] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0119] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0120] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0121] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0122] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0123] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0124] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0125] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0126] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0127] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0128] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0129] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0130] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0131] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0132] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0133] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0134] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0135] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0136] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0137] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0138] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0139] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0140] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0141] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0142] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0143] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0144] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0145] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0146] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0147] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0148] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0149] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0150] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0151] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0152] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0153] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0154] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0155] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0156] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0157] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0158] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0159] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0160] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0161] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0162] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0163] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0164] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0165] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0166] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0167] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0168] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0169] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0170] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0171] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0172] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0173] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0174] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0175] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0176] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0177] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0178] [Explanation of symbols]

[0179] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a registration unit for registering a face image of a user; a storage unit for storing the face images registered by the registration unit; a matching unit that matches the generated image with the face image stored in the storage unit when the generation AI generates the image; a control unit that prohibits the use of a facial image when a matching facial image is detected by the matching unit. A system characterized by:

2. The registration unit The user takes a facial image using a dedicated application and uploads it to the system.

2. The system of claim 1.

3. The storage unit Facial images are encrypted and stored so that only authorized individuals can access them.

2. The system of claim 1.

4. The collation unit Using facial recognition technology based on feature point extraction or deep learning, the AI ​​generates an image and matches it with facial images registered in a database.

2. The system of claim 1.

5. The control unit If a matching face image is found, the face image is prohibited from use.

2. The system of claim 1.

6. The storage unit Anonymize facial images so they cannot be linked to personal information 2. The system of claim 1.

7. The storage unit Regularly update the face images registered in the database and add newly registered face images.

2. The system of claim 1.

8. The storage unit If the user wishes to delete the face image, he or she should contact the system administrator and request the deletion.

2. The system of claim 1.

Images