system

Abstract

Provide a system. [Solution] a means for uploading the generated image file; A means of detecting bias or errors in uploaded image files; a means of selecting raters with different backgrounds; a means for obtaining feedback from the selected raters; means for presenting the obtained feedback to the user; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Describe the "problem that the invention aims to solve" and the "means for solving the problem."

[0005] While image-generating AI technology is rapidly evolving, there are numerous biases and errors in the images generated. This increases the risk of unfair and biased artwork being distributed. Another issue is the inefficient process for properly evaluating images generated by artists and obtaining feedback. Furthermore, language and cultural barriers exist, making it difficult to obtain sufficient feedback from evaluators with diverse backgrounds. [Means for solving the problem]

[0006] The present invention solves the above-mentioned problems by providing a system that includes a means for uploading generated image files, a means for detecting bias and errors from the uploaded image files, a means for selecting evaluators with different backgrounds, a means for obtaining feedback from the selected evaluators, and a means for presenting the obtained feedback to a user. This system automatically detects bias and errors and enables feedback from evaluators with diverse backgrounds. Furthermore, by using an automatic translation function for the feedback, language barriers are eliminated, allowing users to create artwork that reflects opinions from unbiased and diverse perspectives.

[0007] A "generated image file" is a digital image file generated using image generation AI technology.

[0008] "Means for uploading" refers to the mechanism and function that allows a user to transfer image files from their own device to the server.

[0009] "Measures to detect bias and errors" refers to an AI detection engine and its algorithms for identifying bias and errors contained in uploaded image files.

[0010] "Evaluators with different backgrounds" are human evaluators with various fields of expertise and cultural backgrounds, or AI that simulates them.

[0011] "Means of selection" refers to the interface and functionality that allows a user to select a specific evaluator on the platform.

[0012] "Means for obtaining feedback" refers to a communication system and associated functions for collecting opinions and evaluations from selected evaluators.

[0013] "Presentation means" refers to a display function or interface for providing the obtained feedback to the user in a visual or other form. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators, allowing users to create artwork that incorporates unbiased and diverse perspectives.

[0036] Program processing

[0037] 1. User uploads an image

[0038] Subject: User

[0039] Users upload images generated using the generative AI from their devices to the server, where the uploaded image files are stored.

[0040] 2. The server detects bias and errors

[0041] Subject: Server

[0042] The server receives the uploaded images and automatically detects bias and errors using a bias detection engine, which uses pre-trained models.

[0043] 3. The user selects the evaluator

[0044] Subject: User

[0045] After reviewing the results of bias and error detection, users can select the evaluator from whom they would like to receive feedback from a list of evaluators displayed on the system. The list displays the evaluators' fields of expertise and profiles, and they come from a variety of backgrounds.

[0046] 4. The server collects feedback

[0047] Subject: Server

[0048] The server sends the uploaded images and bias detection results to the selected evaluators and asks for their feedback. The evaluators then check the images and information on their own devices and enter their feedback.

[0049] 5. Evaluators provide feedback

[0050] Subject: Terminal (evaluator)

[0051] Evaluators use their devices to send feedback to the server, including specific comments, like / dislike ratings, and unpleasantness indices.

[0052] 6. The server aggregates and presents the feedback

[0053] Subject: Server

[0054] The server aggregates the feedback collected from the evaluators and presents it to the user. If necessary, the server automatically translates the feedback and provides it in multiple languages ​​in a format that is easy for the user to understand.

[0055] 7. Tweak the image based on user feedback

[0056] Subject: User

[0057] Users can then use the feedback they provide to fine-tune the images and generative AI models, and the process is repeated until the end result is a less biased and more diverse piece of art.

[0058] Specific examples

[0059] Example 1: Evaluating and correcting artist-generated portraits

[0060] Artist X creates a portrait using generative AI and uploads it to the platform, where the server detects that the portrait contains unofficial gender bias.

[0061] Artist X selects evaluator Y, an expert on gender perspectives. The server sends the image and bias information to evaluator Y and asks for feedback.

[0062] Evaluator Y uses a device to check the image and submits feedback including specific suggestions for correction. The server presents the feedback to artist X and performs automatic translation if translation is required.

[0063] Based on the feedback, Artist X adjusts the generative AI model to recreate portraits with less gender bias.

[0064] Example 2: Advertising agency bias correction of advertising images

[0065] A designer at an advertising agency uses generative AI to create and upload images for advertisements, and the server detects if the images contain racial stereotypes.

[0066] The designer selects Evaluator Z, an expert on racial diversity, and asks for feedback. The server sends the image and the detection results to Evaluator Z.

[0067] The server receives the feedback from rater Z and presents it to the designer, translating it if necessary. The designer then adjusts the generative AI model based on the feedback to recreate the advertising image with fewer racial stereotypes.

[0068] As described above, the system of the present invention effectively identifies and corrects biases and errors in generated images, helping users create artworks based on fair and diverse perspectives.

[0069] The processing flow will be explained below.

[0070] Step 1:

[0071] Users prepare the image generated by the generative AI on their device, then drag and drop the image file into the platform's upload interface and click the upload button, which sends the image file to the server.

[0072] Step 2:

[0073] The server receives the uploaded image file, saves it in storage, and records the path and metadata of the saved file in the database.

[0074] Step 3:

[0075] The server then sends the stored image files to a bias detection engine, which uses pre-trained models to detect bias and errors in the images and generates a report containing the type and location of identified bias.

[0076] Step 4:

[0077] The server stores the results of the detection in a database and notifies the user, including a summary of the bias and instructions for next steps.

[0078] Step 5:

[0079] Users check the bias detection results on the platform, and then select the evaluator they would like to receive feedback from from the list of evaluators provided by the system.

[0080] Step 6:

[0081] The server sends the uploaded image file and bias detection results to the evaluator selected by the user and generates a feedback request.

[0082] Step 7:

[0083] The evaluator will use their own device to check the submitted images and bias information, enter comments, specific indications of bias, and suggestions for correction in the feedback form, and then press the submit button.

[0084] Step 8:

[0085] The server receives feedback data returned by the evaluators, aggregates and stores it in a database, and automatically translates the feedback data if necessary.

[0086] Step 9:

[0087] The server then presents the aggregated feedback to the user, including the reviewer's comments, Like / Dislike ratings, and an unpleasantness index. The feedback is presented in a format that is easy for the user to understand.

[0088] Step 10:

[0089] Users can fine-tune the image and generative AI model based on the feedback they provide, and then upload the regenerated image using the same process, seeking further feedback if needed.

[0090] This is the specific process flow, which effectively corrects bias and errors in the generated images and produces artwork that reflects fair and diverse perspectives.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] Current image generation processes using generative AI technology may contain biases and errors in the generated images. These biases and errors can result in unintended losses in diversity and fairness. Furthermore, the process of evaluating and correcting generated images is cumbersome and requires a lot of time and effort. Furthermore, there is a lack of systems that can collect feedback from evaluators with different backgrounds and provide it in a format that is easy for users to understand. There is a need for a system that can resolve these issues and support the generation of high-quality images based on fair and diverse perspectives.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes a means for uploading generated image files, a means for detecting biases and errors from the uploaded image files, a means for presenting the results of the bias and error detection to a user and selecting evaluators with different backgrounds, a means for collecting feedback from the selected evaluators, and a means for providing the collected feedback to a user and supporting fine-tuning of the generated images. This makes it possible to effectively detect and correct biases and errors in the generated images and generate high-quality images based on diverse viewpoints.

[0096] An "image file" is digital data containing visual information generated using a generative AI model.

[0097] "Uploading" is the act of sending digital data (image files) from a user's device to a server and storing it.

[0098] "Bias" refers to biased viewpoints or inappropriate elements contained in the content of image files, which have an unfair influence on a particular classification or evaluation.

[0099] "Error" refers to any inaccuracy or inappropriate element in the content of an image file that deviates from the intended result.

[0100] "Evaluators" are experts or users with different backgrounds who provide feedback on the generated image files.

[0101] "Feedback" is a response, including comments, ratings, and suggested modifications, provided by a reviewer to an image file.

[0102] "User" refers to an individual or organization that uses the system to generate image files using a generative AI model.

[0103] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate image files based on input such as prompts from users.

[0104] A "prompt sentence" is text information entered by the user to convey instructions and intentions for image generation to the generative AI model.

[0105] The present invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators. This system allows users to create artwork that incorporates unbiased and diverse perspectives. Specific embodiments of the present invention are described below.

[0106] Overall system overview

[0107] The system is implemented using the following hardware and software:

[0108] User device: For generating and uploading images and checking feedback

[0109] Server: Image file storage, bias and error detection, feedback collection and presentation

[0110] Generative AI models: Examples include OpenAI's DALL-E and MidJourney

[0111] Bias detection engine: For example, OpenAI's CLIP

[0112] Translation Engine: Automatic translation tool for multilingual feedback

[0113] System Operation

[0114] User generates and uploads an image

[0115] The user uses a device to operate a generative AI model (e.g., DALL-E or MidJourney) and generate an image by entering the desired prompt. An example prompt might be: "A cat dancing under the night sky." The generated image is saved on the user's device.

[0116] Next, the user uploads the image file to the server via the system's web interface. By pressing the upload button, the image file is sent to the server and saved in the server's storage.

[0117] Detecting bias and error

[0118] The server receives the uploaded image file and analyzes the image content using a bias detection engine (e.g., OpenAI's CLIP). This analysis engine uses pre-trained models to detect known bias and error patterns. The detection results are saved in a format such as JSON and later provided to the user.

[0119] User-selected raters

[0120] The user checks the results of bias and error detection through the system's web interface. The interface displays a list of evaluators, each of which includes their area of ​​expertise and profile. The user selects an appropriate evaluator from the list and presses the evaluation request button.

[0121] Gathering feedback from evaluators

[0122] The server notifies the selected evaluator of the uploaded image and the bias detection results. The evaluator logs into the system on their own device and checks the image and detection results. The evaluator enters specific comments and evaluations into the feedback form and presses the submit button to send the feedback to the server.

[0123] Providing feedback and fine-tuning images

[0124] The server consolidates the collected feedback and provides it to the user. If necessary, the feedback is translated into multiple languages ​​using a translation engine. The user then uses the provided feedback to fine-tune the image using a generative AI model or image editing software. By repeating the same process, the resulting artwork is based on a less biased, fairer, and more diverse perspective.

[0125] As a concrete example:

[0126] Example 1: Evaluating and correcting artist-generated portraits

[0127] Artists create portraits using generative AI models and upload them to the system, where the server detects any gender bias in the portraits.

[0128] The artist selects evaluators who are experts in gender perspectives, and the server sends the images and bias information to the evaluators and asks for their feedback.

[0129] The evaluator submits feedback with specific suggestions for correction, and the server provides the feedback to the artist, translating it if necessary.

[0130] Based on the feedback, the artist adjusts the generative AI model to generate new portraits.

[0131] Using these steps, the system of the present invention can effectively detect and correct biases and errors in the image generation process, helping users to create high-quality artwork that incorporates diverse perspectives.

[0132] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0133] Step 1:

[0134] Image generation and upload

[0135] The user uses the generative AI model to generate images by entering a prompt.

[0136] Input: Prompt sentence (e.g. "Cats dancing under the night sky")

[0137] Output: Generated image file

[0138] Users save the generated image files to their own devices, then log in to the system using a dedicated web interface and upload the saved image files.

[0139] Input: Generated image file

[0140] Output: Image file stored on the server

[0141] Step 2:

[0142] Detecting bias and error

[0143] The server receives the uploaded image file and then analyzes the image using a bias detection engine (which includes a pre-trained model).

[0144] Input: Uploaded image file

[0145] Output: Bias and error detection results (e.g., JSON format)

[0146] Specifically, the server passes the image to an analysis engine, which detects known patterns of bias and error, and stores the results on the server for later review by the user.

[0147] Step 3:

[0148] Presenting bias and error findings

[0149] The server presents the bias and error detection results to the user, who can review these results through a web interface.

[0150] Input: Bias and error detection results

[0151] Output: User-viewable reports

[0152] Specifically, the server displays the detection results on a web interface in a format that is easy for the user to understand.

[0153] Step 4:

[0154] Selection of raters

[0155] The user selects the evaluator from whom they wish to receive feedback from the list of evaluators displayed on the system.

[0156] Input: Rater list and user selection of rater

[0157] Output: Selected raters

[0158] Specifically, the evaluator list includes the evaluator's fields of expertise and profiles, and the user makes a selection based on these.

[0159] Step 5:

[0160] Requesting Feedback

[0161] The server sends the uploaded images and bias detection results to the selected evaluators and requests their feedback.

[0162] Input: Selected evaluators, image files, bias detection results

[0163] Output: Feedback request sent to evaluator

[0164] Specifically, the server sends emails and notifications to the evaluators, who can then view the images and detection results on their own devices.

[0165] Step 6:

[0166] Gathering feedback

[0167] Evaluators access the feedback form on their own devices and provide feedback based on the image files and detection results.

[0168] Input: Image files, bias detection results, evaluator feedback

[0169] Output: Feedback sent to the server

[0170] Specifically, the evaluator enters specific comments and evaluations into the feedback form and presses the submit button. The feedback is then saved on the server.

[0171] Step 7:

[0172] Providing feedback and fine-tuning images

[0173] The server consolidates the collected feedback and provides it to the user, and if necessary, translates the feedback into multiple languages ​​using a translation engine.

[0174] Input: Collected feedback

[0175] Output: User-viewable feedback report

[0176] Based on the feedback, the user can fine-tune the image using a generative AI model or image editing software, and then repeat the process again to ultimately generate a high-quality image with minimal bias.

[0177] Input: Feedback report, user image tweaks

[0178] Output: Modified image file

[0179] Through these steps, the system of the present invention effectively detects and corrects biases and errors in the image generation process, and supports the creation of artworks based on diverse perspectives.

[0180] (Application example 1)

[0181] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0182] This invention relates to a system for producing fair and diverse artworks and advertising images by automatically detecting biases and errors contained in image files created using generative AI and collecting feedback from diverse perspectives. Conventional methods require significant time and effort to manually detect biases contained in generated images, making it difficult to incorporate diverse perspectives. Furthermore, there was a lack of a method for efficiently collecting feedback from evaluators and presenting it to users in multiple languages. This made it difficult to detect and correct biases in image production using generative AI, resulting in the risk of inappropriate images being published.

[0183] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0184] In this invention, the server includes a means for uploading generated image files, a means for detecting bias and errors from the uploaded image files, and a means for selecting evaluators with different backgrounds. This allows users to quickly detect bias and errors in uploaded images, effectively collect feedback from evaluators with diverse backgrounds, and present the results in multiple languages. The server also includes a means for users to adjust images based on the feedback, ultimately producing high-quality images that reflect fair and diverse perspectives.

[0185] "Generated Image File" refers to visual data created using generative AI.

[0186] "Bias and Errors" refers to inappropriate biases and mistakes contained in the generated image files.

[0187] "Evaluator" refers to a third party individual or entity with a different background or expertise whose role is to evaluate and provide feedback on images.

[0188] "Feedback" refers to information such as specific comments, evaluations, and suggested corrections obtained from the evaluator.

[0189] "User" refers to the entity that uploads the generated image files and receives bias and error detection and feedback.

[0190] "Server" refers to the computer system that uploads images, detects bias and errors, solicits feedback from raters, and aggregates and presents the feedback.

[0191] A "trained model" refers to an algorithm or neural network that has been pre-trained using large amounts of data.

[0192] "Machine translation" refers to technology that mechanically translates text and feedback between different languages.

[0193] "Means for adjusting images" refers to methods or tools that allow users to modify or regenerate generative AI models or images based on user feedback.

[0194] The system of the present invention allows users to upload generated image files, automatically detect bias and errors, and collect and present feedback from evaluators in order to create fair and diverse artworks and advertising images.

[0195] Hardware and software used:

[0196] Hardware:

[0197] Server: Uploads images, detects bias and errors, requests feedback from raters, and aggregates and presents the feedback.

[0198] Client device: A smartphone is the primary device used to upload images and view feedback.

[0199] software:

[0200] Flask: Used as a Python web framework to process requests on the server.

[0201] JSON: Used to exchange data.

[0202] Trained models: Use pre-trained algorithms or neural networks to detect bias and errors.

[0203] Image storage library: Used to store uploaded images on the server.

[0204] Specific system processing description:

[0205] The user uploads the generated image file from their smartphone to the server, which receives the image using Flask and stores it in local storage.The server then uses the trained model to detect biases and errors in the uploaded image file.

[0206] Based on the detected biases and errors, the server presents a list of evaluators to the user. The evaluators are third parties with different backgrounds and expertise, and the user can review the evaluator profiles and select the evaluators they want to receive the necessary feedback from.

[0207] The server sends the images and bias detection results to the selected evaluators. The evaluators then review the images and input their feedback using their own devices. Once the feedback is collected, the server aggregates it and presents it to the user in multiple languages.

[0208] Based on the feedback provided, users can adjust the images and generative AI models, and then repeat the process to create high-quality images with less bias.

[0209] Examples:

[0210] For example, a designer at an advertising agency uses generative AI to create an image for an advertisement and upload it to the system. The server detects that the image contains racial stereotypes. The designer selects an expert on racial diversity from the evaluators presented on the system and asks for their feedback. The server then presents the evaluator's feedback to the designer, who then adjusts the generative AI model based on the feedback and recreates the advertisement image with fewer racial stereotypes.

[0211] Example prompts to input to a generative AI model:

[0212] "Generate a scene of an adult woman working in an office. The image should reflect a modern, diverse work environment."

[0213] Images generated based on this prompt can then be uploaded to the system, where bias detection and feedback collection can be performed to produce fair and diverse images.

[0214] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0215] Step 1:

[0216] Upload user-generated images from your smartphone to the server.

[0217] Input: Generated image files on your smartphone.

[0218] Processing: Use the Flask framework to save the image received by the server to local storage.

[0219] Output: An image file saved on the server (e.g. uploaded_image.jpg).

[0220] Step 2:

[0221] The server detects bias and errors in the uploaded images.

[0222] Input: Image files stored on the server.

[0223] Processing: Using a trained model (a pre-trained neural network) to analyze the image for biases and errors.

[0224] Output: Bias or error detection results (e.g., "Gender bias detected").

[0225] Step 3:

[0226] The server presents the results of the bias and error detection to the user and allows them to select an evaluator.

[0227] Input: Bias and error detection results, evaluator list.

[0228] Processing: The detection results and the list of evaluators are displayed on the user's smartphone, and the user is prompted to select an evaluator.

[0229] Output: Information about the rater selected by the user.

[0230] Step 4:

[0231] The server sends the images and detection results to the selected evaluators and requests their feedback.

[0232] Input: User-selected evaluator information, image files stored on the server, and bias and error detection results.

[0233] Processing: Send the images and detection results to the selected evaluators and issue a request for feedback.

[0234] Output: The request sent to the evaluator.

[0235] Step 5:

[0236] The evaluator enters the feedback and sends it to the server.

[0237] Input: Rater feedback (specific comments and ratings)

[0238] Processing: Feedback data is sent to the server via the evaluator's terminal.

[0239] Output: Feedback received by the server.

[0240] Step 6:

[0241] The server aggregates the feedback and presents it to the user in multiple languages.

[0242] Input: Multiple pieces of feedback received from raters.

[0243] Processing: Feedback is collected, automatically translated if necessary, and displayed on the user's smartphone in multiple languages.

[0244] Output: A multilingual translated feedback list.

[0245] Step 7:

[0246] Users adjust the images and generative AI models based on feedback.

[0247] Input: Feedback received by the user.

[0248] Processing: Use the feedback to adjust the parameters of the generative AI model, enter new prompts to regenerate the image, or adjust the image directly using image editing tools.

[0249] Output: Adjusted or regenerated image files.

[0250] Following these specific processing steps, the system quickly detects bias and errors in user-generated image files and enables the system to recreate fair and diverse images based on feedback from evaluators.

[0251] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0252] This invention relates to a system that uploads generated image files, automatically detects bias and errors, collects and presents feedback from evaluators, and provides feedback that takes into account the user's emotional state by combining it with an emotion engine. This system allows users to create artwork that incorporates fair and diverse perspectives, and receives appropriate feedback according to the user's emotional state.

[0253] Program processing

[0254] 1. User uploads an image

[0255] Subject: User

[0256] Users prepare images generated using generative AI on their devices and upload the image files to the platform, where they are sent to the server and stored.

[0257] 2. The server detects bias and errors

[0258] Subject: Server

[0259] The server receives the stored image files and uses a bias detection engine to automatically detect biases and errors, generating a report containing the type and location of identified biases and storing it in a database.

[0260] 3. The user selects the evaluator

[0261] Subject: User

[0262] The user reviews the bias detection results and selects the evaluator from whom they would like to receive feedback from a list of evaluators provided on the platform.

[0263] 4. The server collects feedback

[0264] Subject: Server

[0265] The server sends the uploaded image files and bias detection results to the selected evaluators and asks for their feedback.

[0266] 5. Evaluators provide feedback

[0267] Subject: Terminal (evaluator)

[0268] The evaluator uses a terminal to check the submitted images and bias information, enters comments, specific indications of bias, and suggestions for correction into the feedback form, and sends it to the server.

[0269] 6. The server receives and translates the feedback

[0270] Subject: Server

[0271] The server receives feedback data returned by the evaluators, aggregates and stores it in a database, and automatically translates the feedback data if necessary.

[0272] 7. The server uses an emotion engine to recognize the user's emotions.

[0273] Subject: Server

[0274] The server uses an emotion engine to analyze facial and voice data collected from the user's device to recognize the user's current emotional state, which is then stored in a database.

[0275] 8. The server provides feedback

[0276] Subject: Server

[0277] The server optimizes the display method of the feedback content based on the recognized emotional state, and the aggregated feedback is presented to the user in a form that corresponds to the user's emotional state.

[0278] 9. Tweak the image based on user feedback

[0279] Subject: User

[0280] Users can fine-tune the image and generative AI model based on the feedback they provide, and then upload the regenerated image using the same process, seeking further feedback if needed.

[0281] Specific examples

[0282] Example 1: Evaluating and correcting artist-generated illustrations

[0283] Artist X creates an illustration using generative AI and uploads it to the platform, where the server detects that the illustration contains gender bias.

[0284] Artist X selects evaluator Y, an expert on gender perspectives. The server sends the image and bias information to evaluator Y and asks for feedback.

[0285] Evaluator Y uses a device to check the image and send feedback including specific suggestions for correction. The server receives and translates the feedback and presents it to artist X.

[0286] Before presenting the illustration, the server uses an emotion engine to investigate Artist X's emotional state and select an appropriate feedback display method. Based on the feedback, Artist X adjusts the generative AI model and recreates the illustration with less gender bias.

[0287] Example 2: Advertising agency bias correction of advertising images

[0288] A designer at an advertising agency uses generative AI to create and upload images for advertisements, and the server detects if the images contain racial stereotypes.

[0289] The designer selects Evaluator Z, an expert on racial diversity, and asks for feedback. The server sends the image and the detection results to Evaluator Z.

[0290] The server receives feedback from evaluator Z, aggregates and translates it, and presents it to the designer. Before presenting it, the server uses an emotion engine to analyze the designer's emotional state and select the optimal display method.

[0291] Designers use the feedback to adjust the generative AI model to recreate advertising images with fewer racial stereotypes.

[0292] This is the specific processing flow, which effectively corrects biases and errors in the generated images while taking into account the user's emotional state, allowing for the creation of artwork that reflects fair and diverse perspectives.

[0293] The processing flow will be explained below.

[0294] Step 1:

[0295] Users prepare the image files generated by the generative AI on their devices, then drag and drop the image files into the platform's upload interface and click the upload button to send them to the server.

[0296] Step 2:

[0297] The server receives the uploaded image file, saves it in storage, and records the path and metadata of the saved file in the database.

[0298] Step 3:

[0299] The server then sends the saved image files to a bias detection engine, which uses pre-trained models to automatically detect biases and errors in the images and generates a report containing the type and location of any biases identified.

[0300] Step 4:

[0301] The server saves the bias detection result report in a database and notifies the user, including a summary of the bias and instructions for next steps.

[0302] Step 5:

[0303] Users open the platform on their device, check the bias detection results, and then select the evaluator they wish to provide feedback to from the list of evaluators provided by the system. The list displays each evaluator's area of ​​expertise and profile.

[0304] Step 6:

[0305] The server sends the uploaded image file and bias detection results to the evaluator selected by the user and requests feedback.

[0306] Step 7:

[0307] The evaluator uses their own device to check the submitted images and bias information, enters comments, specific indications of bias, and suggestions for correction in the feedback form, and sends the feedback to the server.

[0308] Step 8:

[0309] The server receives feedback data from evaluators, aggregates it in a database, and stores it. If necessary, it automatically translates the feedback data and makes it understandable to users.

[0310] Step 9:

[0311] The server uses an emotion engine to recognize the user's emotions. It analyzes facial expressions and voice data while the user is using the platform and evaluates the user's current emotional state. The evaluation data is stored in a database.

[0312] Step 10:

[0313] The server determines how to display the feedback based on the collected feedback data and the user's emotional state. For example, if the user is feeling stressed, the server may display the feedback in a softer tone.

[0314] Step 11:

[0315] The server presents the optimized feedback to the user, who can review it on their device and is also provided with a detailed view function to review the feedback again if necessary.

[0316] Step 12:

[0317] Users can use the feedback provided to fine-tune the generative AI model and the image itself, then re-upload the generated image using the same process and ask for further feedback if necessary.

[0318] These are the specific processing steps of the system that combines the emotion engine. This series of processes makes it possible to effectively create artwork that reflects fair and diverse viewpoints while taking into account the user's emotional state.

[0319] Example 2

[0320] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0321] In recent years, image generation using generative AI models has become popular, but the generated images can contain biases and errors, such as gender and race bias. Therefore, there is a need to evaluate and correct these images from a fair and diverse perspective. However, existing systems lack the means to collect feedback from evaluators and present that feedback appropriately to users. Furthermore, the lack of feedback based on the user's emotional state poses a challenge in improving the user experience.

[0322] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for uploading generated image files, a means for detecting bias and errors from the uploaded image files, a means for selecting evaluators with different backgrounds, a means for obtaining feedback from the selected evaluators, a means for automatically translating the feedback obtained from the evaluators, a means for recognizing the emotional state of the user, and a means for presenting the obtained feedback in a form corresponding to the emotional state of the user. This makes it possible to realize fair and diverse evaluations of images containing bias and errors and to provide appropriate feedback corresponding to the emotional state of the user.

[0323] "Image File" refers to visual data generated using a generative AI model.

[0324] "Bias" refers to errors that involve particular perspectives or prejudices, such as those related to gender, race, or religion.

[0325] "Errors" refer to factually incorrect or inappropriate elements in the content of an image.

[0326] "Evaluators" refer to people or entities with different backgrounds and expertise who are tasked with evaluating and providing feedback on the content of images.

[0327] "Feedback" refers to information such as comments, suggested corrections, and findings provided by the evaluator.

[0328] "Machine translation" refers to the process of mechanically converting the content of feedback obtained from evaluators into another language.

[0329] "Emotional state" refers to the user's current mental and emotional state.

[0330] "Generative AI model" refers to an artificial intelligence algorithm that creates a generated visual representation based on an input prompt.

[0331] A "prompt" refers to a text instruction or explanation input to a generative AI model.

[0332] The present invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators. The system includes, among other things, a means of providing feedback that takes into account the user's emotional state.

[0333] The overall flow of the system is as follows: First, the user generates an image using a generative AI model (e.g., DALL-E or Midjourney). The generated image is saved on the user's device. Next, the user logs in to the platform using a web browser and uploads the image file. The uploaded image file is sent to the server and saved.

[0334] The server analyzes the saved image files using a bias detection engine (e.g., IBM Watson® Visual Recognition) to automatically detect bias and errors. A report is generated as a result of the detection and saved in a database. The user reviews this report and selects the evaluator they wish to receive feedback from from a list of evaluators provided on the platform.

[0335] The server sends the image files and bias detection results to the selected evaluators. The evaluators enter comments and suggestions for corrections in a feedback form and send it to the server. The server receives feedback from the evaluators, translates it as needed using an automatic translation engine (e.g., Google (registered trademark) Translate API), and aggregates it in a database.

[0336] The server then uses an emotion engine (e.g., Affectiva or Microsoft® Azure® Emotion API) to analyze the user's emotional state. The analysis is performed using facial expression and voice data collected from the user's device, and the information is stored in a database.

[0337] Finally, the server optimizes the feedback display method based on the user's emotional state and presents the feedback to the user.The user can then use the provided feedback to modify the generated images and prompts and input them back into the generative AI model to regenerate images with less bias and errors.

[0338] Specific examples

[0339] Example 1: Evaluating and correcting artist-generated illustrations

[0340] 1. The user (artist) uses DALL-E to input the following prompt sentence and generate an illustration.

[0341] Prompt: Draw a gender-neutral fantasy character.

[0342] 2. The user uploads the generated image to the platform.

[0343] 3. The server uses a bias detection engine to detect gender bias.

[0344] 4. The user selects a gender perspective expert (evaluator) and requests feedback.

[0345] 5. The evaluator fills out the feedback form with specific suggestions for correction and submits it.

[0346] 6. The server receives and translates the feedback and presents it to the user based on their emotional state.

[0347] 7. The user modifies the prompt based on the feedback and regenerates a new image.

[0348] Example 2: Advertising agency bias correction of advertising images

[0349] 1. A designer at an advertising agency uses Midjourney to enter the following prompt text and create an image for an advertisement.

[0350] Prompt: Draw a scene of a diverse business meeting.

[0351] 2. The designer uploads the generated image to the platform.

[0352] 3. The server detects that the image contains racial stereotypes.

[0353] 4. The designer selects racial diversity experts (evaluators) and solicits their feedback.

[0354] 5. Receive feedback from evaluators, automatically translate it, and present it to the designer.

[0355] 6. The designer adjusts the generative AI model based on the presented feedback and emotional state, and recreates a new advertising image.

[0356] The above is a specific embodiment of the invention, which allows users to create artwork or advertising materials that incorporate unbiased and diverse perspectives, and effectively correct biases and errors based on appropriate feedback.

[0357] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0358] Step 1:

[0359] The user generates an image using a generative AI model.

[0360] Users input a prompt (e.g., "Draw a gender-neutral fantasy character") into their device and run a generative AI model (e.g., DALL-E or Midjourney). The generated image file is saved locally on the device.

[0361] Input: prompt statement

[0362] Output: Generated image file

[0363] Step 2:

[0364] A user uploads an image to the platform.

[0365] The user opens a web browser and logs in to the platform. After logging in, they use the image upload function to select an image file and click the upload button. The image file is sent to the server and saved in a specified folder.

[0366] Input: Generated image file

[0367] Output: Image file saved on the server

[0368] Step 3:

[0369] The server detects bias and errors.

[0370] The server invokes a bias detection engine (e.g., IBM Watson Visual Recognition) to analyze the uploaded image file. If bias or errors are detected, a report is generated identifying the type and location of the errors and stored in a database.

[0371] Input: Saved image file

[0372] Output: Bias detection report

[0373] Step 4:

[0374] The user selects the evaluator.

[0375] The user reviews the bias detection report, selects the evaluator they wish to receive feedback from from the list of evaluators provided on the platform, and clicks a button to confirm their selection.

[0376] Input: Bias detection report, evaluator list

[0377] Output: Selected raters

[0378] Step 5:

[0379] The server collects the feedback.

[0380] The server will send the uploaded image files and bias detection results to the selected raters, who will be notified and receive a request for feedback from their account.

[0381] Input: Selected evaluators, image files, bias detection results

[0382] Output: Feedback request notification

[0383] Step 6:

[0384] The evaluator provides feedback.

[0385] The evaluator receives the notification and logs in to the platform. They check the image to be evaluated and the bias information, and enter comments and suggestions for corrections in the feedback form. Once they have completed the input, they click the submit button to send the feedback to the server.

[0386] Input: Feedback request notification, bias information

[0387] Output: Feedback data

[0388] Step 7:

[0389] The server receives and translates the feedback.

[0390] The server receives feedback data sent by evaluators and saves it in a designated folder. It then aggregates the feedback data in a database and translates the feedback content using an automatic translation engine (e.g., Google Translate API) if necessary. The translated data is then saved in the database.

[0391] Input: Feedback data

[0392] Output: Translated feedback data

[0393] Step 8:

[0394] The server uses an emotion engine to recognize the user's emotion.

[0395] The server calls an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) to analyze facial and voice data collected from the user's device. As a result of the analysis, the user's current emotional state (e.g., joy, surprise, sadness) is identified and stored in a database.

[0396] Input: facial expression data, voice data

[0397] Output: User's emotional state

[0398] Step 9:

[0399] The server provides feedback.

[0400] The server optimizes the feedback display method based on the user's emotional state, and the feedback can be presented in the form of a pop-up, notification bar, voice message, etc. The feedback will be displayed when the user logs in to the platform and accesses the dashboard.

[0401] Input: User's emotional state, translated feedback data

[0402] Output: Feedback presented to the user

[0403] Step 10:

[0404] The user fine-tunes the image based on the feedback.

[0405] The user reviews the provided feedback and modifies the generated image and prompt text. For example, if gender bias is identified, the prompt text can be modified to explicitly state "gender-neutral." The modified prompt is then input into the generative AI model to generate a new image. The generated image can then be uploaded back to the platform, and additional feedback can be sought if necessary.

[0406] Input: Feedback

[0407] Output: Modified image file

[0408] The above are the specific processing steps.

[0409] (Application example 2)

[0410] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0411] Existing image generation systems face the challenge of effectively detecting and correcting biases and errors in generated image files. Furthermore, they face the challenge of not being able to provide appropriate feedback that takes into account the user's emotional state. This makes it difficult to create images that incorporate fair and diverse perspectives, and to create advertising images with minimal user bias.

[0412] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0413] In this invention, the server includes means for uploading generated image files, means for detecting bias or errors from the uploaded image files, means for selecting evaluators with different backgrounds, means for obtaining feedback from the selected evaluators, means for presenting the obtained feedback to the user, means for recognizing the emotional state of the user, and means for optimizing the display format of the feedback content based on the recognized emotional state. This makes it possible to present optimal feedback according to the emotional state of the user, thereby realizing image production that is fair and incorporates diverse perspectives.

[0414] "Image file" refers to a digitally generated still image that contains visual information about a subject or scene.

[0415] "Uploading" is the process of transferring or sending a digital file from a user's device to a server via the Internet.

[0416] "Bias" refers to the phenomenon in which certain prejudices or stereotypes influence data or systems, and is a factor that can lead to unfair results.

[0417] "Error" refers to an unintended inaccuracy or inconsistency in data or results.

[0418] "Evaluator" means a person who has specialized knowledge and experience in the uploaded image and provides evaluation and feedback on the image.

[0419] "Feedback" refers to information such as comments, suggestions, and improvements provided by the evaluator about an image.

[0420] "User" refers to the person who uses the system to generate and upload images and receive feedback.

[0421] "Emotional state" refers to the user's current psychological and emotional state, and is analyzed from data such as facial expressions and voice.

[0422] "Optimization" refers to the process of achieving the most effective state or arrangement for a specific purpose or condition.

[0423] "Display method" refers to the format or method for visually presenting information.

[0424] "Database" refers to a system for systematically storing and managing specific data.

[0425] A "server" is a computer system that provides services and data to clients over a network.

[0426] This invention begins with a user uploading an image file they have generated to the system using a device such as a smartphone or computer. The user generates an image using a generative AI model and uploads it to the system. The uploaded image file is then stored on a server.

[0427] The server automatically detects biases and errors in the stored image files using pre-trained models. For example, if the image contains gender bias or racial stereotypes, it will detect them and generate a report containing the type and location of identified bias. This report is stored in a database.

[0428] Next, users select their desired evaluators from a list of evaluators provided on the platform. Evaluators are experts with different backgrounds who can provide unbiased and diverse perspectives on some aspects. The server then sends the uploaded image files and bias detection results to the selected evaluators and requests their feedback.

[0429] Evaluators use their devices to check the images and bias information, and then enter comments, specific suggestions, and corrections in the feedback form. The entered feedback is sent to the server, which receives and translates it, then aggregates and stores it in a database.

[0430] Furthermore, the server uses an emotion engine to analyze facial expression and voice data collected from the user's device to recognize the user's current emotional state, which is then stored in a database and used to optimize the way feedback is displayed.

[0431] Finally, the server presents the aggregated feedback according to the user's emotional state (for example, a positive tone if the user is happy, or a comforting tone if the user is sad).The user can use this feedback to fine-tune the generative AI model and the image, then upload the image again for further feedback.

[0432] For example, an advertising designer uploads an advertising image created using a generative AI model, and the server detects racial stereotypes in the image. The designer selects evaluators knowledgeable about diversity and receives feedback. The server checks the designer's emotional state and selects an appropriate feedback display format. The designer adjusts the generative AI model based on this feedback.

[0433] Examples of prompt sentences include the following:

[0434] "Please advise on creating advertising images that are gender-bias-free. The current image contains the following biases: {List of biases detected}"

[0435] As described above, this system incorporates a fair and diverse range of viewpoints and provides optimal feedback according to the user's emotional state, enabling high-quality image production.

[0436] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0437] Step 1:

[0438] The user prepares an image file generated using the generative AI model on their device and uploads the image file to the platform. The server receives and stores the uploaded image file.

[0439] Input: Image file (created by the user using a generative AI model)

[0440] Output: Image files are saved to the server

[0441] Step 2:

[0442] The server detects bias and errors in the stored image files using pre-trained models. The bias detection model identifies prejudices and stereotypes in the images and generates a report based on this. The report includes the type and location of bias detected.

[0443] Input: Saved image file

[0444] Output: Bias detection report (including type and location of bias)

[0445] Step 3:

[0446] Users select the evaluators they wish to receive feedback from from a list of evaluators provided on the platform. Users choose evaluators with different backgrounds, with the aim of obtaining fair and diverse perspectives.

[0447] Input: Rater list, user selection

[0448] Output: Selected raters

[0449] Step 4:

[0450] The server then sends the uploaded image files and bias detection results to the selected evaluators and requests their feedback, preparing the server to receive specific feedback and suggestions for corrections from the evaluators.

[0451] Input: Image file, bias detection report, selected rater

[0452] Output: Feedback request to evaluator

[0453] Step 5:

[0454] The evaluator uses the device to review the submitted images and bias information, and enters comments, specific suggestions, and correction suggestions in the feedback form. The evaluator's feedback is then sent from the device to the server.

[0455] Input: Image file, bias detection report

[0456] Output: Rater feedback

[0457] Step 6:

[0458] The server receives feedback data returned by evaluators, aggregates and stores it in a database, and automatically translates the feedback data as needed, providing users with feedback in a format that is easy to understand.

[0459] Input: Rater feedback

[0460] Output: translated feedback, aggregated feedback data

[0461] Step 7:

[0462] The server uses an emotion engine to analyze facial and voice data collected from the user's device and recognize the user's current emotional state. The emotion engine is implemented using, for example, the DeepFace library.

[0463] Input: User's facial expressions and voice data

[0464] Output: User's emotional state

[0465] Step 8:

[0466] The server optimizes the display of feedback content based on the user's emotional state. The aggregated feedback is presented in a way that reflects the user's emotional state. For example, positive feedback is displayed prominently.

[0467] Input: Emotional state, translated feedback

[0468] Output: Optimized feedback display

[0469] Step 9:

[0470] Based on the feedback provided by the user, the image and generative AI model are fine-tuned, and the corrected image is uploaded again and can receive feedback again through the same process.

[0471] Input: Optimized feedback display

[0472] Output: Fine-tuned image

[0473] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0474] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0475] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0476] [Second embodiment]

[0477] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0478] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0479] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0480] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0481] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0482] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0483] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0484] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0485] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0486] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0487] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0488] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0489] This invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators, allowing users to create artwork that incorporates unbiased and diverse perspectives.

[0490] Program processing

[0491] 1. User uploads an image

[0492] Subject: User

[0493] Users upload images generated using the generative AI from their devices to the server, where the uploaded image files are stored.

[0494] 2. The server detects bias and errors

[0495] Subject: Server

[0496] The server receives the uploaded images and automatically detects bias and errors using a bias detection engine, which uses pre-trained models.

[0497] 3. The user selects the evaluator

[0498] Subject: User

[0499] After reviewing the results of bias and error detection, users can select the evaluator from whom they would like to receive feedback from a list of evaluators displayed on the system. The list displays the evaluators' fields of expertise and profiles, and they come from a variety of backgrounds.

[0500] 4. The server collects feedback

[0501] Subject: Server

[0502] The server sends the uploaded images and bias detection results to the selected evaluators and asks for their feedback. The evaluators then check the images and information on their own devices and enter their feedback.

[0503] 5. Evaluators provide feedback

[0504] Subject: Terminal (evaluator)

[0505] Evaluators use their devices to send feedback to the server, including specific comments, like / dislike ratings, and unpleasantness indices.

[0506] 6. The server aggregates and presents the feedback

[0507] Subject: Server

[0508] The server aggregates the feedback collected from the evaluators and presents it to the user. If necessary, the server automatically translates the feedback and provides it in multiple languages ​​in a format that is easy for the user to understand.

[0509] 7. Tweak the image based on user feedback

[0510] Subject: User

[0511] Users can then use the feedback they provide to fine-tune the images and generative AI models, and the process is repeated until the end result is a less biased and more diverse piece of art.

[0512] Specific examples

[0513] Example 1: Evaluating and correcting artist-generated portraits

[0514] Artist X creates a portrait using generative AI and uploads it to the platform, where the server detects that the portrait contains unofficial gender bias.

[0515] Artist X selects evaluator Y, an expert on gender perspectives. The server sends the image and bias information to evaluator Y and asks for feedback.

[0516] Evaluator Y uses a device to check the image and submits feedback including specific suggestions for correction. The server presents the feedback to artist X and performs automatic translation if translation is required.

[0517] Based on the feedback, Artist X adjusts the generative AI model to recreate portraits with less gender bias.

[0518] Example 2: Advertising agency bias correction of advertising images

[0519] A designer at an advertising agency uses generative AI to create and upload images for advertisements, and the server detects if the images contain racial stereotypes.

[0520] The designer selects Evaluator Z, an expert on racial diversity, and asks for feedback. The server sends the image and the detection results to Evaluator Z.

[0521] The server receives the feedback from rater Z and presents it to the designer, translating it if necessary. The designer then adjusts the generative AI model based on the feedback to recreate the advertising image with fewer racial stereotypes.

[0522] As described above, the system of the present invention effectively identifies and corrects biases and errors in generated images, helping users create artworks based on fair and diverse perspectives.

[0523] The processing flow will be explained below.

[0524] Step 1:

[0525] Users prepare the image generated by the generative AI on their device, then drag and drop the image file into the platform's upload interface and click the upload button, which sends the image file to the server.

[0526] Step 2:

[0527] The server receives the uploaded image file, saves it in storage, and records the path and metadata of the saved file in the database.

[0528] Step 3:

[0529] The server then sends the stored image files to a bias detection engine, which uses pre-trained models to detect bias and errors in the images and generates a report containing the type and location of identified bias.

[0530] Step 4:

[0531] The server stores the results of the detection in a database and notifies the user, including a summary of the bias and instructions for next steps.

[0532] Step 5:

[0533] Users check the bias detection results on the platform, and then select the evaluator they would like to receive feedback from from the list of evaluators provided by the system.

[0534] Step 6:

[0535] The server sends the uploaded image file and bias detection results to the evaluator selected by the user and generates a feedback request.

[0536] Step 7:

[0537] The evaluator will use their own device to check the submitted images and bias information, enter comments, specific indications of bias, and suggestions for correction in the feedback form, and then press the submit button.

[0538] Step 8:

[0539] The server receives feedback data returned by the evaluators, aggregates and stores it in a database, and automatically translates the feedback data if necessary.

[0540] Step 9:

[0541] The server then presents the aggregated feedback to the user, including the reviewer's comments, Like / Dislike ratings, and an unpleasantness index. The feedback is presented in a format that is easy for the user to understand.

[0542] Step 10:

[0543] Users can fine-tune the image and generative AI model based on the feedback they provide, and then upload the regenerated image using the same process, seeking further feedback if needed.

[0544] This is the specific process flow, which effectively corrects bias and errors in the generated images and produces artwork that reflects fair and diverse perspectives.

[0545] Example 1

[0546] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0547] Current image generation processes using generative AI technology may contain biases and errors in the generated images. These biases and errors can result in unintended losses in diversity and fairness. Furthermore, the process of evaluating and correcting generated images is cumbersome and requires a lot of time and effort. Furthermore, there is a lack of systems that can collect feedback from evaluators with different backgrounds and provide it in a format that is easy for users to understand. There is a need for a system that can resolve these issues and support the generation of high-quality images based on fair and diverse perspectives.

[0548] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0549] In this invention, the server includes a means for uploading generated image files, a means for detecting biases and errors from the uploaded image files, a means for presenting the results of the bias and error detection to a user and selecting evaluators with different backgrounds, a means for collecting feedback from the selected evaluators, and a means for providing the collected feedback to a user and supporting fine-tuning of the generated images. This makes it possible to effectively detect and correct biases and errors in the generated images and generate high-quality images based on diverse viewpoints.

[0550] An "image file" is digital data containing visual information generated using a generative AI model.

[0551] "Uploading" is the act of sending digital data (image files) from a user's device to a server and storing it.

[0552] "Bias" refers to biased viewpoints or inappropriate elements contained in the content of image files, which have an unfair influence on a particular classification or evaluation.

[0553] "Error" refers to any inaccuracy or inappropriate element in the content of an image file that deviates from the intended result.

[0554] "Evaluators" are experts or users with different backgrounds who provide feedback on the generated image files.

[0555] "Feedback" is a response, including comments, ratings, and suggested modifications, provided by a reviewer to an image file.

[0556] "User" refers to an individual or organization that uses the system to generate image files using a generative AI model.

[0557] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate image files based on input such as prompts from users.

[0558] A "prompt sentence" is text information entered by the user to convey instructions and intentions for image generation to the generative AI model.

[0559] The present invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators. This system allows users to create artwork that incorporates unbiased and diverse perspectives. Specific embodiments of the present invention are described below.

[0560] Overall system overview

[0561] The system is implemented using the following hardware and software:

[0562] User device: For generating and uploading images and checking feedback

[0563] Server: Image file storage, bias and error detection, feedback collection and presentation

[0564] Generative AI models: Examples include OpenAI's DALL-E and MidJourney

[0565] Bias detection engine: For example, OpenAI's CLIP

[0566] Translation Engine: Automatic translation tool for multilingual feedback

[0567] System Operation

[0568] User generates and uploads an image

[0569] The user uses a device to operate a generative AI model (e.g., DALL-E or MidJourney) and generate an image by entering the desired prompt. An example prompt might be: "A cat dancing under the night sky." The generated image is saved on the user's device.

[0570] Next, the user uploads the image file to the server via the system's web interface. By pressing the upload button, the image file is sent to the server and saved in the server's storage.

[0571] Detecting bias and error

[0572] The server receives the uploaded image file and analyzes the image content using a bias detection engine (e.g., OpenAI's CLIP). This analysis engine uses pre-trained models to detect known bias and error patterns. The detection results are saved in a format such as JSON and later provided to the user.

[0573] User-selected raters

[0574] The user checks the results of bias and error detection through the system's web interface. The interface displays a list of evaluators, each of which includes their area of ​​expertise and profile. The user selects an appropriate evaluator from the list and presses the evaluation request button.

[0575] Gathering feedback from evaluators

[0576] The server notifies the selected evaluator of the uploaded image and the bias detection results. The evaluator logs into the system on their own device and checks the image and detection results. The evaluator enters specific comments and evaluations into the feedback form and presses the submit button to send the feedback to the server.

[0577] Providing feedback and fine-tuning images

[0578] The server consolidates the collected feedback and provides it to the user. If necessary, the feedback is translated into multiple languages ​​using a translation engine. The user then uses the provided feedback to fine-tune the image using a generative AI model or image editing software. By repeating the same process, the resulting artwork is based on a less biased, fairer, and more diverse perspective.

[0579] As a concrete example:

[0580] Example 1: Evaluating and correcting artist-generated portraits

[0581] Artists create portraits using generative AI models and upload them to the system, where the server detects any gender bias in the portraits.

[0582] The artist selects evaluators who are experts in gender perspectives, and the server sends the images and bias information to the evaluators and asks for their feedback.

[0583] The evaluator submits feedback with specific suggestions for correction, and the server provides the feedback to the artist, translating it if necessary.

[0584] Based on the feedback, the artist adjusts the generative AI model to generate new portraits.

[0585] Using these steps, the system of the present invention can effectively detect and correct biases and errors in the image generation process, helping users to create high-quality artwork that incorporates diverse perspectives.

[0586] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0587] Step 1:

[0588] Image generation and upload

[0589] The user uses the generative AI model to generate images by entering a prompt.

[0590] Input: Prompt sentence (e.g. "Cats dancing under the night sky")

[0591] Output: Generated image file

[0592] Users save the generated image files to their own devices, then log in to the system using a dedicated web interface and upload the saved image files.

[0593] Input: Generated image file

[0594] Output: Image file stored on the server

[0595] Step 2:

[0596] Detecting bias and error

[0597] The server receives the uploaded image file and then analyzes the image using a bias detection engine (which includes a pre-trained model).

[0598] Input: Uploaded image file

[0599] Output: Bias and error detection results (e.g., JSON format)

[0600] Specifically, the server passes the image to an analysis engine, which detects known patterns of bias and error, and stores the results on the server for later review by the user.

[0601] Step 3:

[0602] Presenting bias and error findings

[0603] The server presents the bias and error detection results to the user, who can review these results through a web interface.

[0604] Input: Bias and error detection results

[0605] Output: User-viewable reports

[0606] Specifically, the server displays the detection results on a web interface in a format that is easy for the user to understand.

[0607] Step 4:

[0608] Selection of raters

[0609] The user selects the evaluator from whom they wish to receive feedback from the list of evaluators displayed on the system.

[0610] Input: Rater list and user selection of rater

[0611] Output: Selected raters

[0612] Specifically, the evaluator list includes the evaluator's fields of expertise and profiles, and the user makes a selection based on these.

[0613] Step 5:

[0614] Requesting Feedback

[0615] The server sends the uploaded images and bias detection results to the selected evaluators and requests their feedback.

[0616] Input: Selected evaluators, image files, bias detection results

[0617] Output: Feedback request sent to evaluator

[0618] Specifically, the server sends emails and notifications to the evaluators, who can then view the images and detection results on their own devices.

[0619] Step 6:

[0620] Gathering feedback

[0621] Evaluators access the feedback form on their own devices and provide feedback based on the image files and detection results.

[0622] Input: Image files, bias detection results, evaluator feedback

[0623] Output: Feedback sent to the server

[0624] Specifically, the evaluator enters specific comments and evaluations into the feedback form and presses the submit button. The feedback is then saved on the server.

[0625] Step 7:

[0626] Providing feedback and fine-tuning images

[0627] The server consolidates the collected feedback and provides it to the user, and if necessary, translates the feedback into multiple languages ​​using a translation engine.

[0628] Input: Collected feedback

[0629] Output: User-viewable feedback report

[0630] Based on the feedback, the user can fine-tune the image using a generative AI model or image editing software, and then repeat the process again to ultimately generate a high-quality image with minimal bias.

[0631] Input: Feedback report, user image tweaks

[0632] Output: Modified image file

[0633] Through these steps, the system of the present invention effectively detects and corrects biases and errors in the image generation process, and supports the creation of artworks based on diverse perspectives.

[0634] (Application example 1)

[0635] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0636] This invention relates to a system for producing fair and diverse artworks and advertising images by automatically detecting biases and errors contained in image files created using generative AI and collecting feedback from diverse perspectives. Conventional methods require significant time and effort to manually detect biases contained in generated images, making it difficult to incorporate diverse perspectives. Furthermore, there was a lack of a method for efficiently collecting feedback from evaluators and presenting it to users in multiple languages. This made it difficult to detect and correct biases in image production using generative AI, resulting in the risk of inappropriate images being published.

[0637] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0638] In this invention, the server includes a means for uploading generated image files, a means for detecting bias and errors from the uploaded image files, and a means for selecting evaluators with different backgrounds. This allows users to quickly detect bias and errors in uploaded images, effectively collect feedback from evaluators with diverse backgrounds, and present the results in multiple languages. The server also includes a means for users to adjust images based on the feedback, ultimately producing high-quality images that reflect fair and diverse perspectives.

[0639] "Generated Image File" refers to visual data created using generative AI.

[0640] "Bias and Errors" refers to inappropriate biases and mistakes contained in the generated image files.

[0641] "Evaluator" refers to a third party individual or entity with a different background or expertise whose role is to evaluate and provide feedback on images.

[0642] "Feedback" refers to information such as specific comments, evaluations, and suggested corrections obtained from the evaluator.

[0643] "User" refers to the entity that uploads the generated image files and receives bias and error detection and feedback.

[0644] "Server" refers to the computer system that uploads images, detects bias and errors, solicits feedback from raters, and aggregates and presents the feedback.

[0645] A "trained model" refers to an algorithm or neural network that has been pre-trained using large amounts of data.

[0646] "Machine translation" refers to technology that mechanically translates text and feedback between different languages.

[0647] "Means for adjusting images" refers to methods or tools that allow users to modify or regenerate generative AI models or images based on user feedback.

[0648] The system of the present invention allows users to upload generated image files, automatically detect bias and errors, and collect and present feedback from evaluators in order to create fair and diverse artworks and advertising images.

[0649] Hardware and software used:

[0650] Hardware:

[0651] Server: Uploads images, detects bias and errors, requests feedback from raters, and aggregates and presents the feedback.

[0652] Client device: A smartphone is the primary device used to upload images and view feedback.

[0653] software:

[0654] Flask: Used as a Python web framework to process requests on the server.

[0655] JSON: Used to exchange data.

[0656] Trained models: Use pre-trained algorithms or neural networks to detect bias and errors.

[0657] Image storage library: Used to store uploaded images on the server.

[0658] Specific system processing description:

[0659] The user uploads the generated image file from their smartphone to the server, which receives the image using Flask and stores it in local storage.The server then uses the trained model to detect biases and errors in the uploaded image file.

[0660] Based on the detected biases and errors, the server presents a list of evaluators to the user. The evaluators are third parties with different backgrounds and expertise, and the user can review the evaluator profiles and select the evaluators they want to receive the necessary feedback from.

[0661] The server sends the images and bias detection results to the selected evaluators. The evaluators then review the images and input their feedback using their own devices. Once the feedback is collected, the server aggregates it and presents it to the user in multiple languages.

[0662] Based on the feedback provided, users can adjust the images and generative AI models, and then repeat the process to create high-quality images with less bias.

[0663] Examples:

[0664] For example, a designer at an advertising agency uses generative AI to create an image for an advertisement and upload it to the system. The server detects that the image contains racial stereotypes. The designer selects an expert on racial diversity from the evaluators presented on the system and asks for their feedback. The server then presents the evaluator's feedback to the designer, who then adjusts the generative AI model based on the feedback and recreates the advertisement image with fewer racial stereotypes.

[0665] Example prompts to input to a generative AI model:

[0666] "Generate a scene of an adult woman working in an office. The image should reflect a modern, diverse work environment."

[0667] Images generated based on this prompt can then be uploaded to the system, where bias detection and feedback collection can be performed to produce fair and diverse images.

[0668] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0669] Step 1:

[0670] Upload user-generated images from your smartphone to the server.

[0671] Input: Generated image files on your smartphone.

[0672] Processing: Use the Flask framework to save the image received by the server to local storage.

[0673] Output: An image file saved on the server (e.g. uploaded_image.jpg).

[0674] Step 2:

[0675] The server detects bias and errors in the uploaded images.

[0676] Input: Image files stored on the server.

[0677] Processing: Using a trained model (a pre-trained neural network) to analyze the image for biases and errors.

[0678] Output: Bias or error detection results (e.g., "Gender bias detected").

[0679] Step 3:

[0680] The server presents the results of the bias and error detection to the user and allows them to select an evaluator.

[0681] Input: Bias and error detection results, evaluator list.

[0682] Processing: The detection results and the list of evaluators are displayed on the user's smartphone, and the user is prompted to select an evaluator.

[0683] Output: Information about the rater selected by the user.

[0684] Step 4:

[0685] The server sends the images and detection results to the selected evaluators and requests their feedback.

[0686] Input: User-selected evaluator information, image files stored on the server, and bias and error detection results.

[0687] Processing: Send the images and detection results to the selected evaluators and issue a request for feedback.

[0688] Output: The request sent to the evaluator.

[0689] Step 5:

[0690] The evaluator enters the feedback and sends it to the server.

[0691] Input: Rater feedback (specific comments and ratings)

[0692] Processing: Feedback data is sent to the server via the evaluator's terminal.

[0693] Output: Feedback received by the server.

[0694] Step 6:

[0695] The server aggregates the feedback and presents it to the user in multiple languages.

[0696] Input: Multiple pieces of feedback received from raters.

[0697] Processing: Feedback is collected, automatically translated if necessary, and displayed on the user's smartphone in multiple languages.

[0698] Output: A multilingual translated feedback list.

[0699] Step 7:

[0700] Users adjust the images and generative AI models based on feedback.

[0701] Input: Feedback received by the user.

[0702] Processing: Use the feedback to adjust the parameters of the generative AI model, enter new prompts to regenerate the image, or adjust the image directly using image editing tools.

[0703] Output: Adjusted or regenerated image files.

[0704] Following these specific processing steps, the system quickly detects bias and errors in user-generated image files and enables the system to recreate fair and diverse images based on feedback from evaluators.

[0705] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0706] This invention relates to a system that uploads generated image files, automatically detects bias and errors, collects and presents feedback from evaluators, and provides feedback that takes into account the user's emotional state by combining it with an emotion engine. This system allows users to create artwork that incorporates fair and diverse perspectives, and receives appropriate feedback according to the user's emotional state.

[0707] Program processing

[0708] 1. User uploads an image

[0709] Subject: User

[0710] Users prepare images generated using generative AI on their devices and upload the image files to the platform, where they are sent to the server and stored.

[0711] 2. The server detects bias and errors

[0712] Subject: Server

[0713] The server receives the stored image files and uses a bias detection engine to automatically detect biases and errors, generating a report containing the type and location of identified biases and storing it in a database.

[0714] 3. The user selects the evaluator

[0715] Subject: User

[0716] The user reviews the bias detection results and selects the evaluator from whom they would like to receive feedback from a list of evaluators provided on the platform.

[0717] 4. The server collects feedback

[0718] Subject: Server

[0719] The server sends the uploaded image files and bias detection results to the selected evaluators and asks for their feedback.

[0720] 5. Evaluators provide feedback

[0721] Subject: Terminal (evaluator)

[0722] The evaluator uses a terminal to check the submitted images and bias information, enters comments, specific indications of bias, and suggestions for correction into the feedback form, and sends it to the server.

[0723] 6. The server receives and translates the feedback

[0724] Subject: Server

[0725] The server receives feedback data returned by the evaluators, aggregates and stores it in a database, and automatically translates the feedback data if necessary.

[0726] 7. The server uses an emotion engine to recognize the user's emotions.

[0727] Subject: Server

[0728] The server uses an emotion engine to analyze facial and voice data collected from the user's device to recognize the user's current emotional state, which is then stored in a database.

[0729] 8. The server provides feedback

[0730] Subject: Server

[0731] The server optimizes the display method of the feedback content based on the recognized emotional state, and the aggregated feedback is presented to the user in a form that corresponds to the user's emotional state.

[0732] 9. Tweak the image based on user feedback

[0733] Subject: User

[0734] Users can fine-tune the image and generative AI model based on the feedback they provide, and then upload the regenerated image using the same process, seeking further feedback if needed.

[0735] Specific examples

[0736] Example 1: Evaluating and correcting artist-generated illustrations

[0737] Artist X creates an illustration using generative AI and uploads it to the platform, where the server detects that the illustration contains gender bias.

[0738] Artist X selects evaluator Y, an expert on gender perspectives. The server sends the image and bias information to evaluator Y and asks for feedback.

[0739] Evaluator Y uses a device to check the image and send feedback including specific suggestions for correction. The server receives and translates the feedback and presents it to artist X.

[0740] Before presenting the illustration, the server uses an emotion engine to investigate Artist X's emotional state and select an appropriate feedback display method. Based on the feedback, Artist X adjusts the generative AI model and recreates the illustration with less gender bias.

[0741] Example 2: Advertising agency bias correction of advertising images

[0742] A designer at an advertising agency uses generative AI to create and upload images for advertisements, and the server detects if the images contain racial stereotypes.

[0743] The designer selects Evaluator Z, an expert on racial diversity, and asks for feedback. The server sends the image and the detection results to Evaluator Z.

[0744] The server receives feedback from evaluator Z, aggregates and translates it, and presents it to the designer. Before presenting it, the server uses an emotion engine to analyze the designer's emotional state and select the optimal display method.

[0745] Designers use the feedback to adjust the generative AI model to recreate advertising images with fewer racial stereotypes.

[0746] This is the specific processing flow, which effectively corrects biases and errors in the generated images while taking into account the user's emotional state, allowing for the creation of artwork that reflects fair and diverse perspectives.

[0747] The processing flow will be explained below.

[0748] Step 1:

[0749] Users prepare the image files generated by the generative AI on their devices, then drag and drop the image files into the platform's upload interface and click the upload button to send them to the server.

[0750] Step 2:

[0751] The server receives the uploaded image file, saves it in storage, and records the path and metadata of the saved file in the database.

[0752] Step 3:

[0753] The server then sends the saved image files to a bias detection engine, which uses pre-trained models to automatically detect biases and errors in the images and generates a report containing the type and location of any biases identified.

[0754] Step 4:

[0755] The server saves the bias detection result report in a database and notifies the user, including a summary of the bias and instructions for next steps.

[0756] Step 5:

[0757] Users open the platform on their device, check the bias detection results, and then select the evaluator they wish to provide feedback to from the list of evaluators provided by the system. The list displays each evaluator's area of ​​expertise and profile.

[0758] Step 6:

[0759] The server sends the uploaded image file and bias detection results to the evaluator selected by the user and requests feedback.

[0760] Step 7:

[0761] The evaluator uses their own device to check the submitted images and bias information, enters comments, specific indications of bias, and suggestions for correction in the feedback form, and sends the feedback to the server.

[0762] Step 8:

[0763] The server receives feedback data from evaluators, aggregates it in a database, and stores it. If necessary, it automatically translates the feedback data and makes it understandable to users.

[0764] Step 9:

[0765] The server uses an emotion engine to recognize the user's emotions. It analyzes facial expressions and voice data while the user is using the platform and evaluates the user's current emotional state. The evaluation data is stored in a database.

[0766] Step 10:

[0767] The server determines how to display the feedback based on the collected feedback data and the user's emotional state. For example, if the user is feeling stressed, the server may display the feedback in a softer tone.

[0768] Step 11:

[0769] The server presents the optimized feedback to the user, who can review it on their device and is also provided with a detailed view function to review the feedback again if necessary.

[0770] Step 12:

[0771] Users can use the feedback provided to fine-tune the generative AI model and the image itself, then re-upload the generated image using the same process and ask for further feedback if necessary.

[0772] These are the specific processing steps of the system that combines the emotion engine. This series of processes makes it possible to effectively create artwork that reflects fair and diverse viewpoints while taking into account the user's emotional state.

[0773] Example 2

[0774] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0775] In recent years, image generation using generative AI models has become popular, but the generated images can contain biases and errors, such as gender and race bias. Therefore, there is a need to evaluate and correct these images from a fair and diverse perspective. However, existing systems lack the means to collect feedback from evaluators and present that feedback appropriately to users. Furthermore, the lack of feedback based on the user's emotional state poses a challenge in improving the user experience.

[0776] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for uploading generated image files, a means for detecting bias and errors from the uploaded image files, a means for selecting evaluators with different backgrounds, a means for obtaining feedback from the selected evaluators, a means for automatically translating the feedback obtained from the evaluators, a means for recognizing the emotional state of the user, and a means for presenting the obtained feedback in a form corresponding to the emotional state of the user. This makes it possible to realize fair and diverse evaluations of images containing bias and errors and to provide appropriate feedback corresponding to the emotional state of the user.

[0777] "Image File" refers to visual data generated using a generative AI model.

[0778] "Bias" refers to errors that involve particular perspectives or prejudices, such as those related to gender, race, or religion.

[0779] "Errors" refer to factually incorrect or inappropriate elements in the content of an image.

[0780] "Evaluators" refer to people or entities with different backgrounds and expertise who are tasked with evaluating and providing feedback on the content of images.

[0781] "Feedback" refers to information such as comments, suggested corrections, and findings provided by the evaluator.

[0782] "Machine translation" refers to the process of mechanically converting the content of feedback obtained from evaluators into another language.

[0783] "Emotional state" refers to the user's current mental and emotional state.

[0784] "Generative AI model" refers to an artificial intelligence algorithm that creates a generated visual representation based on an input prompt.

[0785] A "prompt" refers to a text instruction or explanation input to a generative AI model.

[0786] The present invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators. The system includes, among other things, a means of providing feedback that takes into account the user's emotional state.

[0787] The overall flow of the system is as follows: First, the user generates an image using a generative AI model (e.g., DALL-E or Midjourney). The generated image is saved on the user's device. Next, the user logs in to the platform using a web browser and uploads the image file. The uploaded image file is sent to the server and saved.

[0788] The server analyzes the saved image files using a bias detection engine (e.g., IBM Watson Visual Recognition) to automatically detect bias and errors. A report is generated as a result of the detection and stored in a database. The user reviews this report and selects the evaluator they wish to receive feedback from from a list of evaluators provided on the platform.

[0789] The server sends the image files and bias detection results to the selected evaluators. The evaluators enter comments and suggestions for corrections in the feedback form and send it to the server. The server receives the feedback from the evaluators, translates it as needed using an automatic translation engine (e.g., Google Translate API), and aggregates it in a database.

[0790] The server then uses an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) to analyze the user's emotional state using facial expression and voice data collected from the user's device, and the information is stored in a database.

[0791] Finally, the server optimizes the feedback display method based on the user's emotional state and presents the feedback to the user.The user can then use the provided feedback to modify the generated images and prompts and input them back into the generative AI model to regenerate images with less bias and errors.

[0792] Specific examples

[0793] Example 1: Evaluating and correcting artist-generated illustrations

[0794] 1. The user (artist) uses DALL-E to input the following prompt sentence and generate an illustration.

[0795] Prompt: Draw a gender-neutral fantasy character.

[0796] 2. The user uploads the generated image to the platform.

[0797] 3. The server uses a bias detection engine to detect gender bias.

[0798] 4. The user selects a gender perspective expert (evaluator) and requests feedback.

[0799] 5. The evaluator fills out the feedback form with specific suggestions for correction and submits it.

[0800] 6. The server receives and translates the feedback and presents it to the user based on their emotional state.

[0801] 7. The user modifies the prompt based on the feedback and regenerates a new image.

[0802] Example 2: Advertising agency bias correction of advertising images

[0803] 1. A designer at an advertising agency uses Midjourney to enter the following prompt text and create an image for an advertisement.

[0804] Prompt: Draw a scene of a diverse business meeting.

[0805] 2. The designer uploads the generated image to the platform.

[0806] 3. The server detects that the image contains racial stereotypes.

[0807] 4. The designer selects racial diversity experts (evaluators) and solicits their feedback.

[0808] 5. Receive feedback from evaluators, automatically translate it, and present it to the designer.

[0809] 6. The designer adjusts the generative AI model based on the presented feedback and emotional state, and recreates a new advertising image.

[0810] The above is a specific embodiment of the invention, which allows users to create artwork or advertising materials that incorporate unbiased and diverse perspectives, and effectively correct biases and errors based on appropriate feedback.

[0811] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0812] Step 1:

[0813] The user generates an image using a generative AI model.

[0814] Users input a prompt (e.g., "Draw a gender-neutral fantasy character") into their device and run a generative AI model (e.g., DALL-E or Midjourney). The generated image file is saved locally on the device.

[0815] Input: prompt statement

[0816] Output: Generated image file

[0817] Step 2:

[0818] A user uploads an image to the platform.

[0819] The user opens a web browser and logs in to the platform. After logging in, they use the image upload function to select an image file and click the upload button. The image file is sent to the server and saved in a specified folder.

[0820] Input: Generated image file

[0821] Output: Image file saved on the server

[0822] Step 3:

[0823] The server detects bias and errors.

[0824] The server invokes a bias detection engine (e.g., IBM Watson Visual Recognition) to analyze the uploaded image file. If bias or errors are detected, a report is generated identifying the type and location of the errors and stored in a database.

[0825] Input: Saved image file

[0826] Output: Bias detection report

[0827] Step 4:

[0828] The user selects the evaluator.

[0829] The user reviews the bias detection report, selects the evaluator they wish to receive feedback from from the list of evaluators provided on the platform, and clicks a button to confirm their selection.

[0830] Input: Bias detection report, evaluator list

[0831] Output: Selected raters

[0832] Step 5:

[0833] The server collects the feedback.

[0834] The server will send the uploaded image files and bias detection results to the selected raters, who will be notified and receive a request for feedback from their account.

[0835] Input: Selected evaluators, image files, bias detection results

[0836] Output: Feedback request notification

[0837] Step 6:

[0838] The evaluator provides feedback.

[0839] The evaluator receives the notification and logs in to the platform. They check the image to be evaluated and the bias information, and enter comments and suggestions for corrections in the feedback form. Once they have completed the input, they click the submit button to send the feedback to the server.

[0840] Input: Feedback request notification, bias information

[0841] Output: Feedback data

[0842] Step 7:

[0843] The server receives and translates the feedback.

[0844] The server receives feedback data sent by evaluators and saves it in a designated folder. It then aggregates the feedback data in a database and translates the feedback content using an automatic translation engine (e.g., Google Translate API) if necessary. The translated data is then saved in the database.

[0845] Input: Feedback data

[0846] Output: Translated feedback data

[0847] Step 8:

[0848] The server uses an emotion engine to recognize the user's emotion.

[0849] The server calls an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) to analyze facial and voice data collected from the user's device. As a result of the analysis, the user's current emotional state (e.g., joy, surprise, sadness) is identified and stored in a database.

[0850] Input: facial expression data, voice data

[0851] Output: User's emotional state

[0852] Step 9:

[0853] The server provides feedback.

[0854] The server optimizes the feedback display method based on the user's emotional state, and the feedback can be presented in the form of a pop-up, notification bar, voice message, etc. The feedback will be displayed when the user logs in to the platform and accesses the dashboard.

[0855] Input: User's emotional state, translated feedback data

[0856] Output: Feedback presented to the user

[0857] Step 10:

[0858] The user fine-tunes the image based on the feedback.

[0859] The user reviews the provided feedback and modifies the generated image and prompt text. For example, if gender bias is identified, the prompt text can be modified to explicitly state "gender-neutral." The modified prompt is then input into the generative AI model to generate a new image. The generated image can then be uploaded back to the platform, and additional feedback can be sought if necessary.

[0860] Input: Feedback

[0861] Output: Modified image file

[0862] The above are the specific processing steps.

[0863] (Application example 2)

[0864] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0865] Existing image generation systems face the challenge of effectively detecting and correcting biases and errors in generated image files. Furthermore, they face the challenge of not being able to provide appropriate feedback that takes into account the user's emotional state. This makes it difficult to create images that incorporate fair and diverse perspectives, and to create advertising images with minimal user bias.

[0866] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0867] In this invention, the server includes means for uploading generated image files, means for detecting bias or errors from the uploaded image files, means for selecting evaluators with different backgrounds, means for obtaining feedback from the selected evaluators, means for presenting the obtained feedback to the user, means for recognizing the emotional state of the user, and means for optimizing the display format of the feedback content based on the recognized emotional state. This makes it possible to present optimal feedback according to the emotional state of the user, thereby realizing image production that is fair and incorporates diverse perspectives.

[0868] "Image file" refers to a digitally generated still image that contains visual information about a subject or scene.

[0869] "Uploading" is the process of transferring or sending a digital file from a user's device to a server via the Internet.

[0870] "Bias" refers to the phenomenon in which certain prejudices or stereotypes influence data or systems, and is a factor that can lead to unfair results.

[0871] "Error" refers to an unintended inaccuracy or inconsistency in data or results.

[0872] "Evaluator" means a person who has specialized knowledge and experience in the uploaded image and provides evaluation and feedback on the image.

[0873] "Feedback" refers to information such as comments, suggestions, and improvements provided by the evaluator about an image.

[0874] "User" refers to the person who uses the system to generate and upload images and receive feedback.

[0875] "Emotional state" refers to the user's current psychological and emotional state, and is analyzed from data such as facial expressions and voice.

[0876] "Optimization" refers to the process of achieving the most effective state or arrangement for a specific purpose or condition.

[0877] "Display method" refers to the format or method for visually presenting information.

[0878] "Database" refers to a system for systematically storing and managing specific data.

[0879] A "server" is a computer system that provides services and data to clients over a network.

[0880] This invention begins with a user uploading an image file they have generated to the system using a device such as a smartphone or computer. The user generates an image using a generative AI model and uploads it to the system. The uploaded image file is then stored on a server.

[0881] The server automatically detects biases and errors in the stored image files using pre-trained models. For example, if the image contains gender bias or racial stereotypes, it will detect them and generate a report containing the type and location of identified bias. This report is stored in a database.

[0882] Next, users select their desired evaluators from a list of evaluators provided on the platform. Evaluators are experts with different backgrounds who can provide unbiased and diverse perspectives on some aspects. The server then sends the uploaded image files and bias detection results to the selected evaluators and requests their feedback.

[0883] Evaluators use their devices to check the images and bias information, and then enter comments, specific suggestions, and corrections in the feedback form. The entered feedback is sent to the server, which receives and translates it, then aggregates and stores it in a database.

[0884] Furthermore, the server uses an emotion engine to analyze facial expression and voice data collected from the user's device to recognize the user's current emotional state, which is then stored in a database and used to optimize the way feedback is displayed.

[0885] Finally, the server presents the aggregated feedback according to the user's emotional state (for example, a positive tone if the user is happy, or a comforting tone if the user is sad).The user can use this feedback to fine-tune the generative AI model and the image, then upload the image again for further feedback.

[0886] For example, an advertising designer uploads an advertising image created using a generative AI model, and the server detects racial stereotypes in the image. The designer selects evaluators knowledgeable about diversity and receives feedback. The server checks the designer's emotional state and selects an appropriate feedback display format. The designer adjusts the generative AI model based on this feedback.

[0887] Examples of prompt sentences include the following:

[0888] "Please advise on creating advertising images that are gender-bias-free. The current image contains the following biases: {List of biases detected}"

[0889] As described above, this system incorporates a fair and diverse range of viewpoints and provides optimal feedback according to the user's emotional state, enabling high-quality image production.

[0890] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0891] Step 1:

[0892] The user prepares an image file generated using the generative AI model on their device and uploads the image file to the platform. The server receives and stores the uploaded image file.

[0893] Input: Image file (created by the user using a generative AI model)

[0894] Output: Image files are saved to the server

[0895] Step 2:

[0896] The server detects bias and errors in the stored image files using pre-trained models. The bias detection model identifies prejudices and stereotypes in the images and generates a report based on this. The report includes the type and location of bias detected.

[0897] Input: Saved image file

[0898] Output: Bias detection report (including type and location of bias)

[0899] Step 3:

[0900] Users select the evaluators they wish to receive feedback from from a list of evaluators provided on the platform. Users choose evaluators with different backgrounds, with the aim of obtaining fair and diverse perspectives.

[0901] Input: Rater list, user selection

[0902] Output: Selected raters

[0903] Step 4:

[0904] The server then sends the uploaded image files and bias detection results to the selected evaluators and requests their feedback, preparing the server to receive specific feedback and suggestions for corrections from the evaluators.

[0905] Input: Image file, bias detection report, selected rater

[0906] Output: Feedback request to evaluator

[0907] Step 5:

[0908] The evaluator uses the device to review the submitted images and bias information, and enters comments, specific suggestions, and correction suggestions in the feedback form. The evaluator's feedback is then sent from the device to the server.

[0909] Input: Image file, bias detection report

[0910] Output: Rater feedback

[0911] Step 6:

[0912] The server receives feedback data returned by evaluators, aggregates and stores it in a database, and automatically translates the feedback data as needed, providing users with feedback in a format that is easy to understand.

[0913] Input: Rater feedback

[0914] Output: translated feedback, aggregated feedback data

[0915] Step 7:

[0916] The server uses an emotion engine to analyze facial and voice data collected from the user's device and recognize the user's current emotional state. The emotion engine is implemented using, for example, the DeepFace library.

[0917] Input: User's facial expressions and voice data

[0918] Output: User's emotional state

[0919] Step 8:

[0920] The server optimizes the display of feedback content based on the user's emotional state. The aggregated feedback is presented in a way that reflects the user's emotional state. For example, positive feedback is displayed prominently.

[0921] Input: Emotional state, translated feedback

[0922] Output: Optimized feedback display

[0923] Step 9:

[0924] Based on the feedback provided by the user, the image and generative AI model are fine-tuned, and the corrected image is uploaded again and can receive feedback again through the same process.

[0925] Input: Optimized feedback display

[0926] Output: Fine-tuned image

[0927] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0928] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0929] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0930] [Third embodiment]

[0931] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0932] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0933] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0934] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0935] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0936] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0937] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0938] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0939] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0940] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0941] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0942] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0943] This invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators, allowing users to create artwork that incorporates unbiased and diverse perspectives.

[0944] Program processing

[0945] 1. User uploads an image

[0946] Subject: User

[0947] Users upload images generated using the generative AI from their devices to the server, where the uploaded image files are stored.

[0948] 2. The server detects bias and errors

[0949] Subject: Server

[0950] The server receives the uploaded images and automatically detects bias and errors using a bias detection engine, which uses pre-trained models.

[0951] 3. The user selects the evaluator

[0952] Subject: User

[0953] After reviewing the results of bias and error detection, users can select the evaluator from whom they would like to receive feedback from a list of evaluators displayed on the system. The list displays the evaluators' fields of expertise and profiles, and they come from a variety of backgrounds.

[0954] 4. The server collects feedback

[0955] Subject: Server

[0956] The server sends the uploaded images and bias detection results to the selected evaluators and asks for their feedback. The evaluators then check the images and information on their own devices and enter their feedback.

[0957] 5. Evaluators provide feedback

[0958] Subject: Terminal (evaluator)

[0959] Evaluators use their devices to send feedback to the server, including specific comments, like / dislike ratings, and unpleasantness indices.

[0960] 6. The server aggregates and presents the feedback

[0961] Subject: Server

[0962] The server aggregates the feedback collected from the evaluators and presents it to the user. If necessary, the server automatically translates the feedback and provides it in multiple languages ​​in a format that is easy for the user to understand.

[0963] 7. Tweak the image based on user feedback

[0964] Subject: User

[0965] Users can then use the feedback they provide to fine-tune the images and generative AI models, and the process is repeated until the end result is a less biased and more diverse piece of art.

[0966] Specific examples

[0967] Example 1: Evaluating and correcting artist-generated portraits

[0968] Artist X creates a portrait using generative AI and uploads it to the platform, where the server detects that the portrait contains unofficial gender bias.

[0969] Artist X selects evaluator Y, an expert on gender perspectives. The server sends the image and bias information to evaluator Y and asks for feedback.

[0970] Evaluator Y uses a device to check the image and submits feedback including specific suggestions for correction. The server presents the feedback to artist X and performs automatic translation if translation is required.

[0971] Based on the feedback, Artist X adjusts the generative AI model to recreate portraits with less gender bias.

[0972] Example 2: Advertising agency bias correction of advertising images

[0973] A designer at an advertising agency uses generative AI to create and upload images for advertisements, and the server detects if the images contain racial stereotypes.

[0974] The designer selects Evaluator Z, an expert on racial diversity, and asks for feedback. The server sends the image and the detection results to Evaluator Z.

[0975] The server receives the feedback from rater Z and presents it to the designer, translating it if necessary. The designer then adjusts the generative AI model based on the feedback to recreate the advertising image with fewer racial stereotypes.

[0976] As described above, the system of the present invention effectively identifies and corrects biases and errors in generated images, helping users create artworks based on fair and diverse perspectives.

[0977] The processing flow will be explained below.

[0978] Step 1:

[0979] Users prepare the image generated by the generative AI on their device, then drag and drop the image file into the platform's upload interface and click the upload button, which sends the image file to the server.

[0980] Step 2:

[0981] The server receives the uploaded image file, saves it in storage, and records the path and metadata of the saved file in the database.

[0982] Step 3:

[0983] The server then sends the stored image files to a bias detection engine, which uses pre-trained models to detect bias and errors in the images and generates a report containing the type and location of identified bias.

[0984] Step 4:

[0985] The server stores the results of the detection in a database and notifies the user, including a summary of the bias and instructions for next steps.

[0986] Step 5:

[0987] Users check the bias detection results on the platform, and then select the evaluator they would like to receive feedback from from the list of evaluators provided by the system.

[0988] Step 6:

[0989] The server sends the uploaded image file and bias detection results to the evaluator selected by the user and generates a feedback request.

[0990] Step 7:

[0991] The evaluator will use their own device to check the submitted images and bias information, enter comments, specific indications of bias, and suggestions for correction in the feedback form, and then press the submit button.

[0992] Step 8:

[0993] The server receives feedback data returned by the evaluators, aggregates and stores it in a database, and automatically translates the feedback data if necessary.

[0994] Step 9:

[0995] The server then presents the aggregated feedback to the user, including the reviewer's comments, Like / Dislike ratings, and an unpleasantness index. The feedback is presented in a format that is easy for the user to understand.

[0996] Step 10:

[0997] Users can fine-tune the image and generative AI model based on the feedback they provide, and then upload the regenerated image using the same process, seeking further feedback if needed.

[0998] This is the specific process flow, which effectively corrects bias and errors in the generated images and produces artwork that reflects fair and diverse perspectives.

[0999] Example 1

[1000] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1001] Current image generation processes using generative AI technology may contain biases and errors in the generated images. These biases and errors can result in unintended losses in diversity and fairness. Furthermore, the process of evaluating and correcting generated images is cumbersome and requires a lot of time and effort. Furthermore, there is a lack of systems that can collect feedback from evaluators with different backgrounds and provide it in a format that is easy for users to understand. There is a need for a system that can resolve these issues and support the generation of high-quality images based on fair and diverse perspectives.

[1002] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1003] In this invention, the server includes a means for uploading generated image files, a means for detecting biases and errors from the uploaded image files, a means for presenting the results of the bias and error detection to a user and selecting evaluators with different backgrounds, a means for collecting feedback from the selected evaluators, and a means for providing the collected feedback to a user and supporting fine-tuning of the generated images. This makes it possible to effectively detect and correct biases and errors in the generated images and generate high-quality images based on diverse viewpoints.

[1004] An "image file" is digital data containing visual information generated using a generative AI model.

[1005] "Uploading" is the act of sending digital data (image files) from a user's device to a server and storing it.

[1006] "Bias" refers to biased viewpoints or inappropriate elements contained in the content of image files, which have an unfair influence on a particular classification or evaluation.

[1007] "Error" refers to any inaccuracy or inappropriate element in the content of an image file that deviates from the intended result.

[1008] "Evaluators" are experts or users with different backgrounds who provide feedback on the generated image files.

[1009] "Feedback" is a response, including comments, ratings, and suggested modifications, provided by a reviewer to an image file.

[1010] "User" refers to an individual or organization that uses the system to generate image files using a generative AI model.

[1011] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate image files based on input such as prompts from users.

[1012] A "prompt sentence" is text information entered by the user to convey instructions and intentions for image generation to the generative AI model.

[1013] The present invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators. This system allows users to create artwork that incorporates unbiased and diverse perspectives. Specific embodiments of the present invention are described below.

[1014] Overall system overview

[1015] The system is implemented using the following hardware and software:

[1016] User device: For generating and uploading images and checking feedback

[1017] Server: Image file storage, bias and error detection, feedback collection and presentation

[1018] Generative AI models: Examples include OpenAI's DALL-E and MidJourney

[1019] Bias detection engine: For example, OpenAI's CLIP

[1020] Translation Engine: Automatic translation tool for multilingual feedback

[1021] System Operation

[1022] User generates and uploads an image

[1023] The user uses a device to operate a generative AI model (e.g., DALL-E or MidJourney) and generate an image by entering the desired prompt. An example prompt might be: "A cat dancing under the night sky." The generated image is saved on the user's device.

[1024] Next, the user uploads the image file to the server via the system's web interface. By pressing the upload button, the image file is sent to the server and saved in the server's storage.

[1025] Detecting bias and error

[1026] The server receives the uploaded image file and analyzes the image content using a bias detection engine (e.g., OpenAI's CLIP). This analysis engine uses pre-trained models to detect known bias and error patterns. The detection results are saved in a format such as JSON and later provided to the user.

[1027] User-selected raters

[1028] The user checks the results of bias and error detection through the system's web interface. The interface displays a list of evaluators, each of which includes their area of ​​expertise and profile. The user selects an appropriate evaluator from the list and presses the evaluation request button.

[1029] Gathering feedback from evaluators

[1030] The server notifies the selected evaluator of the uploaded image and the bias detection results. The evaluator logs into the system on their own device and checks the image and detection results. The evaluator enters specific comments and evaluations into the feedback form and presses the submit button to send the feedback to the server.

[1031] Providing feedback and fine-tuning images

[1032] The server consolidates the collected feedback and provides it to the user. If necessary, the feedback is translated into multiple languages ​​using a translation engine. The user then uses the provided feedback to fine-tune the image using a generative AI model or image editing software. By repeating the same process, the resulting artwork is based on a less biased, fairer, and more diverse perspective.

[1033] As a concrete example:

[1034] Example 1: Evaluating and correcting artist-generated portraits

[1035] Artists create portraits using generative AI models and upload them to the system, where the server detects any gender bias in the portraits.

[1036] The artist selects evaluators who are experts in gender perspectives, and the server sends the images and bias information to the evaluators and asks for their feedback.

[1037] The evaluator submits feedback with specific suggestions for correction, and the server provides the feedback to the artist, translating it if necessary.

[1038] Based on the feedback, the artist adjusts the generative AI model to generate new portraits.

[1039] Using these steps, the system of the present invention can effectively detect and correct biases and errors in the image generation process, helping users to create high-quality artwork that incorporates diverse perspectives.

[1040] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1041] Step 1:

[1042] Image generation and upload

[1043] The user uses the generative AI model to generate images by entering a prompt.

[1044] Input: Prompt sentence (e.g. "Cats dancing under the night sky")

[1045] Output: Generated image file

[1046] Users save the generated image files to their own devices, then log in to the system using a dedicated web interface and upload the saved image files.

[1047] Input: Generated image file

[1048] Output: Image file stored on the server

[1049] Step 2:

[1050] Detecting bias and error

[1051] The server receives the uploaded image file and then analyzes the image using a bias detection engine (which includes a pre-trained model).

[1052] Input: Uploaded image file

[1053] Output: Bias and error detection results (e.g., JSON format)

[1054] Specifically, the server passes the image to an analysis engine, which detects known patterns of bias and error, and stores the results on the server for later review by the user.

[1055] Step 3:

[1056] Presenting bias and error findings

[1057] The server presents the bias and error detection results to the user, who can review these results through a web interface.

[1058] Input: Bias and error detection results

[1059] Output: User-viewable reports

[1060] Specifically, the server displays the detection results on a web interface in a format that is easy for the user to understand.

[1061] Step 4:

[1062] Selection of raters

[1063] The user selects the evaluator from whom they wish to receive feedback from the list of evaluators displayed on the system.

[1064] Input: Rater list and user selection of rater

[1065] Output: Selected raters

[1066] Specifically, the evaluator list includes the evaluator's fields of expertise and profiles, and the user makes a selection based on these.

[1067] Step 5:

[1068] Requesting Feedback

[1069] The server sends the uploaded images and bias detection results to the selected evaluators and requests their feedback.

[1070] Input: Selected evaluators, image files, bias detection results

[1071] Output: Feedback request sent to evaluator

[1072] Specifically, the server sends emails and notifications to the evaluators, who can then view the images and detection results on their own devices.

[1073] Step 6:

[1074] Gathering feedback

[1075] Evaluators access the feedback form on their own devices and provide feedback based on the image files and detection results.

[1076] Input: Image files, bias detection results, evaluator feedback

[1077] Output: Feedback sent to the server

[1078] Specifically, the evaluator enters specific comments and evaluations into the feedback form and presses the submit button. The feedback is then saved on the server.

[1079] Step 7:

[1080] Providing feedback and fine-tuning images

[1081] The server consolidates the collected feedback and provides it to the user, and if necessary, translates the feedback into multiple languages ​​using a translation engine.

[1082] Input: Collected feedback

[1083] Output: User-viewable feedback report

[1084] Based on the feedback, the user can fine-tune the image using a generative AI model or image editing software, and then repeat the process again to ultimately generate a high-quality image with minimal bias.

[1085] Input: Feedback report, user image tweaks

[1086] Output: Modified image file

[1087] Through these steps, the system of the present invention effectively detects and corrects biases and errors in the image generation process, and supports the creation of artworks based on diverse perspectives.

[1088] (Application example 1)

[1089] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1090] This invention relates to a system for producing fair and diverse artworks and advertising images by automatically detecting biases and errors contained in image files created using generative AI and collecting feedback from diverse perspectives. Conventional methods require significant time and effort to manually detect biases contained in generated images, making it difficult to incorporate diverse perspectives. Furthermore, there was a lack of a method for efficiently collecting feedback from evaluators and presenting it to users in multiple languages. This made it difficult to detect and correct biases in image production using generative AI, resulting in the risk of inappropriate images being published.

[1091] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1092] In this invention, the server includes a means for uploading generated image files, a means for detecting bias and errors from the uploaded image files, and a means for selecting evaluators with different backgrounds. This allows users to quickly detect bias and errors in uploaded images, effectively collect feedback from evaluators with diverse backgrounds, and present the results in multiple languages. The server also includes a means for users to adjust images based on the feedback, ultimately producing high-quality images that reflect fair and diverse perspectives.

[1093] "Generated Image File" refers to visual data created using generative AI.

[1094] "Bias and Errors" refers to inappropriate biases and mistakes contained in the generated image files.

[1095] "Evaluator" refers to a third party individual or entity with a different background or expertise whose role is to evaluate and provide feedback on images.

[1096] "Feedback" refers to information such as specific comments, evaluations, and suggested corrections obtained from the evaluator.

[1097] "User" refers to the entity that uploads the generated image files and receives bias and error detection and feedback.

[1098] "Server" refers to the computer system that uploads images, detects bias and errors, solicits feedback from raters, and aggregates and presents the feedback.

[1099] A "trained model" refers to an algorithm or neural network that has been pre-trained using large amounts of data.

[1100] "Machine translation" refers to technology that mechanically translates text and feedback between different languages.

[1101] "Means for adjusting images" refers to methods or tools that allow users to modify or regenerate generative AI models or images based on user feedback.

[1102] The system of the present invention allows users to upload generated image files, automatically detect bias and errors, and collect and present feedback from evaluators in order to create fair and diverse artworks and advertising images.

[1103] Hardware and software used:

[1104] Hardware:

[1105] Server: Uploads images, detects bias and errors, requests feedback from raters, and aggregates and presents the feedback.

[1106] Client device: A smartphone is the primary device used to upload images and view feedback.

[1107] software:

[1108] Flask: Used as a Python web framework to process requests on the server.

[1109] JSON: Used to exchange data.

[1110] Trained models: Use pre-trained algorithms or neural networks to detect bias and errors.

[1111] Image storage library: Used to store uploaded images on the server.

[1112] Specific system processing description:

[1113] The user uploads the generated image file from their smartphone to the server, which receives the image using Flask and stores it in local storage.The server then uses the trained model to detect biases and errors in the uploaded image file.

[1114] Based on the detected biases and errors, the server presents a list of evaluators to the user. The evaluators are third parties with different backgrounds and expertise, and the user can review the evaluator profiles and select the evaluators they want to receive the necessary feedback from.

[1115] The server sends the images and bias detection results to the selected evaluators. The evaluators then review the images and input their feedback using their own devices. Once the feedback is collected, the server aggregates it and presents it to the user in multiple languages.

[1116] Based on the feedback provided, users can adjust the images and generative AI models, and then repeat the process to create high-quality images with less bias.

[1117] Examples:

[1118] For example, a designer at an advertising agency uses generative AI to create an image for an advertisement and upload it to the system. The server detects that the image contains racial stereotypes. The designer selects an expert on racial diversity from the evaluators presented on the system and asks for their feedback. The server then presents the evaluator's feedback to the designer, who then adjusts the generative AI model based on the feedback and recreates the advertisement image with fewer racial stereotypes.

[1119] Example prompts to input to a generative AI model:

[1120] "Generate a scene of an adult woman working in an office. The image should reflect a modern, diverse work environment."

[1121] Images generated based on this prompt can then be uploaded to the system, where bias detection and feedback collection can be performed to produce fair and diverse images.

[1122] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1123] Step 1:

[1124] Upload user-generated images from your smartphone to the server.

[1125] Input: Generated image files on your smartphone.

[1126] Processing: Use the Flask framework to save the image received by the server to local storage.

[1127] Output: An image file saved on the server (e.g. uploaded_image.jpg).

[1128] Step 2:

[1129] The server detects bias and errors in the uploaded images.

[1130] Input: Image files stored on the server.

[1131] Processing: Using a trained model (a pre-trained neural network) to analyze the image for biases and errors.

[1132] Output: Bias or error detection results (e.g., "Gender bias detected").

[1133] Step 3:

[1134] The server presents the results of the bias and error detection to the user and allows them to select an evaluator.

[1135] Input: Bias and error detection results, evaluator list.

[1136] Processing: The detection results and the list of evaluators are displayed on the user's smartphone, and the user is prompted to select an evaluator.

[1137] Output: Information about the rater selected by the user.

[1138] Step 4:

[1139] The server sends the images and detection results to the selected evaluators and requests their feedback.

[1140] Input: User-selected evaluator information, image files stored on the server, and bias and error detection results.

[1141] Processing: Send the images and detection results to the selected evaluators and issue a request for feedback.

[1142] Output: The request sent to the evaluator.

[1143] Step 5:

[1144] The evaluator enters the feedback and sends it to the server.

[1145] Input: Rater feedback (specific comments and ratings)

[1146] Processing: Feedback data is sent to the server via the evaluator's terminal.

[1147] Output: Feedback received by the server.

[1148] Step 6:

[1149] The server aggregates the feedback and presents it to the user in multiple languages.

[1150] Input: Multiple pieces of feedback received from raters.

[1151] Processing: Feedback is collected, automatically translated if necessary, and displayed on the user's smartphone in multiple languages.

[1152] Output: A multilingual translated feedback list.

[1153] Step 7:

[1154] Users adjust the images and generative AI models based on feedback.

[1155] Input: Feedback received by the user.

[1156] Processing: Use the feedback to adjust the parameters of the generative AI model, enter new prompts to regenerate the image, or adjust the image directly using image editing tools.

[1157] Output: Adjusted or regenerated image files.

[1158] Following these specific processing steps, the system quickly detects bias and errors in user-generated image files and enables the system to recreate fair and diverse images based on feedback from evaluators.

[1159] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1160] This invention relates to a system that uploads generated image files, automatically detects bias and errors, collects and presents feedback from evaluators, and provides feedback that takes into account the user's emotional state by combining it with an emotion engine. This system allows users to create artwork that incorporates fair and diverse perspectives, and receives appropriate feedback according to the user's emotional state.

[1161] Program processing

[1162] 1. User uploads an image

[1163] Subject: User

[1164] Users prepare images generated using generative AI on their devices and upload the image files to the platform, where they are sent to the server and stored.

[1165] 2. The server detects bias and errors

[1166] Subject: Server

[1167] The server receives the stored image files and uses a bias detection engine to automatically detect biases and errors, generating a report containing the type and location of identified biases and storing it in a database.

[1168] 3. The user selects the evaluator

[1169] Subject: User

[1170] The user reviews the bias detection results and selects the evaluator from whom they would like to receive feedback from a list of evaluators provided on the platform.

[1171] 4. The server collects feedback

[1172] Subject: Server

[1173] The server sends the uploaded image files and bias detection results to the selected evaluators and asks for their feedback.

[1174] 5. Evaluators provide feedback

[1175] Subject: Terminal (evaluator)

[1176] The evaluator uses a terminal to check the submitted images and bias information, enters comments, specific indications of bias, and suggestions for correction into the feedback form, and sends it to the server.

[1177] 6. The server receives and translates the feedback

[1178] Subject: Server

[1179] The server receives feedback data returned by the evaluators, aggregates and stores it in a database, and automatically translates the feedback data if necessary.

[1180] 7. The server uses an emotion engine to recognize the user's emotions.

[1181] Subject: Server

[1182] The server uses an emotion engine to analyze facial and voice data collected from the user's device to recognize the user's current emotional state, which is then stored in a database.

[1183] 8. The server provides feedback

[1184] Subject: Server

[1185] The server optimizes the display method of the feedback content based on the recognized emotional state, and the aggregated feedback is presented to the user in a form that corresponds to the user's emotional state.

[1186] 9. Tweak the image based on user feedback

[1187] Subject: User

[1188] Users can fine-tune the image and generative AI model based on the feedback they provide, and then upload the regenerated image using the same process, seeking further feedback if needed.

[1189] Specific examples

[1190] Example 1: Evaluating and correcting artist-generated illustrations

[1191] Artist X creates an illustration using generative AI and uploads it to the platform, where the server detects that the illustration contains gender bias.

[1192] Artist X selects evaluator Y, an expert on gender perspectives. The server sends the image and bias information to evaluator Y and asks for feedback.

[1193] Evaluator Y uses a device to check the image and send feedback including specific suggestions for correction. The server receives and translates the feedback and presents it to artist X.

[1194] Before presenting the illustration, the server uses an emotion engine to investigate Artist X's emotional state and select an appropriate feedback display method. Based on the feedback, Artist X adjusts the generative AI model and recreates the illustration with less gender bias.

[1195] Example 2: Advertising agency bias correction of advertising images

[1196] A designer at an advertising agency uses generative AI to create and upload images for advertisements, and the server detects if the images contain racial stereotypes.

[1197] The designer selects Evaluator Z, an expert on racial diversity, and asks for feedback. The server sends the image and the detection results to Evaluator Z.

[1198] The server receives feedback from evaluator Z, aggregates and translates it, and presents it to the designer. Before presenting it, the server uses an emotion engine to analyze the designer's emotional state and select the optimal display method.

[1199] Designers use the feedback to adjust the generative AI model to recreate advertising images with fewer racial stereotypes.

[1200] This is the specific processing flow, which effectively corrects biases and errors in the generated images while taking into account the user's emotional state, allowing for the creation of artwork that reflects fair and diverse perspectives.

[1201] The processing flow will be explained below.

[1202] Step 1:

[1203] Users prepare the image files generated by the generative AI on their devices, then drag and drop the image files into the platform's upload interface and click the upload button to send them to the server.

[1204] Step 2:

[1205] The server receives the uploaded image file, saves it in storage, and records the path and metadata of the saved file in the database.

[1206] Step 3:

[1207] The server then sends the saved image files to a bias detection engine, which uses pre-trained models to automatically detect biases and errors in the images and generates a report containing the type and location of any biases identified.

[1208] Step 4:

[1209] The server saves the bias detection result report in a database and notifies the user, including a summary of the bias and instructions for next steps.

[1210] Step 5:

[1211] Users open the platform on their device, check the bias detection results, and then select the evaluator they wish to provide feedback to from the list of evaluators provided by the system. The list displays each evaluator's area of ​​expertise and profile.

[1212] Step 6:

[1213] The server sends the uploaded image file and bias detection results to the evaluator selected by the user and requests feedback.

[1214] Step 7:

[1215] The evaluator uses their own device to check the submitted images and bias information, enters comments, specific indications of bias, and suggestions for correction in the feedback form, and sends the feedback to the server.

[1216] Step 8:

[1217] The server receives feedback data from evaluators, aggregates it in a database, and stores it. If necessary, it automatically translates the feedback data and makes it understandable to users.

[1218] Step 9:

[1219] The server uses an emotion engine to recognize the user's emotions. It analyzes facial expressions and voice data while the user is using the platform and evaluates the user's current emotional state. The evaluation data is stored in a database.

[1220] Step 10:

[1221] The server determines how to display the feedback based on the collected feedback data and the user's emotional state. For example, if the user is feeling stressed, the server may display the feedback in a softer tone.

[1222] Step 11:

[1223] The server presents the optimized feedback to the user, who can review it on their device and is also provided with a detailed view function to review the feedback again if necessary.

[1224] Step 12:

[1225] Users can use the feedback provided to fine-tune the generative AI model and the image itself, then re-upload the generated image using the same process and ask for further feedback if necessary.

[1226] These are the specific processing steps of the system that combines the emotion engine. This series of processes makes it possible to effectively create artwork that reflects fair and diverse viewpoints while taking into account the user's emotional state.

[1227] Example 2

[1228] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1229] In recent years, image generation using generative AI models has become popular, but the generated images can contain biases and errors, such as gender and race bias. Therefore, there is a need to evaluate and correct these images from a fair and diverse perspective. However, existing systems lack the means to collect feedback from evaluators and present that feedback appropriately to users. Furthermore, the lack of feedback based on the user's emotional state poses a challenge in improving the user experience.

[1230] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for uploading generated image files, a means for detecting bias and errors from the uploaded image files, a means for selecting evaluators with different backgrounds, a means for obtaining feedback from the selected evaluators, a means for automatically translating the feedback obtained from the evaluators, a means for recognizing the emotional state of the user, and a means for presenting the obtained feedback in a form corresponding to the emotional state of the user. This makes it possible to realize fair and diverse evaluations of images containing bias and errors and to provide appropriate feedback corresponding to the emotional state of the user.

[1231] "Image File" refers to visual data generated using a generative AI model.

[1232] "Bias" refers to errors that involve particular perspectives or prejudices, such as those related to gender, race, or religion.

[1233] "Errors" refer to factually incorrect or inappropriate elements in the content of an image.

[1234] "Evaluators" refer to people or entities with different backgrounds and expertise who are tasked with evaluating and providing feedback on the content of images.

[1235] "Feedback" refers to information such as comments, suggested corrections, and findings provided by the evaluator.

[1236] "Machine translation" refers to the process of mechanically converting the content of feedback obtained from evaluators into another language.

[1237] "Emotional state" refers to the user's current mental and emotional state.

[1238] "Generative AI model" refers to an artificial intelligence algorithm that creates a generated visual representation based on an input prompt.

[1239] A "prompt" refers to a text instruction or explanation input to a generative AI model.

[1240] The present invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators. The system includes, among other things, a means of providing feedback that takes into account the user's emotional state.

[1241] The overall flow of the system is as follows: First, the user generates an image using a generative AI model (e.g., DALL-E or Midjourney). The generated image is saved on the user's device. Next, the user logs in to the platform using a web browser and uploads the image file. The uploaded image file is sent to the server and saved.

[1242] The server analyzes the saved image files using a bias detection engine (e.g., IBM Watson Visual Recognition) to automatically detect bias and errors. A report is generated as a result of the detection and stored in a database. The user reviews this report and selects the evaluator they wish to receive feedback from from a list of evaluators provided on the platform.

[1243] The server sends the image files and bias detection results to the selected evaluators. The evaluators enter comments and suggestions for corrections in the feedback form and send it to the server. The server receives the feedback from the evaluators, translates it as needed using an automatic translation engine (e.g., Google Translate API), and aggregates it in a database.

[1244] The server then uses an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) to analyze the user's emotional state using facial expression and voice data collected from the user's device, and the information is stored in a database.

[1245] Finally, the server optimizes the feedback display method based on the user's emotional state and presents the feedback to the user.The user can then use the provided feedback to modify the generated images and prompts and input them back into the generative AI model to regenerate images with less bias and errors.

[1246] Specific examples

[1247] Example 1: Evaluating and correcting artist-generated illustrations

[1248] 1. The user (artist) uses DALL-E to input the following prompt sentence and generate an illustration.

[1249] Prompt: Draw a gender-neutral fantasy character.

[1250] 2. The user uploads the generated image to the platform.

[1251] 3. The server uses a bias detection engine to detect gender bias.

[1252] 4. The user selects a gender perspective expert (evaluator) and requests feedback.

[1253] 5. The evaluator fills out the feedback form with specific suggestions for correction and submits it.

[1254] 6. The server receives and translates the feedback and presents it to the user based on their emotional state.

[1255] 7. The user modifies the prompt based on the feedback and regenerates a new image.

[1256] Example 2: Advertising agency bias correction of advertising images

[1257] 1. A designer at an advertising agency uses Midjourney to enter the following prompt text and create an image for an advertisement.

[1258] Prompt: Draw a scene of a diverse business meeting.

[1259] 2. The designer uploads the generated image to the platform.

[1260] 3. The server detects that the image contains racial stereotypes.

[1261] 4. The designer selects racial diversity experts (evaluators) and solicits their feedback.

[1262] 5. Receive feedback from evaluators, automatically translate it, and present it to the designer.

[1263] 6. The designer adjusts the generative AI model based on the presented feedback and emotional state, and recreates a new advertising image.

[1264] The above is a specific embodiment of the invention, which allows users to create artwork or advertising materials that incorporate unbiased and diverse perspectives, and effectively correct biases and errors based on appropriate feedback.

[1265] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1266] Step 1:

[1267] The user generates an image using a generative AI model.

[1268] Users input a prompt (e.g., "Draw a gender-neutral fantasy character") into their device and run a generative AI model (e.g., DALL-E or Midjourney). The generated image file is saved locally on the device.

[1269] Input: prompt statement

[1270] Output: Generated image file

[1271] Step 2:

[1272] A user uploads an image to the platform.

[1273] The user opens a web browser and logs in to the platform. After logging in, they use the image upload function to select an image file and click the upload button. The image file is sent to the server and saved in a specified folder.

[1274] Input: Generated image file

[1275] Output: Image file saved on the server

[1276] Step 3:

[1277] The server detects bias and errors.

[1278] The server invokes a bias detection engine (e.g., IBM Watson Visual Recognition) to analyze the uploaded image file. If bias or errors are detected, a report is generated identifying the type and location of the errors and stored in a database.

[1279] Input: Saved image file

[1280] Output: Bias detection report

[1281] Step 4:

[1282] The user selects the evaluator.

[1283] The user reviews the bias detection report, selects the evaluator they wish to receive feedback from from the list of evaluators provided on the platform, and clicks a button to confirm their selection.

[1284] Input: Bias detection report, evaluator list

[1285] Output: Selected raters

[1286] Step 5:

[1287] The server collects the feedback.

[1288] The server will send the uploaded image files and bias detection results to the selected raters, who will be notified and receive a request for feedback from their account.

[1289] Input: Selected evaluators, image files, bias detection results

[1290] Output: Feedback request notification

[1291] Step 6:

[1292] The evaluator provides feedback.

[1293] The evaluator receives the notification and logs in to the platform. They check the image to be evaluated and the bias information, and enter comments and suggestions for corrections in the feedback form. Once they have completed the input, they click the submit button to send the feedback to the server.

[1294] Input: Feedback request notification, bias information

[1295] Output: Feedback data

[1296] Step 7:

[1297] The server receives and translates the feedback.

[1298] The server receives feedback data sent by evaluators and saves it in a designated folder. It then aggregates the feedback data in a database and translates the feedback content using an automatic translation engine (e.g., Google Translate API) if necessary. The translated data is then saved in the database.

[1299] Input: Feedback data

[1300] Output: Translated feedback data

[1301] Step 8:

[1302] The server uses an emotion engine to recognize the user's emotion.

[1303] The server calls an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) to analyze facial and voice data collected from the user's device. As a result of the analysis, the user's current emotional state (e.g., joy, surprise, sadness) is identified and stored in a database.

[1304] Input: facial expression data, voice data

[1305] Output: User's emotional state

[1306] Step 9:

[1307] The server provides feedback.

[1308] The server optimizes the feedback display method based on the user's emotional state, and the feedback can be presented in the form of a pop-up, notification bar, voice message, etc. The feedback will be displayed when the user logs in to the platform and accesses the dashboard.

[1309] Input: User's emotional state, translated feedback data

[1310] Output: Feedback presented to the user

[1311] Step 10:

[1312] The user fine-tunes the image based on the feedback.

[1313] The user reviews the provided feedback and modifies the generated image and prompt text. For example, if gender bias is identified, the prompt text can be modified to explicitly state "gender-neutral." The modified prompt is then input into the generative AI model to generate a new image. The generated image can then be uploaded back to the platform, and additional feedback can be sought if necessary.

[1314] Input: Feedback

[1315] Output: Modified image file

[1316] The above are the specific processing steps.

[1317] (Application example 2)

[1318] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1319] Existing image generation systems face the challenge of effectively detecting and correcting biases and errors in generated image files. Furthermore, they face the challenge of not being able to provide appropriate feedback that takes into account the user's emotional state. This makes it difficult to create images that incorporate fair and diverse perspectives, and to create advertising images with minimal user bias.

[1320] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1321] In this invention, the server includes means for uploading generated image files, means for detecting bias or errors from the uploaded image files, means for selecting evaluators with different backgrounds, means for obtaining feedback from the selected evaluators, means for presenting the obtained feedback to the user, means for recognizing the emotional state of the user, and means for optimizing the display format of the feedback content based on the recognized emotional state. This makes it possible to present optimal feedback according to the emotional state of the user, thereby realizing image production that is fair and incorporates diverse perspectives.

[1322] "Image file" refers to a digitally generated still image that contains visual information about a subject or scene.

[1323] "Uploading" is the process of transferring or sending a digital file from a user's device to a server via the Internet.

[1324] "Bias" refers to the phenomenon in which certain prejudices or stereotypes influence data or systems, and is a factor that can lead to unfair results.

[1325] "Error" refers to an unintended inaccuracy or inconsistency in data or results.

[1326] "Evaluator" means a person who has specialized knowledge and experience in the uploaded image and provides evaluation and feedback on the image.

[1327] "Feedback" refers to information such as comments, suggestions, and improvements provided by the evaluator about an image.

[1328] "User" refers to the person who uses the system to generate and upload images and receive feedback.

[1329] "Emotional state" refers to the user's current psychological and emotional state, and is analyzed from data such as facial expressions and voice.

[1330] "Optimization" refers to the process of achieving the most effective state or arrangement for a specific purpose or condition.

[1331] "Display method" refers to the format or method for visually presenting information.

[1332] "Database" refers to a system for systematically storing and managing specific data.

[1333] A "server" is a computer system that provides services and data to clients over a network.

[1334] This invention begins with a user uploading an image file they have generated to the system using a device such as a smartphone or computer. The user generates an image using a generative AI model and uploads it to the system. The uploaded image file is then stored on a server.

[1335] The server automatically detects biases and errors in the stored image files using pre-trained models. For example, if the image contains gender bias or racial stereotypes, it will detect them and generate a report containing the type and location of identified bias. This report is stored in a database.

[1336] Next, users select their desired evaluators from a list of evaluators provided on the platform. Evaluators are experts with different backgrounds who can provide unbiased and diverse perspectives on some aspects. The server then sends the uploaded image files and bias detection results to the selected evaluators and requests their feedback.

[1337] Evaluators use their devices to check the images and bias information, and then enter comments, specific suggestions, and corrections in the feedback form. The entered feedback is sent to the server, which receives and translates it, then aggregates and stores it in a database.

[1338] Furthermore, the server uses an emotion engine to analyze facial expression and voice data collected from the user's device to recognize the user's current emotional state, which is then stored in a database and used to optimize the way feedback is displayed.

[1339] Finally, the server presents the aggregated feedback according to the user's emotional state (for example, a positive tone if the user is happy, or a comforting tone if the user is sad).The user can use this feedback to fine-tune the generative AI model and the image, then upload the image again for further feedback.

[1340] For example, an advertising designer uploads an advertising image created using a generative AI model, and the server detects racial stereotypes in the image. The designer selects evaluators knowledgeable about diversity and receives feedback. The server checks the designer's emotional state and selects an appropriate feedback display format. The designer adjusts the generative AI model based on this feedback.

[1341] Examples of prompt sentences include the following:

[1342] "Please advise on creating advertising images that are gender-bias-free. The current image contains the following biases: {List of biases detected}"

[1343] As described above, this system incorporates a fair and diverse range of viewpoints and provides optimal feedback according to the user's emotional state, enabling high-quality image production.

[1344] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1345] Step 1:

[1346] The user prepares an image file generated using the generative AI model on their device and uploads the image file to the platform. The server receives and stores the uploaded image file.

[1347] Input: Image file (created by the user using a generative AI model)

[1348] Output: Image files are saved to the server

[1349] Step 2:

[1350] The server detects bias and errors in the stored image files using pre-trained models. The bias detection model identifies prejudices and stereotypes in the images and generates a report based on this. The report includes the type and location of bias detected.

[1351] Input: Saved image file

[1352] Output: Bias detection report (including type and location of bias)

[1353] Step 3:

[1354] Users select the evaluators they wish to receive feedback from from a list of evaluators provided on the platform. Users choose evaluators with different backgrounds, with the aim of obtaining fair and diverse perspectives.

[1355] Input: Rater list, user selection

[1356] Output: Selected raters

[1357] Step 4:

[1358] The server then sends the uploaded image files and bias detection results to the selected evaluators and requests their feedback, preparing the server to receive specific feedback and suggestions for corrections from the evaluators.

[1359] Input: Image file, bias detection report, selected rater

[1360] Output: Feedback request to evaluator

[1361] Step 5:

[1362] The evaluator uses the device to review the submitted images and bias information, and enters comments, specific suggestions, and correction suggestions in the feedback form. The evaluator's feedback is then sent from the device to the server.

[1363] Input: Image file, bias detection report

[1364] Output: Rater feedback

[1365] Step 6:

[1366] The server receives feedback data returned by evaluators, aggregates and stores it in a database, and automatically translates the feedback data as needed, providing users with feedback in a format that is easy to understand.

[1367] Input: Rater feedback

[1368] Output: translated feedback, aggregated feedback data

[1369] Step 7:

[1370] The server uses an emotion engine to analyze facial and voice data collected from the user's device and recognize the user's current emotional state. The emotion engine is implemented using, for example, the DeepFace library.

[1371] Input: User's facial expressions and voice data

[1372] Output: User's emotional state

[1373] Step 8:

[1374] The server optimizes the display of feedback content based on the user's emotional state. The aggregated feedback is presented in a way that reflects the user's emotional state. For example, positive feedback is displayed prominently.

[1375] Input: Emotional state, translated feedback

[1376] Output: Optimized feedback display

[1377] Step 9:

[1378] Based on the feedback provided by the user, the image and generative AI model are fine-tuned, and the corrected image is uploaded again and can receive feedback again through the same process.

[1379] Input: Optimized feedback display

[1380] Output: Fine-tuned image

[1381] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1382] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1383] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1384] [Fourth embodiment]

[1385] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1386] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1387] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1388] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1389] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1390] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1391] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1392] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1393] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1394] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1395] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1396] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1397] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1398] This invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators, allowing users to create artwork that incorporates unbiased and diverse perspectives.

[1399] Program processing

[1400] 1. User uploads an image

[1401] Subject: User

[1402] Users upload images generated using the generative AI from their devices to the server, where the uploaded image files are stored.

[1403] 2. The server detects bias and errors

[1404] Subject: Server

[1405] The server receives the uploaded images and automatically detects bias and errors using a bias detection engine, which uses pre-trained models.

[1406] 3. The user selects the evaluator

[1407] Subject: User

[1408] After reviewing the results of bias and error detection, users can select the evaluator from whom they would like to receive feedback from a list of evaluators displayed on the system. The list displays the evaluators' fields of expertise and profiles, and they come from a variety of backgrounds.

[1409] 4. The server collects feedback

[1410] Subject: Server

[1411] The server sends the uploaded images and bias detection results to the selected evaluators and asks for their feedback. The evaluators then check the images and information on their own devices and enter their feedback.

[1412] 5. Evaluators provide feedback

[1413] Subject: Terminal (evaluator)

[1414] Evaluators use their devices to send feedback to the server, including specific comments, like / dislike ratings, and unpleasantness indices.

[1415] 6. The server aggregates and presents the feedback

[1416] Subject: Server

[1417] The server aggregates the feedback collected from the evaluators and presents it to the user. If necessary, the server automatically translates the feedback and provides it in multiple languages ​​in a format that is easy for the user to understand.

[1418] 7. Tweak the image based on user feedback

[1419] Subject: User

[1420] Users can then use the feedback they provide to fine-tune the images and generative AI models, and the process is repeated until the end result is a less biased and more diverse piece of art.

[1421] Specific examples

[1422] Example 1: Evaluating and correcting artist-generated portraits

[1423] Artist X creates a portrait using generative AI and uploads it to the platform, where the server detects that the portrait contains unofficial gender bias.

[1424] Artist X selects evaluator Y, an expert on gender perspectives. The server sends the image and bias information to evaluator Y and asks for feedback.

[1425] Evaluator Y uses a device to check the image and submits feedback including specific suggestions for correction. The server presents the feedback to artist X and performs automatic translation if translation is required.

[1426] Based on the feedback, Artist X adjusts the generative AI model to recreate portraits with less gender bias.

[1427] Example 2: Advertising agency bias correction of advertising images

[1428] A designer at an advertising agency uses generative AI to create and upload images for advertisements, and the server detects if the images contain racial stereotypes.

[1429] The designer selects Evaluator Z, an expert on racial diversity, and asks for feedback. The server sends the image and the detection results to Evaluator Z.

[1430] The server receives the feedback from rater Z and presents it to the designer, translating it if necessary. The designer then adjusts the generative AI model based on the feedback to recreate the advertising image with fewer racial stereotypes.

[1431] As described above, the system of the present invention effectively identifies and corrects biases and errors in generated images, helping users create artworks based on fair and diverse perspectives.

[1432] The processing flow will be explained below.

[1433] Step 1:

[1434] Users prepare the image generated by the generative AI on their device, then drag and drop the image file into the platform's upload interface and click the upload button, which sends the image file to the server.

[1435] Step 2:

[1436] The server receives the uploaded image file, saves it in storage, and records the path and metadata of the saved file in the database.

[1437] Step 3:

[1438] The server then sends the stored image files to a bias detection engine, which uses pre-trained models to detect bias and errors in the images and generates a report containing the type and location of identified bias.

[1439] Step 4:

[1440] The server stores the results of the detection in a database and notifies the user, including a summary of the bias and instructions for next steps.

[1441] Step 5:

[1442] Users check the bias detection results on the platform, and then select the evaluator they would like to receive feedback from from the list of evaluators provided by the system.

[1443] Step 6:

[1444] The server sends the uploaded image file and bias detection results to the evaluator selected by the user and generates a feedback request.

[1445] Step 7:

[1446] The evaluator will use their own device to check the submitted images and bias information, enter comments, specific indications of bias, and suggestions for correction in the feedback form, and then press the submit button.

[1447] Step 8:

[1448] The server receives feedback data returned by the evaluators, aggregates and stores it in a database, and automatically translates the feedback data if necessary.

[1449] Step 9:

[1450] The server then presents the aggregated feedback to the user, including the reviewer's comments, Like / Dislike ratings, and an unpleasantness index. The feedback is presented in a format that is easy for the user to understand.

[1451] Step 10:

[1452] Users can fine-tune the image and generative AI model based on the feedback they provide, and then upload the regenerated image using the same process, seeking further feedback if needed.

[1453] This is the specific process flow, which effectively corrects bias and errors in the generated images and produces artwork that reflects fair and diverse perspectives.

[1454] Example 1

[1455] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1456] Current image generation processes using generative AI technology may contain biases and errors in the generated images. These biases and errors can result in unintended losses in diversity and fairness. Furthermore, the process of evaluating and correcting generated images is cumbersome and requires a lot of time and effort. Furthermore, there is a lack of systems that can collect feedback from evaluators with different backgrounds and provide it in a format that is easy for users to understand. There is a need for a system that can resolve these issues and support the generation of high-quality images based on fair and diverse perspectives.

[1457] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1458] In this invention, the server includes a means for uploading generated image files, a means for detecting biases and errors from the uploaded image files, a means for presenting the results of the bias and error detection to a user and selecting evaluators with different backgrounds, a means for collecting feedback from the selected evaluators, and a means for providing the collected feedback to a user and supporting fine-tuning of the generated images. This makes it possible to effectively detect and correct biases and errors in the generated images and generate high-quality images based on diverse viewpoints.

[1459] An "image file" is digital data containing visual information generated using a generative AI model.

[1460] "Uploading" is the act of sending digital data (image files) from a user's device to a server and storing it.

[1461] "Bias" refers to biased viewpoints or inappropriate elements contained in the content of image files, which have an unfair influence on a particular classification or evaluation.

[1462] "Error" refers to any inaccuracy or inappropriate element in the content of an image file that deviates from the intended result.

[1463] "Evaluators" are experts or users with different backgrounds who provide feedback on the generated image files.

[1464] "Feedback" is a response, including comments, ratings, and suggested modifications, provided by a reviewer to an image file.

[1465] "User" refers to an individual or organization that uses the system to generate image files using a generative AI model.

[1466] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate image files based on input such as prompts from users.

[1467] A "prompt sentence" is text information entered by the user to convey instructions and intentions for image generation to the generative AI model.

[1468] The present invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators. This system allows users to create artwork that incorporates unbiased and diverse perspectives. Specific embodiments of the present invention are described below.

[1469] Overall system overview

[1470] The system is implemented using the following hardware and software:

[1471] User device: For generating and uploading images and checking feedback

[1472] Server: Image file storage, bias and error detection, feedback collection and presentation

[1473] Generative AI models: Examples include OpenAI's DALL-E and MidJourney

[1474] Bias detection engine: For example, OpenAI's CLIP

[1475] Translation Engine: Automatic translation tool for multilingual feedback

[1476] System Operation

[1477] User generates and uploads an image

[1478] The user uses a device to operate a generative AI model (e.g., DALL-E or MidJourney) and generate an image by entering the desired prompt. An example prompt might be: "A cat dancing under the night sky." The generated image is saved on the user's device.

[1479] Next, the user uploads the image file to the server via the system's web interface. By pressing the upload button, the image file is sent to the server and saved in the server's storage.

[1480] Detecting bias and error

[1481] The server receives the uploaded image file and analyzes the image content using a bias detection engine (e.g., OpenAI's CLIP). This analysis engine uses pre-trained models to detect known bias and error patterns. The detection results are saved in a format such as JSON and later provided to the user.

[1482] User-selected raters

[1483] The user checks the results of bias and error detection through the system's web interface. The interface displays a list of evaluators, each of which includes their area of ​​expertise and profile. The user selects an appropriate evaluator from the list and presses the evaluation request button.

[1484] Gathering feedback from evaluators

[1485] The server notifies the selected evaluator of the uploaded image and the bias detection results. The evaluator logs into the system on their own device and checks the image and detection results. The evaluator enters specific comments and evaluations into the feedback form and presses the submit button to send the feedback to the server.

[1486] Providing feedback and fine-tuning images

[1487] The server consolidates the collected feedback and provides it to the user. If necessary, the feedback is translated into multiple languages ​​using a translation engine. The user then uses the provided feedback to fine-tune the image using a generative AI model or image editing software. By repeating the same process, the resulting artwork is based on a less biased, fairer, and more diverse perspective.

[1488] As a concrete example:

[1489] Example 1: Evaluating and correcting artist-generated portraits

[1490] Artists create portraits using generative AI models and upload them to the system, where the server detects any gender bias in the portraits.

[1491] The artist selects evaluators who are experts in gender perspectives, and the server sends the images and bias information to the evaluators and asks for their feedback.

[1492] The evaluator submits feedback with specific suggestions for correction, and the server provides the feedback to the artist, translating it if necessary.

[1493] Based on the feedback, the artist adjusts the generative AI model to generate new portraits.

[1494] Using these steps, the system of the present invention can effectively detect and correct biases and errors in the image generation process, helping users to create high-quality artwork that incorporates diverse perspectives.

[1495] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1496] Step 1:

[1497] Image generation and upload

[1498] The user uses the generative AI model to generate images by entering a prompt.

[1499] Input: Prompt sentence (e.g. "Cats dancing under the night sky")

[1500] Output: Generated image file

[1501] Users save the generated image files to their own devices, then log in to the system using a dedicated web interface and upload the saved image files.

[1502] Input: Generated image file

[1503] Output: Image file stored on the server

[1504] Step 2:

[1505] Detecting bias and error

[1506] The server receives the uploaded image file and then analyzes the image using a bias detection engine (which includes a pre-trained model).

[1507] Input: Uploaded image file

[1508] Output: Bias and error detection results (e.g., JSON format)

[1509] Specifically, the server passes the image to an analysis engine, which detects known patterns of bias and error, and stores the results on the server for later review by the user.

[1510] Step 3:

[1511] Presenting bias and error findings

[1512] The server presents the bias and error detection results to the user, who can review these results through a web interface.

[1513] Input: Bias and error detection results

[1514] Output: User-viewable reports

[1515] Specifically, the server displays the detection results on a web interface in a format that is easy for the user to understand.

[1516] Step 4:

[1517] Selection of raters

[1518] The user selects the evaluator from whom they wish to receive feedback from the list of evaluators displayed on the system.

[1519] Input: Rater list and user selection of rater

[1520] Output: Selected raters

[1521] Specifically, the evaluator list includes the evaluator's fields of expertise and profiles, and the user makes a selection based on these.

[1522] Step 5:

[1523] Requesting Feedback

[1524] The server sends the uploaded images and bias detection results to the selected evaluators and requests their feedback.

[1525] Input: Selected evaluators, image files, bias detection results

[1526] Output: Feedback request sent to evaluator

[1527] Specifically, the server sends emails and notifications to the evaluators, who can then view the images and detection results on their own devices.

[1528] Step 6:

[1529] Gathering feedback

[1530] Evaluators access the feedback form on their own devices and provide feedback based on the image files and detection results.

[1531] Input: Image files, bias detection results, evaluator feedback

[1532] Output: Feedback sent to the server

[1533] Specifically, the evaluator enters specific comments and evaluations into the feedback form and presses the submit button. The feedback is then saved on the server.

[1534] Step 7:

[1535] Providing feedback and fine-tuning images

[1536] The server consolidates the collected feedback and provides it to the user, and if necessary, translates the feedback into multiple languages ​​using a translation engine.

[1537] Input: Collected feedback

[1538] Output: User-viewable feedback report

[1539] Based on the feedback, the user can fine-tune the image using a generative AI model or image editing software, and then repeat the process again to ultimately generate a high-quality image with minimal bias.

[1540] Input: Feedback report, user image tweaks

[1541] Output: Modified image file

[1542] Through these steps, the system of the present invention effectively detects and corrects biases and errors in the image generation process, and supports the creation of artworks based on diverse perspectives.

[1543] (Application example 1)

[1544] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1545] This invention relates to a system for producing fair and diverse artworks and advertising images by automatically detecting biases and errors contained in image files created using generative AI and collecting feedback from diverse perspectives. Conventional methods require significant time and effort to manually detect biases contained in generated images, making it difficult to incorporate diverse perspectives. Furthermore, there was a lack of a method for efficiently collecting feedback from evaluators and presenting it to users in multiple languages. This made it difficult to detect and correct biases in image production using generative AI, resulting in the risk of inappropriate images being published.

[1546] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1547] In this invention, the server includes a means for uploading generated image files, a means for detecting bias and errors from the uploaded image files, and a means for selecting evaluators with different backgrounds. This allows users to quickly detect bias and errors in uploaded images, effectively collect feedback from evaluators with diverse backgrounds, and present the results in multiple languages. The server also includes a means for users to adjust images based on the feedback, ultimately producing high-quality images that reflect fair and diverse perspectives.

[1548] "Generated Image File" refers to visual data created using generative AI.

[1549] "Bias and Errors" refers to inappropriate biases and mistakes contained in the generated image files.

[1550] "Evaluator" refers to a third party individual or entity with a different background or expertise whose role is to evaluate and provide feedback on images.

[1551] "Feedback" refers to information such as specific comments, evaluations, and suggested corrections obtained from the evaluator.

[1552] "User" refers to the entity that uploads the generated image files and receives bias and error detection and feedback.

[1553] "Server" refers to the computer system that uploads images, detects bias and errors, solicits feedback from raters, and aggregates and presents the feedback.

[1554] A "trained model" refers to an algorithm or neural network that has been pre-trained using large amounts of data.

[1555] "Machine translation" refers to technology that mechanically translates text and feedback between different languages.

[1556] "Means for adjusting images" refers to methods or tools that allow users to modify or regenerate generative AI models or images based on user feedback.

[1557] The system of the present invention allows users to upload generated image files, automatically detect bias and errors, and collect and present feedback from evaluators in order to create fair and diverse artworks and advertising images.

[1558] Hardware and software used:

[1559] Hardware:

[1560] Server: Uploads images, detects bias and errors, requests feedback from raters, and aggregates and presents the feedback.

[1561] Client device: A smartphone is the primary device used to upload images and view feedback.

[1562] software:

[1563] Flask: Used as a Python web framework to process requests on the server.

[1564] JSON: Used to exchange data.

[1565] Trained models: Use pre-trained algorithms or neural networks to detect bias and errors.

[1566] Image storage library: Used to store uploaded images on the server.

[1567] Specific system processing description:

[1568] The user uploads the generated image file from their smartphone to the server, which receives the image using Flask and stores it in local storage.The server then uses the trained model to detect biases and errors in the uploaded image file.

[1569] Based on the detected biases and errors, the server presents a list of evaluators to the user. The evaluators are third parties with different backgrounds and expertise, and the user can review the evaluator profiles and select the evaluators they want to receive the necessary feedback from.

[1570] The server sends the images and bias detection results to the selected evaluators. The evaluators then review the images and input their feedback using their own devices. Once the feedback is collected, the server aggregates it and presents it to the user in multiple languages.

[1571] Based on the feedback provided, users can adjust the images and generative AI models, and then repeat the process to create high-quality images with less bias.

[1572] Examples:

[1573] For example, a designer at an advertising agency uses generative AI to create an image for an advertisement and upload it to the system. The server detects that the image contains racial stereotypes. The designer selects an expert on racial diversity from the evaluators presented on the system and asks for their feedback. The server then presents the evaluator's feedback to the designer, who then adjusts the generative AI model based on the feedback and recreates the advertisement image with fewer racial stereotypes.

[1574] Example prompts to input to a generative AI model:

[1575] "Generate a scene of an adult woman working in an office. The image should reflect a modern, diverse work environment."

[1576] Images generated based on this prompt can then be uploaded to the system, where bias detection and feedback collection can be performed to produce fair and diverse images.

[1577] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1578] Step 1:

[1579] Upload user-generated images from your smartphone to the server.

[1580] Input: Generated image files on your smartphone.

[1581] Processing: Use the Flask framework to save the image received by the server to local storage.

[1582] Output: An image file saved on the server (e.g. uploaded_image.jpg).

[1583] Step 2:

[1584] The server detects bias and errors in the uploaded images.

[1585] Input: Image files stored on the server.

[1586] Processing: Using a trained model (a pre-trained neural network) to analyze the image for biases and errors.

[1587] Output: Bias or error detection results (e.g., "Gender bias detected").

[1588] Step 3:

[1589] The server presents the results of the bias and error detection to the user and allows them to select an evaluator.

[1590] Input: Bias and error detection results, evaluator list.

[1591] Processing: The detection results and the list of evaluators are displayed on the user's smartphone, and the user is prompted to select an evaluator.

[1592] Output: Information about the rater selected by the user.

[1593] Step 4:

[1594] The server sends the images and detection results to the selected evaluators and requests their feedback.

[1595] Input: User-selected evaluator information, image files stored on the server, and bias and error detection results.

[1596] Processing: Send the images and detection results to the selected evaluators and issue a request for feedback.

[1597] Output: The request sent to the evaluator.

[1598] Step 5:

[1599] The evaluator enters the feedback and sends it to the server.

[1600] Input: Rater feedback (specific comments and ratings)

[1601] Processing: Feedback data is sent to the server via the evaluator's terminal.

[1602] Output: Feedback received by the server.

[1603] Step 6:

[1604] The server aggregates the feedback and presents it to the user in multiple languages.

[1605] Input: Multiple pieces of feedback received from raters.

[1606] Processing: Feedback is collected, automatically translated if necessary, and displayed on the user's smartphone in multiple languages.

[1607] Output: A multilingual translated feedback list.

[1608] Step 7:

[1609] Users adjust the images and generative AI models based on feedback.

[1610] Input: Feedback received by the user.

[1611] Processing: Use the feedback to adjust the parameters of the generative AI model, enter new prompts to regenerate the image, or adjust the image directly using image editing tools.

[1612] Output: Adjusted or regenerated image files.

[1613] Following these specific processing steps, the system quickly detects bias and errors in user-generated image files and enables the system to recreate fair and diverse images based on feedback from evaluators.

[1614] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1615] This invention relates to a system that uploads generated image files, automatically detects bias and errors, collects and presents feedback from evaluators, and provides feedback that takes into account the user's emotional state by combining it with an emotion engine. This system allows users to create artwork that incorporates fair and diverse perspectives, and receives appropriate feedback according to the user's emotional state.

[1616] Program processing

[1617] 1. User uploads an image

[1618] Subject: User

[1619] Users prepare images generated using generative AI on their devices and upload the image files to the platform, where they are sent to the server and stored.

[1620] 2. The server detects bias and errors

[1621] Subject: Server

[1622] The server receives the stored image files and uses a bias detection engine to automatically detect biases and errors, generating a report containing the type and location of identified biases and storing it in a database.

[1623] 3. The user selects the evaluator

[1624] Subject: User

[1625] The user reviews the bias detection results and selects the evaluator from whom they would like to receive feedback from a list of evaluators provided on the platform.

[1626] 4. The server collects feedback

[1627] Subject: Server

[1628] The server sends the uploaded image files and bias detection results to the selected evaluators and asks for their feedback.

[1629] 5. Evaluators provide feedback

[1630] Subject: Terminal (evaluator)

[1631] The evaluator uses a terminal to check the submitted images and bias information, enters comments, specific indications of bias, and suggestions for correction into the feedback form, and sends it to the server.

[1632] 6. The server receives and translates the feedback

[1633] Subject: Server

[1634] The server receives feedback data returned by the evaluators, aggregates and stores it in a database, and automatically translates the feedback data if necessary.

[1635] 7. The server uses an emotion engine to recognize the user's emotions.

[1636] Subject: Server

[1637] The server uses an emotion engine to analyze facial and voice data collected from the user's device to recognize the user's current emotional state, which is then stored in a database.

[1638] 8. The server provides feedback

[1639] Subject: Server

[1640] The server optimizes the display method of the feedback content based on the recognized emotional state, and the aggregated feedback is presented to the user in a form that corresponds to the user's emotional state.

[1641] 9. Tweak the image based on user feedback

[1642] Subject: User

[1643] Users can fine-tune the image and generative AI model based on the feedback they provide, and then upload the regenerated image using the same process, seeking further feedback if needed.

[1644] Specific examples

[1645] Example 1: Evaluating and correcting artist-generated illustrations

[1646] Artist X creates an illustration using generative AI and uploads it to the platform, where the server detects that the illustration contains gender bias.

[1647] Artist X selects evaluator Y, an expert on gender perspectives. The server sends the image and bias information to evaluator Y and asks for feedback.

[1648] Evaluator Y uses a device to check the image and send feedback including specific suggestions for correction. The server receives and translates the feedback and presents it to artist X.

[1649] Before presenting the illustration, the server uses an emotion engine to investigate Artist X's emotional state and select an appropriate feedback display method. Based on the feedback, Artist X adjusts the generative AI model and recreates the illustration with less gender bias.

[1650] Example 2: Advertising agency bias correction of advertising images

[1651] A designer at an advertising agency uses generative AI to create and upload images for advertisements, and the server detects if the images contain racial stereotypes.

[1652] The designer selects Evaluator Z, an expert on racial diversity, and asks for feedback. The server sends the image and the detection results to Evaluator Z.

[1653] The server receives feedback from evaluator Z, aggregates and translates it, and presents it to the designer. Before presenting it, the server uses an emotion engine to analyze the designer's emotional state and select the optimal display method.

[1654] Designers use the feedback to adjust the generative AI model to recreate advertising images with fewer racial stereotypes.

[1655] This is the specific processing flow, which effectively corrects biases and errors in the generated images while taking into account the user's emotional state, allowing for the creation of artwork that reflects fair and diverse perspectives.

[1656] The processing flow will be explained below.

[1657] Step 1:

[1658] Users prepare the image files generated by the generative AI on their devices, then drag and drop the image files into the platform's upload interface and click the upload button to send them to the server.

[1659] Step 2:

[1660] The server receives the uploaded image file, saves it in storage, and records the path and metadata of the saved file in the database.

[1661] Step 3:

[1662] The server then sends the saved image files to a bias detection engine, which uses pre-trained models to automatically detect biases and errors in the images and generates a report containing the type and location of any biases identified.

[1663] Step 4:

[1664] The server saves the bias detection result report in a database and notifies the user, including a summary of the bias and instructions for next steps.

[1665] Step 5:

[1666] Users open the platform on their device, check the bias detection results, and then select the evaluator they wish to provide feedback to from the list of evaluators provided by the system. The list displays each evaluator's area of ​​expertise and profile.

[1667] Step 6:

[1668] The server sends the uploaded image file and bias detection results to the evaluator selected by the user and requests feedback.

[1669] Step 7:

[1670] The evaluator uses their own device to check the submitted images and bias information, enters comments, specific indications of bias, and suggestions for correction in the feedback form, and sends the feedback to the server.

[1671] Step 8:

[1672] The server receives feedback data from evaluators, aggregates it in a database, and stores it. If necessary, it automatically translates the feedback data and makes it understandable to users.

[1673] Step 9:

[1674] The server uses an emotion engine to recognize the user's emotions. It analyzes facial expressions and voice data while the user is using the platform and evaluates the user's current emotional state. The evaluation data is stored in a database.

[1675] Step 10:

[1676] The server determines how to display the feedback based on the collected feedback data and the user's emotional state. For example, if the user is feeling stressed, the server may display the feedback in a softer tone.

[1677] Step 11:

[1678] The server presents the optimized feedback to the user, who can review it on their device and is also provided with a detailed view function to review the feedback again if necessary.

[1679] Step 12:

[1680] Users can use the feedback provided to fine-tune the generative AI model and the image itself, then re-upload the generated image using the same process and ask for further feedback if necessary.

[1681] These are the specific processing steps of the system that combines the emotion engine. This series of processes makes it possible to effectively create artwork that reflects fair and diverse viewpoints while taking into account the user's emotional state.

[1682] Example 2

[1683] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1684] In recent years, image generation using generative AI models has become popular, but the generated images can contain biases and errors, such as gender and race bias. Therefore, there is a need to evaluate and correct these images from a fair and diverse perspective. However, existing systems lack the means to collect feedback from evaluators and present that feedback appropriately to users. Furthermore, the lack of feedback based on the user's emotional state poses a challenge in improving the user experience.

[1685] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for uploading generated image files, a means for detecting bias and errors from the uploaded image files, a means for selecting evaluators with different backgrounds, a means for obtaining feedback from the selected evaluators, a means for automatically translating the feedback obtained from the evaluators, a means for recognizing the emotional state of the user, and a means for presenting the obtained feedback in a form corresponding to the emotional state of the user. This makes it possible to realize fair and diverse evaluations of images containing bias and errors and to provide appropriate feedback corresponding to the emotional state of the user.

[1686] "Image File" refers to visual data generated using a generative AI model.

[1687] "Bias" refers to errors that involve particular perspectives or prejudices, such as those related to gender, race, or religion.

[1688] "Errors" refer to factually incorrect or inappropriate elements in the content of an image.

[1689] "Evaluators" refer to people or entities with different backgrounds and expertise who are tasked with evaluating and providing feedback on the content of images.

[1690] "Feedback" refers to information such as comments, suggested corrections, and findings provided by the evaluator.

[1691] "Machine translation" refers to the process of mechanically converting the content of feedback obtained from evaluators into another language.

[1692] "Emotional state" refers to the user's current mental and emotional state.

[1693] "Generative AI model" refers to an artificial intelligence algorithm that creates a generated visual representation based on an input prompt.

[1694] A "prompt" refers to a text instruction or explanation input to a generative AI model.

[1695] The present invention relates to a system that uploads generated image files, automatically detects bias and errors, and collects and presents feedback from evaluators. The system includes, among other things, a means of providing feedback that takes into account the user's emotional state.

[1696] The overall flow of the system is as follows: First, the user generates an image using a generative AI model (e.g., DALL-E or Midjourney). The generated image is saved on the user's device. Next, the user logs in to the platform using a web browser and uploads the image file. The uploaded image file is sent to the server and saved.

[1697] The server analyzes the saved image files using a bias detection engine (e.g., IBM Watson Visual Recognition) to automatically detect bias and errors. A report is generated as a result of the detection and stored in a database. The user reviews this report and selects the evaluator they wish to receive feedback from from a list of evaluators provided on the platform.

[1698] The server sends the image files and bias detection results to the selected evaluators. The evaluators enter comments and suggestions for corrections in the feedback form and send it to the server. The server receives the feedback from the evaluators, translates it as needed using an automatic translation engine (e.g., Google Translate API), and aggregates it in a database.

[1699] The server then uses an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) to analyze the user's emotional state using facial expression and voice data collected from the user's device, and the information is stored in a database.

[1700] Finally, the server optimizes the feedback display method based on the user's emotional state and presents the feedback to the user.The user can then use the provided feedback to modify the generated images and prompts and input them back into the generative AI model to regenerate images with less bias and errors.

[1701] Specific examples

[1702] Example 1: Evaluating and correcting artist-generated illustrations

[1703] 1. The user (artist) uses DALL-E to input the following prompt sentence and generate an illustration.

[1704] Prompt: Draw a gender-neutral fantasy character.

[1705] 2. The user uploads the generated image to the platform.

[1706] 3. The server uses a bias detection engine to detect gender bias.

[1707] 4. The user selects a gender perspective expert (evaluator) and requests feedback.

[1708] 5. The evaluator fills out the feedback form with specific suggestions for correction and submits it.

[1709] 6. The server receives and translates the feedback and presents it to the user based on their emotional state.

[1710] 7. The user modifies the prompt based on the feedback and regenerates a new image.

[1711] Example 2: Advertising agency bias correction of advertising images

[1712] 1. A designer at an advertising agency uses Midjourney to enter the following prompt text and create an image for an advertisement.

[1713] Prompt: Draw a scene of a diverse business meeting.

[1714] 2. The designer uploads the generated image to the platform.

[1715] 3. The server detects that the image contains racial stereotypes.

[1716] 4. The designer selects racial diversity experts (evaluators) and solicits their feedback.

[1717] 5. Receive feedback from evaluators, automatically translate it, and present it to the designer.

[1718] 6. The designer adjusts the generative AI model based on the presented feedback and emotional state, and recreates a new advertising image.

[1719] The above is a specific embodiment of the invention, which allows users to create artwork or advertising materials that incorporate unbiased and diverse perspectives, and effectively correct biases and errors based on appropriate feedback.

[1720] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1721] Step 1:

[1722] The user generates an image using a generative AI model.

[1723] Users input a prompt (e.g., "Draw a gender-neutral fantasy character") into their device and run a generative AI model (e.g., DALL-E or Midjourney). The generated image file is saved locally on the device.

[1724] Input: prompt statement

[1725] Output: Generated image file

[1726] Step 2:

[1727] A user uploads an image to the platform.

[1728] The user opens a web browser and logs in to the platform. After logging in, they use the image upload function to select an image file and click the upload button. The image file is sent to the server and saved in a specified folder.

[1729] Input: Generated image file

[1730] Output: Image file saved on the server

[1731] Step 3:

[1732] The server detects bias and errors.

[1733] The server invokes a bias detection engine (e.g., IBM Watson Visual Recognition) to analyze the uploaded image file. If bias or errors are detected, a report is generated identifying the type and location of the errors and stored in a database.

[1734] Input: Saved image file

[1735] Output: Bias detection report

[1736] Step 4:

[1737] The user selects the evaluator.

[1738] The user reviews the bias detection report, selects the evaluator they wish to receive feedback from from the list of evaluators provided on the platform, and clicks a button to confirm their selection.

[1739] Input: Bias detection report, evaluator list

[1740] Output: Selected raters

[1741] Step 5:

[1742] The server collects the feedback.

[1743] The server will send the uploaded image files and bias detection results to the selected raters, who will be notified and receive a request for feedback from their account.

[1744] Input: Selected evaluators, image files, bias detection results

[1745] Output: Feedback request notification

[1746] Step 6:

[1747] The evaluator provides feedback.

[1748] The evaluator receives the notification and logs in to the platform. They check the image to be evaluated and the bias information, and enter comments and suggestions for corrections in the feedback form. Once they have completed the input, they click the submit button to send the feedback to the server.

[1749] Input: Feedback request notification, bias information

[1750] Output: Feedback data

[1751] Step 7:

[1752] The server receives and translates the feedback.

[1753] The server receives feedback data sent by evaluators and saves it in a designated folder. It then aggregates the feedback data in a database and translates the feedback content using an automatic translation engine (e.g., Google Translate API) if necessary. The translated data is then saved in the database.

[1754] Input: Feedback data

[1755] Output: Translated feedback data

[1756] Step 8:

[1757] The server uses an emotion engine to recognize the user's emotion.

[1758] The server calls an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) to analyze facial and voice data collected from the user's device. As a result of the analysis, the user's current emotional state (e.g., joy, surprise, sadness) is identified and stored in a database.

[1759] Input: facial expression data, voice data

[1760] Output: User's emotional state

[1761] Step 9:

[1762] The server provides feedback.

[1763] The server optimizes the feedback display method based on the user's emotional state, and the feedback can be presented in the form of a pop-up, notification bar, voice message, etc. The feedback will be displayed when the user logs in to the platform and accesses the dashboard.

[1764] Input: User's emotional state, translated feedback data

[1765] Output: Feedback presented to the user

[1766] Step 10:

[1767] The user fine-tunes the image based on the feedback.

[1768] The user reviews the provided feedback and modifies the generated image and prompt text. For example, if gender bias is identified, the prompt text can be modified to explicitly state "gender-neutral." The modified prompt is then input into the generative AI model to generate a new image. The generated image can then be uploaded back to the platform, and additional feedback can be sought if necessary.

[1769] Input: Feedback

[1770] Output: Modified image file

[1771] The above are the specific processing steps.

[1772] (Application example 2)

[1773] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1774] Existing image generation systems face the challenge of effectively detecting and correcting biases and errors in generated image files. Furthermore, they face the challenge of not being able to provide appropriate feedback that takes into account the user's emotional state. This makes it difficult to create images that incorporate fair and diverse perspectives, and to create advertising images with minimal user bias.

[1775] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1776] In this invention, the server includes means for uploading generated image files, means for detecting bias or errors from the uploaded image files, means for selecting evaluators with different backgrounds, means for obtaining feedback from the selected evaluators, means for presenting the obtained feedback to the user, means for recognizing the emotional state of the user, and means for optimizing the display format of the feedback content based on the recognized emotional state. This makes it possible to present optimal feedback according to the emotional state of the user, thereby realizing image production that is fair and incorporates diverse perspectives.

[1777] "Image file" refers to a digitally generated still image that contains visual information about a subject or scene.

[1778] "Uploading" is the process of transferring or sending a digital file from a user's device to a server via the Internet.

[1779] "Bias" refers to the phenomenon in which certain prejudices or stereotypes influence data or systems, and is a factor that can lead to unfair results.

[1780] "Error" refers to an unintended inaccuracy or inconsistency in data or results.

[1781] "Evaluator" means a person who has specialized knowledge and experience in the uploaded image and provides evaluation and feedback on the image.

[1782] "Feedback" refers to information such as comments, suggestions, and improvements provided by the evaluator about an image.

[1783] "User" refers to the person who uses the system to generate and upload images and receive feedback.

[1784] "Emotional state" refers to the user's current psychological and emotional state, and is analyzed from data such as facial expressions and voice.

[1785] "Optimization" refers to the process of achieving the most effective state or arrangement for a specific purpose or condition.

[1786] "Display method" refers to the format or method for visually presenting information.

[1787] "Database" refers to a system for systematically storing and managing specific data.

[1788] A "server" is a computer system that provides services and data to clients over a network.

[1789] This invention begins with a user uploading an image file they have generated to the system using a device such as a smartphone or computer. The user generates an image using a generative AI model and uploads it to the system. The uploaded image file is then stored on a server.

[1790] The server automatically detects biases and errors in the stored image files using pre-trained models. For example, if the image contains gender bias or racial stereotypes, it will detect them and generate a report containing the type and location of identified bias. This report is stored in a database.

[1791] Next, users select their desired evaluators from a list of evaluators provided on the platform. Evaluators are experts with different backgrounds who can provide unbiased and diverse perspectives on some aspects. The server then sends the uploaded image files and bias detection results to the selected evaluators and requests their feedback.

[1792] Evaluators use their devices to check the images and bias information, and then enter comments, specific suggestions, and corrections in the feedback form. The entered feedback is sent to the server, which receives and translates it, then aggregates and stores it in a database.

[1793] Furthermore, the server uses an emotion engine to analyze facial expression and voice data collected from the user's device to recognize the user's current emotional state, which is then stored in a database and used to optimize the way feedback is displayed.

[1794] Finally, the server presents the aggregated feedback according to the user's emotional state (for example, a positive tone if the user is happy, or a comforting tone if the user is sad).The user can use this feedback to fine-tune the generative AI model and the image, then upload the image again for further feedback.

[1795] For example, an advertising designer uploads an advertising image created using a generative AI model, and the server detects racial stereotypes in the image. The designer selects evaluators knowledgeable about diversity and receives feedback. The server checks the designer's emotional state and selects an appropriate feedback display format. The designer adjusts the generative AI model based on this feedback.

[1796] Examples of prompt sentences include the following:

[1797] "Please advise on creating advertising images that are gender-bias-free. The current image contains the following biases: {List of biases detected}"

[1798] As described above, this system incorporates a fair and diverse range of viewpoints and provides optimal feedback according to the user's emotional state, enabling high-quality image production.

[1799] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1800] Step 1:

[1801] The user prepares an image file generated using the generative AI model on their device and uploads the image file to the platform. The server receives and stores the uploaded image file.

[1802] Input: Image file (created by the user using a generative AI model)

[1803] Output: Image files are saved to the server

[1804] Step 2:

[1805] The server detects bias and errors in the stored image files using pre-trained models. The bias detection model identifies prejudices and stereotypes in the images and generates a report based on this. The report includes the type and location of bias detected.

[1806] Input: Saved image file

[1807] Output: Bias detection report (including type and location of bias)

[1808] Step 3:

[1809] Users select the evaluators they wish to receive feedback from from a list of evaluators provided on the platform. Users choose evaluators with different backgrounds, with the aim of obtaining fair and diverse perspectives.

[1810] Input: Rater list, user selection

[1811] Output: Selected raters

[1812] Step 4:

[1813] The server then sends the uploaded image files and bias detection results to the selected evaluators and requests their feedback, preparing the server to receive specific feedback and suggestions for corrections from the evaluators.

[1814] Input: Image file, bias detection report, selected rater

[1815] Output: Feedback request to evaluator

[1816] Step 5:

[1817] The evaluator uses the device to review the submitted images and bias information, and enters comments, specific suggestions, and correction suggestions in the feedback form. The evaluator's feedback is then sent from the device to the server.

[1818] Input: Image file, bias detection report

[1819] Output: Rater feedback

[1820] Step 6:

[1821] The server receives feedback data returned by evaluators, aggregates and stores it in a database, and automatically translates the feedback data as needed, providing users with feedback in a format that is easy to understand.

[1822] Input: Rater feedback

[1823] Output: translated feedback, aggregated feedback data

[1824] Step 7:

[1825] The server uses an emotion engine to analyze facial and voice data collected from the user's device and recognize the user's current emotional state. The emotion engine is implemented using, for example, the DeepFace library.

[1826] Input: User's facial expressions and voice data

[1827] Output: User's emotional state

[1828] Step 8:

[1829] The server optimizes the display of feedback content based on the user's emotional state. The aggregated feedback is presented in a way that reflects the user's emotional state. For example, positive feedback is displayed prominently.

[1830] Input: Emotional state, translated feedback

[1831] Output: Optimized feedback display

[1832] Step 9:

[1833] Based on the feedback provided by the user, the image and generative AI model are fine-tuned, and the corrected image is uploaded again and can receive feedback again through the same process.

[1834] Input: Optimized feedback display

[1835] Output: Fine-tuned image

[1836] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1837] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1838] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1839] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1840] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1841] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1842] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1843] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1844] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1845] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1846] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1847] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1848] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1849] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1850] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1851] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1852] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1853] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1854] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1855] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1856] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1857] The following is further disclosed regarding the above embodiment.

[1858] (Claim 1)

[1859] a means for uploading the generated image file;

[1860] A means of detecting bias or errors in uploaded image files;

[1861] a means of selecting raters with different backgrounds;

[1862] a means for obtaining feedback from the selected raters;

[1863] means for presenting the obtained feedback to the user;

[1864] A system including:

[1865] (Claim 2)

[1866] 10. The system of claim 1, wherein the means for detecting bias or error in uploaded image files uses a pre-trained model to detect bias or error.

[1867] (Claim 3)

[1868] 10. The system of claim 1, wherein the means for obtaining feedback includes means for automatically translating feedback from the evaluators.

[1869] "Example 1"

[1870] (Claim 1)

[1871] a means for uploading the generated image file;

[1872] A means of detecting bias or errors in uploaded image files;

[1873] A means for presenting the results of bias and error detection to the user and selecting evaluators with different backgrounds;

[1874] a means of collecting feedback from the selected raters;

[1875] a means for providing collected feedback to the user to assist in fine-tuning the generated image;

[1876] A system including:

[1877] (Claim 2)

[1878] 2. The system of claim 1, wherein the means for detecting bias or errors from uploaded image files uses a pre-trained model to detect bias or errors.

[1879] (Claim 3)

[1880] 2. The system according to claim 1, wherein the means for collecting feedback includes means for automatically translating feedback from evaluators and providing the translated feedback to the user.

[1881] "Application Example 1"

[1882] (Claim 1)

[1883] a means for uploading the generated image file;

[1884] A means of detecting bias or errors in uploaded image files;

[1885] a means of selecting raters with different backgrounds;

[1886] a means for obtaining feedback from the selected raters;

[1887] means for presenting the obtained feedback to the user;

[1888] a means for the user to adjust the image based on the feedback;

[1889] A system including:

[1890] (Claim 2)

[1891] 10. The system of claim 1, wherein the means for detecting bias or error in uploaded image files uses a pre-trained model to detect bias or error.

[1892] (Claim 3)

[1893] 10. The system of claim 1, wherein the means for obtaining feedback includes means for automatically translating feedback from the evaluators, and the collected feedback is presented to the user in multiple languages.

[1894] "Example 2: Combining Emotion Engines"

[1895] (Claim 1)

[1896] a means for uploading the generated image file;

[1897] A means of detecting bias or errors in uploaded image files;

[1898] a means of selecting raters with different backgrounds;

[1899] a means for obtaining feedback from the selected raters;

[1900] a means for automatically translating feedback obtained from evaluators;

[1901] means for recognizing the emotional state of a user;

[1902] means for presenting the obtained feedback in a form corresponding to the emotional state of the user;

[1903] A system including:

[1904] (Claim 2)

[1905] 10. The system of claim 1, wherein the means for detecting bias or error in uploaded image files uses a pre-trained model to detect bias or error.

[1906] (Claim 3)

[1907] 10. The system of claim 1, wherein the means for recognizing the user's emotional state analyzes facial expression and voice data collected from the user's terminal.

[1908] "Application example 2 when combining emotion engines"

[1909] (Claim 1)

[1910] a means for uploading the generated image file;

[1911] A means of detecting bias or errors in uploaded image files;

[1912] a means of selecting raters with different backgrounds;

[1913] a means for obtaining feedback from the selected raters;

[1914] means for presenting the obtained feedback to the user;

[1915] means for recognizing the emotional state of a user;

[1916] means for optimizing the display manner of the feedback content based on the recognized emotional state;

[1917] A system including:

[1918] (Claim 2)

[1919] 10. The system of claim 1, wherein the means for detecting bias or error in uploaded image files uses a pre-trained model to detect bias or error.

[1920] (Claim 3)

[1921] 10. The system of claim 1, wherein the means for obtaining feedback includes means for automatically translating feedback from the evaluators. [Explanation of symbols]

[1922] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for uploading the generated image file; A means of detecting bias or errors in uploaded image files; a means of selecting raters with different backgrounds; a means for obtaining feedback from the selected raters; means for presenting the obtained feedback to the user; A system including:

2. 10. The system of claim 1, wherein the means for detecting bias or error in uploaded image files uses a pre-trained model to detect bias or error.

3. The system of claim 1 , wherein the means for obtaining feedback includes means for automatically translating the feedback from the evaluator.

Images