System

Abstract

A system is provided.SOLUTION: The system includes a means for receiving an existing material, a means for analyzing the received material and grasping the emphasis of colors, a means for learning the combination of colors which are difficult for a colorblind person to view, a means for learning colors which are easy for the colorblind person to view, a means for changing the material to color arrangement friendly to the colorblind person on the basis of a learning result, and a means for generating and transmitting the changed material.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional document creation, red generally indicates negative elements and blue indicates positive elements, but these colors are difficult for color-blind people to see, making it difficult for them to accurately grasp the information. Color-blind people often have particular difficulty distinguishing between red and green, making it difficult for them to understand the documents. To solve this problem, a tool is needed that can automatically change the color scheme of existing documents to make them more visible to color-blind people. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for receiving and analyzing existing materials to determine color emphasis, a means for learning color combinations that are difficult and easy for color-blind people to see, and a means for modifying the materials to be color-blind-friendly based on the learning results. Specifically, the system also includes a means for referencing a database related to color blindness to acquire information on color combinations that are difficult and easy to see, and a means for receiving a material generation request from a user, generating a modified material file, and sending the modified file to the user's terminal. This system automatically generates barrier-free materials that allow color-blind people to accurately understand the information.

[0006] "Existing materials" refers to material files and documents that the user has already created.

[0007] "Means for receiving" refers to a mechanism for executing a process in which the server receives the document file sent by the user from the terminal.

[0008] "Means of analysis" refers to the mechanism by which the system analyzes the contents of the received document and determines which color represents which information.

[0009] "Color highlighting" refers to colors used to highlight specific information within a document.

[0010] "Means of learning" refers to the mechanism by which the generative AI goes through a process to understand how colors are used, and which color combinations are difficult and easy for color-blind people to see.

[0011] "Color blind" refers to people who see colors differently from people with normal color vision.

[0012] "Color-blind friendly" refers to a state in which colors are selected and arranged with consideration for people with color-blindness, making them easy to see and understand.

[0013] The "means of change" refers to a system that automatically converts the color scheme of existing materials into one that is easy to understand for people with color blindness.

[0014] The "means for generating" refers to a mechanism that executes a process for creating a new document file with a modified color scheme.

[0015] The "means for transmitting" refers to a mechanism for executing the process of transmitting the generated color-blind-friendly material file to the user's terminal.

[0016] A "color blindness database" is a database that stores information about color combinations that are difficult or easy for color blind people to see.

[0017] "Color combination information" refers to information about whether a combination of specific colors makes it difficult or easy for color-blind people to see.

[0018] "User Device" means an electronic device, such as a computer or smartphone, used to upload or download materials. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The system of this invention includes three main elements: a server, a terminal, and a user, each of which plays a specific role to realize the function of automatically generating materials that are considerate of color blindness.

[0041] First, a user uploads existing materials to the server using a terminal, and the terminal provides an interface for sending selected material files to the server.

[0042] The server then receives the uploaded file and temporarily stores it. The server then passes the file to the AI ​​generator, which analyzes the content of the file to determine the color emphasis. This analysis process determines what information each color represents.

[0043] The server then references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. The server then passes this information to the generation AI, which then learns which color combinations are difficult and easy for color-blind people to see. Specifically, the server obtains information from the database that indicates that the combination of red and green is difficult to see, and that dark orange and navy blue are easy to see.

[0044] After the learning is complete, the user requests the generation of color-blind-friendly materials from their device. When the server receives this request, it instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. Based on the learning results, the generation AI appropriately converts the colors of the materials and generates materials with a new color scheme.

[0045] The server saves the new color file and sends it to the user's device. The device receives the new file and allows the user to download it. This provides materials that can be accurately understood by color-blind people.

[0046] As a concrete example, let's take a sales report. When a user uploads a sales report from their device to the server, the server passes the report to a generation AI, which analyzes the negative aspects indicated in red and the positive aspects indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. As a result, the converted document is sent to the user's device, and the user can download a color-blind-friendly sales report.

[0047] The processing flow will be explained below.

[0048] Step 1:

[0049] The user selects a file on the device and clicks the upload button. The device sends the selected file to the server.

[0050] Step 2:

[0051] The server receives the uploaded file and temporarily stores it.

[0052] Step 3:

[0053] The server passes the saved document file to the generation AI, which analyzes the content of the document and determines the color emphasis. Specifically, it analyzes whether red indicates negative elements and blue indicates positive elements.

[0054] Step 4:

[0055] The server references a database about color blindness and obtains information about color combinations that are difficult for color-blind people to see. Specifically, it checks whether the combination of red and green is difficult for color-blind people to see.

[0056] Step 5:

[0057] The server passes the acquired color combination information that is difficult for color-blind people to see to the generation AI, which then learns from it.

[0058] Step 6:

[0059] The server references a database of colors that are easy for colorblind people to see and obtains color combinations that are easy to see. Specifically, it checks whether deep orange and navy blue are easy to see.

[0060] Step 7:

[0061] The server passes the acquired information on easy-to-see color combinations to the generation AI, which then learns from it.

[0062] Step 8:

[0063] The user sends a request for generating color-blind friendly materials from the device.

[0064] Step 9:

[0065] The server instructs the AI ​​to change the color scheme of existing documents to be color-blind-friendly. The AI ​​then appropriately converts the colors of the documents based on the learning results. Specifically, it changes red to dark orange and blue to navy blue.

[0066] Step 10:

[0067] The server saves the converted new color scheme data file.

[0068] Step 11:

[0069] The server sends the new file to the user's device, and the user receives the new file and can download it.

[0070] Example 1

[0071] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0072] Conventional document creation methods can make it difficult for color-blind people to accurately understand information. Specifically, in documents where color emphasis is important, color-blind people have difficulty distinguishing between colors, which can lead to communication problems. Furthermore, manually adjusting color schemes is cumbersome and requires specialized knowledge, making it inefficient. To solve these issues, a system that automatically generates documents that take color-blind people into consideration is needed.

[0073] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0074] In this invention, the server includes means for uploading existing materials to the server using a terminal, means for temporarily storing received material files on the server, means for passing the materials stored by the server to a generation AI and analyzing the contents of the materials, means for referencing a database on color deficiency to obtain information on color combinations that are difficult or easy for color-blind people to see, means for instructing the generation AI based on the obtained information to change the color scheme of the materials to colors that are easy for color-blind people to see, means for the generation AI to generate color-converted materials, and means for temporarily storing the changed materials on the server and sending them to the terminal. This makes it possible to efficiently and automatically generate materials with color schemes that are easy for color-blind people to see.

[0075] A "terminal" is an information device that a user uses to upload materials to a server or download generated materials.

[0076] The "server" is a computer system that receives and temporarily stores document files and interacts with the generating AI and database.

[0077] "Document files" are existing documents and reports that users upload to the server.

[0078] "Generative AI" is a program that uses artificial intelligence to analyze the contents of documents and change the color scheme for people with color blindness.

[0079] "Analysis" is the process in which the generating AI reads the contents of the document file and extracts color information.

[0080] "Color accents" refer to colors used to highlight specific information within a document.

[0081] The "color blindness database" is a database that stores information on color combinations that are difficult or easy for color-blind people to see.

[0082] "Color conversion" is the process of changing the colors in existing materials to make them easier to see for color-blind people.

[0083] The "modified document file" refers to the new document generated by the generating AI with the color scheme changed to suit color-blind people.

[0084] "Upload" refers to the act of a user sending a document file to a server via a terminal.

[0085] "Downloading" refers to the act of a user receiving a modified material file from a server via a terminal.

[0086] The system of the present invention includes a terminal, a server, a generative AI model, and a database related to color blindness, and provides a method and means for automatically generating materials that are considerate of color blindness.

[0087] A user uses a terminal to upload existing documents to the server. At this time, the terminal selects the document file and provides an interface for uploading. For example, a user selects a sales report file (e.g., report.pdf) and clicks the "Upload" button.

[0088] The server temporarily stores the document files received from the user. The stored document files are then analyzed by the generative AI model. The generative AI model analyzes the content of the document, identifies color emphasis, and determines what information each color represents.

[0089] Next, the server references a database on color blindness to obtain information on color combinations that are difficult or easy for color-blind people to see. For example, it obtains information from the database that "the combination of red and green is difficult to see" and "dark orange and navy blue are easy to see."

[0090] Based on the acquired information, the server issues instructions to the generative AI model to change the color scheme of the document to make it easier for people with color blindness to see. Based on the results of the analysis and learning, the generative AI model appropriately converts the colors of the document and generates a document with a new color scheme.

[0091] The generated modified document file is temporarily stored on the server and then sent to the user's device. The user can then use the device to receive and download the new document file. For example, the user can download the generated report_processed.pdf file.

[0092] Examples of specific prompts include requests such as "Please convert the red and blue colors in this report to color-blind-friendly colors" or "Please convert this sales report so that the reds and greens are easy to see."

[0093] As described above, this invention provides a system that can efficiently and automatically generate documents with color schemes that are easy to see for people with color blindness. By utilizing a generative AI model and a database on color blindness, it is possible to provide high-quality documents that take color blindness into consideration while saving time and effort.

[0094] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0095] Step 1:

[0096] User uploads materials

[0097] A user uses a terminal to select an existing document file and upload it to the server. As a concrete example, a user selects a sales report file (e.g., report.pdf) and clicks the "Upload" button. The input is the document file selected by the user, and the output is that file being sent to the server.

[0098] Step 2:

[0099] The server receives the data and temporarily stores it

[0100] The server receives the file uploaded by the user and temporarily stores it. The server stores the received file in the / uploads directory, for example. The input is the file sent, and the output is the file stored on the server.

[0101] Step 3:

[0102] The server passes the data to the generated AI for analysis.

[0103] The server passes the saved document file to the generation AI, which then analyzes the contents of the document. The generation AI extracts the colors used in the document and analyzes which colors represent which information. The input is the saved document file, and the output is the analysis results, including the document's color information. As a concrete example, a prompt such as "Analyze report.pdf and extract the color information within the document" is sent to the generation AI.

[0104] Step 4:

[0105] The server consults a database of color blindness information

[0106] The server references a database on color blindness and retrieves information about color combinations that are difficult and easy for color-blind people to see. The input is a query to the database, and the output is information about color combinations that are difficult and easy for color-blind people to see. For example, the server retrieves information from the database that "the combination of red and green is difficult to see" and "dark orange and navy blue are easy to see."

[0107] Step 5:

[0108] The server instructs the AI ​​to convert colors based on the learning results.

[0109] The server instructs the generation AI to convert the color of the material based on information obtained from a database on color blindness. The input is the analysis result and information from the database, and the output is a color conversion instruction. As a concrete example, the prompt statement "Convert the red in the analysis result to dark orange and the blue to navy blue" is sent to the generation AI.

[0110] Step 6:

[0111] Generative AI generates color-converted materials

[0112] The generation AI changes the color of the document based on instructions from the server and generates a document with a new color scheme. The input is the color conversion instruction, and the output is the document after the color change. The generation AI receives the instructions and generates a new document in which the red parts of report.pdf are converted to dark orange and the blue parts to navy blue.

[0113] Step 7:

[0114] The server saves the new material and sends it to the device.

[0115] The server temporarily stores the new material generated by the AI ​​and sends it to the user's device. The server stores the new material in, for example, the / processed directory, and sends a notification and a download link to the user's device. The input is the new material generated, and the output is to store and send the material.

[0116] Step 8:

[0117] User downloads new material

[0118] The user receives and downloads new documents using a device. The user clicks on a notification or link to download report_processed.pdf, which is generated with the new color scheme. The input is the download request from the device, and the output is the downloaded new documents.

[0119] The above processing steps make it possible to efficiently and automatically generate materials with color schemes that are easy to see even for color-blind people.

[0120] (Application example 1)

[0121] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] There is a need for a system that can change the colors of existing documents and video content to be color-blind-friendly so that people with color blindness can accurately understand the information. Analyzing the colors of each frame and converting them into a color scheme that is considerate to people with color blindness is a particularly difficult technical challenge in the case of video content. A system that can solve this problem is needed.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0124] In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and grasping color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for changing the materials to a color-blind-friendly color scheme based on the learning results, means for receiving video content and analyzing the colors of each frame, means for generating and transmitting the changed materials, means for converting the colors of each video frame to a color-blind-friendly color scheme, and means for generating and transmitting the converted video. This makes it possible to provide materials and video content that even color-blind people can accurately understand.

[0125] "Existing materials" refers to documents and data files that were created in the past and are currently available.

[0126] "Color emphasis" refers to the frequency and prominence of a particular color.

[0127] "Color blind" refers to people who have difficulty distinguishing colors.

[0128] A "database" is a collection of data that is systematically organized and stored in a form that is easily accessible and searchable.

[0129] "Video Content" refers to multimedia files containing video and audio.

[0130] A "frame" refers to a single still image in a video.

[0131] "Learning results" refers to the knowledge and patterns that a generative AI model has acquired based on training data.

[0132] A "prompt sentence" refers to an instruction or input sentence to a generative AI model.

[0133] "Receiving" refers to the act of taking in data or information from outside.

[0134] "Analysis" refers to the process of examining and understanding data or information.

[0135] "Modification" refers to the act of changing something that already exists into a different state or form.

[0136] "Downloading" refers to the act of importing data from a remote location, such as a server, into your own device.

[0137] The system for implementing this invention consists of three main components: a server, a terminal, and a user. This system is designed to automatically convert existing materials and video content into color-blind-friendly color schemes.

[0138] First, a user uses a terminal to upload existing materials or video content to a server, and the terminal provides an interface for transmitting selected material files or video content to the server.

[0139] The server receives and temporarily stores the uploaded files and video content. The server then passes the files and video content to the generative AI model, which analyzes the files and each frame to understand color emphasis. This analysis process determines which information or scene each color represents.

[0140] The server then references a database of color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see.The generative AI model then uses this information to learn which color combinations are difficult and easy for color-blind people to see.

[0141] After the learning process is complete, the user can request the generation of color-blind-friendly materials or video conversions from their device. When the server receives this request, it instructs the generative AI model to change the color scheme of each frame of the existing materials or video to be color-blind-friendly. Based on the learning results, the generative AI model appropriately converts the colors of the materials or videos and generates materials or videos with the new color scheme.

[0142] The document files and video content created with the new color scheme are stored on the server and sent to the user's device. The user can then receive and download the new document files and video content. In this way, documents and video content are provided that even color-blind people can accurately understand.

[0143] As a concrete example, let's take the following prompt sentence as input to a generative AI model:

[0144] Analyze the uploaded video and identify the color emphasis of each frame. To accommodate color-blind users, convert red to orange and blue to navy blue. Combine the converted frames to create a new video file.

[0145] The hardware used is a smartphone (iOS or Android) or a high-performance server (e.g., NVIDIA Tesla T4), and the software used is a cloud-based generative AI model (e.g., Google Cloud AI, AWS SageMaker), which makes it possible to provide materials and video content that can be accurately understood even by people with color blindness.

[0146] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0147] Step 1:

[0148] A user uses a terminal to select existing materials or video content and upload it to the server. The terminal provides an interface for sending the selected file to the server. The input is the material file or video content selected by the user, and the output is the file sent to the server.

[0149] Step 2:

[0150] The server receives document files and video content uploaded by users and temporarily stores them. The input here is the uploaded file, and the output is the temporarily stored file. Specifically, the server analyzes the file reception request and stores the file in the appropriate storage.

[0151] Step 3:

[0152] The server passes the document files and video content to the generative AI model for analysis. The generative AI model processes the documents and the color emphasis of each frame. The input is the saved file, and the output is the analysis result including color emphasis information. In this step, the generative AI model receives the file as input and performs data analysis to extract color characteristics.

[0153] Step 4:

[0154] The server references a database about color blindness to obtain information about difficult and easy-to-see color combinations. The input is a database query, and the output is color information relevant to color blind people. In this step, the server executes the database query to obtain the required color information.

[0155] Step 5:

[0156] The generative AI model learns combinations of colors that are difficult and easy for colorblind people to see based on color information obtained from a database. The input is color information obtained from the database, and the output is the learning result. In this step, the generative AI model establishes color conversion rules using a learning algorithm.

[0157] Step 6:

[0158] A user requests the generation of color-blind-friendly materials or video conversion. The device sends the request to the server. The input is the user's request, and the output is the requested data sent to the server.

[0159] Step 7:

[0160] The server receives the user's request and issues instructions to the generative AI model. The generative AI model processes the analyzed materials and each frame of the video to change the color scheme to be color-blind-friendly. The input is instructions to the generative AI model and the analyzed data, and the output is the color-changed materials and video frames. In this step, the generative AI model performs the color conversion process based on the prompt text.

[0161] Step 8:

[0162] The generative AI model generates new materials and videos based on the learning results. The generated materials and videos are stored on the server. The input is the color-changed data, and the output is the newly generated file. In this step, the generative AI model completes the conversion process and creates the product.

[0163] Step 9:

[0164] The server sends new document files or video content to the user's device. The input is the new file that has been saved, and the output is the data to be sent to the user's device. Specifically, the server generates a file transfer request and delivers the file to the user's device.

[0165] Step 10:

[0166] The user receives and downloads new material files or video content on their device. The input is the file sent from the server, and the output is the new material or video stored on the device. In this step, the user confirms the receipt of the file and downloads it to local storage.

[0167] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0168] The system of this invention includes a server, a terminal, a user, and an emotion engine, each of which plays a specific role to realize the function of automatically generating materials that are considerate of color blindness while taking into account the user's emotions.

[0169] First, a user uploads existing materials to the server using a device. The device provides an interface for sending selected material files to the server, and the emotion engine analyzes the user's facial expressions, voice, or text input to recognize the user's emotions.

[0170] The server then receives the uploaded file and temporarily stores it. The server then passes the file to the AI ​​generator, which analyzes the content of the file to determine the color emphasis. This analysis process determines what information each color represents.

[0171] The server then references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. The server then passes this information to the generation AI, which then learns which color combinations are difficult and easy for color-blind people to see. Specifically, the server obtains information from the database that indicates that the combination of red and green is difficult to see, and that dark orange and navy blue are easy to see.

[0172] After the learning is complete, the user requests the generation of color-blind-friendly materials from their device. When the server receives this request, it instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. The emotion engine then works again to reconfirm the user's emotions. If the user is feeling stressed or dissatisfied, the generation AI further adjusts the color scheme to match the color scheme that the user finds most comfortable.

[0173] Based on the learning results and the user's emotional information, the generative AI appropriately converts the colors of the materials and generates materials with new color schemes. Specifically, it changes red to dark orange and blue to navy blue, selecting a color scheme that the user finds most appealing.

[0174] The document file generated with the new color scheme is stored by the server and sent to the user's device. The user can then receive and download the new document file on their device. The emotion engine also has a feedback function that allows the user to input their satisfaction with the generated document, allowing the engine to provide a more appropriate color scheme the next time a document is generated.

[0175] As a concrete example, let's take a sales report. When a user uploads a sales report from their device to the server, the emotion engine analyzes the user's input and facial expressions to determine their current emotional state. The server then passes the report to the generation AI, which analyzes the negative elements indicated in red and the positive elements indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. The converted document is then sent to the user's device, where it is available for download. The user can also provide feedback about the document to the emotion engine, which can then further improve the document generation the next time.

[0176] This will not only enable color-blind people to accurately understand information, but will also create a system that provides barrier-free materials tailored to the user's emotions.

[0177] The processing flow will be explained below.

[0178] Step 1:

[0179] The user selects a file on their device and clicks the upload button. The device then sends the selected file to the server. At this time, the emotion engine captures the user's facial expressions with a webcam and analyzes their voice tone and text input to recognize their emotional state.

[0180] Step 2:

[0181] The server receives the uploaded file and temporarily stores it. At the same time, the emotion engine analyzes the acquired user emotion data and determines the user's current emotional state (e.g., stress, dissatisfaction, satisfaction, etc.).

[0182] Step 3:

[0183] The server passes the saved document file to the generation AI, which analyzes the content of the document and identifies color emphasis. Specifically, it identifies negative elements shown in red and positive elements shown in blue.

[0184] Step 4:

[0185] The server references a database of color blindness information to obtain information about color combinations (e.g., red and green) that are difficult for color-blind people to see. The server passes this information to the generation AI.

[0186] Step 5:

[0187] The server references a database of colors that are easy for colorblind people to see and obtains information about color combinations that are easy to see (e.g., dark orange and navy blue). The server passes this information to the generation AI, which then learns from it.

[0188] Step 6:

[0189] The user sends a request for generating color-blind-friendly documents from their device. The emotion engine again captures the user's facial expressions and voice to confirm their emotional state.

[0190] Step 7:

[0191] The server instructs the AI ​​to change existing materials to a color-blind-friendly color scheme. The AI ​​then converts the materials based on its learning results and the user's latest emotional information. For example, if the user is feeling stressed, it will adjust the color tone by softening it.

[0192] Step 8:

[0193] The AI ​​generates a file containing the new color scheme, specifically changing red to dark orange and blue to navy blue.

[0194] Step 9:

[0195] The server stores the converted data files.

[0196] Step 10:

[0197] The server sends the new material file to the user's terminal, and the terminal receives the new material file and allows the user to download it.

[0198] Step 11:

[0199] The device asks the user for feedback on new materials. The user provides the feedback through the emotion engine. The emotion engine analyzes this feedback and stores the data to be reflected in future material generation.

[0200] This provides color-blind-friendly materials that are optimized for the user's emotions.

[0201] Example 2

[0202] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0203] Existing materials are often difficult for people with color blindness to view, especially when highlighting colors. Furthermore, the readability and comprehension of materials are also affected by the user's emotional state. However, existing systems lack the technology to automatically generate materials that take into account the readability of color blind people and the user's emotional state. Therefore, there is a need for a method of generating materials that provides materials that are easy to view for people with color blindness and that also takes into account the user's emotional state.

[0204] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0205] In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and grasping color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for recognizing and analyzing user emotions, means for changing the materials to a color-blind-friendly color scheme based on the learning results and user emotion information, and means for generating and transmitting the changed materials. This enables the automatic generation of materials that are easy for color-blind people to see and that take user emotions into consideration.

[0206] The "means for receiving existing materials" is a system component that allows the server to receive existing documents or data provided by the user.

[0207] "Means for analyzing received materials and understanding color emphasis" is a function for analyzing the use of colors and the importance of each color in received materials and understanding how they are emphasized in the materials.

[0208] "Means for learning color combinations that are difficult for color-blind people to see" is a function that collects information about color combinations that are difficult for color-blind people to see from a database, etc., and uses that information for learning.

[0209] "Means for learning colors that are easy for color-blind people to see" is a function that collects information from a database, etc. about color combinations that are easy for color-blind people to see, and uses that information for learning.

[0210] "Means for recognizing and analyzing user emotions" refers to a function that uses the device's camera, microphone, and text input interface to analyze the user's facial expressions, voice, and text, and recognize the user's current emotional state.

[0211] "Means for changing the color scheme of documents to be color-blind-friendly based on the learning results and the user's emotional information" is a function for changing the color scheme of documents to be color-blind-friendly based on the acquired learning results and the user's emotional information.

[0212] The "means for generating and transmitting modified materials" is a function for generating new materials with a modified color scheme and transmitting them to the user's terminal.

[0213] A "color blindness database" is an information source that stores information about color combinations that are difficult or easy for color blind people to see.

[0214] "Means for receiving user feedback and reflecting it in subsequent material generation" is a function for receiving evaluations and opinions on the generated materials provided by users and using them to help with subsequent material generation processes.

[0215] The system of this invention includes a server, a terminal, a user, and an emotion engine. Each component plays a specific role and realizes the function of automatically generating materials that are color-blind-friendly, taking into account the user's emotions. The specific operation of each component is described below.

[0216] Server Operation

[0217] The server first receives and temporarily stores existing documents uploaded by users on their devices. After receiving the documents, the server passes them to the AI ​​generator, which analyzes their contents. The AI ​​then identifies highlighted areas and determines what information each color represents. For example, red often indicates a warning, while blue indicates positive information.

[0218] Next, the server references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. This includes information such as the combination of red and green being difficult to see, and that dark orange and navy blue are easy to see. Based on this information, it instructs the generation AI on color-conscious learning.

[0219] Based on the learning results, the generative AI changes the color scheme of the document to be color-blind-friendly. Specifically, it changes red to dark orange and blue to navy blue. This makes the document easier to read for people with color blindness.

[0220] Furthermore, the server receives a request from the user to generate a document and instructs the AI ​​to reconfigure the existing document to have a color-blind-friendly color scheme. The emotion engine then reconfirms the user's emotions, and if the user feels stressed or dissatisfied, the AI ​​further adjusts the color scheme. The generated new document is then saved by the server and sent to the user's device.

[0221] Device behavior

[0222] The device provides an interface for users to upload existing documents. The emotion engine uses the device's camera, microphone, and text input interface to analyze the user's facial expressions, voice, and text to recognize their emotions. Emotional information is also sent to the server at the same time as the documents are uploaded.

[0223] The device receives the new color scheme file sent from the server and allows the user to download it. The emotion engine also has a feedback function that allows the user to input their satisfaction with the generated material.

[0224] User operations

[0225] The user uses the device to select an existing document and begin uploading it. For example, they select a sales report file and send it to the server. At this time, the user inputs facial expressions, voice, and text using the device's camera and microphone, and their emotional state is recognized.

[0226] Once new materials are generated and sent from the server to the user's device, the user can download them, check their content, and enter feedback such as their satisfaction level and requests through the device, which will help improve future material generation.

[0227] Specific examples

[0228] Take a sales report as an example. When a user uploads a sales report from their device to the server, the emotion engine analyzes the user's input and facial expressions to determine their current emotional state. The server then passes the report to the generation AI, which analyzes the negative aspects indicated in red and the positive aspects indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. The converted document is then sent to the user's device, where it is available for download. The user can also provide feedback on the document to the emotion engine, which can then further improve the document generation the next time.

[0229] Prompt Sentence Examples

[0230] "Please change the red parts of the following materials to dark orange and the blue parts to navy blue. Also, please adjust the colors to reflect the user's current emotions."

[0231] By implementing this invention, not only can color-blind people accurately understand information, but also barrier-free materials that match the user's emotions can be provided.

[0232] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0233] Step 1:

[0234] The user uses the terminal to select an existing document and start uploading. The operation is performed by selecting the document file and clicking the upload button in the terminal. The input is the user's existing document file, and the output is the document file sent to the server.

[0235] Step 2:

[0236] The device sends the selected data file to the server. The device uploads the data file, and at the same time the emotion engine analyzes the user's facial expressions, voice, and text using the camera, microphone, and text input interface. The input is the data file and the user's facial expressions, voice, and text, and the output is the data file and emotion data sent to the server.

[0237] Step 3:

[0238] The server receives the uploaded data files and temporarily stores them. The input is the data files and emotion data sent from the device, and the output is the saved data files. This storage makes the data available for subsequent processing.

[0239] Step 4:

[0240] The server passes the document file to the generation AI and requests it to analyze it. Specifically, it sends the AI ​​a prompt saying, "Analyze the colors in this document and identify which parts are emphasized." The input is the saved document file and the prompt, and the output is the analysis result of the highlighted color parts.

[0241] Step 5:

[0242] The generative AI receives the document, scans every page, and analyzes the use of color. For example, it identifies areas where red indicates a warning and blue indicates positive information. It returns this information to the server. The input is the document file, and the output is the color emphasis analysis results.

[0243] Step 6:

[0244] The server references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. For example, it obtains information on the combination of red and green that is considered difficult to see. The input is a database lookup query, and the output is information on color combinations that are difficult and easy to see.

[0245] Step 7:

[0246] The server passes the acquired information about color blindness to the generation AI, which then learns how to use colors in a way that is considerate to color-blind people. The input is information about combinations of difficult-to-see and easy-to-see colors, and the output is the learned color-conscious model. Specifically, the generation AI learns how to convert red to dark orange and green to light blue.

[0247] Step 8:

[0248] The user requests the generation of color-blind-friendly documents from the terminal. The input is a new prompt statement called the document generation request, and the output is instructions to the server.

[0249] Step 9:

[0250] The server instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. The emotion engine then reconfirms the user's emotions. The input is the material generation request and emotion data, and the output is the prompt and the user's emotion information.

[0251] Step 10:

[0252] Based on the learning results and the user's emotional information, the generative AI appropriately converts the colors of the materials and generates materials with new color schemes. Specifically, it changes red to dark orange and blue to navy blue. The input is the trained color consideration model and emotional data, and the output is materials with new color schemes.

[0253] Step 11:

[0254] The data file generated with the new color scheme is saved by the server and sent to the user's device. The input is the data for the new color scheme, and the output is the data sent to the device. This prepares the user to download the new data.

[0255] Step 12:

[0256] Users can receive and download new data files on their devices. They can also input feedback on the data through the emotion engine, which will be used to generate data for future projects. The input is the new data file and the user's feedback, and the output is the feedback data sent to the server.

[0257] (Application example 2)

[0258] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0259] In factories, work instruction screens that do not adequately take into account the visual characteristics and emotional state of workers are causing visual misperception and increased stress, which has a negative impact on work efficiency and safety. To solve this problem, it is necessary to automatically generate appropriate work instruction screens that take into consideration the visual characteristics and emotional state of workers and color-blind people.

[0260] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and understanding color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for changing the materials to a color-blind-friendly color scheme based on the learning results, means for generating and transmitting the changed materials, emotion analysis means for recognizing and analyzing the user's emotions, means for adjusting the color scheme of the materials based on the emotion analysis results, and means for automatically generating and displaying a work instruction screen tailored to the visual characteristics of the worker. This prevents visual misinterpretation, reduces worker stress, and enables improved work efficiency and safety.

[0261] "Existing materials" refers to documents, files, etc. that the user has already created.

[0262] "Means for receiving" refers to devices or software that have the function of sending and receiving data from a server.

[0263] "Means for analyzing" refers to a process or device for analyzing received data and understanding its structure and content.

[0264] "Color emphasis" refers to the state in which a particular color is more prominent or emphasized than other colors.

[0265] "Color blind" refers to people who cannot perceive certain colors normally.

[0266] "Difficult-to-see color combinations" refer to color combinations that are difficult for people with color blindness to distinguish.

[0267] "Easy-to-see color combinations" refer to color combinations that are easy to distinguish and highly visible for people with color blindness.

[0268] "Learning tools" refer to the processes and devices used to acquire and apply new knowledge based on past data and experience.

[0269] A "color-blind-friendly color scheme" refers to a color scheme in which the color combinations are adjusted so that even people with color blindness can accurately recognize information.

[0270] "Emotion analysis means" refers to a process or device for recognizing a user's emotions and understanding their state.

[0271] "Adjusting means" refers to a process or device for changing a setting or state based on specific conditions or criteria.

[0272] "Work instruction screen" refers to a display screen for providing specific instructions and information to workers.

[0273] "Display means" refers to devices or software that allow information to be visually confirmed.

[0274] This invention is a system that automatically generates work instruction screens that take into consideration color-blind people and the emotional state of workers in order to improve work efficiency and safety in factories.

[0275] System configuration

[0276] This system mainly includes a server, a terminal, a user, an emotion analysis means, and a generative AI model.

[0277] 1. Server

[0278] The server has the following functions:

[0279] Means of receiving existing materials: The user uploads the work instruction screen from smart glasses or other devices.

[0280] Means for analyzing materials and grasping color emphasis: Analyze the content of the received materials and extract the color emphasis parts.

[0281] A method for learning color combinations that are difficult for color-blind people to see: Color combinations that are difficult for color-blind people to see are learned from a database on color deficiency.

[0282] A way for color-blind people to learn colors that are easy to see: Learn color combinations that are easy to see.

[0283] A method for changing the color scheme of materials to be color-blind-friendly based on the learning results: Based on the analysis results and learning results, the color scheme of materials is changed to be easy for color-blind people to see.

[0284] Means for generating and transmitting modified materials: The materials modified to be color-blind friendly are transmitted to the user terminal.

[0285] Emotion analysis means to recognize and analyze the user's emotions: Analyze the user's emotional state from facial expressions and voice.

[0286] A means for adjusting the color scheme of the document based on the result of the emotion analysis: The color scheme of the document is further adjusted based on the result of the emotion analysis of the user.

[0287] A means for automatically generating and displaying a work instruction screen that is tailored to the visual characteristics of the worker: A work instruction screen optimized for smart glasses, etc. is generated and displayed.

[0288] Hardware and Software

[0289] Smart glasses: Uses a facial recognition camera, microphone, and display.

[0290] Cloud servers: Google Cloud Platform (GCP), Amazon Web Services (AWS), Microsoft Azure, etc.

[0291] Sentiment analysis software: Microsoft Azure Emotion API, IBM Watson, etc.

[0292] Generative AI model: OpenAI's GPT-4.

[0293] Processing Details

[0294] 1. Uploading the work instruction screen: The user uploads the work instruction screen to the server through a device such as smart glasses.

[0295] 2. Emotion analysis: The emotion analysis means recognizes the user's facial expressions and voice and analyzes their current emotional state.

[0296] 3. Document analysis and color emphasis recognition: The server analyzes the uploaded document and identifies the color emphasis areas.

[0297] 4. Learning color correction: Refer to a database on color blindness to learn which color combinations are difficult to see and which are easy to see.

[0298] 5. Generating color-blind-friendly materials: Based on the learning results and sentiment analysis results, the materials are changed to a color-blind-friendly color scheme.

[0299] 6. Generate and send new material: Generate the revised material and send it to the user's device.

[0300] 7. Gather feedback and improve: Use user feedback to improve the next version of the material.

[0301] Examples and prompts

[0302] For example, if a worker uploads a work instruction screen for a factory using smart glasses, the server analyzes the screen and changes the color combination to an appropriate one based on the user's emotional state while taking into consideration color blindness. The user can provide feedback, and the server will reflect this information in subsequent document generation.

[0303] Example prompts for generative AI models

[0304] "Please change the current color combination of this work instruction screen to one that is easier for people with color blindness to see, and convert it into an instruction screen for barrier-free monitoring that aims to reduce stress using emotional data. Please change the current red and blue to dark orange and navy blue."

[0305] In this way, the present invention makes it possible to provide a work instruction screen that is not only easy to view for people with color blindness, but also takes into consideration the emotional state of the worker.

[0306] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0307] Step 1: The user uploads an existing work instruction screen to the server through the smart glasses. The input is image data of the work instruction screen, and the output is the image data received by the server. Specifically, the worker captures the work instruction screen with the camera on the smart glasses and sends the data to the server.

[0308] Step 2: The server receives the uploaded work instruction screen and temporarily stores it. The input is the image data of the work instruction screen, and the output is the saved image file. The server stores this data in cloud storage (e.g., AWS S3).

[0309] Step 3: The emotion analysis means recognizes the user's facial expressions and voice and analyzes their emotional state. The input is the user's facial expression images and voice data, and the output is the analysis result, which is emotion data. Emotion analysis is performed using the camera and microphone of the smart glasses and Microsoft Azure's Emotion API.

[0310] Step 4: The server analyzes the received work instruction screen and extracts the color-enhanced parts. The input is an image file, and the output is the data of the color-enhanced parts. An image analysis library such as OpenCV is used to detect the color regions of the image.

[0311] Step 5: The server references a database of color blindness data to obtain the color combinations that are difficult and easy for color blind people to see. The input is the color highlight data, and the output is the combinations of difficult and easy-to-see colors. The server then executes a database query to obtain the required information.

[0312] Step 6: The generative AI model uses the color enhancement data, information about color blindness, and emotion data to generate a color-blind-friendly work instruction screen. The input is the color enhancement area, data on difficult and easy-to-see colors, and emotion data, and the output is the modified work instruction screen. Specific prompt sentences are used to issue instructions to the generative AI model (e.g., OpenAI GPT-4).

[0313] Step 7: The server generates a modified work instruction screen and sends it to the user's smart glasses. The input is the modified work instruction screen, and the output is an image file sent to the user's device. Data is transferred using a cloud messaging service (e.g., Firebase Cloud Messaging).

[0314] Step 8: The user performs the task and then provides feedback to the server through the smart glasses. The input is feedback data (text or voice), and the output is the received feedback data. The server analyzes this feedback and uses it to generate materials for future use.

[0315] Step 9: The server uses the feedback data to learn data to further optimize the next document generation. The input is the feedback data, and the output is the learning result. Using a machine learning algorithm (e.g., TensorFlow), important features are extracted from the feedback data and reflected in the next document generation.

[0316] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0317] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0318] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0319] [Second embodiment]

[0320] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0321] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0322] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0323] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0324] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0325] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0326] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0327] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0328] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0329] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0330] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0331] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0332] The system of this invention includes three main elements: a server, a terminal, and a user, each of which plays a specific role to realize the function of automatically generating materials that are considerate of color blindness.

[0333] First, a user uploads existing materials to the server using a terminal, and the terminal provides an interface for sending selected material files to the server.

[0334] The server then receives the uploaded file and temporarily stores it. The server then passes the file to the AI ​​generator, which analyzes the content of the file to determine the color emphasis. This analysis process determines what information each color represents.

[0335] The server then references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. The server then passes this information to the generation AI, which then learns which color combinations are difficult and easy for color-blind people to see. Specifically, the server obtains information from the database that indicates that the combination of red and green is difficult to see, and that dark orange and navy blue are easy to see.

[0336] After the learning is complete, the user requests the generation of color-blind-friendly materials from their device. When the server receives this request, it instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. Based on the learning results, the generation AI appropriately converts the colors of the materials and generates materials with a new color scheme.

[0337] The server saves the new color file and sends it to the user's device. The device receives the new file and allows the user to download it. This provides materials that can be accurately understood by color-blind people.

[0338] As a concrete example, let's take a sales report. When a user uploads a sales report from their device to the server, the server passes the report to a generation AI, which analyzes the negative aspects indicated in red and the positive aspects indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. As a result, the converted document is sent to the user's device, and the user can download a color-blind-friendly sales report.

[0339] The processing flow will be explained below.

[0340] Step 1:

[0341] The user selects a file on the device and clicks the upload button. The device sends the selected file to the server.

[0342] Step 2:

[0343] The server receives the uploaded file and temporarily stores it.

[0344] Step 3:

[0345] The server passes the saved document file to the generation AI, which analyzes the content of the document and determines the color emphasis. Specifically, it analyzes whether red indicates negative elements and blue indicates positive elements.

[0346] Step 4:

[0347] The server references a database about color blindness and obtains information about color combinations that are difficult for color-blind people to see. Specifically, it checks whether the combination of red and green is difficult for color-blind people to see.

[0348] Step 5:

[0349] The server passes the acquired color combination information that is difficult for color-blind people to see to the generation AI, which then learns from it.

[0350] Step 6:

[0351] The server references a database of colors that are easy for colorblind people to see and obtains color combinations that are easy to see. Specifically, it checks whether deep orange and navy blue are easy to see.

[0352] Step 7:

[0353] The server passes the acquired information on easy-to-see color combinations to the generation AI, which then learns from it.

[0354] Step 8:

[0355] The user sends a request for generating color-blind friendly materials from the device.

[0356] Step 9:

[0357] The server instructs the AI ​​to change the color scheme of existing documents to be color-blind-friendly. The AI ​​then appropriately converts the colors of the documents based on the learning results. Specifically, it changes red to dark orange and blue to navy blue.

[0358] Step 10:

[0359] The server saves the converted new color scheme data file.

[0360] Step 11:

[0361] The server sends the new file to the user's device, and the user receives the new file and can download it.

[0362] Example 1

[0363] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0364] Conventional document creation methods can make it difficult for color-blind people to accurately understand information. Specifically, in documents where color emphasis is important, color-blind people have difficulty distinguishing between colors, which can lead to communication problems. Furthermore, manually adjusting color schemes is cumbersome and requires specialized knowledge, making it inefficient. To solve these issues, a system that automatically generates documents that take color-blind people into consideration is needed.

[0365] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0366] In this invention, the server includes means for uploading existing materials to the server using a terminal, means for temporarily storing received material files on the server, means for passing the materials stored by the server to a generation AI and analyzing the contents of the materials, means for referencing a database on color deficiency to obtain information on color combinations that are difficult or easy for color-blind people to see, means for instructing the generation AI based on the obtained information to change the color scheme of the materials to colors that are easy for color-blind people to see, means for the generation AI to generate color-converted materials, and means for temporarily storing the changed materials on the server and sending them to the terminal. This makes it possible to efficiently and automatically generate materials with color schemes that are easy for color-blind people to see.

[0367] A "terminal" is an information device that a user uses to upload materials to a server or download generated materials.

[0368] The "server" is a computer system that receives and temporarily stores document files and interacts with the generating AI and database.

[0369] "Document files" are existing documents and reports that users upload to the server.

[0370] "Generative AI" is a program that uses artificial intelligence to analyze the contents of documents and change the color scheme for people with color blindness.

[0371] "Analysis" is the process in which the generating AI reads the contents of the document file and extracts color information.

[0372] "Color accents" refer to colors used to highlight specific information within a document.

[0373] The "color blindness database" is a database that stores information on color combinations that are difficult or easy for color-blind people to see.

[0374] "Color conversion" is the process of changing the colors in existing materials to make them easier to see for color-blind people.

[0375] The "modified document file" refers to the new document generated by the generating AI with the color scheme changed to suit color-blind people.

[0376] "Upload" refers to the act of a user sending a document file to a server via a terminal.

[0377] "Downloading" refers to the act of a user receiving a modified material file from a server via a terminal.

[0378] The system of the present invention includes a terminal, a server, a generative AI model, and a database related to color blindness, and provides a method and means for automatically generating materials that are considerate of color blindness.

[0379] A user uses a terminal to upload existing documents to the server. At this time, the terminal selects the document file and provides an interface for uploading. For example, a user selects a sales report file (e.g., report.pdf) and clicks the "Upload" button.

[0380] The server temporarily stores the document files received from the user. The stored document files are then analyzed by the generative AI model. The generative AI model analyzes the content of the document, identifies color emphasis, and determines what information each color represents.

[0381] Next, the server references a database on color blindness to obtain information on color combinations that are difficult or easy for color-blind people to see. For example, it obtains information from the database that "the combination of red and green is difficult to see" and "dark orange and navy blue are easy to see."

[0382] Based on the acquired information, the server issues instructions to the generative AI model to change the color scheme of the document to make it easier for people with color blindness to see. Based on the results of the analysis and learning, the generative AI model appropriately converts the colors of the document and generates a document with a new color scheme.

[0383] The generated modified document file is temporarily stored on the server and then sent to the user's device. The user can then use the device to receive and download the new document file. For example, the user can download the generated report_processed.pdf file.

[0384] Examples of specific prompts include requests such as "Please convert the red and blue colors in this report to color-blind-friendly colors" or "Please convert this sales report so that the reds and greens are easy to see."

[0385] As described above, this invention provides a system that can efficiently and automatically generate documents with color schemes that are easy to view for people with color blindness. By utilizing a generative AI model and a database on color blindness, it is possible to provide high-quality documents that take color blindness into consideration while saving time and effort.

[0386] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0387] Step 1:

[0388] User uploads materials

[0389] A user uses a terminal to select an existing document file and upload it to the server. As a concrete example, a user selects a sales report file (e.g., report.pdf) and clicks the "Upload" button. The input is the document file selected by the user, and the output is that file being sent to the server.

[0390] Step 2:

[0391] The server receives the data and temporarily stores it

[0392] The server receives the file uploaded by the user and temporarily stores it. The server stores the received file in the / uploads directory, for example. The input is the file sent, and the output is the file stored on the server.

[0393] Step 3:

[0394] The server passes the data to the generated AI for analysis.

[0395] The server passes the saved document file to the generation AI, which then analyzes the contents of the document. The generation AI extracts the colors used in the document and analyzes which colors represent which information. The input is the saved document file, and the output is the analysis results, including the document's color information. As a concrete example, a prompt such as "Analyze report.pdf and extract the color information within the document" is sent to the generation AI.

[0396] Step 4:

[0397] The server consults a database of color blindness information

[0398] The server references a database on color blindness and retrieves information about color combinations that are difficult and easy for color-blind people to see. The input is a query to the database, and the output is information about color combinations that are difficult and easy for color-blind people to see. For example, the server retrieves information from the database that "the combination of red and green is difficult to see" and "dark orange and navy blue are easy to see."

[0399] Step 5:

[0400] The server instructs the AI ​​to convert colors based on the learning results.

[0401] The server instructs the generation AI to convert the color of the material based on information obtained from a database on color blindness. The input is the analysis result and information from the database, and the output is a color conversion instruction. As a concrete example, the prompt statement "Convert the red in the analysis result to dark orange and the blue to navy blue" is sent to the generation AI.

[0402] Step 6:

[0403] Generative AI generates color-converted materials

[0404] The generation AI changes the color of the document based on instructions from the server and generates a document with a new color scheme. The input is the color conversion instruction, and the output is the document after the color change. The generation AI receives the instructions and generates a new document in which the red parts of report.pdf are converted to dark orange and the blue parts to navy blue.

[0405] Step 7:

[0406] The server saves the new material and sends it to the device.

[0407] The server temporarily stores the new material generated by the AI ​​and sends it to the user's device. The server stores the new material in, for example, the / processed directory, and sends a notification and a download link to the user's device. The input is the new material generated, and the output is to store and send the material.

[0408] Step 8:

[0409] User downloads new material

[0410] The user receives and downloads new documents using a device. The user clicks on a notification or link to download report_processed.pdf, which is generated with the new color scheme. The input is the download request from the device, and the output is the downloaded new documents.

[0411] The above processing steps make it possible to efficiently and automatically generate materials with color schemes that are easy to see even for color-blind people.

[0412] (Application example 1)

[0413] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0414] There is a need for a system that can change the colors of existing documents and video content to be color-blind-friendly so that people with color blindness can accurately understand the information. Analyzing the colors of each frame and converting them into a color scheme that is considerate to people with color blindness is a particularly difficult technical challenge in the case of video content. A system that can solve this problem is needed.

[0415] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0416] In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and grasping color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for changing the materials to a color-blind-friendly color scheme based on the learning results, means for receiving video content and analyzing the colors of each frame, means for generating and transmitting the changed materials, means for converting the colors of each video frame to a color-blind-friendly color scheme, and means for generating and transmitting the converted video. This makes it possible to provide materials and video content that even color-blind people can accurately understand.

[0417] "Existing materials" refers to documents and data files that were created in the past and are currently available.

[0418] "Color emphasis" refers to the frequency and prominence of a particular color.

[0419] "Color blind" refers to people who have difficulty distinguishing colors.

[0420] A "database" is a collection of data that is systematically organized and stored in a form that is easily accessible and searchable.

[0421] "Video Content" refers to multimedia files containing video and audio.

[0422] A "frame" refers to a single still image in a video.

[0423] "Learning results" refers to the knowledge and patterns that a generative AI model has acquired based on training data.

[0424] A "prompt sentence" refers to an instruction or input sentence to a generative AI model.

[0425] "Receiving" refers to the act of taking in data or information from outside.

[0426] "Analysis" refers to the process of examining and understanding data or information.

[0427] "Modification" refers to the act of changing something that already exists into a different state or form.

[0428] "Downloading" refers to the act of importing data from a remote location, such as a server, into your own device.

[0429] The system for implementing this invention consists of three main components: a server, a terminal, and a user. This system is designed to automatically convert existing materials and video content into color-blind-friendly color schemes.

[0430] First, a user uses a terminal to upload existing materials or video content to a server, and the terminal provides an interface for transmitting selected material files or video content to the server.

[0431] The server receives and temporarily stores the uploaded files and video content. The server then passes the files and video content to the generative AI model, which analyzes the files and each frame to understand color emphasis. This analysis process determines which information or scene each color represents.

[0432] The server then references a database of color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see.The generative AI model then uses this information to learn which color combinations are difficult and easy for color-blind people to see.

[0433] After the learning process is complete, the user can request the generation of color-blind-friendly materials or video conversions from their device. When the server receives this request, it instructs the generative AI model to change the color scheme of each frame of the existing materials or video to be color-blind-friendly. Based on the learning results, the generative AI model appropriately converts the colors of the materials or videos and generates materials or videos with the new color scheme.

[0434] The document files and video content created with the new color scheme are stored on the server and sent to the user's device. The user can then receive and download the new document files and video content. In this way, documents and video content are provided that even color-blind people can accurately understand.

[0435] As a concrete example, let's take the following prompt sentence as input to a generative AI model:

[0436] Analyze the uploaded video and identify the color emphasis of each frame. To accommodate color-blind users, convert red to orange and blue to navy blue. Combine the converted frames to create a new video file.

[0437] The hardware used is a smartphone (iOS or Android) or a high-performance server (e.g., NVIDIA Tesla T4), and the software used is a cloud-based generative AI model (e.g., Google Cloud AI, AWS SageMaker), which makes it possible to provide materials and video content that can be accurately understood even by people with color blindness.

[0438] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0439] Step 1:

[0440] A user uses a terminal to select existing materials or video content and upload it to the server. The terminal provides an interface for sending the selected file to the server. The input is the material file or video content selected by the user, and the output is the file sent to the server.

[0441] Step 2:

[0442] The server receives document files and video content uploaded by users and temporarily stores them. The input here is the uploaded file, and the output is the temporarily stored file. Specifically, the server analyzes the file reception request and stores the file in the appropriate storage.

[0443] Step 3:

[0444] The server passes the document files and video content to the generative AI model for analysis. The generative AI model processes the documents and the color emphasis of each frame. The input is the saved file, and the output is the analysis result including color emphasis information. In this step, the generative AI model receives the file as input and performs data analysis to extract color characteristics.

[0445] Step 4:

[0446] The server references a database about color blindness to obtain information about difficult and easy-to-see color combinations. The input is a database query, and the output is color information relevant to color blind people. In this step, the server executes the database query to obtain the required color information.

[0447] Step 5:

[0448] The generative AI model learns combinations of colors that are difficult and easy for colorblind people to see based on color information obtained from a database. The input is color information obtained from the database, and the output is the learning result. In this step, the generative AI model establishes color conversion rules using a learning algorithm.

[0449] Step 6:

[0450] A user requests the generation of color-blind-friendly materials or video conversion. The device sends the request to the server. The input is the user's request, and the output is the requested data sent to the server.

[0451] Step 7:

[0452] The server receives the user's request and issues instructions to the generative AI model. The generative AI model processes the analyzed materials and each frame of the video to change the color scheme to be color-blind-friendly. The input is instructions to the generative AI model and the analyzed data, and the output is the color-changed materials and video frames. In this step, the generative AI model performs the color conversion process based on the prompt text.

[0453] Step 8:

[0454] The generative AI model generates new materials and videos based on the learning results. The generated materials and videos are stored on the server. The input is the color-changed data, and the output is the newly generated file. In this step, the generative AI model completes the conversion process and creates the product.

[0455] Step 9:

[0456] The server sends new document files or video content to the user's device. The input is the new file that has been saved, and the output is the data to be sent to the user's device. Specifically, the server generates a file transfer request and delivers the file to the user's device.

[0457] Step 10:

[0458] The user receives and downloads new material files or video content on their device. The input is the file sent from the server, and the output is the new material or video stored on the device. In this step, the user confirms the receipt of the file and downloads it to local storage.

[0459] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0460] The system of this invention includes a server, a terminal, a user, and an emotion engine, each of which plays a specific role to realize the function of automatically generating materials that are considerate of color blindness while taking into account the user's emotions.

[0461] First, a user uploads existing materials to the server using a device. The device provides an interface for sending selected material files to the server, and the emotion engine analyzes the user's facial expressions, voice, or text input to recognize the user's emotions.

[0462] The server then receives the uploaded file and temporarily stores it. The server then passes the file to the AI ​​generator, which analyzes the content of the file to determine the color emphasis. This analysis process determines what information each color represents.

[0463] The server then references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. The server then passes this information to the generation AI, which then learns which color combinations are difficult and easy for color-blind people to see. Specifically, the server obtains information from the database that indicates that the combination of red and green is difficult to see, and that dark orange and navy blue are easy to see.

[0464] After the learning is complete, the user requests the generation of color-blind-friendly materials from their device. When the server receives this request, it instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. The emotion engine then works again to reconfirm the user's emotions. If the user is feeling stressed or dissatisfied, the generation AI further adjusts the color scheme to match the color scheme that the user finds most comfortable.

[0465] Based on the learning results and the user's emotional information, the generative AI appropriately converts the colors of the materials and generates materials with new color schemes. Specifically, it changes red to dark orange and blue to navy blue, selecting a color scheme that the user finds most appealing.

[0466] The document file generated with the new color scheme is stored by the server and sent to the user's device. The user can then receive and download the new document file on their device. The emotion engine also has a feedback function that allows the user to input their satisfaction with the generated document, allowing the engine to provide a more appropriate color scheme the next time a document is generated.

[0467] As a concrete example, let's take a sales report. When a user uploads a sales report from their device to the server, the emotion engine analyzes the user's input and facial expressions to determine their current emotional state. The server then passes the report to the generation AI, which analyzes the negative elements indicated in red and the positive elements indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. The converted document is then sent to the user's device, where it is available for download. The user can also provide feedback about the document to the emotion engine, which can then further improve the document generation the next time.

[0468] This will not only enable color-blind people to accurately understand information, but will also create a system that provides barrier-free materials tailored to the user's emotions.

[0469] The processing flow will be explained below.

[0470] Step 1:

[0471] The user selects a file on their device and clicks the upload button. The device then sends the selected file to the server. At this time, the emotion engine captures the user's facial expressions with a webcam and analyzes their voice tone and text input to recognize their emotional state.

[0472] Step 2:

[0473] The server receives the uploaded file and temporarily stores it. At the same time, the emotion engine analyzes the acquired user emotion data and determines the user's current emotional state (e.g., stress, dissatisfaction, satisfaction, etc.).

[0474] Step 3:

[0475] The server passes the saved document file to the generation AI, which analyzes the content of the document and identifies color emphasis. Specifically, it identifies negative elements shown in red and positive elements shown in blue.

[0476] Step 4:

[0477] The server references a database of color blindness information to obtain information about color combinations (e.g., red and green) that are difficult for color-blind people to see. The server passes this information to the generation AI.

[0478] Step 5:

[0479] The server references a database of colors that are easy for colorblind people to see and obtains information about color combinations that are easy to see (e.g., dark orange and navy blue). The server passes this information to the generation AI, which then learns from it.

[0480] Step 6:

[0481] The user sends a request for generating color-blind-friendly documents from their device. The emotion engine again captures the user's facial expressions and voice to confirm their emotional state.

[0482] Step 7:

[0483] The server instructs the AI ​​to change existing materials to a color-blind-friendly color scheme. The AI ​​then converts the materials based on its learning results and the user's latest emotional information. For example, if the user is feeling stressed, it will adjust the color tone by softening it.

[0484] Step 8:

[0485] The AI ​​generates a file containing the new color scheme, specifically changing red to dark orange and blue to navy blue.

[0486] Step 9:

[0487] The server stores the converted data files.

[0488] Step 10:

[0489] The server sends the new material file to the user's terminal, and the terminal receives the new material file and allows the user to download it.

[0490] Step 11:

[0491] The device asks the user for feedback on new materials. The user provides the feedback through the emotion engine. The emotion engine analyzes this feedback and stores the data to be reflected in future material generation.

[0492] This provides color-blind-friendly materials that are optimized for the user's emotions.

[0493] Example 2

[0494] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0495] Existing materials are often difficult for people with color blindness to view, especially when highlighting colors. Furthermore, the readability and comprehension of materials are also affected by the user's emotional state. However, existing systems lack the technology to automatically generate materials that take into account the readability of color blind people and the user's emotional state. Therefore, there is a need for a method of generating materials that provides materials that are easy to view for people with color blindness and that also takes into account the user's emotional state.

[0496] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0497] In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and grasping color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for recognizing and analyzing user emotions, means for changing the materials to a color-blind-friendly color scheme based on the learning results and user emotion information, and means for generating and transmitting the changed materials. This enables the automatic generation of materials that are easy for color-blind people to see and that take user emotions into consideration.

[0498] The "means for receiving existing materials" is a system component that allows the server to receive existing documents or data provided by the user.

[0499] "Means for analyzing received materials and understanding color emphasis" is a function for analyzing the use of colors and the importance of each color in received materials and understanding how they are emphasized in the materials.

[0500] "Means for learning color combinations that are difficult for color-blind people to see" is a function that collects information about color combinations that are difficult for color-blind people to see from a database, etc., and uses that information for learning.

[0501] "Means for learning colors that are easy for color-blind people to see" is a function that collects information from a database, etc. about color combinations that are easy for color-blind people to see, and uses that information for learning.

[0502] "Means for recognizing and analyzing user emotions" refers to a function that uses the device's camera, microphone, and text input interface to analyze the user's facial expressions, voice, and text, and recognize the user's current emotional state.

[0503] "Means for changing the color scheme of documents to be color-blind-friendly based on the learning results and the user's emotional information" is a function for changing the color scheme of documents to be color-blind-friendly based on the acquired learning results and the user's emotional information.

[0504] The "means for generating and transmitting modified materials" is a function for generating new materials with a modified color scheme and transmitting them to the user's terminal.

[0505] A "color blindness database" is an information source that stores information about color combinations that are difficult or easy for color blind people to see.

[0506] "Means for receiving user feedback and reflecting it in subsequent material generation" is a function for receiving evaluations and opinions on the generated materials provided by users and using them to help with subsequent material generation processes.

[0507] The system of this invention includes a server, a terminal, a user, and an emotion engine. Each component plays a specific role and realizes the function of automatically generating materials that are color-blind-friendly, taking into account the user's emotions. The specific operation of each component is described below.

[0508] Server Operation

[0509] The server first receives and temporarily stores existing documents uploaded by users on their devices. After receiving the documents, the server passes them to the AI ​​generator, which analyzes their contents. The AI ​​then identifies highlighted areas and determines what information each color represents. For example, red often indicates a warning, while blue indicates positive information.

[0510] Next, the server references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. This includes information such as the combination of red and green being difficult to see, and that dark orange and navy blue are easy to see. Based on this information, it instructs the generation AI on color-conscious learning.

[0511] Based on the learning results, the generative AI changes the color scheme of the document to be color-blind-friendly. Specifically, it changes red to dark orange and blue to navy blue. This makes the document easier to read for people with color blindness.

[0512] Furthermore, the server receives a request from the user to generate a document and instructs the AI ​​to reconfigure the existing document to have a color-blind-friendly color scheme. The emotion engine then reconfirms the user's emotions, and if the user feels stressed or dissatisfied, the AI ​​further adjusts the color scheme. The generated new document is then saved by the server and sent to the user's device.

[0513] Device behavior

[0514] The device provides an interface for users to upload existing documents. The emotion engine uses the device's camera, microphone, and text input interface to analyze the user's facial expressions, voice, and text to recognize their emotions. Emotional information is also sent to the server at the same time as the documents are uploaded.

[0515] The device receives the new color scheme file sent from the server and allows the user to download it. The emotion engine also has a feedback function that allows the user to input their satisfaction with the generated material.

[0516] User operations

[0517] The user uses the device to select an existing document and begin uploading it. For example, they select a sales report file and send it to the server. At this time, the user inputs facial expressions, voice, and text using the device's camera and microphone, and their emotional state is recognized.

[0518] Once new materials are generated and sent from the server to the user's device, the user can download them, check their content, and enter feedback such as their satisfaction level and requests through the device, which will help improve future material generation.

[0519] Specific examples

[0520] Take a sales report as an example. When a user uploads a sales report from their device to the server, the emotion engine analyzes the user's input and facial expressions to determine their current emotional state. The server then passes the report to the generation AI, which analyzes the negative aspects indicated in red and the positive aspects indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. The converted document is then sent to the user's device, where it is available for download. The user can also provide feedback on the document to the emotion engine, which can then further improve the document generation the next time.

[0521] Prompt Sentence Examples

[0522] "Please change the red parts of the following materials to dark orange and the blue parts to navy blue. Also, please adjust the colors to reflect the user's current emotions."

[0523] By implementing this invention, not only can color-blind people accurately understand information, but also barrier-free materials that match the user's emotions can be provided.

[0524] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0525] Step 1:

[0526] The user uses the terminal to select an existing document and start uploading. The operation is performed by selecting the document file and clicking the upload button in the terminal. The input is the user's existing document file, and the output is the document file sent to the server.

[0527] Step 2:

[0528] The device sends the selected data file to the server. The device uploads the data file, and at the same time the emotion engine analyzes the user's facial expressions, voice, and text using the camera, microphone, and text input interface. The input is the data file and the user's facial expressions, voice, and text, and the output is the data file and emotion data sent to the server.

[0529] Step 3:

[0530] The server receives the uploaded data files and temporarily stores them. The input is the data files and emotion data sent from the device, and the output is the saved data files. This storage makes the data available for subsequent processing.

[0531] Step 4:

[0532] The server passes the document file to the generation AI and requests it to analyze it. Specifically, it sends the AI ​​a prompt saying, "Analyze the colors in this document and identify which parts are emphasized." The input is the saved document file and the prompt, and the output is the analysis result of the highlighted color parts.

[0533] Step 5:

[0534] The generative AI receives the document, scans every page, and analyzes the use of color. For example, it identifies areas where red indicates a warning and blue indicates positive information. It returns this information to the server. The input is the document file, and the output is the color emphasis analysis results.

[0535] Step 6:

[0536] The server references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. For example, it obtains information on the combination of red and green that is considered difficult to see. The input is a database lookup query, and the output is information on color combinations that are difficult and easy to see.

[0537] Step 7:

[0538] The server passes the acquired information about color blindness to the generation AI, which then learns how to use colors in a way that is considerate to color-blind people. The input is information about combinations of difficult-to-see and easy-to-see colors, and the output is the learned color-conscious model. Specifically, the generation AI learns how to convert red to dark orange and green to light blue.

[0539] Step 8:

[0540] The user requests the generation of color-blind-friendly documents from the terminal. The input is a new prompt statement called the document generation request, and the output is instructions to the server.

[0541] Step 9:

[0542] The server instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. The emotion engine then reconfirms the user's emotions. The input is the material generation request and emotion data, and the output is the prompt and the user's emotion information.

[0543] Step 10:

[0544] Based on the learning results and the user's emotional information, the generative AI appropriately converts the colors of the materials and generates materials with new color schemes. Specifically, it changes red to dark orange and blue to navy blue. The input is the trained color consideration model and emotional data, and the output is materials with new color schemes.

[0545] Step 11:

[0546] The data file generated with the new color scheme is saved by the server and sent to the user's device. The input is the data for the new color scheme, and the output is the data sent to the device. This prepares the user to download the new data.

[0547] Step 12:

[0548] Users can receive and download new data files on their devices. They can also input feedback on the data through the emotion engine, which will be used to generate data for future projects. The input is the new data file and the user's feedback, and the output is the feedback data sent to the server.

[0549] (Application example 2)

[0550] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0551] In factories, work instruction screens that do not adequately take into account the visual characteristics and emotional state of workers are causing visual misperception and increased stress, which has a negative impact on work efficiency and safety. To solve this problem, it is necessary to automatically generate appropriate work instruction screens that take into consideration the visual characteristics and emotional state of workers and color-blind people.

[0552] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and understanding color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for changing the materials to a color-blind-friendly color scheme based on the learning results, means for generating and transmitting the changed materials, emotion analysis means for recognizing and analyzing the user's emotions, means for adjusting the color scheme of the materials based on the emotion analysis results, and means for automatically generating and displaying a work instruction screen tailored to the visual characteristics of the worker. This prevents visual misinterpretation, reduces worker stress, and enables improved work efficiency and safety.

[0553] "Existing materials" refers to documents, files, etc. that the user has already created.

[0554] "Means for receiving" refers to devices or software that have the function of sending and receiving data from a server.

[0555] "Means for analyzing" refers to a process or device for analyzing received data and understanding its structure and content.

[0556] "Color emphasis" refers to the state in which a particular color is more prominent or emphasized than other colors.

[0557] "Color blind" refers to people who cannot perceive certain colors normally.

[0558] "Difficult-to-see color combinations" refer to color combinations that are difficult for people with color blindness to distinguish.

[0559] "Easy-to-see color combinations" refer to color combinations that are easy to distinguish and highly visible for people with color blindness.

[0560] "Learning tools" refer to the processes and devices used to acquire and apply new knowledge based on past data and experience.

[0561] A "color-blind-friendly color scheme" refers to a color scheme in which the color combinations are adjusted so that even people with color blindness can accurately recognize information.

[0562] "Emotion analysis means" refers to a process or device for recognizing a user's emotions and understanding their state.

[0563] "Adjusting means" refers to a process or device for changing a setting or state based on specific conditions or criteria.

[0564] "Work instruction screen" refers to a display screen for providing specific instructions and information to workers.

[0565] "Display means" refers to devices or software that allow information to be visually confirmed.

[0566] This invention is a system that automatically generates work instruction screens that take into consideration color-blind people and the emotional state of workers in order to improve work efficiency and safety in factories.

[0567] System configuration

[0568] This system mainly includes a server, a terminal, a user, an emotion analysis means, and a generative AI model.

[0569] 1. Server

[0570] The server has the following functions:

[0571] Means of receiving existing materials: The user uploads the work instruction screen from smart glasses or other devices.

[0572] Means for analyzing materials and grasping color emphasis: Analyze the content of the received materials and extract the color emphasis parts.

[0573] A method for learning color combinations that are difficult for color-blind people to see: Color combinations that are difficult for color-blind people to see are learned from a database on color deficiency.

[0574] A way for color-blind people to learn colors that are easy to see: Learn color combinations that are easy to see.

[0575] A method for changing the color scheme of materials to be color-blind-friendly based on the learning results: Based on the analysis results and learning results, the color scheme of materials is changed to be easy for color-blind people to see.

[0576] Means for generating and transmitting modified materials: The materials modified to be color-blind friendly are transmitted to the user terminal.

[0577] Emotion analysis means to recognize and analyze the user's emotions: Analyze the user's emotional state from facial expressions and voice.

[0578] A means for adjusting the color scheme of the document based on the result of the emotion analysis: The color scheme of the document is further adjusted based on the result of the emotion analysis of the user.

[0579] A means for automatically generating and displaying a work instruction screen that is tailored to the visual characteristics of the worker: A work instruction screen optimized for smart glasses, etc. is generated and displayed.

[0580] Hardware and Software

[0581] Smart glasses: Uses a facial recognition camera, microphone, and display.

[0582] Cloud servers: Google Cloud Platform (GCP), Amazon Web Services (AWS), Microsoft Azure, etc.

[0583] Sentiment analysis software: Microsoft Azure Emotion API, IBM Watson, etc.

[0584] Generative AI model: OpenAI's GPT-4.

[0585] Processing Details

[0586] 1. Uploading the work instruction screen: The user uploads the work instruction screen to the server through a device such as smart glasses.

[0587] 2. Emotion analysis: The emotion analysis means recognizes the user's facial expressions and voice and analyzes their current emotional state.

[0588] 3. Document analysis and color emphasis recognition: The server analyzes the uploaded document and identifies the color emphasis areas.

[0589] 4. Learning color correction: Refer to a database on color blindness to learn which color combinations are difficult to see and which are easy to see.

[0590] 5. Generating color-blind-friendly materials: Based on the learning results and sentiment analysis results, the materials are changed to a color-blind-friendly color scheme.

[0591] 6. Generate and send new material: Generate the revised material and send it to the user's device.

[0592] 7. Gather feedback and improve: Use user feedback to improve the next version of the material.

[0593] Examples and prompts

[0594] For example, if a worker uploads a work instruction screen for a factory using smart glasses, the server analyzes the screen and changes the color combination to an appropriate one based on the user's emotional state while taking into consideration color blindness. The user can provide feedback, and the server will reflect this information in subsequent document generation.

[0595] Example prompts for generative AI models

[0596] "Please change the current color combination of this work instruction screen to one that is easier for people with color blindness to see, and convert it into an instruction screen for barrier-free monitoring that aims to reduce stress using emotional data. Please change the current red and blue to dark orange and navy blue."

[0597] In this way, the present invention makes it possible to provide a work instruction screen that is not only easy to view for people with color blindness, but also takes into consideration the emotional state of the worker.

[0598] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0599] Step 1: The user uploads an existing work instruction screen to the server through the smart glasses. The input is image data of the work instruction screen, and the output is the image data received by the server. Specifically, the worker captures the work instruction screen with the camera on the smart glasses and sends the data to the server.

[0600] Step 2: The server receives the uploaded work instruction screen and temporarily stores it. The input is the image data of the work instruction screen, and the output is the saved image file. The server stores this data in cloud storage (e.g., AWS S3).

[0601] Step 3: The emotion analysis means recognizes the user's facial expressions and voice and analyzes their emotional state. The input is the user's facial expression images and voice data, and the output is the analysis result, which is emotion data. Emotion analysis is performed using the camera and microphone of the smart glasses and Microsoft Azure's Emotion API.

[0602] Step 4: The server analyzes the received work instruction screen and extracts the color-enhanced parts. The input is an image file, and the output is the data of the color-enhanced parts. An image analysis library such as OpenCV is used to detect the color regions of the image.

[0603] Step 5: The server references a database of color blindness data to obtain the color combinations that are difficult and easy for color blind people to see. The input is the color highlight data, and the output is the combinations of difficult and easy-to-see colors. The server then executes a database query to obtain the required information.

[0604] Step 6: The generative AI model uses the color enhancement data, information about color blindness, and emotion data to generate a color-blind-friendly work instruction screen. The input is the color enhancement area, data on difficult and easy-to-see colors, and emotion data, and the output is the modified work instruction screen. Specific prompt sentences are used to issue instructions to the generative AI model (e.g., OpenAI GPT-4).

[0605] Step 7: The server generates a modified work instruction screen and sends it to the user's smart glasses. The input is the modified work instruction screen, and the output is an image file sent to the user's device. Data is transferred using a cloud messaging service (e.g., Firebase Cloud Messaging).

[0606] Step 8: The user performs the task and then provides feedback to the server through the smart glasses. The input is feedback data (text or voice), and the output is the received feedback data. The server analyzes this feedback and uses it to generate materials for future use.

[0607] Step 9: The server uses the feedback data to learn data to further optimize the next document generation. The input is the feedback data, and the output is the learning result. Using a machine learning algorithm (e.g., TensorFlow), important features are extracted from the feedback data and reflected in the next document generation.

[0608] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0609] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0610] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0611] [Third embodiment]

[0612] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0613] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0614] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0615] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0616] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0617] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0618] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0619] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0620] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0621] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0622] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0623] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0624] The system of this invention includes three main elements: a server, a terminal, and a user, each of which plays a specific role to realize the function of automatically generating materials that are considerate of color blindness.

[0625] First, a user uploads existing materials to the server using a terminal, and the terminal provides an interface for sending selected material files to the server.

[0626] The server then receives the uploaded file and temporarily stores it. The server then passes the file to the AI ​​generator, which analyzes the content of the file to determine the color emphasis. This analysis process determines what information each color represents.

[0627] The server then references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. The server then passes this information to the generation AI, which then learns which color combinations are difficult and easy for color-blind people to see. Specifically, the server obtains information from the database that indicates that the combination of red and green is difficult to see, and that dark orange and navy blue are easy to see.

[0628] After the learning is complete, the user requests the generation of color-blind-friendly materials from their device. When the server receives this request, it instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. Based on the learning results, the generation AI appropriately converts the colors of the materials and generates materials with a new color scheme.

[0629] The server saves the new color file and sends it to the user's device. The device receives the new file and allows the user to download it. This provides materials that can be accurately understood by color-blind people.

[0630] As a concrete example, let's take a sales report. When a user uploads a sales report from their device to the server, the server passes the report to a generation AI, which analyzes the negative aspects indicated in red and the positive aspects indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. As a result, the converted document is sent to the user's device, and the user can download a color-blind-friendly sales report.

[0631] The processing flow will be explained below.

[0632] Step 1:

[0633] The user selects a file on the device and clicks the upload button. The device sends the selected file to the server.

[0634] Step 2:

[0635] The server receives the uploaded file and temporarily stores it.

[0636] Step 3:

[0637] The server passes the saved document file to the generation AI, which analyzes the content of the document and determines the color emphasis. Specifically, it analyzes whether red indicates negative elements and blue indicates positive elements.

[0638] Step 4:

[0639] The server references a database about color blindness and obtains information about color combinations that are difficult for color-blind people to see. Specifically, it checks whether the combination of red and green is difficult for color-blind people to see.

[0640] Step 5:

[0641] The server passes the acquired color combination information that is difficult for color-blind people to see to the generation AI, which then learns from it.

[0642] Step 6:

[0643] The server references a database of colors that are easy for colorblind people to see and obtains color combinations that are easy to see. Specifically, it checks whether deep orange and navy blue are easy to see.

[0644] Step 7:

[0645] The server passes the acquired information on easy-to-see color combinations to the generation AI, which then learns from it.

[0646] Step 8:

[0647] The user sends a request for generating color-blind friendly materials from the device.

[0648] Step 9:

[0649] The server instructs the AI ​​to change the color scheme of existing documents to be color-blind-friendly. The AI ​​then appropriately converts the colors of the documents based on the learning results. Specifically, it changes red to dark orange and blue to navy blue.

[0650] Step 10:

[0651] The server saves the converted new color scheme data file.

[0652] Step 11:

[0653] The server sends the new file to the user's device, and the user receives the new file and can download it.

[0654] Example 1

[0655] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0656] Conventional document creation methods can make it difficult for color-blind people to accurately understand information. Specifically, in documents where color emphasis is important, color-blind people have difficulty distinguishing between colors, which can lead to communication problems. Furthermore, manually adjusting color schemes is cumbersome and requires specialized knowledge, making it inefficient. To solve these issues, a system that automatically generates documents that take color-blind people into consideration is needed.

[0657] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0658] In this invention, the server includes means for uploading existing materials to the server using a terminal, means for temporarily storing received material files on the server, means for passing the materials stored by the server to a generation AI and analyzing the contents of the materials, means for referencing a database on color deficiency to obtain information on color combinations that are difficult or easy for color-blind people to see, means for instructing the generation AI based on the obtained information to change the color scheme of the materials to colors that are easy for color-blind people to see, means for the generation AI to generate color-converted materials, and means for temporarily storing the changed materials on the server and sending them to the terminal. This makes it possible to efficiently and automatically generate materials with color schemes that are easy for color-blind people to see.

[0659] A "terminal" is an information device that a user uses to upload materials to a server or download generated materials.

[0660] The "server" is a computer system that receives and temporarily stores document files and interacts with the generating AI and database.

[0661] "Document files" are existing documents and reports that users upload to the server.

[0662] "Generative AI" is a program that uses artificial intelligence to analyze the contents of documents and change the color scheme for people with color blindness.

[0663] "Analysis" is the process in which the generating AI reads the contents of the document file and extracts color information.

[0664] "Color accents" refer to colors used to highlight specific information within a document.

[0665] The "color blindness database" is a database that stores information on color combinations that are difficult or easy for color-blind people to see.

[0666] "Color conversion" is the process of changing the colors in existing materials to make them easier to see for color-blind people.

[0667] The "modified document file" refers to the new document generated by the generating AI with the color scheme changed to suit color-blind people.

[0668] "Upload" refers to the act of a user sending a document file to a server via a terminal.

[0669] "Downloading" refers to the act of a user receiving a modified material file from a server via a terminal.

[0670] The system of the present invention includes a terminal, a server, a generative AI model, and a database related to color blindness, and provides a method and means for automatically generating materials that are considerate of color blindness.

[0671] A user uses a terminal to upload existing documents to the server. At this time, the terminal selects the document file and provides an interface for uploading. For example, a user selects a sales report file (e.g., report.pdf) and clicks the "Upload" button.

[0672] The server temporarily stores the document files received from the user. The stored document files are then analyzed by the generative AI model. The generative AI model analyzes the content of the document, identifies color emphasis, and determines what information each color represents.

[0673] Next, the server references a database on color blindness to obtain information on color combinations that are difficult or easy for color-blind people to see. For example, it obtains information from the database that "the combination of red and green is difficult to see" and "dark orange and navy blue are easy to see."

[0674] Based on the acquired information, the server issues instructions to the generative AI model to change the color scheme of the document to make it easier for people with color blindness to see. Based on the results of the analysis and learning, the generative AI model appropriately converts the colors of the document and generates a document with a new color scheme.

[0675] The generated modified document file is temporarily stored on the server and then sent to the user's device. The user can then use the device to receive and download the new document file. For example, the user can download the generated report_processed.pdf file.

[0676] Examples of specific prompts include requests such as "Please convert the red and blue colors in this report to color-blind-friendly colors" or "Please convert this sales report so that the reds and greens are easy to see."

[0677] As described above, this invention provides a system that can efficiently and automatically generate documents with color schemes that are easy to view for people with color blindness. By utilizing a generative AI model and a database on color blindness, it is possible to provide high-quality documents that take color blindness into consideration while saving time and effort.

[0678] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0679] Step 1:

[0680] User uploads materials

[0681] A user uses a terminal to select an existing document file and upload it to the server. As a concrete example, a user selects a sales report file (e.g., report.pdf) and clicks the "Upload" button. The input is the document file selected by the user, and the output is that file being sent to the server.

[0682] Step 2:

[0683] The server receives the data and temporarily stores it

[0684] The server receives the file uploaded by the user and temporarily stores it. The server stores the received file in the / uploads directory, for example. The input is the file sent, and the output is the file stored on the server.

[0685] Step 3:

[0686] The server passes the data to the generated AI for analysis.

[0687] The server passes the saved document file to the generation AI, which then analyzes the contents of the document. The generation AI extracts the colors used in the document and analyzes which colors represent which information. The input is the saved document file, and the output is the analysis results, including the document's color information. As a concrete example, a prompt such as "Analyze report.pdf and extract the color information within the document" is sent to the generation AI.

[0688] Step 4:

[0689] The server consults a database of color blindness information

[0690] The server references a database on color blindness and retrieves information about color combinations that are difficult and easy for color-blind people to see. The input is a query to the database, and the output is information about color combinations that are difficult and easy for color-blind people to see. For example, the server retrieves information from the database that "the combination of red and green is difficult to see" and "dark orange and navy blue are easy to see."

[0691] Step 5:

[0692] The server instructs the AI ​​to convert colors based on the learning results.

[0693] The server instructs the generation AI to convert the color of the material based on information obtained from a database on color blindness. The input is the analysis result and information from the database, and the output is a color conversion instruction. As a concrete example, the prompt statement "Convert the red in the analysis result to dark orange and the blue to navy blue" is sent to the generation AI.

[0694] Step 6:

[0695] Generative AI generates color-converted materials

[0696] The generation AI changes the color of the document based on instructions from the server and generates a document with a new color scheme. The input is the color conversion instruction, and the output is the document after the color change. The generation AI receives the instructions and generates a new document in which the red parts of report.pdf are converted to dark orange and the blue parts to navy blue.

[0697] Step 7:

[0698] The server saves the new material and sends it to the device.

[0699] The server temporarily stores the new material generated by the AI ​​and sends it to the user's device. The server stores the new material in, for example, the / processed directory, and sends a notification and a download link to the user's device. The input is the new material generated, and the output is to store and send the material.

[0700] Step 8:

[0701] User downloads new material

[0702] The user receives and downloads new documents using a device. The user clicks on a notification or link to download report_processed.pdf, which is generated with the new color scheme. The input is the download request from the device, and the output is the downloaded new documents.

[0703] The above processing steps make it possible to efficiently and automatically generate materials with color schemes that are easy to see even for color-blind people.

[0704] (Application example 1)

[0705] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0706] There is a need for a system that can change the colors of existing documents and video content to be color-blind-friendly so that people with color blindness can accurately understand the information. Analyzing the colors of each frame and converting them into a color scheme that is considerate to people with color blindness is a particularly difficult technical challenge in the case of video content. A system that can solve this problem is needed.

[0707] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0708] In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and grasping color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for changing the materials to a color-blind-friendly color scheme based on the learning results, means for receiving video content and analyzing the colors of each frame, means for generating and transmitting the changed materials, means for converting the colors of each video frame to a color-blind-friendly color scheme, and means for generating and transmitting the converted video. This makes it possible to provide materials and video content that even color-blind people can accurately understand.

[0709] "Existing materials" refers to documents and data files that were created in the past and are currently available.

[0710] "Color emphasis" refers to the frequency and prominence of a particular color.

[0711] "Color blind" refers to people who have difficulty distinguishing colors.

[0712] A "database" is a collection of data that is systematically organized and stored in a form that is easily accessible and searchable.

[0713] "Video Content" refers to multimedia files containing video and audio.

[0714] A "frame" refers to a single still image in a video.

[0715] "Learning results" refers to the knowledge and patterns that a generative AI model has acquired based on training data.

[0716] A "prompt sentence" refers to an instruction or input sentence to a generative AI model.

[0717] "Receiving" refers to the act of taking in data or information from outside.

[0718] "Analysis" refers to the process of examining and understanding data or information.

[0719] "Modification" refers to the act of changing something that already exists into a different state or form.

[0720] "Downloading" refers to the act of importing data from a remote location, such as a server, into your own device.

[0721] The system for implementing this invention consists of three main components: a server, a terminal, and a user. This system is designed to automatically convert existing materials and video content into color-blind-friendly color schemes.

[0722] First, a user uses a terminal to upload existing materials or video content to a server, and the terminal provides an interface for transmitting selected material files or video content to the server.

[0723] The server receives and temporarily stores the uploaded files and video content. The server then passes the files and video content to the generative AI model, which analyzes the files and each frame to understand color emphasis. This analysis process determines which information or scene each color represents.

[0724] The server then references a database of color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see.The generative AI model then uses this information to learn which color combinations are difficult and easy for color-blind people to see.

[0725] After the learning process is complete, the user can request the generation of color-blind-friendly materials or video conversions from their device. When the server receives this request, it instructs the generative AI model to change the color scheme of each frame of the existing materials or video to be color-blind-friendly. Based on the learning results, the generative AI model appropriately converts the colors of the materials or videos and generates materials or videos with the new color scheme.

[0726] The document files and video content created with the new color scheme are stored on the server and sent to the user's device. The user can then receive and download the new document files and video content. In this way, documents and video content are provided that even color-blind people can accurately understand.

[0727] As a concrete example, let's take the following prompt sentence as input to a generative AI model:

[0728] Analyze the uploaded video and identify the color emphasis of each frame. To accommodate color-blind users, convert red to orange and blue to navy blue. Combine the converted frames to create a new video file.

[0729] The hardware used is a smartphone (iOS or Android) or a high-performance server (e.g., NVIDIA Tesla T4), and the software used is a cloud-based generative AI model (e.g., Google Cloud AI, AWS SageMaker), which makes it possible to provide materials and video content that can be accurately understood even by people with color blindness.

[0730] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0731] Step 1:

[0732] A user uses a terminal to select existing materials or video content and upload it to the server. The terminal provides an interface for sending the selected file to the server. The input is the material file or video content selected by the user, and the output is the file sent to the server.

[0733] Step 2:

[0734] The server receives document files and video content uploaded by users and temporarily stores them. The input here is the uploaded file, and the output is the temporarily stored file. Specifically, the server analyzes the file reception request and stores the file in the appropriate storage.

[0735] Step 3:

[0736] The server passes the document files and video content to the generative AI model for analysis. The generative AI model processes the documents and the color emphasis of each frame. The input is the saved file, and the output is the analysis result including color emphasis information. In this step, the generative AI model receives the file as input and performs data analysis to extract color characteristics.

[0737] Step 4:

[0738] The server references a database about color blindness to obtain information about difficult and easy-to-see color combinations. The input is a database query, and the output is color information relevant to color blind people. In this step, the server executes the database query to obtain the required color information.

[0739] Step 5:

[0740] The generative AI model learns combinations of colors that are difficult and easy for colorblind people to see based on color information obtained from a database. The input is color information obtained from the database, and the output is the learning result. In this step, the generative AI model establishes color conversion rules using a learning algorithm.

[0741] Step 6:

[0742] A user requests the generation of color-blind-friendly materials or video conversion. The device sends the request to the server. The input is the user's request, and the output is the requested data sent to the server.

[0743] Step 7:

[0744] The server receives the user's request and issues instructions to the generative AI model. The generative AI model processes the analyzed materials and each frame of the video to change the color scheme to be color-blind-friendly. The input is instructions to the generative AI model and the analyzed data, and the output is the color-changed materials and video frames. In this step, the generative AI model performs the color conversion process based on the prompt text.

[0745] Step 8:

[0746] The generative AI model generates new materials and videos based on the learning results. The generated materials and videos are stored on the server. The input is the color-changed data, and the output is the newly generated file. In this step, the generative AI model completes the conversion process and creates the product.

[0747] Step 9:

[0748] The server sends new document files or video content to the user's device. The input is the new file that has been saved, and the output is the data to be sent to the user's device. Specifically, the server generates a file transfer request and delivers the file to the user's device.

[0749] Step 10:

[0750] The user receives and downloads new material files or video content on their device. The input is the file sent from the server, and the output is the new material or video stored on the device. In this step, the user confirms the receipt of the file and downloads it to local storage.

[0751] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0752] The system of this invention includes a server, a terminal, a user, and an emotion engine, each of which plays a specific role to realize the function of automatically generating materials that are considerate of color blindness while taking into account the user's emotions.

[0753] First, a user uploads existing materials to the server using a device. The device provides an interface for sending selected material files to the server, and the emotion engine analyzes the user's facial expressions, voice, or text input to recognize the user's emotions.

[0754] The server then receives the uploaded file and temporarily stores it. The server then passes the file to the AI ​​generator, which analyzes the content of the file to determine the color emphasis. This analysis process determines what information each color represents.

[0755] The server then references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. The server then passes this information to the generation AI, which then learns which color combinations are difficult and easy for color-blind people to see. Specifically, the server obtains information from the database that indicates that the combination of red and green is difficult to see, and that dark orange and navy blue are easy to see.

[0756] After the learning is complete, the user requests the generation of color-blind-friendly materials from their device. When the server receives this request, it instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. The emotion engine then works again to reconfirm the user's emotions. If the user is feeling stressed or dissatisfied, the generation AI further adjusts the color scheme to match the color scheme that the user finds most comfortable.

[0757] Based on the learning results and the user's emotional information, the generative AI appropriately converts the colors of the materials and generates materials with new color schemes. Specifically, it changes red to dark orange and blue to navy blue, selecting a color scheme that the user finds most appealing.

[0758] The document file generated with the new color scheme is stored by the server and sent to the user's device. The user can then receive and download the new document file on their device. The emotion engine also has a feedback function that allows the user to input their satisfaction with the generated document, allowing the engine to provide a more appropriate color scheme the next time a document is generated.

[0759] As a concrete example, let's take a sales report. When a user uploads a sales report from their device to the server, the emotion engine analyzes the user's input and facial expressions to determine their current emotional state. The server then passes the report to the generation AI, which analyzes the negative elements indicated in red and the positive elements indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. The converted document is then sent to the user's device, where it is available for download. The user can also provide feedback about the document to the emotion engine, which can then further improve the document generation the next time.

[0760] This will not only enable color-blind people to accurately understand information, but will also create a system that provides barrier-free materials tailored to the user's emotions.

[0761] The processing flow will be explained below.

[0762] Step 1:

[0763] The user selects a file on their device and clicks the upload button. The device then sends the selected file to the server. At this time, the emotion engine captures the user's facial expressions with a webcam and analyzes their voice tone and text input to recognize their emotional state.

[0764] Step 2:

[0765] The server receives the uploaded file and temporarily stores it. At the same time, the emotion engine analyzes the acquired user emotion data and determines the user's current emotional state (e.g., stress, dissatisfaction, satisfaction, etc.).

[0766] Step 3:

[0767] The server passes the saved document file to the generation AI, which analyzes the content of the document and identifies color emphasis. Specifically, it identifies negative elements shown in red and positive elements shown in blue.

[0768] Step 4:

[0769] The server references a database of color blindness information to obtain information about color combinations (e.g., red and green) that are difficult for color-blind people to see. The server passes this information to the generation AI.

[0770] Step 5:

[0771] The server references a database of colors that are easy for colorblind people to see and obtains information about color combinations that are easy to see (e.g., dark orange and navy blue). The server passes this information to the generation AI, which then learns from it.

[0772] Step 6:

[0773] The user sends a request for generating color-blind-friendly documents from their device. The emotion engine again captures the user's facial expressions and voice to confirm their emotional state.

[0774] Step 7:

[0775] The server instructs the AI ​​to change existing materials to a color-blind-friendly color scheme. The AI ​​then converts the materials based on its learning results and the user's latest emotional information. For example, if the user is feeling stressed, it will adjust the color tone by softening it.

[0776] Step 8:

[0777] The AI ​​generates a file containing the new color scheme, specifically changing red to dark orange and blue to navy blue.

[0778] Step 9:

[0779] The server stores the converted data files.

[0780] Step 10:

[0781] The server sends the new material file to the user's terminal, and the terminal receives the new material file and allows the user to download it.

[0782] Step 11:

[0783] The device asks the user for feedback on new materials. The user provides the feedback through the emotion engine. The emotion engine analyzes this feedback and stores the data to be reflected in future material generation.

[0784] This provides color-blind-friendly materials that are optimized for the user's emotions.

[0785] Example 2

[0786] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0787] Existing materials are often difficult for people with color blindness to view, especially when highlighting colors. Furthermore, the readability and comprehension of materials are also affected by the user's emotional state. However, existing systems lack the technology to automatically generate materials that take into account the readability of color blind people and the user's emotional state. Therefore, there is a need for a method of generating materials that provides materials that are easy to view for people with color blindness and that also takes into account the user's emotional state.

[0788] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0789] In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and grasping color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for recognizing and analyzing user emotions, means for changing the materials to a color-blind-friendly color scheme based on the learning results and user emotion information, and means for generating and transmitting the changed materials. This enables the automatic generation of materials that are easy for color-blind people to see and that take user emotions into consideration.

[0790] The "means for receiving existing materials" is a system component that allows the server to receive existing documents or data provided by the user.

[0791] "Means for analyzing received materials and understanding color emphasis" is a function for analyzing the use of colors and the importance of each color in received materials and understanding how they are emphasized in the materials.

[0792] "Means for learning color combinations that are difficult for color-blind people to see" is a function that collects information about color combinations that are difficult for color-blind people to see from a database, etc., and uses that information for learning.

[0793] "Means for learning colors that are easy for color-blind people to see" is a function that collects information from a database, etc. about color combinations that are easy for color-blind people to see, and uses that information for learning.

[0794] "Means for recognizing and analyzing user emotions" refers to a function that uses the device's camera, microphone, and text input interface to analyze the user's facial expressions, voice, and text, and recognize the user's current emotional state.

[0795] "Means for changing the color scheme of documents to be color-blind-friendly based on the learning results and the user's emotional information" is a function for changing the color scheme of documents to be color-blind-friendly based on the acquired learning results and the user's emotional information.

[0796] The "means for generating and transmitting modified materials" is a function for generating new materials with a modified color scheme and transmitting them to the user's terminal.

[0797] A "color blindness database" is an information source that stores information about color combinations that are difficult or easy for color blind people to see.

[0798] "Means for receiving user feedback and reflecting it in subsequent material generation" is a function for receiving evaluations and opinions on the generated materials provided by users and using them to help with subsequent material generation processes.

[0799] The system of this invention includes a server, a terminal, a user, and an emotion engine. Each component plays a specific role and realizes the function of automatically generating materials that are color-blind-friendly, taking into account the user's emotions. The specific operation of each component is described below.

[0800] Server Operation

[0801] The server first receives and temporarily stores existing documents uploaded by users on their devices. After receiving the documents, the server passes them to the AI ​​generator, which analyzes their contents. The AI ​​then identifies highlighted areas and determines what information each color represents. For example, red often indicates a warning, while blue indicates positive information.

[0802] Next, the server references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. This includes information such as the combination of red and green being difficult to see, and that dark orange and navy blue are easy to see. Based on this information, it instructs the generation AI on color-conscious learning.

[0803] Based on the learning results, the generative AI changes the color scheme of the document to be color-blind-friendly. Specifically, it changes red to dark orange and blue to navy blue. This makes the document easier to read for people with color blindness.

[0804] Furthermore, the server receives a request from the user to generate a document and instructs the AI ​​to reconfigure the existing document to have a color-blind-friendly color scheme. The emotion engine then reconfirms the user's emotions, and if the user feels stressed or dissatisfied, the AI ​​further adjusts the color scheme. The generated new document is then saved by the server and sent to the user's device.

[0805] Device behavior

[0806] The device provides an interface for users to upload existing documents. The emotion engine uses the device's camera, microphone, and text input interface to analyze the user's facial expressions, voice, and text to recognize their emotions. Emotional information is also sent to the server at the same time as the documents are uploaded.

[0807] The device receives the new color scheme file sent from the server and allows the user to download it. The emotion engine also has a feedback function that allows the user to input their satisfaction with the generated material.

[0808] User operations

[0809] The user uses the device to select an existing document and begin uploading it. For example, they select a sales report file and send it to the server. At this time, the user inputs facial expressions, voice, and text using the device's camera and microphone, and their emotional state is recognized.

[0810] Once new materials are generated and sent from the server to the user's device, the user can download them, check their content, and enter feedback such as their satisfaction level and requests through the device, which will help improve future material generation.

[0811] Specific examples

[0812] Take a sales report as an example. When a user uploads a sales report from their device to the server, the emotion engine analyzes the user's input and facial expressions to determine their current emotional state. The server then passes the report to the generation AI, which analyzes the negative aspects indicated in red and the positive aspects indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. The converted document is then sent to the user's device, where it is available for download. The user can also provide feedback on the document to the emotion engine, which can then further improve the document generation the next time.

[0813] Prompt Sentence Examples

[0814] "Please change the red parts of the following materials to dark orange and the blue parts to navy blue. Also, please adjust the colors to reflect the user's current feelings."

[0815] By implementing this invention, not only can color-blind people accurately understand information, but also barrier-free materials that match the user's emotions can be provided.

[0816] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0817] Step 1:

[0818] The user uses the terminal to select an existing document and start uploading. The operation is performed by selecting the document file and clicking the upload button in the terminal. The input is the user's existing document file, and the output is the document file sent to the server.

[0819] Step 2:

[0820] The device sends the selected data file to the server. The device uploads the data file, and at the same time the emotion engine analyzes the user's facial expressions, voice, and text using the camera, microphone, and text input interface. The input is the data file and the user's facial expressions, voice, and text, and the output is the data file and emotion data sent to the server.

[0821] Step 3:

[0822] The server receives the uploaded data files and temporarily stores them. The input is the data files and emotion data sent from the device, and the output is the saved data files. This storage makes the data available for subsequent processing.

[0823] Step 4:

[0824] The server passes the document file to the generation AI and requests it to analyze it. Specifically, it sends the AI ​​a prompt saying, "Analyze the colors in this document and identify which parts are emphasized." The input is the saved document file and the prompt, and the output is the analysis result of the highlighted color parts.

[0825] Step 5:

[0826] The generative AI receives the document, scans every page, and analyzes the use of color. For example, it identifies areas where red indicates a warning and blue indicates positive information. It returns this information to the server. The input is the document file, and the output is the color emphasis analysis results.

[0827] Step 6:

[0828] The server references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. For example, it obtains information on the combination of red and green that is considered difficult to see. The input is a database lookup query, and the output is information on color combinations that are difficult and easy to see.

[0829] Step 7:

[0830] The server passes the acquired information about color blindness to the generation AI, which then learns how to use colors in a way that is considerate to color-blind people. The input is information about combinations of difficult-to-see and easy-to-see colors, and the output is the learned color-conscious model. Specifically, the generation AI learns how to convert red to dark orange and green to light blue.

[0831] Step 8:

[0832] The user requests the generation of color-blind-friendly documents from the terminal. The input is a new prompt statement called the document generation request, and the output is instructions to the server.

[0833] Step 9:

[0834] The server instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. The emotion engine then reconfirms the user's emotions. The input is the material generation request and emotion data, and the output is the prompt and the user's emotion information.

[0835] Step 10:

[0836] Based on the learning results and the user's emotional information, the generative AI appropriately converts the colors of the materials and generates materials with new color schemes. Specifically, it changes red to dark orange and blue to navy blue. The input is the trained color consideration model and emotional data, and the output is materials with new color schemes.

[0837] Step 11:

[0838] The data file generated with the new color scheme is saved by the server and sent to the user's device. The input is the data for the new color scheme, and the output is the data sent to the device. This prepares the user to download the new data.

[0839] Step 12:

[0840] Users can receive and download new data files on their devices. They can also input feedback on the data through the emotion engine, which will be used to generate data for future projects. The input is the new data file and the user's feedback, and the output is the feedback data sent to the server.

[0841] (Application example 2)

[0842] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0843] In factories, work instruction screens that do not adequately take into account the visual characteristics and emotional state of workers are causing visual misperception and increased stress, which has a negative impact on work efficiency and safety. To solve this problem, it is necessary to automatically generate appropriate work instruction screens that take into consideration the visual characteristics and emotional state of workers and color-blind people.

[0844] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and understanding color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for changing the materials to a color-blind-friendly color scheme based on the learning results, means for generating and transmitting the changed materials, emotion analysis means for recognizing and analyzing the user's emotions, means for adjusting the color scheme of the materials based on the emotion analysis results, and means for automatically generating and displaying a work instruction screen tailored to the visual characteristics of the worker. This prevents visual misinterpretation, reduces worker stress, and enables improved work efficiency and safety.

[0845] "Existing materials" refers to documents, files, etc. that the user has already created.

[0846] "Means for receiving" refers to devices or software that have the function of sending and receiving data from a server.

[0847] "Means for analyzing" refers to a process or device for analyzing received data and understanding its structure and content.

[0848] "Color emphasis" refers to the state in which a particular color is more prominent or emphasized than other colors.

[0849] "Color blind" refers to people who cannot perceive certain colors normally.

[0850] "Difficult-to-see color combinations" refer to color combinations that are difficult for people with color blindness to distinguish.

[0851] "Easy-to-see color combinations" refer to color combinations that are easy to distinguish and highly visible for people with color blindness.

[0852] "Learning tools" refer to the processes and devices used to acquire and apply new knowledge based on past data and experience.

[0853] A "color-blind-friendly color scheme" refers to a color scheme in which the color combinations are adjusted so that even people with color blindness can accurately recognize information.

[0854] "Emotion analysis means" refers to a process or device for recognizing a user's emotions and understanding their state.

[0855] "Adjusting means" refers to a process or device for changing a setting or state based on specific conditions or criteria.

[0856] "Work instruction screen" refers to a display screen for providing specific instructions and information to workers.

[0857] "Display means" refers to devices or software that allow information to be visually confirmed.

[0858] This invention is a system that automatically generates work instruction screens that take into consideration color-blind people and the emotional state of workers in order to improve work efficiency and safety in factories.

[0859] System configuration

[0860] This system mainly includes a server, a terminal, a user, an emotion analysis means, and a generative AI model.

[0861] 1. Server

[0862] The server has the following functions:

[0863] Means of receiving existing materials: The user uploads the work instruction screen from smart glasses or other devices.

[0864] Means for analyzing materials and grasping color emphasis: Analyze the content of the received materials and extract the color emphasis parts.

[0865] A method for learning color combinations that are difficult for color-blind people to see: Color combinations that are difficult for color-blind people to see are learned from a database on color deficiency.

[0866] A way for color-blind people to learn colors that are easy to see: Learn color combinations that are easy to see.

[0867] A method for changing the color scheme of materials to be color-blind-friendly based on the learning results: Based on the analysis results and learning results, the color scheme of materials is changed to be easy for color-blind people to see.

[0868] Means for generating and transmitting modified materials: The materials modified to be color-blind friendly are transmitted to the user terminal.

[0869] Emotion analysis means to recognize and analyze the user's emotions: Analyze the user's emotional state from facial expressions and voice.

[0870] A means for adjusting the color scheme of the document based on the result of the emotion analysis: The color scheme of the document is further adjusted based on the result of the emotion analysis of the user.

[0871] A means for automatically generating and displaying a work instruction screen that is tailored to the visual characteristics of the worker: A work instruction screen optimized for smart glasses, etc. is generated and displayed.

[0872] Hardware and Software

[0873] Smart glasses: Uses a facial recognition camera, microphone, and display.

[0874] Cloud servers: Google Cloud Platform (GCP), Amazon Web Services (AWS), Microsoft Azure, etc.

[0875] Sentiment analysis software: Microsoft Azure Emotion API, IBM Watson, etc.

[0876] Generative AI model: OpenAI's GPT-4.

[0877] Processing Details

[0878] 1. Uploading the work instruction screen: The user uploads the work instruction screen to the server through a device such as smart glasses.

[0879] 2. Emotion analysis: The emotion analysis means recognizes the user's facial expressions and voice and analyzes their current emotional state.

[0880] 3. Document analysis and color emphasis recognition: The server analyzes the uploaded document and identifies the color emphasis areas.

[0881] 4. Learning color correction: Refer to a database on color blindness to learn which color combinations are difficult to see and which are easy to see.

[0882] 5. Generating color-blind-friendly materials: Based on the learning results and sentiment analysis results, the materials are changed to a color-blind-friendly color scheme.

[0883] 6. Generate and send new material: Generate the revised material and send it to the user's device.

[0884] 7. Gather feedback and improve: Use user feedback to improve the next version of the material.

[0885] Examples and prompts

[0886] For example, if a worker uploads a work instruction screen for a factory using smart glasses, the server analyzes the screen and changes the color combination to an appropriate one based on the user's emotional state while taking into consideration color blindness. The user can provide feedback, and the server will reflect this information in subsequent document generation.

[0887] Example prompts for generative AI models

[0888] "Please change the current color combination of this work instruction screen to one that is easier for people with color blindness to see, and convert it into an instruction screen for barrier-free monitoring that aims to reduce stress using emotional data. Please change the current red and blue to dark orange and navy blue."

[0889] In this way, the present invention makes it possible to provide a work instruction screen that is not only easy to view for people with color blindness, but also takes into consideration the emotional state of the worker.

[0890] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0891] Step 1: The user uploads an existing work instruction screen to the server through the smart glasses. The input is image data of the work instruction screen, and the output is the image data received by the server. Specifically, the worker captures the work instruction screen with the camera on the smart glasses and sends the data to the server.

[0892] Step 2: The server receives the uploaded work instruction screen and temporarily stores it. The input is the image data of the work instruction screen, and the output is the saved image file. The server stores this data in cloud storage (e.g., AWS S3).

[0893] Step 3: The emotion analysis means recognizes the user's facial expressions and voice and analyzes their emotional state. The input is the user's facial expression images and voice data, and the output is the analysis result, which is emotion data. Emotion analysis is performed using the camera and microphone of the smart glasses and Microsoft Azure's Emotion API.

[0894] Step 4: The server analyzes the received work instruction screen and extracts the color-enhanced parts. The input is an image file, and the output is the data of the color-enhanced parts. An image analysis library such as OpenCV is used to detect the color regions of the image.

[0895] Step 5: The server references a database of color blindness data to obtain the color combinations that are difficult and easy for color blind people to see. The input is the color highlight data, and the output is the combinations of difficult and easy-to-see colors. The server then executes a database query to obtain the required information.

[0896] Step 6: The generative AI model uses the color enhancement data, information about color blindness, and emotion data to generate a color-blind-friendly work instruction screen. The input is the color enhancement area, data on difficult and easy-to-see colors, and emotion data, and the output is the modified work instruction screen. Specific prompt sentences are used to issue instructions to the generative AI model (e.g., OpenAI GPT-4).

[0897] Step 7: The server generates a modified work instruction screen and sends it to the user's smart glasses. The input is the modified work instruction screen, and the output is an image file sent to the user's device. Data is transferred using a cloud messaging service (e.g., Firebase Cloud Messaging).

[0898] Step 8: The user performs the task and then provides feedback to the server through the smart glasses. The input is feedback data (text or voice), and the output is the received feedback data. The server analyzes this feedback and uses it to generate materials for future use.

[0899] Step 9: The server uses the feedback data to learn data to further optimize the next document generation. The input is the feedback data, and the output is the learning result. Using a machine learning algorithm (e.g., TensorFlow), important features are extracted from the feedback data and reflected in the next document generation.

[0900] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0901] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0902] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0903] [Fourth embodiment]

[0904] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0905] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0906] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0907] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0908] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0909] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0910] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0911] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0912] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0913] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0914] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0915] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0916] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0917] The system of this invention includes three main elements: a server, a terminal, and a user, each of which plays a specific role to realize the function of automatically generating materials that are considerate of color blindness.

[0918] First, a user uploads existing materials to the server using a terminal, and the terminal provides an interface for sending selected material files to the server.

[0919] The server then receives the uploaded file and temporarily stores it. The server then passes the file to the AI ​​generator, which analyzes the content of the file to determine the color emphasis. This analysis process determines what information each color represents.

[0920] The server then references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. The server then passes this information to the generation AI, which then learns which color combinations are difficult and easy for color-blind people to see. Specifically, the server obtains information from the database that indicates that the combination of red and green is difficult to see, and that dark orange and navy blue are easy to see.

[0921] After the learning is complete, the user requests the generation of color-blind-friendly materials from their device. When the server receives this request, it instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. Based on the learning results, the generation AI appropriately converts the colors of the materials and generates materials with a new color scheme.

[0922] The server saves the new color file and sends it to the user's device. The device receives the new file and allows the user to download it. This provides materials that can be accurately understood by color-blind people.

[0923] As a concrete example, let's take a sales report. When a user uploads a sales report from their device to the server, the server passes the report to a generation AI, which analyzes the negative aspects indicated in red and the positive aspects indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. As a result, the converted document is sent to the user's device, and the user can download a color-blind-friendly sales report.

[0924] The processing flow will be explained below.

[0925] Step 1:

[0926] The user selects a file on the device and clicks the upload button. The device sends the selected file to the server.

[0927] Step 2:

[0928] The server receives the uploaded file and temporarily stores it.

[0929] Step 3:

[0930] The server passes the saved document file to the generation AI, which analyzes the content of the document and determines the color emphasis. Specifically, it analyzes whether red indicates negative elements and blue indicates positive elements.

[0931] Step 4:

[0932] The server references a database about color blindness and obtains information about color combinations that are difficult for color-blind people to see. Specifically, it checks whether the combination of red and green is difficult for color-blind people to see.

[0933] Step 5:

[0934] The server passes the acquired color combination information that is difficult for color-blind people to see to the generation AI, which then learns from it.

[0935] Step 6:

[0936] The server references a database of colors that are easy for colorblind people to see and obtains color combinations that are easy to see. Specifically, it checks whether deep orange and navy blue are easy to see.

[0937] Step 7:

[0938] The server passes the acquired information on easy-to-see color combinations to the generation AI, which then learns from it.

[0939] Step 8:

[0940] The user sends a request for generating color-blind friendly materials from the device.

[0941] Step 9:

[0942] The server instructs the AI ​​to change the color scheme of existing documents to be color-blind-friendly. The AI ​​then appropriately converts the colors of the documents based on the learning results. Specifically, it changes red to dark orange and blue to navy blue.

[0943] Step 10:

[0944] The server saves the converted new color scheme data file.

[0945] Step 11:

[0946] The server sends the new file to the user's device, and the user receives the new file and can download it.

[0947] Example 1

[0948] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0949] Conventional document creation methods can make it difficult for color-blind people to accurately understand information. Specifically, in documents where color emphasis is important, color-blind people have difficulty distinguishing between colors, which can lead to communication problems. Furthermore, manually adjusting color schemes is cumbersome and requires specialized knowledge, making it inefficient. To solve these issues, a system that automatically generates documents that take color-blind people into consideration is needed.

[0950] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0951] In this invention, the server includes means for uploading existing materials to the server using a terminal, means for temporarily storing received material files on the server, means for passing the materials stored by the server to a generation AI and analyzing the contents of the materials, means for referencing a database on color deficiency to obtain information on color combinations that are difficult or easy for color-blind people to see, means for instructing the generation AI based on the obtained information to change the color scheme of the materials to colors that are easy for color-blind people to see, means for the generation AI to generate color-converted materials, and means for temporarily storing the changed materials on the server and sending them to the terminal. This makes it possible to efficiently and automatically generate materials with color schemes that are easy for color-blind people to see.

[0952] A "terminal" is an information device that a user uses to upload materials to a server or download generated materials.

[0953] The "server" is a computer system that receives and temporarily stores document files and interacts with the generating AI and database.

[0954] "Document files" are existing documents and reports that users upload to the server.

[0955] "Generative AI" is a program that uses artificial intelligence to analyze the contents of documents and change the color scheme for people with color blindness.

[0956] "Analysis" is the process in which the generating AI reads the contents of the document file and extracts color information.

[0957] "Color accents" refer to colors used to highlight specific information within a document.

[0958] The "color blindness database" is a database that stores information on color combinations that are difficult or easy for color-blind people to see.

[0959] "Color conversion" is the process of changing the colors in existing materials to make them easier to see for color-blind people.

[0960] The "modified document file" refers to the new document generated by the generating AI with the color scheme changed to suit color-blind people.

[0961] "Upload" refers to the act of a user sending a document file to a server via a terminal.

[0962] "Downloading" refers to the act of a user receiving a modified material file from a server via a terminal.

[0963] The system of the present invention includes a terminal, a server, a generative AI model, and a database related to color blindness, and provides a method and means for automatically generating materials that are considerate of color blindness.

[0964] A user uses a terminal to upload existing documents to the server. At this time, the terminal selects the document file and provides an interface for uploading. For example, a user selects a sales report file (e.g., report.pdf) and clicks the "Upload" button.

[0965] The server temporarily stores the document files received from the user. The stored document files are then analyzed by the generative AI model. The generative AI model analyzes the content of the document, identifies color emphasis, and determines what information each color represents.

[0966] Next, the server references a database on color blindness to obtain information on color combinations that are difficult or easy for color-blind people to see. For example, it obtains information from the database that "the combination of red and green is difficult to see" and "dark orange and navy blue are easy to see."

[0967] Based on the acquired information, the server issues instructions to the generative AI model to change the color scheme of the document to make it easier for people with color blindness to see. Based on the results of the analysis and learning, the generative AI model appropriately converts the colors of the document and generates a document with a new color scheme.

[0968] The generated modified document file is temporarily stored on the server and then sent to the user's device. The user can then use the device to receive and download the new document file. For example, the user can download the generated report_processed.pdf file.

[0969] Examples of specific prompts include requests such as "Please convert the red and blue colors in this report to color-blind-friendly colors" or "Please convert this sales report so that the reds and greens are easy to see."

[0970] As described above, this invention provides a system that can efficiently and automatically generate documents with color schemes that are easy to view for people with color blindness. By utilizing a generative AI model and a database on color blindness, it is possible to provide high-quality documents that take color blindness into consideration while saving time and effort.

[0971] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0972] Step 1:

[0973] User uploads materials

[0974] A user uses a terminal to select an existing document file and upload it to the server. As a concrete example, a user selects a sales report file (e.g., report.pdf) and clicks the "Upload" button. The input is the document file selected by the user, and the output is that file being sent to the server.

[0975] Step 2:

[0976] The server receives the data and temporarily stores it

[0977] The server receives the file uploaded by the user and temporarily stores it. The server stores the received file in the / uploads directory, for example. The input is the file sent, and the output is the file stored on the server.

[0978] Step 3:

[0979] The server passes the data to the generated AI for analysis.

[0980] The server passes the saved document file to the generation AI, which then analyzes the contents of the document. The generation AI extracts the colors used in the document and analyzes which colors represent which information. The input is the saved document file, and the output is the analysis results, including the document's color information. As a concrete example, a prompt such as "Analyze report.pdf and extract the color information within the document" is sent to the generation AI.

[0981] Step 4:

[0982] The server consults a database of color blindness information

[0983] The server references a database on color blindness and retrieves information about color combinations that are difficult and easy for color-blind people to see. The input is a query to the database, and the output is information about color combinations that are difficult and easy for color-blind people to see. For example, the server retrieves information from the database that "the combination of red and green is difficult to see" and "dark orange and navy blue are easy to see."

[0984] Step 5:

[0985] The server instructs the AI ​​to convert colors based on the learning results.

[0986] The server instructs the generation AI to convert the color of the material based on information obtained from a database on color blindness. The input is the analysis result and information from the database, and the output is a color conversion instruction. As a concrete example, the prompt statement "Convert the red in the analysis result to dark orange and the blue to navy blue" is sent to the generation AI.

[0987] Step 6:

[0988] Generative AI generates color-converted materials

[0989] The generation AI changes the color of the document based on instructions from the server and generates a document with a new color scheme. The input is the color conversion instruction, and the output is the document after the color change. The generation AI receives the instructions and generates a new document in which the red parts of report.pdf are converted to dark orange and the blue parts to navy blue.

[0990] Step 7:

[0991] The server saves the new material and sends it to the device.

[0992] The server temporarily stores the new material generated by the AI ​​and sends it to the user's device. The server stores the new material in, for example, the / processed directory, and sends a notification and a download link to the user's device. The input is the new material generated, and the output is to store and send the material.

[0993] Step 8:

[0994] User downloads new material

[0995] The user receives and downloads new documents using a device. The user clicks on a notification or link to download report_processed.pdf, which is generated with the new color scheme. The input is the download request from the device, and the output is the downloaded new documents.

[0996] The above processing steps make it possible to efficiently and automatically generate materials with color schemes that are easy to see even for color-blind people.

[0997] (Application example 1)

[0998] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0999] There is a need for a system that can change the colors of existing documents and video content to be color-blind-friendly so that people with color blindness can accurately understand the information. Analyzing the colors of each frame and converting them into a color scheme that is considerate to people with color blindness is a particularly difficult technical challenge in the case of video content. A system that can solve this problem is needed.

[1000] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1001] In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and grasping color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for changing the materials to a color-blind-friendly color scheme based on the learning results, means for receiving video content and analyzing the colors of each frame, means for generating and transmitting the changed materials, means for converting the colors of each video frame to a color-blind-friendly color scheme, and means for generating and transmitting the converted video. This makes it possible to provide materials and video content that even color-blind people can accurately understand.

[1002] "Existing materials" refers to documents and data files that were created in the past and are currently available.

[1003] "Color emphasis" refers to the frequency and prominence of a particular color.

[1004] "Color blind" refers to people who have difficulty distinguishing colors.

[1005] A "database" is a collection of data that is systematically organized and stored in a form that is easily accessible and searchable.

[1006] "Video Content" refers to multimedia files containing video and audio.

[1007] A "frame" refers to a single still image in a video.

[1008] "Learning results" refers to the knowledge and patterns that a generative AI model has acquired based on training data.

[1009] A "prompt sentence" refers to an instruction or input sentence to a generative AI model.

[1010] "Receiving" refers to the act of taking in data or information from outside.

[1011] "Analysis" refers to the process of examining and understanding data or information.

[1012] "Modification" refers to the act of changing something that already exists into a different state or form.

[1013] "Downloading" refers to the act of importing data from a remote location, such as a server, into your own device.

[1014] The system for implementing this invention consists of three main components: a server, a terminal, and a user. This system is designed to automatically convert existing materials and video content into color-blind-friendly color schemes.

[1015] First, a user uses a terminal to upload existing materials or video content to a server, and the terminal provides an interface for transmitting selected material files or video content to the server.

[1016] The server receives and temporarily stores the uploaded files and video content. The server then passes the files and video content to the generative AI model, which analyzes the files and each frame to understand color emphasis. This analysis process determines which information or scene each color represents.

[1017] The server then references a database of color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see.The generative AI model then uses this information to learn which color combinations are difficult and easy for color-blind people to see.

[1018] After the learning process is complete, the user can request the generation of color-blind-friendly materials or video conversions from their device. When the server receives this request, it instructs the generative AI model to change the color scheme of each frame of the existing materials or video to be color-blind-friendly. Based on the learning results, the generative AI model appropriately converts the colors of the materials or videos and generates materials or videos with the new color scheme.

[1019] The document files and video content created with the new color scheme are stored on the server and sent to the user's device. The user can then receive and download the new document files and video content. In this way, documents and video content are provided that even color-blind people can accurately understand.

[1020] As a concrete example, let's take the following prompt sentence as input to a generative AI model:

[1021] Analyze the uploaded video and identify the color emphasis of each frame. To accommodate color-blind users, convert red to orange and blue to navy blue. Combine the converted frames to create a new video file.

[1022] The hardware used is a smartphone (iOS or Android) or a high-performance server (e.g., NVIDIA Tesla T4), and the software used is a cloud-based generative AI model (e.g., Google Cloud AI, AWS SageMaker), which makes it possible to provide materials and video content that can be accurately understood even by people with color blindness.

[1023] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1024] Step 1:

[1025] A user uses a terminal to select existing materials or video content and upload it to the server. The terminal provides an interface for sending the selected file to the server. The input is the material file or video content selected by the user, and the output is the file sent to the server.

[1026] Step 2:

[1027] The server receives document files and video content uploaded by users and temporarily stores them. The input here is the uploaded file, and the output is the temporarily stored file. Specifically, the server analyzes the file reception request and stores the file in the appropriate storage.

[1028] Step 3:

[1029] The server passes the document files and video content to the generative AI model for analysis. The generative AI model processes the documents and the color emphasis of each frame. The input is the saved file, and the output is the analysis result including color emphasis information. In this step, the generative AI model receives the file as input and performs data analysis to extract color characteristics.

[1030] Step 4:

[1031] The server references a database about color blindness to obtain information about difficult and easy-to-see color combinations. The input is a database query, and the output is color information relevant to color blind people. In this step, the server executes the database query to obtain the required color information.

[1032] Step 5:

[1033] The generative AI model learns combinations of colors that are difficult and easy for colorblind people to see based on color information obtained from a database. The input is color information obtained from the database, and the output is the learning result. In this step, the generative AI model establishes color conversion rules using a learning algorithm.

[1034] Step 6:

[1035] A user requests the generation of color-blind-friendly materials or video conversion. The device sends the request to the server. The input is the user's request, and the output is the requested data sent to the server.

[1036] Step 7:

[1037] The server receives the user's request and issues instructions to the generative AI model. The generative AI model processes the analyzed materials and each frame of the video to change the color scheme to be color-blind-friendly. The input is instructions to the generative AI model and the analyzed data, and the output is the color-changed materials and video frames. In this step, the generative AI model performs the color conversion process based on the prompt text.

[1038] Step 8:

[1039] The generative AI model generates new materials and videos based on the learning results. The generated materials and videos are stored on the server. The input is the color-changed data, and the output is the newly generated file. In this step, the generative AI model completes the conversion process and creates the product.

[1040] Step 9:

[1041] The server sends new document files or video content to the user's device. The input is the new file that has been saved, and the output is the data to be sent to the user's device. Specifically, the server generates a file transfer request and delivers the file to the user's device.

[1042] Step 10:

[1043] The user receives and downloads new material files or video content on their device. The input is the file sent from the server, and the output is the new material or video stored on the device. In this step, the user confirms the receipt of the file and downloads it to local storage.

[1044] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1045] The system of this invention includes a server, a terminal, a user, and an emotion engine, each of which plays a specific role to realize the function of automatically generating materials that are considerate of color blindness while taking into account the user's emotions.

[1046] First, a user uploads existing materials to the server using a device. The device provides an interface for sending selected material files to the server, and the emotion engine analyzes the user's facial expressions, voice, or text input to recognize the user's emotions.

[1047] The server then receives the uploaded file and temporarily stores it. The server then passes the file to the AI ​​generator, which analyzes the content of the file to determine the color emphasis. This analysis process determines what information each color represents.

[1048] The server then references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. The server then passes this information to the generation AI, which then learns which color combinations are difficult and easy for color-blind people to see. Specifically, the server obtains information from the database that indicates that the combination of red and green is difficult to see, and that dark orange and navy blue are easy to see.

[1049] After the learning is complete, the user requests the generation of color-blind-friendly materials from their device. When the server receives this request, it instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. The emotion engine then works again to reconfirm the user's emotions. If the user is feeling stressed or dissatisfied, the generation AI further adjusts the color scheme to match the color scheme that the user finds most comfortable.

[1050] Based on the learning results and the user's emotional information, the generative AI appropriately converts the colors of the materials and generates materials with new color schemes. Specifically, it changes red to dark orange and blue to navy blue, selecting a color scheme that the user finds most appealing.

[1051] The document file generated with the new color scheme is stored by the server and sent to the user's device. The user can then receive and download the new document file on their device. The emotion engine also has a feedback function that allows the user to input their satisfaction with the generated document, allowing the engine to provide a more appropriate color scheme the next time a document is generated.

[1052] As a concrete example, let's take a sales report. When a user uploads a sales report from their device to the server, the emotion engine analyzes the user's input and facial expressions to determine their current emotional state. The server then passes the report to the generation AI, which analyzes the negative elements indicated in red and the positive elements indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. The converted document is then sent to the user's device, where it is available for download. The user can also provide feedback about the document to the emotion engine, which can then further improve the document generation the next time.

[1053] This will not only enable color-blind people to accurately understand information, but will also create a system that provides barrier-free materials tailored to the user's emotions.

[1054] The processing flow will be explained below.

[1055] Step 1:

[1056] The user selects a file on their device and clicks the upload button. The device then sends the selected file to the server. At this time, the emotion engine captures the user's facial expressions with a webcam and analyzes their voice tone and text input to recognize their emotional state.

[1057] Step 2:

[1058] The server receives the uploaded file and temporarily stores it. At the same time, the emotion engine analyzes the acquired user emotion data and determines the user's current emotional state (e.g., stress, dissatisfaction, satisfaction, etc.).

[1059] Step 3:

[1060] The server passes the saved document file to the generation AI, which analyzes the content of the document and identifies color emphasis. Specifically, it identifies negative elements shown in red and positive elements shown in blue.

[1061] Step 4:

[1062] The server references a database of color blindness information to obtain information about color combinations (e.g., red and green) that are difficult for color-blind people to see. The server passes this information to the generation AI.

[1063] Step 5:

[1064] The server references a database of colors that are easy for colorblind people to see and obtains information about color combinations that are easy to see (e.g., dark orange and navy blue). The server passes this information to the generation AI, which then learns from it.

[1065] Step 6:

[1066] The user sends a request for generating color-blind-friendly documents from their device. The emotion engine again captures the user's facial expressions and voice to confirm their emotional state.

[1067] Step 7:

[1068] The server instructs the AI ​​to change existing materials to a color-blind-friendly color scheme. The AI ​​then converts the materials based on its learning results and the user's latest emotional information. For example, if the user is feeling stressed, it will adjust the color tone by softening it.

[1069] Step 8:

[1070] The AI ​​generates a file containing the new color scheme, specifically changing red to dark orange and blue to navy blue.

[1071] Step 9:

[1072] The server stores the converted data files.

[1073] Step 10:

[1074] The server sends the new material file to the user's terminal, and the terminal receives the new material file and allows the user to download it.

[1075] Step 11:

[1076] The device asks the user for feedback on new materials. The user provides the feedback through the emotion engine. The emotion engine analyzes this feedback and stores the data to be reflected in future material generation.

[1077] This provides color-blind-friendly materials that are optimized for the user's emotions.

[1078] Example 2

[1079] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1080] Existing materials are often difficult for people with color blindness to view, especially when highlighting colors. Furthermore, the readability and comprehension of materials are also affected by the user's emotional state. However, existing systems lack the technology to automatically generate materials that take into account the readability of color blind people and the user's emotional state. Therefore, there is a need for a method of generating materials that provides materials that are easy to view for people with color blindness and that also takes into account the user's emotional state.

[1081] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1082] In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and grasping color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for recognizing and analyzing user emotions, means for changing the materials to a color-blind-friendly color scheme based on the learning results and user emotion information, and means for generating and transmitting the changed materials. This enables the automatic generation of materials that are easy for color-blind people to see and that take user emotions into consideration.

[1083] The "means for receiving existing materials" is a system component that allows the server to receive existing documents or data provided by the user.

[1084] "Means for analyzing received materials and understanding color emphasis" is a function for analyzing the use of colors and the importance of each color in received materials and understanding how they are emphasized in the materials.

[1085] "Means for learning color combinations that are difficult for color-blind people to see" is a function that collects information about color combinations that are difficult for color-blind people to see from a database, etc., and uses that information for learning.

[1086] "Means for learning colors that are easy for color-blind people to see" is a function that collects information from a database, etc. about color combinations that are easy for color-blind people to see, and uses that information for learning.

[1087] "Means for recognizing and analyzing user emotions" refers to a function that uses the device's camera, microphone, and text input interface to analyze the user's facial expressions, voice, and text, and recognize the user's current emotional state.

[1088] "Means for changing the color scheme of documents to be color-blind-friendly based on the learning results and the user's emotional information" is a function for changing the color scheme of documents to be color-blind-friendly based on the acquired learning results and the user's emotional information.

[1089] The "means for generating and transmitting modified materials" is a function for generating new materials with a modified color scheme and transmitting them to the user's terminal.

[1090] A "color blindness database" is an information source that stores information about color combinations that are difficult or easy for color blind people to see.

[1091] "Means for receiving user feedback and reflecting it in subsequent material generation" is a function for receiving evaluations and opinions on the generated materials provided by users and using them to help with subsequent material generation processes.

[1092] The system of this invention includes a server, a terminal, a user, and an emotion engine. Each component plays a specific role and realizes the function of automatically generating materials that are color-blind-friendly, taking into account the user's emotions. The specific operation of each component is described below.

[1093] Server Operation

[1094] The server first receives and temporarily stores existing documents uploaded by users on their devices. After receiving the documents, the server passes them to the AI ​​generator, which analyzes their contents. The AI ​​then identifies highlighted areas and determines what information each color represents. For example, red often indicates a warning, while blue indicates positive information.

[1095] Next, the server references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. This includes information such as the combination of red and green being difficult to see, and that dark orange and navy blue are easy to see. Based on this information, it instructs the generation AI on color-conscious learning.

[1096] Based on the learning results, the generative AI changes the color scheme of the document to be color-blind-friendly. Specifically, it changes red to dark orange and blue to navy blue. This makes the document easier to read for people with color blindness.

[1097] Furthermore, the server receives a request from the user to generate a document and instructs the AI ​​to reconfigure the existing document to have a color-blind-friendly color scheme. The emotion engine then reconfirms the user's emotions, and if the user feels stressed or dissatisfied, the AI ​​further adjusts the color scheme. The generated new document is then saved by the server and sent to the user's device.

[1098] Device behavior

[1099] The device provides an interface for users to upload existing documents. The emotion engine uses the device's camera, microphone, and text input interface to analyze the user's facial expressions, voice, and text to recognize their emotions. Emotional information is also sent to the server at the same time as the documents are uploaded.

[1100] The device receives the new color scheme file sent from the server and allows the user to download it. The emotion engine also has a feedback function that allows the user to input their satisfaction with the generated material.

[1101] User operations

[1102] The user uses the device to select an existing document and begin uploading it. For example, they select a sales report file and send it to the server. At this time, the user inputs facial expressions, voice, and text using the device's camera and microphone, and their emotional state is recognized.

[1103] Once new materials are generated and sent from the server to the user's device, the user can download them, check their content, and enter feedback such as their satisfaction level and requests through the device, which will help improve future material generation.

[1104] Specific examples

[1105] Take a sales report as an example. When a user uploads a sales report from their device to the server, the emotion engine analyzes the user's input and facial expressions to determine their current emotional state. The server then passes the report to the generation AI, which analyzes the negative aspects indicated in red and the positive aspects indicated in blue. Based on information obtained from a database on color blindness, the generation AI converts the document to use dark orange instead of red and navy blue instead of blue. The converted document is then sent to the user's device, where it is available for download. The user can also provide feedback on the document to the emotion engine, which can then further improve the document generation the next time.

[1106] Prompt Sentence Examples

[1107] "Please change the red parts of the following materials to dark orange and the blue parts to navy blue. Also, please adjust the colors to reflect the user's current feelings."

[1108] By implementing this invention, not only can color-blind people accurately understand information, but also barrier-free materials that match the user's emotions can be provided.

[1109] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1110] Step 1:

[1111] The user uses the terminal to select an existing document and start uploading. The operation is performed by selecting the document file and clicking the upload button in the terminal. The input is the user's existing document file, and the output is the document file sent to the server.

[1112] Step 2:

[1113] The device sends the selected data file to the server. The device uploads the data file, and at the same time the emotion engine analyzes the user's facial expressions, voice, and text using the camera, microphone, and text input interface. The input is the data file and the user's facial expressions, voice, and text, and the output is the data file and emotion data sent to the server.

[1114] Step 3:

[1115] The server receives the uploaded data files and temporarily stores them. The input is the data files and emotion data sent from the device, and the output is the saved data files. This storage makes the data available for subsequent processing.

[1116] Step 4:

[1117] The server passes the document file to the generation AI and requests it to analyze it. Specifically, it sends the AI ​​a prompt saying, "Analyze the colors in this document and identify which parts are emphasized." The input is the saved document file and the prompt, and the output is the analysis result of the highlighted color parts.

[1118] Step 5:

[1119] The generative AI receives the document, scans every page, and analyzes the use of color. For example, it identifies areas where red indicates a warning and blue indicates positive information. It returns this information to the server. The input is the document file, and the output is the color emphasis analysis results.

[1120] Step 6:

[1121] The server references a database on color blindness to obtain information on color combinations that are difficult and easy for color-blind people to see. For example, it obtains information on the combination of red and green that is considered difficult to see. The input is a database lookup query, and the output is information on color combinations that are difficult and easy to see.

[1122] Step 7:

[1123] The server passes the acquired information about color blindness to the generation AI, which then learns how to use colors in a way that is considerate to color-blind people. The input is information about combinations of difficult-to-see and easy-to-see colors, and the output is the learned color-conscious model. Specifically, the generation AI learns how to convert red to dark orange and green to light blue.

[1124] Step 8:

[1125] The user requests the generation of color-blind-friendly documents from the terminal. The input is a new prompt statement called the document generation request, and the output is instructions to the server.

[1126] Step 9:

[1127] The server instructs the generation AI to change the existing materials to a color-blind-friendly color scheme. The emotion engine then reconfirms the user's emotions. The input is the material generation request and emotion data, and the output is the prompt and the user's emotion information.

[1128] Step 10:

[1129] Based on the learning results and the user's emotional information, the generative AI appropriately converts the colors of the materials and generates materials with new color schemes. Specifically, it changes red to dark orange and blue to navy blue. The input is the trained color consideration model and emotional data, and the output is materials with new color schemes.

[1130] Step 11:

[1131] The data file generated with the new color scheme is saved by the server and sent to the user's device. The input is the data for the new color scheme, and the output is the data sent to the device. This prepares the user to download the new data.

[1132] Step 12:

[1133] Users can receive and download new data files on their devices. They can also input feedback on the data through the emotion engine, which will be used to generate data for future projects. The input is the new data file and the user's feedback, and the output is the feedback data sent to the server.

[1134] (Application example 2)

[1135] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1136] In factories, work instruction screens that do not adequately take into account the visual characteristics and emotional state of workers are causing visual misperception and increased stress, which has a negative impact on work efficiency and safety. To solve this problem, it is necessary to automatically generate appropriate work instruction screens that take into consideration the visual characteristics and emotional state of workers and color-blind people.

[1137] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving existing materials, means for analyzing the received materials and understanding color emphasis, means for learning color combinations that are difficult for color-blind people to see, means for learning colors that are easy for color-blind people to see, means for changing the materials to a color-blind-friendly color scheme based on the learning results, means for generating and transmitting the changed materials, emotion analysis means for recognizing and analyzing the user's emotions, means for adjusting the color scheme of the materials based on the emotion analysis results, and means for automatically generating and displaying a work instruction screen tailored to the visual characteristics of the worker. This prevents visual misinterpretation, reduces worker stress, and enables improved work efficiency and safety.

[1138] "Existing materials" refers to documents, files, etc. that the user has already created.

[1139] "Means for receiving" refers to devices or software that have the function of sending and receiving data from a server.

[1140] "Means for analyzing" refers to a process or device for analyzing received data and understanding its structure and content.

[1141] "Color emphasis" refers to the state in which a particular color is more prominent or emphasized than other colors.

[1142] "Color blind" refers to people who cannot perceive certain colors normally.

[1143] "Difficult-to-see color combinations" refer to color combinations that are difficult for people with color blindness to distinguish.

[1144] "Easy-to-see color combinations" refer to color combinations that are easy to distinguish and highly visible for people with color blindness.

[1145] "Learning tools" refer to the processes and devices used to acquire and apply new knowledge based on past data and experience.

[1146] A "color-blind-friendly color scheme" refers to a color scheme in which the color combinations are adjusted so that even people with color blindness can accurately recognize information.

[1147] "Emotion analysis means" refers to a process or device for recognizing a user's emotions and understanding their state.

[1148] "Adjusting means" refers to a process or device for changing a setting or state based on specific conditions or criteria.

[1149] "Work instruction screen" refers to a display screen for providing specific instructions and information to workers.

[1150] "Display means" refers to devices or software that allow information to be visually confirmed.

[1151] This invention is a system that automatically generates work instruction screens that take into consideration color-blind people and the emotional state of workers in order to improve work efficiency and safety in factories.

[1152] System configuration

[1153] This system mainly includes a server, a terminal, a user, an emotion analysis means, and a generative AI model.

[1154] 1. Server

[1155] The server has the following functions:

[1156] Means of receiving existing materials: The user uploads the work instruction screen from smart glasses or other devices.

[1157] Means for analyzing materials and grasping color emphasis: Analyze the content of the received materials and extract the color emphasis parts.

[1158] A method for learning color combinations that are difficult for color-blind people to see: Color combinations that are difficult for color-blind people to see are learned from a database on color deficiency.

[1159] A way for color-blind people to learn colors that are easy to see: Learn color combinations that are easy to see.

[1160] A method for changing the color scheme of materials to be color-blind-friendly based on the learning results: Based on the analysis results and learning results, the color scheme of materials is changed to be easy for color-blind people to see.

[1161] Means for generating and transmitting modified materials: The materials modified to be color-blind friendly are transmitted to the user terminal.

[1162] Emotion analysis means to recognize and analyze the user's emotions: Analyze the user's emotional state from facial expressions and voice.

[1163] A means for adjusting the color scheme of the document based on the result of the emotion analysis: The color scheme of the document is further adjusted based on the result of the emotion analysis of the user.

[1164] A means for automatically generating and displaying a work instruction screen that is tailored to the visual characteristics of the worker: A work instruction screen optimized for smart glasses, etc. is generated and displayed.

[1165] Hardware and Software

[1166] Smart glasses: Uses a facial recognition camera, microphone, and display.

[1167] Cloud servers: Google Cloud Platform (GCP), Amazon Web Services (AWS), Microsoft Azure, etc.

[1168] Sentiment analysis software: Microsoft Azure Emotion API, IBM Watson, etc.

[1169] Generative AI model: OpenAI's GPT-4.

[1170] Processing Details

[1171] 1. Uploading the work instruction screen: The user uploads the work instruction screen to the server through a device such as smart glasses.

[1172] 2. Emotion analysis: The emotion analysis means recognizes the user's facial expressions and voice and analyzes their current emotional state.

[1173] 3. Document analysis and color emphasis recognition: The server analyzes the uploaded document and identifies the color emphasis areas.

[1174] 4. Learning color correction: Refer to a database on color blindness to learn which color combinations are difficult to see and which are easy to see.

[1175] 5. Generating color-blind-friendly materials: Based on the learning results and sentiment analysis results, the materials are changed to a color-blind-friendly color scheme.

[1176] 6. Generate and send new material: Generate the revised material and send it to the user's device.

[1177] 7. Gather feedback and improve: Use user feedback to improve the next version of the material.

[1178] Examples and prompts

[1179] For example, if a worker uploads a work instruction screen for a factory using smart glasses, the server analyzes the screen and changes the color combination to an appropriate one based on the user's emotional state while taking into consideration color blindness. The user can provide feedback, and the server will reflect this information in subsequent document generation.

[1180] Example prompts for generative AI models

[1181] "Please change the current color combination of this work instruction screen to one that is easier for people with color blindness to see, and convert it into an instruction screen for barrier-free monitoring that aims to reduce stress using emotional data. Please change the current red and blue to dark orange and navy blue."

[1182] In this way, the present invention makes it possible to provide a work instruction screen that is not only easy to view for people with color blindness, but also takes into consideration the emotional state of the worker.

[1183] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1184] Step 1: The user uploads an existing work instruction screen to the server through the smart glasses. The input is image data of the work instruction screen, and the output is the image data received by the server. Specifically, the worker captures the work instruction screen with the camera on the smart glasses and sends the data to the server.

[1185] Step 2: The server receives the uploaded work instruction screen and temporarily stores it. The input is the image data of the work instruction screen, and the output is the saved image file. The server stores this data in cloud storage (e.g., AWS S3).

[1186] Step 3: The emotion analysis means recognizes the user's facial expressions and voice and analyzes their emotional state. The input is the user's facial expression images and voice data, and the output is the analysis result, which is emotion data. Emotion analysis is performed using the camera and microphone of the smart glasses and Microsoft Azure's Emotion API.

[1187] Step 4: The server analyzes the received work instruction screen and extracts the color-enhanced parts. The input is an image file, and the output is the data of the color-enhanced parts. An image analysis library such as OpenCV is used to detect the color regions of the image.

[1188] Step 5: The server references a database of color blindness data to obtain the color combinations that are difficult and easy for color blind people to see. The input is the color highlight data, and the output is the combinations of difficult and easy-to-see colors. The server then executes a database query to obtain the required information.

[1189] Step 6: The generative AI model uses the color enhancement data, information about color blindness, and emotion data to generate a color-blind-friendly work instruction screen. The input is the color enhancement area, data on difficult and easy-to-see colors, and emotion data, and the output is the modified work instruction screen. Specific prompt sentences are used to issue instructions to the generative AI model (e.g., OpenAI GPT-4).

[1190] Step 7: The server generates a modified work instruction screen and sends it to the user's smart glasses. The input is the modified work instruction screen, and the output is an image file sent to the user's device. Data is transferred using a cloud messaging service (e.g., Firebase Cloud Messaging).

[1191] Step 8: The user performs the task and then provides feedback to the server through the smart glasses. The input is feedback data (text or voice), and the output is the received feedback data. The server analyzes this feedback and uses it to generate materials for future use.

[1192] Step 9: The server uses the feedback data to learn data to further optimize the next document generation. The input is the feedback data, and the output is the learning result. Using a machine learning algorithm (e.g., TensorFlow), important features are extracted from the feedback data and reflected in the next document generation.

[1193] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1194] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1195] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1196] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1197] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1198] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1199] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1200] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1201] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1202] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1203] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1204] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1205] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1206] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1207] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1208] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1209] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1210] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1211] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1212] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1213] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1214] The following is further disclosed regarding the above embodiment.

[1215] (Claim 1)

[1216] a means of receiving existing materials;

[1217] a means for analyzing received material and determining color emphasis;

[1218] A means of learning color combinations that are difficult for color-blind people to see,

[1219] A method to learn colors that are easy for color-blind people to see,

[1220] A method to change the color scheme of materials to be color-blind friendly based on the learning results,

[1221] means for generating and transmitting the modified materials;

[1222] A system including:

[1223] (Claim 2)

[1224] A means for referencing a database on color blindness to obtain information on color combinations that are difficult to see;

[1225] A means for obtaining easy-to-see color combination information;

[1226] The system of claim 1 further comprising:

[1227] (Claim 3)

[1228] means for receiving a request for generating a document from a user, generating a modified document file, and transmitting the modified file to the user terminal;

[1229] The system of claim 1 further comprising:

[1230] "Example 1"

[1231] (Claim 1)

[1232] a means for uploading existing materials to the server using the terminal;

[1233] A means for temporarily storing the received document file on the server;

[1234] The server passes the stored data to the AI ​​generator and analyzes the data.

[1235] A means for referencing a database on color deficiency to obtain information on color combinations that are difficult or easy for color deficiency people to see;

[1236] A method to instruct the generation AI based on the acquired information to change the color scheme of the document to colors that are easy to see for color-blind people, and

[1237] A means for the generation AI to generate color-converted materials;

[1238] A means for temporarily storing the changed documents on the server and sending them to the terminal;

[1239] A system including:

[1240] (Claim 2)

[1241] A means for referencing a database relating to color blindness to obtain information about difficult-to-see color combinations;

[1242] A means for obtaining easy-to-see color combination information;

[1243] The system of claim 1 further comprising:

[1244] (Claim 3)

[1245] means for receiving a request for generating a document from a user, generating a modified document file, and transmitting the modified file to the user terminal;

[1246] The system of claim 1 further comprising:

[1247] "Application Example 1"

[1248] (Claim 1)

[1249] a means of receiving existing materials;

[1250] a means for analyzing received material and determining color emphasis;

[1251] A means of learning color combinations that are difficult for color-blind people to see,

[1252] A method to learn colors that are easy for color-blind people to see,

[1253] A method to change the color scheme of materials to be color-blind friendly based on the learning results,

[1254] means for receiving video content and analyzing the color of each frame;

[1255] means for generating and transmitting the modified materials;

[1256] A way to convert the colors of each video frame to a color-blind friendly color scheme,

[1257] means for generating and transmitting the converted video;

[1258] A system including:

[1259] (Claim 2)

[1260] A means for referencing a database on color blindness to obtain information on color combinations that are difficult to see;

[1261] A means for obtaining easy-to-see color combination information;

[1262] The system of claim 1 further comprising:

[1263] (Claim 3)

[1264] means for receiving a request for generating a document from a user, generating a modified document file, and transmitting the modified file to the user terminal;

[1265] means for receiving a video conversion request from a user, generating a modified video, and transmitting the modified video to the user terminal;

[1266] The system of claim 1 further comprising:

[1267] "Example 2: Combining Emotion Engines"

[1268] (Claim 1)

[1269] a means of receiving existing materials;

[1270] a means for analyzing received material and determining color emphasis;

[1271] A means of learning color combinations that are difficult for color-blind people to see,

[1272] A method to learn colors that are easy for color-blind people to see,

[1273] means for recognizing and analyzing user emotions;

[1274] A method for changing the color scheme of materials to be color-blind friendly based on the learning results and the user's emotional information.

[1275] means for generating and transmitting the modified materials;

[1276] A system including:

[1277] (Claim 2)

[1278] A means for referencing a database on color blindness to obtain information on color combinations that are difficult to see;

[1279] A means for obtaining easy-to-see color combination information;

[1280] The system of claim 1 further comprising:

[1281] (Claim 3)

[1282] means for receiving a request for generating a document from a user, generating a modified document file, and transmitting the modified file to the user terminal;

[1283] A means for receiving user feedback and reflecting it in subsequent document generation;

[1284] The system of claim 1 further comprising:

[1285] "Application example 2 when combining emotion engines"

[1286] (Claim 1)

[1287] a means of receiving existing materials;

[1288] a means for analyzing received material and determining color emphasis;

[1289] A means of learning color combinations that are difficult for color-blind people to see,

[1290] A method to learn colors that are easy for color-blind people to see,

[1291] A method to change the color scheme of materials to be color-blind friendly based on the learning results,

[1292] means for generating and transmitting the modified materials;

[1293] emotion analysis means for recognizing and analyzing the emotions of a user;

[1294] A method for adjusting the color scheme of documents based on the results of sentiment analysis;

[1295] A system including:

[1296] (Claim 2)

[1297] A means for referencing a database on color blindness to obtain information on color combinations that are difficult to see;

[1298] A means for obtaining easy-to-see color combination information;

[1299] The system of claim 1 further comprising:

[1300] (Claim 3)

[1301] means for receiving a request for generating a document from a user, generating a modified document file, and transmitting the modified file to the user terminal;

[1302] The system of claim 1 further comprising:

[1303] (Claim 4)

[1304] A method for automatically generating and displaying work instruction screens tailored to the visual characteristics of workers is provided.

[1305] The system of claim 1 further comprising: [Explanation of symbols]

[1306] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means of receiving existing materials; a means for analyzing received material and determining color emphasis; A means of learning color combinations that are difficult for color-blind people to see, A method to learn colors that are easy for color-blind people to see, A method to change the color scheme of materials to be color-blind friendly based on the learning results, means for generating and transmitting the modified materials; A system including:

2. A means for referencing a database on color blindness to obtain information on color combinations that are difficult to see; A means for obtaining easy-to-see color combination information; The system of claim 1 further comprising:

3. means for receiving a request for generating a document from a user, generating a modified document file, and transmitting the modified file to the user terminal; The system of claim 1 further comprising:

Images