System

Abstract

A system is provided.SOLUTION: A system comprising: means for providing an interface for inputting an idea or a theme of an investigation target; means for requesting a generative artificial intelligence model to generate an image based on the idea or the theme; means for receiving image data returned from the generative artificial intelligence model; means for providing a user interface for displaying the image data; and means for storing the image data in a state in which a user can edit the image data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In market research, it is necessary to present concept sheets that make it easy for subjects to respond, but finding appropriate images can be time-consuming and labor-intensive. Furthermore, existing image search engines often have copyright issues and lack of image compatibility. This increases the burden on survey participants and reduces the efficiency of the entire survey. [Means for solving the problem]

[0005] The present invention provides a system including means for providing an interface for inputting ideas or themes to be researched, means for requesting a generative AI model to generate an image based on the idea or theme, means for receiving image data returned from the generative AI model, means for providing a user interface for displaying the image data, and means for saving the image data in a user-editable state. This allows the researcher to easily obtain appropriate images and quickly create concept sheets, thereby reducing overall labor hours and improving efficiency.

[0006] The "research subject" refers to the idea, theme, or specific content that is the focus of the research.

[0007] An "interface" is something that provides a screen, input fields, buttons, and other operating means for a user to interact with a system.

[0008] A "generative artificial intelligence model" is an artificial intelligence algorithm or software system that has the ability to generate data such as text or images based on input information.

[0009] The "means for requesting image generation" refers to a function or process for instructing a generative artificial intelligence model to generate an image based on the input information about the subject of investigation.

[0010] "Means for receiving image data" refers to the functions and processes that allow a server or system to receive image data returned from a generative artificial intelligence model.

[0011] The "means for providing a user interface" is a function for providing a screen and operation means for displaying received image data so that the user can check it.

[0012] "Means for saving in an editable format" refers to the function or process of saving image data in a format that allows the user to adjust and edit the image data as needed.

[0013] "Market research" refers to the activity of collecting and analyzing data about a particular market or consumer.

[0014] A "concept sheet" is an explanatory document that visually expresses an idea or theme and is used during market research and product development. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system for generating images of concept sheets used in market research, etc., in which a generative artificial intelligence model automatically generates images in response to user input. A description will be given of an embodiment of this system, including specific examples.

[0037] First, the user accesses the system's homepage through a web browser, which displays a form for inputting ideas or themes to be explored, including input fields and a "Generate" button.

[0038] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters (e.g., user ID, etc.).

[0039] The server analyzes the received data and extracts themes and ideas. Based on this extracted information, the server calls the API of a generative AI model (e.g., DALL-E) to request image generation. The API call requires parameters including the input theme and idea.

[0040] When the generative AI model receives instructions from the server, it generates relevant images based on the input data. The generated image data is sent back to the server in a fixed format (e.g. PNG, JPEG).

[0041] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. This response also includes the image data and other metadata (e.g., creation date and time, related information, etc.) to make it easier for the user to view and confirm the image.

[0042] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image and necessary operation buttons (e.g., regenerate, save, download, etc.), allowing the user to check the generated image.

[0043] The user can check the displayed images and add captions or adjust the layout as necessary. This allows the user to complete a concept sheet that is adjusted to their satisfaction. The completed concept sheet can be saved on the device or printed depending on the purpose of use. The created concept sheet can also be downloaded in PDF or image format.

[0044] This system allows users to efficiently create concept sheets for market research. In particular, by utilizing a generative AI model, appropriate images can be obtained quickly and easily, significantly reducing the amount of work required.

[0045] The processing flow will be explained below.

[0046] Step 1:

[0047] Users open a web browser, access the system's home page, and are presented with a form to input ideas or topics to be researched.

[0048] Step 2:

[0049] The user enters the idea or theme to be researched (e.g., "Design concept for a new smartphone") into the input field and presses the "Generate" button.

[0050] Step 3:

[0051] The device sends the input data (ideas and themes) to the server as an HTTP request, along with any necessary parameters (e.g., input content, user ID, etc.).

[0052] Step 4:

[0053] The server analyzes the received HTTP request and extracts the input themes and ideas.

[0054] Step 5:

[0055] The server calls the API of the generative AI model based on the analyzed data and requests image generation, sending parameters including the theme and idea to the API.

[0056] Step 6:

[0057] The generative AI model generates related images based on the received theme or idea and returns them to the server in the specified format (e.g. PNG, JPEG).

[0058] Step 7:

[0059] The server receives the image data returned from the generative artificial intelligence model and temporarily stores it.

[0060] Step 8:

[0061] The server sends the saved image data to the device as an HTTP response, along with image metadata (e.g., creation date and time, related information, etc.) if necessary.

[0062] Step 9:

[0063] The device analyzes the received image data and displays it in the web browser. At this time, the image display is accompanied by operation buttons (e.g., regenerate, save, download, etc.) so that the user can check the image.

[0064] Step 10:

[0065] The user checks the displayed images and, if necessary, adds captions or adjusts the layout to create the final concept sheet.

[0066] Step 11:

[0067] Users can save the concept sheets they create, print them, or download them in PDF or image format.

[0068] Example 1

[0069] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0070] In conventional market research, visualizing concept sheets and ideas requires a lot of time and effort. Furthermore, there are limitations to manually creating and editing images, making it difficult to quickly obtain appropriate images. This has led to a decrease in efficiency in the early stages of market research and product development.

[0071] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0072] In this invention, the server includes means for a user to input an idea or theme to be researched into an interface, means for transmitting the input idea or theme to the server as an HTTP request, means for the server to analyze the idea or theme and request an image generation from a generative AI model, means for receiving image data returned from the generative AI model, means for transmitting the image data to the user's terminal as an HTTP response, means for the user interface to display the image data, and means for saving the image data in a user-editable state, thereby enabling users to quickly and accurately create concept sheets for market research.

[0073] "User" refers to the entity that utilizes the system to input ideas and themes to be researched and to review and edit the generated images.

[0074] "Interface" refers to elements that include fields for users to enter ideas or themes to be explored and buttons to request image generation.

[0075] An "HTTP request" is a communication protocol for sending data from a user's terminal to a server.

[0076] "Server" refers to a computer system that receives and analyzes data sent by a user, requests image generation from a generative artificial intelligence model, and returns the generated data to the user.

[0077] A "generative artificial intelligence model" refers to an algorithm or computer program that generates relevant images based on an input idea or theme.

[0078] "Image data" refers to data in file format that contains visual information generated by a generative artificial intelligence model.

[0079] "HTTP response" is a communication protocol for sending data from a server to a user's terminal.

[0080] The term "user interface" refers to the screen display and operation means that allow the user to check and edit the generated image data.

[0081] "Saving" refers to the act of storing the generated image data in a state that allows the user to access it later.

[0082] The present invention is a system for generating images of concept sheets used in market research, etc., in which a generative artificial intelligence model automatically generates images in response to user input.

[0083] First, the user accesses the system's homepage through a web browser, which displays a form for inputting ideas or themes to be researched, including input fields and a "Generate" button.

[0084] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters such as the user ID.

[0085] The server analyzes the received data and extracts themes and ideas. Based on this extracted information, the server calls the API of a generative AI model (e.g., DALL-E) to request image generation. The API call requires parameters including the input theme and idea.

[0086] When the generative AI model receives instructions from the server, it generates relevant images based on the input data. The generated image data is sent back to the server in a fixed format (e.g. PNG, JPEG).

[0087] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. This response also includes the image data and other metadata (e.g., creation date and time, related information, etc.) to make it easier for the user to view and confirm the image.

[0088] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image and necessary operation buttons (e.g., regenerate, save, download, etc.), allowing the user to check the generated image.

[0089] The user can check the displayed images and add captions or adjust the layout as necessary. This allows the user to complete a concept sheet that is satisfactory to the user. The completed concept sheet can be saved on the device or printed depending on the purpose of use. The created concept sheet can also be downloaded in PDF or image format.

[0090] For example, if a user enters "Design concept for a new smartphone" as the theme and presses the generate button, the device sends this information to the server. The server then calls DALL-E's API, generates image data based on the theme, and sends it back to the user. The user can then review the generated image, make any necessary adjustments, and finally save or download it.

[0091] Example prompt sentence:

[0092] "Generate modern and futuristic design images based on new smartphone design concepts."

[0093] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0094] Step 1:

[0095] Users access the system's homepage through a web browser. The homepage displays a form containing fields for inputting ideas or themes to be researched and a "Generate" button. When users input a theme and click the "Generate" button, this information becomes input for the next step.

[0096] Step 2:

[0097] The device sends the theme entered by the user to the server as an HTTP request. The input data includes the user ID and the theme entered. When this request is processed, the input data is parsed.

[0098] Step 3:

[0099] The server analyzes the received HTTP request and extracts the input theme or idea. The analyzed data becomes input data to be passed to the generative AI model. Here, the content of the input text is checked and prepared to be passed to the generative AI model as appropriate parameters.

[0100] Step 4:

[0101] The server calls the API of a generative AI model (e.g., DALL-E) to request image generation based on the input theme. It sends an API request containing a prompt and necessary parameters. The input includes analyzed data (themes and ideas), and image data is generated as output.

[0102] Step 5:

[0103] The generative AI model receives an API request from the server and generates relevant images based on the input theme. This process generates image data and sends it back to the server in a certain format (e.g. PNG, JPEG). The generated image is the main output here.

[0104] Step 6:

[0105] The server receives the image data returned from the generative AI model and temporarily stores it. It then sends the stored image data to the user's device as an HTTP response. At this point, the HTTP response contains the image data and metadata (e.g., generation date and time, related information, etc.).

[0106] Step 7:

[0107] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image along with operation buttons such as regenerate, save, and download. The user can check the generated image. Here, the screen display is based on the input image data.

[0108] Step 8:

[0109] The user can check the displayed images and add captions or adjust the layout as necessary. The user can then check the generated images and complete the adjusted concept sheet to their satisfaction. The adjusted concept sheet is obtained as the output.

[0110] Step 9:

[0111] Users can save the completed concept sheet to their device or download it in PDF or image format. Users can perform the respective action by clicking the save or download button, which will result in a file saved as the final output.

[0112] (Application example 1)

[0113] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0114] While image generation technology using artificial intelligence is becoming more widespread, there is still a lack of systems that can quickly create effective advertising visuals. Furthermore, creating advertising materials often requires a lot of time and effort, so there is a growing demand for systems that can handle not only concept images but also editing, saving, and downloading all in one place. Conventional systems have made it difficult for users to efficiently generate and edit visual data for use as advertising materials.

[0115] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0116] In this invention, the server includes means for providing an interface for inputting concepts or themes to be investigated, means for requesting a generative AI model to generate visual data based on the concepts or themes, means for receiving the visual data returned from the generative AI model, means for providing a user interface for displaying the visual data, and means for saving the visual data in a state that allows the user to edit it and for downloading it for use as advertising material, thereby enabling users to quickly and efficiently create advertising visuals and then edit, save, and download them on the spot.

[0117] A "concept" refers to an abstract idea or subject that a user explores.

[0118] "Subject" refers to the specific theme or topic that is the subject of research or image generation.

[0119] "Interface" refers to the UI components that allow users to input concepts and themes, and the method of operating them.

[0120] A "generative artificial intelligence model" refers to an artificial intelligence system that automatically generates relevant visual data based on input concepts or topics.

[0121] "Visual data" refers to images and images generated by generative artificial intelligence models.

[0122] "User interface" refers to UI components that display the generated visual data and allow the user to operate and edit it.

[0123] "Editable" refers to a state in which a user can process the generated visual data, such as by changing the text or layout.

[0124] "Saving" refers to storing the user-edited visual data in a storage device for later reuse.

[0125] "Download" refers to transferring the generated and edited visual data to a terminal and making it available for use.

[0126] This invention is a system for quickly and efficiently creating advertising visuals, which can generate, edit, save, and download visual data using a generative artificial intelligence model based on concepts and themes entered by the user.

[0127] The server has a means for providing an interface for inputting concepts or themes to be researched, a means for requesting the generation of visual data using a generative artificial intelligence model, a means for receiving the returned visual data, and a means for displaying the visual data to the user. In this system, users can generate, edit, and save advertising materials through a smartphone application.

[0128] The hardware required is a smartphone and a server, and the software required includes:

[0129] React Native: Building user interfaces.

[0130] Redux: Application state management.

[0131] Node.js: Server program development.

[0132] Express: API routing.

[0133] DALL-E API: Image generation using generative artificial intelligence models.

[0134] When a user launches the app, a React Native interface is displayed with a field to input a "concept" or "subject." For example, the user enters "advertising campaign for new product launch" and presses the "Generate" button, which sends the information to a Node.js server via an HTTP request.

[0135] The server uses Express to analyze the received data and calls the DALL-E API to generate visual data. The generated visual data is sent back to the server in a specific format (e.g., PNG, JPEG). The server then resends it to the user's smartphone and displays it in a React Native user interface.

[0136] The user interface provides functions for reviewing and editing the generated visual data. For example, users can add ad copy or adjust the layout. After that, users can save the generated ad material to their smartphone by pressing the "Save" or "Download" button and download it as needed.

[0137] For example, the following prompts are used:

[0138] "Generate images for a new smartphone-themed advertising campaign, highlighting the OLED display as a distinctive feature."

[0139] "Generate a visual to advertise a fall fashion sale. Use casual clothing and a background of autumn leaves."

[0140] This system allows users to efficiently generate advertising visuals and edit and save them as needed, significantly reducing the effort and time required to create advertisements.

[0141] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0142] Step 1:

[0143] The user launches the smartphone application.

[0144] Input: None

[0145] Output: Application home screen

[0146] Specific behavior: An application created using React Native starts and displays the user interface.

[0147] Step 2:

[0148] The user inputs the concept or subject to be investigated.

[0149] Input: Concept or subject (e.g., "advertising campaign for new product launch")

[0150] Output: The input data

[0151] What it does: A user enters a concept or subject to research into a React Native form field.

[0152] Step 3:

[0153] The user presses the "Generate" button.

[0154] Input: Concepts and topics typed, button clicks

[0155] Output: HTTP request

[0156] Specific behavior: When the button is clicked, the entered data is sent to the Node.js server as an HTTP request.

[0157] Step 4:

[0158] The server receives the HTTP request and parses the data.

[0159] Input: HTTP request (concept or subject)

[0160] Output: Analyzed data (concepts and themes)

[0161] What it does: Receives requests and parses data using Node.js and Express.

[0162] Step 5:

[0163] The server calls the DALL-E API to request the generation of visual data.

[0164] Input: Analyzed data (concepts and themes)

[0165] Output: Visual data (images)

[0166] Specific operation: The parsed data is sent to the DALL-E API as a prompt, and related visual data (images) are generated.

[0167] Step 6:

[0168] The server receives the generated visual data and returns it to the user's terminal.

[0169] Input: Visual data (images)

[0170] Output: HTTP response (visual data)

[0171] Specific operation: The server receives the visual data returned from the DALL-E API and sends it to the user's device as an HTTP response.

[0172] Step 7:

[0173] The device displays the visual data received.

[0174] Input: HTTP response (visual data)

[0175] Output: The image to be displayed

[0176] What it does: Visual data is displayed in a React Native user interface.

[0177] Step 8:

[0178] The user edits the visual data as needed.

[0179] Input: Editing content (text, layout, etc.)

[0180] Output: Edited visual data

[0181] Specific behavior: The user edits the visual data through the provided user interface.

[0182] Step 9:

[0183] The user saves or downloads the visual data.

[0184] Input: Click the "Save" or "Download" button

[0185] Output: Saved or downloaded file

[0186] Specific behavior: When the user presses the "Save" or "Download" button, the edited visual data is saved or downloaded to the smartphone.

[0187] Through the above processing steps, the user can efficiently generate advertising visuals, and edit, save, and download them as needed.

[0188] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0189] The present invention is a system that dynamically adjusts the interface display content and the style of the generated image according to the user's emotion by combining a system that generates images of concept sheets used in market research, etc. with an emotion engine that recognizes the user's emotion. A description will be given of an embodiment of this system with specific examples.

[0190] First, users access the system's homepage through a web browser and are presented with a form for entering ideas or themes to be researched, which includes input fields and a "Generate" button.

[0191] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters (e.g., user ID, etc.).

[0192] The server analyzes the received data and extracts themes and ideas. The system also incorporates an emotion engine that analyzes the user's emotions and recognizes them from their facial expressions, tone of voice, and input. For example, if the user is using a device with a camera, emotions can be extracted from facial expression analysis and voice input.

[0193] The emotion engine recognizes the user's emotions (e.g., joy, seriousness, impatience, etc.) and adjusts the interface display content and image generation parameters accordingly. For example, if the user is feeling joyful, it can instruct the engine to generate a bright, positive style design.

[0194] The server generates an image by calling the API of the generative AI model based on the analyzed data and data from the emotion engine. At this time, parameters including the user's emotion data along with the theme and idea are sent to the API.

[0195] The generative AI model generates relevant images based on the received theme, idea, and emotional data, and returns them to the server in the specified format (e.g., PNG, JPEG).

[0196] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. If necessary, image metadata (such as the creation date and time, related information, etc.) is also sent.

[0197] The device analyzes the received image data and displays it in the web browser. At this time, the image display is accompanied by operation buttons (regenerate, save, download, etc.) so that the user can check the image.

[0198] The user checks the displayed images and, if necessary, adds captions or adjusts the layout to create the final concept sheet. The emotion engine generates images that match the user's mood, allowing for more satisfying document creation.

[0199] Users can save the concept sheets they create, print them, or download them in PDF or image format. This system allows users to create more sensible and efficient concept sheets for market research. In particular, the combination of a generative AI model and an emotion engine makes it possible to quickly and easily obtain images that reflect the user's emotions, significantly reducing the amount of work required.

[0200] The processing flow will be explained below.

[0201] This invention is a system that dynamically adjusts the interface display content and the style of the generated image according to the user's emotions by combining an emotion engine that recognizes the user's emotions with a system that generates images of concept sheets used in market research, etc. An embodiment of this system will be described below, broken down into processing steps and with specific examples.

[0202] Step 1:

[0203] Users open a web browser, access the system's home page, and are presented with a form to input ideas or topics to be researched.

[0204] Step 2:

[0205] The user enters the idea or theme to be researched (e.g., "Design concept for a new smartphone") into the input field and presses the "Generate" button.

[0206] Step 3:

[0207] The device sends the input data (ideas and themes) to the server as an HTTP request, along with any necessary parameters (e.g., input content, user ID, etc.).

[0208] Step 4:

[0209] The server analyzes the received HTTP request and extracts the input themes and ideas.

[0210] Step 5:

[0211] Using the device's camera and microphone, an emotion engine is activated to recognize emotions from the user's facial expressions, tone of voice, input content, etc.

[0212] Step 6:

[0213] The emotion engine analyzes the recognized emotion data and generates emotion results, for example, recognizing that the user seems to be having fun.

[0214] Step 7:

[0215] The server combines the extracted themes and idea data with the emotion results from the emotion engine, calls the API of the generative AI model, and requests image generation. At this time, the emotion results are also included in the parameters sent.

[0216] Step 8:

[0217] The generative AI model generates relevant images based on the received theme, idea, and emotional data, and returns them to the server in the specified format (e.g., PNG, JPEG).

[0218] Step 9:

[0219] The server receives the image data returned by the generative AI model and temporarily stores it. At this stage, metadata is added as needed.

[0220] Step 10:

[0221] The server sends the saved image data to the device as an HTTP response, along with image metadata (e.g., creation date and time, related information, etc.) if necessary.

[0222] Step 11:

[0223] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image along with necessary operation buttons (regenerate, save, download, etc.).

[0224] Step 12:

[0225] The user checks the displayed images, adds captions as necessary, adjusts the layout, and creates the final concept sheet. The emotion engine generates images that match the user's mood, increasing user satisfaction.

[0226] Step 13:

[0227] Users can save their concept sheets, print them or download them in PDF or image format.

[0228] This system allows users to efficiently create concept sheets for market research. In particular, the combination of a generative AI model and an emotion engine allows users to quickly and easily generate images that reflect their emotions, significantly reducing the amount of work required.

[0229] Example 2

[0230] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0231] Conventional concept sheet creation systems for market research do not take user emotions into account, making it difficult to provide interface display content and generated image styles that match those emotions. This can lead to a decrease in user satisfaction and the completeness of the sheet. Furthermore, conventional systems require manual adjustments to reflect user emotions, which is inefficient.

[0232] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for inputting a theme or idea to be researched, means for requesting a generative AI model to generate an image based on the theme or idea, means for receiving image data returned from the generative AI model, means including an emotion analysis engine for recognizing a user's emotion, means for adjusting the display content of the interface and the style of the generated image based on the emotion data recognized by the emotion analysis engine, and means for saving the image data in a state that can be edited by the user. This makes it possible to dynamically adjust the display content of the interface and the style of the generated image according to the user's emotion, improve user satisfaction, and efficiently create concept sheets for market research.

[0233] "Themes or ideas under investigation" refers to subjects or concepts that are addressed for purposes such as market research or product development.

[0234] "Interface" refers to the screen and input means that users use to access and operate the system.

[0235] A "generative artificial intelligence model" is a type of artificial intelligence that has the ability to generate images from given text or other data.

[0236] "Means for requesting image generation" refers to a method or mechanism for sending specific instructions to an artificial intelligence model to generate an image.

[0237] An "emotion analysis engine" is a software engine that recognizes emotions from a user's facial expressions, voice, input content, etc.

[0238] "Emotion Data" refers to information regarding a user's emotions as recognized by the emotion analysis engine.

[0239] "User interface" refers to the visual and operational elements through which a user interacts with a system.

[0240] "Means for saving in an editable state" refers to a method for saving generated images and concept sheets so that the user can change or modify them later.

[0241] The present invention relates to a system for generating images based on a theme or idea of ​​a research subject, while taking into consideration the emotions of a user. Specific embodiments will be described below.

[0242] This system mainly uses the user's terminal, a server, a generative AI model, and an emotion analysis engine. Each component of the system operates as follows:

[0243] First, the user accesses the system's homepage using a web browser. The homepage provides an interface for entering research topics and ideas. The interface includes a text input field and a "Generate" button. When the user enters a topic, such as "Design concepts for new smartphones," and presses the "Generate" button, the device sends this data to the server. The transmission is made via an HTTP request, and includes the input data along with required parameters (e.g., user ID, etc.).

[0244] The server analyzes the received data and extracts themes and ideas. This analysis includes extracting keywords based on themes and analyzing text. The server also incorporates a sentiment analysis engine that recognizes emotions from the user's facial expressions, tone of voice, and input. If the user is using a device with a camera or microphone, the sentiment analysis engine extracts emotions from input from these devices.

[0245] When the emotion analysis engine recognizes the user's emotions (e.g., joy, seriousness, impatience, etc.), it dynamically adjusts the interface display content and image generation parameters based on the results. For example, if the user seems to be having fun, the emotion analysis engine can instruct the generation of a bright, positive style design.

[0246] The server combines the emotion data obtained from the emotion analysis engine with the analyzed theme data and calls the API of a generative AI model. Examples of the use of generative AI models include OpenAI's DALL-E and MidJourney. When the API is called, a prompt containing emotion data in addition to the theme or idea is sent. For example, a prompt such as "Generate a design concept for a new smartphone in a bright and fun style" is used.

[0247] The generative AI model generates a related image based on the received theme, idea, and emotional data, and returns it to the server in the specified format (e.g., PNG, JPEG). The server receives the generated image data and temporarily stores it. It then sends it to the user's device as an HTTP response. If necessary, image metadata (such as the date and time of creation, related information, etc.) is also sent.

[0248] The device analyzes the received image data and displays it in the user's web browser. At this time, operation buttons (regenerate, save, download, etc.) are also displayed so that the user can check the image. The user can check the displayed image, add captions as needed, adjust the layout, and create the final concept sheet. The emotion analysis engine generates images that match the user's mood, making it possible to create documents that are more satisfying.

[0249] Users can save the concept sheets they create, print them, or download them in PDF or image format. This system enables users to create concept sheets for market research that are tailored to their emotions. In particular, the combination of a generative AI model and an emotion analysis engine makes it possible to quickly and easily obtain images that reflect the user's emotions, significantly reducing the amount of work required.

[0250] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0251] Step 1:

[0252] The user accesses the system's homepage using a web browser. The homepage displays an interface for inputting research topics and ideas. When the user enters "design concept for a new smartphone" in the text input field and presses the "Generate" button, the input data is sent from the device to the server.

[0253] (Input) User input data including themes

[0254] (Output) The HTTP request sent to the server

[0255] Step 2:

[0256] The device sends the submitted data to the server as an HTTP request. This request includes the theme data entered by the user and any necessary parameters (e.g., user ID, etc.). The device performs the appropriate encoding to maintain the integrity of the data.

[0257] (Input) User input data

[0258] (Output) HTTP request to the server

[0259] Step 3:

[0260] The server receives the HTTP request and begins analyzing the data. The analysis process includes extracting themes and ideas, sometimes using natural language processing (NLP) tools. The server extracts keywords based on the themes and stores them in data storage.

[0261] (Input) HTTP request data from the terminal

[0262] (Output) Extracted results of themes and ideas

[0263] Step 4:

[0264] The server uses an emotion analysis engine to recognize emotions from the user's facial expressions, tone of voice, input content, etc. For example, if the user is using a device with a camera, a library such as OpenCV is used to analyze facial expressions. The server saves the emotion analysis results in data storage.

[0265] (Input) Input data from camera and microphone

[0266] (Output) Emotion analysis results

[0267] Step 5:

[0268] The server combines the emotion data obtained from the emotion analysis engine with the analyzed theme data to generate an API request for the generative AI model. For example, the API request might include a prompt such as, "Generate a design for a new smartphone in a bright and fun style."

[0269] (Input) Emotion data and theme data

[0270] (Output) API requests to generative AI models

[0271] Step 6:

[0272] The server calls the API of the generative AI model and requests it to generate an image. After the API request is sent, the model generates an image based on the prompt.

[0273] (Input) API request

[0274] (Output) Generated image data

[0275] Step 7:

[0276] The generative AI model generates an image based on the requested prompt and sends it back to the server in the specified format (e.g. PNG, JPEG). The server receives this data.

[0277] (Input) API request

[0278] (Output) Generated image data

[0279] Step 8:

[0280] The server temporarily stores the generated image data, and then sends this data to the terminal as an HTTP response, including image metadata (such as the generation date and time, related information, etc.) as needed.

[0281] (Input) Generated image data

[0282] (Output) The HTTP response sent to the device

[0283] Step 9:

[0284] The device analyzes the received image data and displays the image in a web browser. The display includes a UI for checking the image (e.g., regenerate, save, download, etc.).

[0285] (Input) HTTP response from the server

[0286] (output) Images and UI displayed in the web browser

[0287] Step 10:

[0288] The user checks the displayed images, adds captions as necessary, and adjusts the layout to create the final concept sheet.

[0289] (Input) Displayed image

[0290] (Output) Concept sheet after user editing

[0291] Step 11:

[0292] Users can save their concept sheets, print them or download them in PDF or image format.

[0293] (Input) User edited concept sheet

[0294] (Output) Saved Concept Sheet file

[0295] (Application example 2)

[0296] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0297] Conventional advertising banner generation tools generate images based on themes entered by the user, but they are unable to provide designs that reflect the user's emotions, making it difficult to generate banners that are optimal for each individual user.In addition, the user interface is fixed, making it impossible to dynamically adjust according to the user's emotions or situation, making it difficult to improve the user experience.

[0298] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for inputting an idea or theme to be investigated, means for requesting image generation from a generative AI model based on the idea or theme and user emotion data, means for receiving image data returned from the generative AI model, means for displaying the image data in a dynamic interface according to the user emotion, and means for saving the image data in a state that can be edited by the user. This makes it possible to dynamically generate advertising banners that reflect the user emotion, improving the user experience.

[0299] The "idea or theme to be investigated" refers to the concept or idea that the user inputs to generate an image based on the generative artificial intelligence model.

[0300] An "interface" is a web page or application screen through which a user inputs an idea or theme and requests image generation.

[0301] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates images based on ideas, themes, and emotional data input by the user.

[0302] "Emotion data" is information about the user's emotional state analyzed from facial expressions, tone of voice, and the like.

[0303] A "dynamic interface" is a flexible user interface whose display content changes depending on the user's emotions and situation.

[0304] "Image data" refers to visual data generated as a result of a generative artificial intelligence model.

[0305] The "means for saving in an editable state" is a function for saving the generated image data in a form that allows the user to re-edit it.

[0306] A "webcam" is a camera device used to capture a user's facial expressions.

[0307] An "emotion recognition engine" is a technology that analyzes emotions from a user's facial expressions and tone of voice and outputs them as data.

[0308] The present invention relates to a system for recognizing a user's emotion and generating an advertising banner based on the emotion. In this embodiment, the system is constructed as follows.

[0309] 1. Provide an interface for users to input ideas and themes to be researched. The interface has a field where users can input themes and press the "Generate" button, as well as a camera and microphone for sentiment analysis.

[0310] 2. When the user inputs an idea or theme through the interface and presses the "Generate" button, the device sends the input data, including the theme and the user's emotional data, to the server.

[0311] 3. The server analyzes the emotion data using an emotion recognition engine that analyzes the user's facial expressions and tone of voice. The emotion recognition engine uses existing face recognition libraries such as OpenCV and Dlib.

[0312] 4. Based on the analyzed emotion data and the theme entered by the user, the server requests the generative AI model to generate an image. The server sends the theme, idea, and emotion data to the generative AI model, which then generates a related image.

[0313] 5. The generative AI model generates an image of the advertising banner based on the received data and sends it back to the server in the specified format (e.g. PNG, JPEG).

[0314] 6. The server sends the generated advertising banner image to the user's device and displays it on the interface so that the user can view it. At this time, a dynamic interface responding to the user's emotions is displayed, with bright colors and relevant operation buttons.

[0315] 7. Users can review the displayed advertising banner and regenerate, save, or edit it as needed. The final banner can be saved in the form desired by the user.

[0316] As a concrete example, suppose a user inputs the theme "Summer Sale Promotion" and smiles at the camera. The system detects a "positive emotion" from the smile and generates a bright, colorful style advertising banner based on this emotion. An example prompt sentence in this scenario is "Theme: Summer Sale Promotion, Emotion: Positive."

[0317] In this way, a system is realized that efficiently generates advertising banners that reflect the user's emotions and supports the creation of advertisements that provide high user satisfaction.

[0318] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0319] Step 1:

[0320] The user inputs the idea or theme to be researched into the interface, and a camera and microphone are used for emotion analysis. The device collects facial expressions captured by the camera and voice recorded by the microphone. The input at this stage is the idea or theme entered by the user, as well as image and audio data for emotion analysis.

[0321] Step 2:

[0322] The device sends the collected data (ideas, themes, facial images, and voice recordings) to the server via HTTP requests. The input data is used by the server for analysis, generating data packets that are sent to the server as output.

[0323] Step 3:

[0324] The server uses an emotion recognition engine (e.g., OpenCV, Dlib) to analyze the received facial expression data and voice data and identify the user's emotions. The input data is facial expression images and voice data, and data processing involves extracting features from facial expressions and analyzing voice tones. The output is emotion data that represents the analyzed user's emotions.

[0325] Step 4:

[0326] The server calls the API of the generative AI model based on the emotion data and the theme entered by the user, requesting image generation. In the API call, the generative AI model receives the theme and emotion data as input. In the data calculation, the model generates a banner image based on the input data. The generated advertising banner image is obtained as the output.

[0327] Step 5:

[0328] The server receives the advertising banner image returned from the generative AI model and sends it to the user's device. At this stage, the banner image is input to the server, and image data is output to the user's device as an HTTP response.

[0329] Step 6:

[0330] The terminal displays the advertising banner image received from the server on the interface, dynamically adjusting the color scheme and display elements of the interface according to the user's emotions. The input data is the received banner image, and the output is displayed on the interface.

[0331] Step 7:

[0332] The user can view the displayed ad banner and regenerate, save, or edit it as needed. During this operation, the user operates the interface to fine-tune the generated banner, and the final banner is saved. The input is the user's operation data, and the output is the final edited ad banner image.

[0333] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0334] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0335] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0336] [Second embodiment]

[0337] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0338] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0339] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0340] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0341] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0342] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0343] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0344] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0345] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0346] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0347] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0348] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0349] The present invention is a system for generating images of concept sheets used in market research, etc., in which a generative artificial intelligence model automatically generates images in response to user input. A description will be given of an embodiment of this system, including specific examples.

[0350] First, the user accesses the system's homepage through a web browser, which displays a form for inputting ideas or themes to be explored, including input fields and a "Generate" button.

[0351] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters (e.g., user ID, etc.).

[0352] The server analyzes the received data and extracts themes and ideas. Based on this extracted information, the server calls the API of a generative AI model (e.g., DALL-E) to request image generation. The API call requires parameters including the input theme and idea.

[0353] When the generative AI model receives instructions from the server, it generates relevant images based on the input data. The generated image data is sent back to the server in a fixed format (e.g. PNG, JPEG).

[0354] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. This response also includes the image data and other metadata (e.g., creation date and time, related information, etc.) to make it easier for the user to view and confirm the image.

[0355] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image and necessary operation buttons (e.g., regenerate, save, download, etc.), allowing the user to check the generated image.

[0356] The user can check the displayed images and add captions or adjust the layout as necessary. This allows the user to complete a concept sheet that is adjusted to their satisfaction. The completed concept sheet can be saved on the device or printed depending on the purpose of use. The created concept sheet can also be downloaded in PDF or image format.

[0357] This system allows users to efficiently create concept sheets for market research. In particular, by utilizing a generative AI model, appropriate images can be obtained quickly and easily, significantly reducing the amount of work required.

[0358] The processing flow will be explained below.

[0359] Step 1:

[0360] Users open a web browser, access the system's home page, and are presented with a form to input ideas or topics to be researched.

[0361] Step 2:

[0362] The user enters the idea or theme to be researched (e.g., "Design concept for a new smartphone") into the input field and presses the "Generate" button.

[0363] Step 3:

[0364] The device sends the input data (ideas and themes) to the server as an HTTP request, along with any necessary parameters (e.g., input content, user ID, etc.).

[0365] Step 4:

[0366] The server analyzes the received HTTP request and extracts the input themes and ideas.

[0367] Step 5:

[0368] The server calls the API of the generative AI model based on the analyzed data and requests image generation, sending parameters including the theme and idea to the API.

[0369] Step 6:

[0370] The generative AI model generates related images based on the received theme or idea and returns them to the server in the specified format (e.g. PNG, JPEG).

[0371] Step 7:

[0372] The server receives the image data returned from the generative artificial intelligence model and temporarily stores it.

[0373] Step 8:

[0374] The server sends the saved image data to the device as an HTTP response, along with image metadata (e.g., creation date and time, related information, etc.) if necessary.

[0375] Step 9:

[0376] The device analyzes the received image data and displays it in the web browser. At this time, the image display is accompanied by operation buttons (e.g., regenerate, save, download, etc.) so that the user can check the image.

[0377] Step 10:

[0378] The user checks the displayed images and, if necessary, adds captions or adjusts the layout to create the final concept sheet.

[0379] Step 11:

[0380] Users can save the concept sheets they create, print them, or download them in PDF or image format.

[0381] Example 1

[0382] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0383] In conventional market research, visualizing concept sheets and ideas requires a lot of time and effort. Furthermore, there are limitations to manually creating and editing images, making it difficult to quickly obtain appropriate images. This has led to a decrease in efficiency in the early stages of market research and product development.

[0384] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0385] In this invention, the server includes means for a user to input an idea or theme to be researched into an interface, means for transmitting the input idea or theme to the server as an HTTP request, means for the server to analyze the idea or theme and request an image generation from a generative AI model, means for receiving image data returned from the generative AI model, means for transmitting the image data to the user's terminal as an HTTP response, means for the user interface to display the image data, and means for saving the image data in a user-editable state, thereby enabling users to quickly and accurately create concept sheets for market research.

[0386] "User" refers to the entity that utilizes the system to input ideas and themes to be researched and to review and edit the generated images.

[0387] "Interface" refers to elements that include fields for users to enter ideas or themes to be explored and buttons to request image generation.

[0388] An "HTTP request" is a communication protocol for sending data from a user's terminal to a server.

[0389] "Server" refers to a computer system that receives and analyzes data sent by a user, requests image generation from a generative artificial intelligence model, and returns the generated data to the user.

[0390] A "generative artificial intelligence model" refers to an algorithm or computer program that generates relevant images based on an input idea or theme.

[0391] "Image data" refers to data in file format that contains visual information generated by a generative artificial intelligence model.

[0392] "HTTP response" is a communication protocol for sending data from a server to a user's terminal.

[0393] The term "user interface" refers to the screen display and operation means that allow the user to check and edit the generated image data.

[0394] "Saving" refers to the act of storing the generated image data in a state that allows the user to access it later.

[0395] The present invention is a system for generating images of concept sheets used in market research, etc., in which a generative artificial intelligence model automatically generates images in response to user input.

[0396] First, the user accesses the system's homepage through a web browser, which displays a form for inputting ideas or themes to be researched, including input fields and a "Generate" button.

[0397] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters such as the user ID.

[0398] The server analyzes the received data and extracts themes and ideas. Based on this extracted information, the server calls the API of a generative AI model (e.g., DALL-E) to request image generation. The API call requires parameters including the input theme and idea.

[0399] When the generative AI model receives instructions from the server, it generates relevant images based on the input data. The generated image data is sent back to the server in a fixed format (e.g. PNG, JPEG).

[0400] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. This response also includes the image data and other metadata (e.g., creation date and time, related information, etc.) to make it easier for the user to view and confirm the image.

[0401] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image and necessary operation buttons (e.g., regenerate, save, download, etc.), allowing the user to check the generated image.

[0402] The user can check the displayed images and add captions or adjust the layout as necessary. This allows the user to complete a concept sheet that is satisfactory to the user. The completed concept sheet can be saved on the device or printed depending on the purpose of use. The created concept sheet can also be downloaded in PDF or image format.

[0403] For example, if a user enters "Design concept for a new smartphone" as the theme and presses the generate button, the device sends this information to the server. The server then calls DALL-E's API, generates image data based on the theme, and sends it back to the user. The user can then review the generated image, make any necessary adjustments, and finally save or download it.

[0404] Example prompt sentence:

[0405] "Generate modern and futuristic design images based on new smartphone design concepts."

[0406] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0407] Step 1:

[0408] Users access the system's homepage through a web browser. The homepage displays a form containing fields for inputting ideas or themes to be researched and a "Generate" button. When users input a theme and click the "Generate" button, this information becomes input for the next step.

[0409] Step 2:

[0410] The device sends the theme entered by the user to the server as an HTTP request. The input data includes the user ID and the theme entered. When this request is processed, the input data is parsed.

[0411] Step 3:

[0412] The server analyzes the received HTTP request and extracts the input theme or idea. The analyzed data becomes input data to be passed to the generative AI model. Here, the content of the input text is checked and prepared to be passed to the generative AI model as appropriate parameters.

[0413] Step 4:

[0414] The server calls the API of a generative AI model (e.g., DALL-E) to request image generation based on the input theme. It sends an API request containing a prompt and necessary parameters. The input includes analyzed data (themes and ideas), and image data is generated as output.

[0415] Step 5:

[0416] The generative AI model receives an API request from the server and generates relevant images based on the input theme. This process generates image data and sends it back to the server in a certain format (e.g. PNG, JPEG). The generated image is the main output here.

[0417] Step 6:

[0418] The server receives the image data returned from the generative AI model and temporarily stores it. It then sends the stored image data to the user's device as an HTTP response. At this point, the HTTP response contains the image data and metadata (e.g., generation date and time, related information, etc.).

[0419] Step 7:

[0420] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image along with operation buttons such as regenerate, save, and download. The user can check the generated image. Here, the screen display is based on the input image data.

[0421] Step 8:

[0422] The user can check the displayed images and add captions or adjust the layout as necessary. The user can then check the generated images and complete the adjusted concept sheet to their satisfaction. The adjusted concept sheet is obtained as the output.

[0423] Step 9:

[0424] Users can save the completed concept sheet to their device or download it in PDF or image format. Users can perform the respective action by clicking the save or download button, which will result in a file saved as the final output.

[0425] (Application example 1)

[0426] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0427] While image generation technology using artificial intelligence is becoming more widespread, there is still a lack of systems that can quickly create effective advertising visuals. Furthermore, creating advertising materials often requires a lot of time and effort, so there is a growing demand for systems that can handle not only concept images but also editing, saving, and downloading all in one place. Conventional systems have made it difficult for users to efficiently generate and edit visual data for use as advertising materials.

[0428] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0429] In this invention, the server includes means for providing an interface for inputting concepts or themes to be investigated, means for requesting a generative AI model to generate visual data based on the concepts or themes, means for receiving the visual data returned from the generative AI model, means for providing a user interface for displaying the visual data, and means for saving the visual data in a state that allows the user to edit it and for downloading it for use as advertising material, thereby enabling users to quickly and efficiently create advertising visuals and then edit, save, and download them on the spot.

[0430] A "concept" refers to an abstract idea or subject that a user explores.

[0431] "Subject" refers to the specific theme or topic that is the subject of research or image generation.

[0432] "Interface" refers to the UI components that allow users to input concepts and themes, and the method of operating them.

[0433] A "generative artificial intelligence model" refers to an artificial intelligence system that automatically generates relevant visual data based on input concepts or topics.

[0434] "Visual data" refers to images and images generated by generative artificial intelligence models.

[0435] "User interface" refers to UI components that display the generated visual data and allow the user to operate and edit it.

[0436] "Editable" refers to a state in which a user can process the generated visual data, such as by changing the text or layout.

[0437] "Saving" refers to storing the user-edited visual data in a storage device for later reuse.

[0438] "Download" refers to transferring the generated and edited visual data to a terminal and making it available for use.

[0439] This invention is a system for quickly and efficiently creating advertising visuals, which can generate, edit, save, and download visual data using a generative artificial intelligence model based on concepts and themes entered by the user.

[0440] The server has a means for providing an interface for inputting concepts or themes to be researched, a means for requesting the generation of visual data using a generative artificial intelligence model, a means for receiving the returned visual data, and a means for displaying the visual data to the user. In this system, users can generate, edit, and save advertising materials through a smartphone application.

[0441] The hardware required is a smartphone and a server, and the software required includes:

[0442] React Native: Building user interfaces.

[0443] Redux: Application state management.

[0444] Node.js: Server program development.

[0445] Express: API routing.

[0446] DALL-E API: Image generation using generative artificial intelligence models.

[0447] When a user launches the app, a React Native interface is displayed with a field to input a "concept" or "subject." For example, the user enters "advertising campaign for new product launch" and presses the "Generate" button, which sends the information to a Node.js server via an HTTP request.

[0448] The server uses Express to analyze the received data and calls the DALL-E API to generate visual data. The generated visual data is sent back to the server in a specific format (e.g., PNG, JPEG). The server then resends it to the user's smartphone and displays it in a React Native user interface.

[0449] The user interface provides functions for reviewing and editing the generated visual data. For example, users can add ad copy or adjust the layout. After that, users can save the generated ad material to their smartphone by pressing the "Save" or "Download" button and download it as needed.

[0450] For example, the following prompts are used:

[0451] "Generate images for a new smartphone-themed advertising campaign, highlighting the OLED display as a distinctive feature."

[0452] "Generate a visual to advertise a fall fashion sale. Use casual clothing and a background of autumn leaves."

[0453] This system allows users to efficiently generate advertising visuals and edit and save them as needed, significantly reducing the effort and time required to create advertisements.

[0454] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0455] Step 1:

[0456] The user launches the smartphone application.

[0457] Input: None

[0458] Output: Application home screen

[0459] Specific behavior: An application created using React Native starts and displays the user interface.

[0460] Step 2:

[0461] The user inputs the concept or subject to be investigated.

[0462] Input: Concept or subject (e.g., "advertising campaign for new product launch")

[0463] Output: The input data

[0464] What it does: A user enters a concept or subject to research into a React Native form field.

[0465] Step 3:

[0466] The user presses the "Generate" button.

[0467] Input: Concepts and topics typed, button clicks

[0468] Output: HTTP request

[0469] Specific behavior: When the button is clicked, the entered data is sent to the Node.js server as an HTTP request.

[0470] Step 4:

[0471] The server receives the HTTP request and parses the data.

[0472] Input: HTTP request (concept or subject)

[0473] Output: Analyzed data (concepts and themes)

[0474] What it does: Receives requests and parses data using Node.js and Express.

[0475] Step 5:

[0476] The server calls the DALL-E API to request the generation of visual data.

[0477] Input: Analyzed data (concepts and themes)

[0478] Output: Visual data (images)

[0479] Specific operation: The parsed data is sent to the DALL-E API as a prompt, and related visual data (images) are generated.

[0480] Step 6:

[0481] The server receives the generated visual data and returns it to the user's terminal.

[0482] Input: Visual data (images)

[0483] Output: HTTP response (visual data)

[0484] Specific operation: The server receives the visual data returned from the DALL-E API and sends it to the user's device as an HTTP response.

[0485] Step 7:

[0486] The device displays the visual data received.

[0487] Input: HTTP response (visual data)

[0488] Output: The image to be displayed

[0489] What it does: Visual data is displayed in a React Native user interface.

[0490] Step 8:

[0491] The user edits the visual data as needed.

[0492] Input: Editing content (text, layout, etc.)

[0493] Output: Edited visual data

[0494] Specific behavior: The user edits the visual data through the provided user interface.

[0495] Step 9:

[0496] The user saves or downloads the visual data.

[0497] Input: Click the "Save" or "Download" button

[0498] Output: Saved or downloaded file

[0499] Specific behavior: When the user presses the "Save" or "Download" button, the edited visual data is saved or downloaded to the smartphone.

[0500] Through the above processing steps, the user can efficiently generate advertising visuals, and edit, save, and download them as needed.

[0501] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0502] The present invention is a system that dynamically adjusts the interface display content and the style of the generated image according to the user's emotion by combining a system that generates images of concept sheets used in market research, etc. with an emotion engine that recognizes the user's emotion. A description will be given of an embodiment of this system with specific examples.

[0503] First, users access the system's homepage through a web browser and are presented with a form for entering ideas or themes to be researched, which includes input fields and a "Generate" button.

[0504] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters (e.g., user ID, etc.).

[0505] The server analyzes the received data and extracts themes and ideas. The system also incorporates an emotion engine that analyzes the user's emotions and recognizes them from their facial expressions, tone of voice, and input. For example, if the user is using a device with a camera, emotions can be extracted from facial expression analysis and voice input.

[0506] The emotion engine recognizes the user's emotions (e.g., joy, seriousness, impatience, etc.) and adjusts the interface display content and image generation parameters accordingly. For example, if the user is feeling joyful, it can instruct the engine to generate a bright, positive style design.

[0507] The server generates an image by calling the API of the generative AI model based on the analyzed data and data from the emotion engine. At this time, parameters including the user's emotion data along with the theme and idea are sent to the API.

[0508] The generative AI model generates relevant images based on the received theme, idea, and emotional data, and returns them to the server in the specified format (e.g., PNG, JPEG).

[0509] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. If necessary, image metadata (such as the creation date and time, related information, etc.) is also sent.

[0510] The device analyzes the received image data and displays it in the web browser. At this time, the image display is accompanied by operation buttons (regenerate, save, download, etc.) so that the user can check the image.

[0511] The user checks the displayed images and, if necessary, adds captions or adjusts the layout to create the final concept sheet. The emotion engine generates images that match the user's mood, allowing for more satisfying document creation.

[0512] Users can save the concept sheets they create, print them, or download them in PDF or image format. This system allows users to create more sensible and efficient concept sheets for market research. In particular, the combination of a generative AI model and an emotion engine makes it possible to quickly and easily obtain images that reflect the user's emotions, significantly reducing the amount of work required.

[0513] The processing flow will be explained below.

[0514] This invention is a system that dynamically adjusts the interface display content and the style of the generated image according to the user's emotions by combining an emotion engine that recognizes the user's emotions with a system that generates images of concept sheets used in market research, etc. An embodiment of this system will be described below, broken down into processing steps and with specific examples.

[0515] Step 1:

[0516] Users open a web browser, access the system's home page, and are presented with a form to input ideas or topics to be researched.

[0517] Step 2:

[0518] The user enters the idea or theme to be researched (e.g., "Design concept for a new smartphone") into the input field and presses the "Generate" button.

[0519] Step 3:

[0520] The device sends the input data (ideas and themes) to the server as an HTTP request, along with any necessary parameters (e.g., input content, user ID, etc.).

[0521] Step 4:

[0522] The server analyzes the received HTTP request and extracts the input themes and ideas.

[0523] Step 5:

[0524] Using the device's camera and microphone, an emotion engine is activated to recognize emotions from the user's facial expressions, tone of voice, input content, etc.

[0525] Step 6:

[0526] The emotion engine analyzes the recognized emotion data and generates emotion results, for example, recognizing that the user seems to be having fun.

[0527] Step 7:

[0528] The server combines the extracted themes and idea data with the emotion results from the emotion engine, calls the API of the generative AI model, and requests image generation. At this time, the emotion results are also included in the parameters sent.

[0529] Step 8:

[0530] The generative AI model generates relevant images based on the received theme, idea, and emotional data, and returns them to the server in the specified format (e.g., PNG, JPEG).

[0531] Step 9:

[0532] The server receives the image data returned by the generative AI model and temporarily stores it. At this stage, metadata is added as needed.

[0533] Step 10:

[0534] The server sends the saved image data to the device as an HTTP response, along with image metadata (e.g., creation date and time, related information, etc.) if necessary.

[0535] Step 11:

[0536] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image along with necessary operation buttons (regenerate, save, download, etc.).

[0537] Step 12:

[0538] The user checks the displayed images, adds captions as necessary, adjusts the layout, and creates the final concept sheet. The emotion engine generates images that match the user's mood, increasing user satisfaction.

[0539] Step 13:

[0540] Users can save their concept sheets, print them or download them in PDF or image format.

[0541] This system allows users to efficiently create concept sheets for market research. In particular, the combination of a generative AI model and an emotion engine allows users to quickly and easily generate images that reflect their emotions, significantly reducing the amount of work required.

[0542] Example 2

[0543] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0544] Conventional concept sheet creation systems for market research do not take user emotions into account, making it difficult to provide interface display content and generated image styles that match those emotions. This can lead to a decrease in user satisfaction and the completeness of the sheet. Furthermore, conventional systems require manual adjustments to reflect user emotions, which is inefficient.

[0545] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for inputting a theme or idea to be researched, means for requesting a generative AI model to generate an image based on the theme or idea, means for receiving image data returned from the generative AI model, means including an emotion analysis engine for recognizing a user's emotion, means for adjusting the display content of the interface and the style of the generated image based on the emotion data recognized by the emotion analysis engine, and means for saving the image data in a state that can be edited by the user. This makes it possible to dynamically adjust the display content of the interface and the style of the generated image according to the user's emotion, improve user satisfaction, and efficiently create concept sheets for market research.

[0546] "Themes or ideas under investigation" refers to subjects or concepts that are addressed for purposes such as market research or product development.

[0547] "Interface" refers to the screen and input means that users use to access and operate the system.

[0548] A "generative artificial intelligence model" is a type of artificial intelligence that has the ability to generate images from given text or other data.

[0549] "Means for requesting image generation" refers to a method or mechanism for sending specific instructions to an artificial intelligence model to generate an image.

[0550] An "emotion analysis engine" is a software engine that recognizes emotions from a user's facial expressions, voice, input content, etc.

[0551] "Emotion Data" refers to information regarding a user's emotions as recognized by the emotion analysis engine.

[0552] "User interface" refers to the visual and operational elements through which a user interacts with a system.

[0553] "Means for saving in an editable state" refers to a method for saving generated images and concept sheets so that the user can change or modify them later.

[0554] The present invention relates to a system for generating images based on a theme or idea of ​​a research subject, while taking into consideration the emotions of a user. Specific embodiments will be described below.

[0555] This system mainly uses the user's terminal, a server, a generative AI model, and an emotion analysis engine. Each component of the system operates as follows:

[0556] First, the user accesses the system's homepage using a web browser. The homepage provides an interface for entering research topics and ideas. The interface includes a text input field and a "Generate" button. When the user enters a topic, such as "Design concepts for new smartphones," and presses the "Generate" button, the device sends this data to the server. The transmission is made via an HTTP request, and includes the input data along with required parameters (e.g., user ID, etc.).

[0557] The server analyzes the received data and extracts themes and ideas. This analysis includes extracting keywords based on themes and analyzing text. The server also incorporates a sentiment analysis engine that recognizes emotions from the user's facial expressions, tone of voice, and input. If the user is using a device with a camera or microphone, the sentiment analysis engine extracts emotions from input from these devices.

[0558] When the emotion analysis engine recognizes the user's emotions (e.g., joy, seriousness, impatience, etc.), it dynamically adjusts the interface display content and image generation parameters based on the results. For example, if the user seems to be having fun, the emotion analysis engine can instruct the generation of a bright, positive style design.

[0559] The server combines the emotion data obtained from the emotion analysis engine with the analyzed theme data and calls the API of a generative AI model. Examples of the use of generative AI models include OpenAI's DALL-E and MidJourney. When the API is called, a prompt containing emotion data in addition to the theme or idea is sent. For example, a prompt such as "Generate a design concept for a new smartphone in a bright and fun style" is used.

[0560] The generative AI model generates a related image based on the received theme, idea, and emotional data, and returns it to the server in the specified format (e.g., PNG, JPEG). The server receives the generated image data and temporarily stores it. It then sends it to the user's device as an HTTP response. If necessary, image metadata (such as the date and time of creation, related information, etc.) is also sent.

[0561] The device analyzes the received image data and displays it in the user's web browser. At this time, operation buttons (regenerate, save, download, etc.) are also displayed so that the user can check the image. The user can check the displayed image, add captions as needed, adjust the layout, and create the final concept sheet. The emotion analysis engine generates images that match the user's mood, making it possible to create documents that are more satisfying.

[0562] Users can save the concept sheets they create, print them, or download them in PDF or image format. This system enables users to create concept sheets for market research that are tailored to their emotions. In particular, the combination of a generative AI model and an emotion analysis engine makes it possible to quickly and easily obtain images that reflect the user's emotions, significantly reducing the amount of work required.

[0563] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0564] Step 1:

[0565] The user accesses the system's homepage using a web browser. The homepage displays an interface for inputting research topics and ideas. When the user enters "design concept for a new smartphone" in the text input field and presses the "Generate" button, the input data is sent from the device to the server.

[0566] (Input) User input data including themes

[0567] (Output) The HTTP request sent to the server

[0568] Step 2:

[0569] The device sends the submitted data to the server as an HTTP request. This request includes the theme data entered by the user and any necessary parameters (e.g., user ID, etc.). The device performs the appropriate encoding to maintain the integrity of the data.

[0570] (Input) User input data

[0571] (Output) HTTP request to the server

[0572] Step 3:

[0573] The server receives the HTTP request and begins analyzing the data. The analysis process includes extracting themes and ideas, sometimes using natural language processing (NLP) tools. The server extracts keywords based on the themes and stores them in data storage.

[0574] (Input) HTTP request data from the terminal

[0575] (Output) Extracted results of themes and ideas

[0576] Step 4:

[0577] The server uses an emotion analysis engine to recognize emotions from the user's facial expressions, tone of voice, input content, etc. For example, if the user is using a device with a camera, a library such as OpenCV is used to analyze facial expressions. The server saves the emotion analysis results in data storage.

[0578] (Input) Input data from camera and microphone

[0579] (Output) Emotion analysis results

[0580] Step 5:

[0581] The server combines the emotion data obtained from the emotion analysis engine with the analyzed theme data to generate an API request for the generative AI model. For example, the API request might include a prompt such as, "Generate a design for a new smartphone in a bright and fun style."

[0582] (Input) Emotion data and theme data

[0583] (Output) API requests to generative AI models

[0584] Step 6:

[0585] The server calls the API of the generative AI model and requests it to generate an image. After the API request is sent, the model generates an image based on the prompt.

[0586] (Input) API request

[0587] (Output) Generated image data

[0588] Step 7:

[0589] The generative AI model generates an image based on the requested prompt and sends it back to the server in the specified format (e.g. PNG, JPEG). The server receives this data.

[0590] (Input) API request

[0591] (Output) Generated image data

[0592] Step 8:

[0593] The server temporarily stores the generated image data, and then sends this data to the terminal as an HTTP response, including image metadata (such as the generation date and time, related information, etc.) as needed.

[0594] (Input) Generated image data

[0595] (Output) The HTTP response sent to the device

[0596] Step 9:

[0597] The device analyzes the received image data and displays the image in a web browser. The display includes a UI for checking the image (e.g., regenerate, save, download, etc.).

[0598] (Input) HTTP response from the server

[0599] (output) Images and UI displayed in the web browser

[0600] Step 10:

[0601] The user checks the displayed images, adds captions as necessary, and adjusts the layout to create the final concept sheet.

[0602] (Input) Displayed image

[0603] (Output) Concept sheet after user editing

[0604] Step 11:

[0605] Users can save their concept sheets, print them or download them in PDF or image format.

[0606] (Input) User edited concept sheet

[0607] (Output) Saved Concept Sheet file

[0608] (Application example 2)

[0609] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0610] Conventional advertising banner generation tools generate images based on themes entered by the user, but they are unable to provide designs that reflect the user's emotions, making it difficult to generate banners that are optimal for each individual user.In addition, the user interface is fixed, making it impossible to dynamically adjust according to the user's emotions or situation, making it difficult to improve the user experience.

[0611] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for inputting an idea or theme to be investigated, means for requesting image generation from a generative AI model based on the idea or theme and user emotion data, means for receiving image data returned from the generative AI model, means for displaying the image data in a dynamic interface according to the user emotion, and means for saving the image data in a state that can be edited by the user. This makes it possible to dynamically generate advertising banners that reflect the user emotion, improving the user experience.

[0612] The "idea or theme to be investigated" refers to the concept or idea that the user inputs to generate an image based on the generative artificial intelligence model.

[0613] An "interface" is a web page or application screen through which a user inputs an idea or theme and requests image generation.

[0614] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates images based on ideas, themes, and emotional data input by the user.

[0615] "Emotion data" is information about the user's emotional state analyzed from facial expressions, tone of voice, and the like.

[0616] A "dynamic interface" is a flexible user interface whose display content changes depending on the user's emotions and situation.

[0617] "Image data" refers to visual data generated as a result of a generative artificial intelligence model.

[0618] The "means for saving in an editable state" is a function for saving the generated image data in a form that allows the user to re-edit it.

[0619] A "webcam" is a camera device used to capture a user's facial expressions.

[0620] An "emotion recognition engine" is a technology that analyzes emotions from a user's facial expressions and tone of voice and outputs them as data.

[0621] The present invention relates to a system for recognizing a user's emotion and generating an advertising banner based on the emotion. In this embodiment, the system is constructed as follows.

[0622] 1. Provide an interface for users to input ideas and themes to be researched. The interface has a field where users can input themes and press the "Generate" button, as well as a camera and microphone for sentiment analysis.

[0623] 2. When the user inputs an idea or theme through the interface and presses the "Generate" button, the device sends the input data, including the theme and the user's emotional data, to the server.

[0624] 3. The server analyzes the emotion data using an emotion recognition engine that analyzes the user's facial expressions and tone of voice. The emotion recognition engine uses existing face recognition libraries such as OpenCV and Dlib.

[0625] 4. Based on the analyzed emotion data and the theme entered by the user, the server requests the generative AI model to generate an image. The server sends the theme, idea, and emotion data to the generative AI model, which then generates a related image.

[0626] 5. The generative AI model generates an image of the advertising banner based on the received data and sends it back to the server in the specified format (e.g. PNG, JPEG).

[0627] 6. The server sends the generated advertising banner image to the user's device and displays it on the interface so that the user can view it. At this time, a dynamic interface responding to the user's emotions is displayed, with bright colors and relevant operation buttons.

[0628] 7. Users can review the displayed advertising banner and regenerate, save, or edit it as needed. The final banner can be saved in the form desired by the user.

[0629] As a concrete example, suppose a user inputs the theme "Summer Sale Promotion" and smiles at the camera. The system detects a "positive emotion" from the smile and generates a bright, colorful style advertising banner based on this emotion. An example prompt sentence in this scenario is "Theme: Summer Sale Promotion, Emotion: Positive."

[0630] In this way, a system is realized that efficiently generates advertising banners that reflect the user's emotions and supports the creation of advertisements that provide high user satisfaction.

[0631] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0632] Step 1:

[0633] The user inputs the idea or theme to be researched into the interface, and a camera and microphone are used for emotion analysis. The device collects facial expressions captured by the camera and voice recorded by the microphone. The input at this stage is the idea or theme entered by the user, as well as image and audio data for emotion analysis.

[0634] Step 2:

[0635] The device sends the collected data (ideas, themes, facial images, and voice recordings) to the server via HTTP requests. The input data is used by the server for analysis, generating data packets that are sent to the server as output.

[0636] Step 3:

[0637] The server uses an emotion recognition engine (e.g., OpenCV, Dlib) to analyze the received facial expression data and voice data and identify the user's emotions. The input data is facial expression images and voice data, and data processing involves extracting features from facial expressions and analyzing voice tones. The output is emotion data that represents the analyzed user's emotions.

[0638] Step 4:

[0639] The server calls the API of the generative AI model based on the emotion data and the theme entered by the user, requesting image generation. In the API call, the generative AI model receives the theme and emotion data as input. In the data calculation, the model generates a banner image based on the input data. The generated advertising banner image is obtained as the output.

[0640] Step 5:

[0641] The server receives the advertising banner image returned from the generative AI model and sends it to the user's device. At this stage, the banner image is input to the server, and image data is output to the user's device as an HTTP response.

[0642] Step 6:

[0643] The terminal displays the advertising banner image received from the server on the interface, dynamically adjusting the color scheme and display elements of the interface according to the user's emotions. The input data is the received banner image, and the output is displayed on the interface.

[0644] Step 7:

[0645] The user can view the displayed ad banner and regenerate, save, or edit it as needed. During this operation, the user operates the interface to fine-tune the generated banner, and the final banner is saved. The input is the user's operation data, and the output is the final edited ad banner image.

[0646] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0647] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0648] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0649] [Third embodiment]

[0650] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0651] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0652] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0653] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0654] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0655] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0656] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0657] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0658] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0659] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0660] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0661] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0662] The present invention is a system for generating images of concept sheets used in market research, etc., in which a generative artificial intelligence model automatically generates images in response to user input. A description will be given of an embodiment of this system, including specific examples.

[0663] First, the user accesses the system's homepage through a web browser, which displays a form for inputting ideas or themes to be explored, including input fields and a "Generate" button.

[0664] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters (e.g., user ID, etc.).

[0665] The server analyzes the received data and extracts themes and ideas. Based on this extracted information, the server calls the API of a generative AI model (e.g., DALL-E) to request image generation. The API call requires parameters including the input theme and idea.

[0666] When the generative AI model receives instructions from the server, it generates relevant images based on the input data. The generated image data is sent back to the server in a fixed format (e.g. PNG, JPEG).

[0667] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. This response also includes the image data and other metadata (e.g., creation date and time, related information, etc.) to make it easier for the user to view and confirm the image.

[0668] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image and necessary operation buttons (e.g., regenerate, save, download, etc.), allowing the user to check the generated image.

[0669] The user can check the displayed images and add captions or adjust the layout as necessary. This allows the user to complete a concept sheet that is adjusted to their satisfaction. The completed concept sheet can be saved on the device or printed depending on the purpose of use. The created concept sheet can also be downloaded in PDF or image format.

[0670] This system allows users to efficiently create concept sheets for market research. In particular, by utilizing a generative AI model, appropriate images can be obtained quickly and easily, significantly reducing the amount of work required.

[0671] The processing flow will be explained below.

[0672] Step 1:

[0673] Users open a web browser, access the system's home page, and are presented with a form to input ideas or topics to be researched.

[0674] Step 2:

[0675] The user enters the idea or theme to be researched (e.g., "Design concept for a new smartphone") into the input field and presses the "Generate" button.

[0676] Step 3:

[0677] The device sends the input data (ideas and themes) to the server as an HTTP request, along with any necessary parameters (e.g., input content, user ID, etc.).

[0678] Step 4:

[0679] The server analyzes the received HTTP request and extracts the input themes and ideas.

[0680] Step 5:

[0681] The server calls the API of the generative AI model based on the analyzed data and requests image generation, sending parameters including the theme and idea to the API.

[0682] Step 6:

[0683] The generative AI model generates related images based on the received theme or idea and returns them to the server in the specified format (e.g. PNG, JPEG).

[0684] Step 7:

[0685] The server receives the image data returned from the generative artificial intelligence model and temporarily stores it.

[0686] Step 8:

[0687] The server sends the saved image data to the device as an HTTP response, along with image metadata (e.g., creation date and time, related information, etc.) if necessary.

[0688] Step 9:

[0689] The device analyzes the received image data and displays it in the web browser. At this time, the image display is accompanied by operation buttons (e.g., regenerate, save, download, etc.) so that the user can check the image.

[0690] Step 10:

[0691] The user checks the displayed images and, if necessary, adds captions or adjusts the layout to create the final concept sheet.

[0692] Step 11:

[0693] Users can save the concept sheets they create, print them, or download them in PDF or image format.

[0694] Example 1

[0695] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0696] In conventional market research, visualizing concept sheets and ideas requires a lot of time and effort. Furthermore, there are limitations to manually creating and editing images, making it difficult to quickly obtain appropriate images. This has led to a decrease in efficiency in the early stages of market research and product development.

[0697] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0698] In this invention, the server includes means for a user to input an idea or theme to be researched into an interface, means for transmitting the input idea or theme to the server as an HTTP request, means for the server to analyze the idea or theme and request an image generation from a generative AI model, means for receiving image data returned from the generative AI model, means for transmitting the image data to the user's terminal as an HTTP response, means for the user interface to display the image data, and means for saving the image data in a user-editable state, thereby enabling users to quickly and accurately create concept sheets for market research.

[0699] "User" refers to the entity that utilizes the system to input ideas and themes to be researched and to review and edit the generated images.

[0700] "Interface" refers to elements that include fields for users to enter ideas or themes to be explored and buttons to request image generation.

[0701] An "HTTP request" is a communication protocol for sending data from a user's terminal to a server.

[0702] "Server" refers to a computer system that receives and analyzes data sent by a user, requests image generation from a generative artificial intelligence model, and returns the generated data to the user.

[0703] A "generative artificial intelligence model" refers to an algorithm or computer program that generates relevant images based on an input idea or theme.

[0704] "Image data" refers to data in file format that contains visual information generated by a generative artificial intelligence model.

[0705] "HTTP response" is a communication protocol for sending data from a server to a user's terminal.

[0706] The term "user interface" refers to the screen display and operation means that allow the user to check and edit the generated image data.

[0707] "Saving" refers to the act of storing the generated image data in a state that allows the user to access it later.

[0708] The present invention is a system for generating images of concept sheets used in market research, etc., in which a generative artificial intelligence model automatically generates images in response to user input.

[0709] First, the user accesses the system's homepage through a web browser, which displays a form for inputting ideas or themes to be researched, including input fields and a "Generate" button.

[0710] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters such as the user ID.

[0711] The server analyzes the received data and extracts themes and ideas. Based on this extracted information, the server calls the API of a generative AI model (e.g., DALL-E) to request image generation. The API call requires parameters including the input theme and idea.

[0712] When the generative AI model receives instructions from the server, it generates relevant images based on the input data. The generated image data is sent back to the server in a fixed format (e.g. PNG, JPEG).

[0713] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. This response also includes the image data and other metadata (e.g., creation date and time, related information, etc.) to make it easier for the user to view and confirm the image.

[0714] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image and necessary operation buttons (e.g., regenerate, save, download, etc.), allowing the user to check the generated image.

[0715] The user can check the displayed images and add captions or adjust the layout as necessary. This allows the user to complete a concept sheet that is satisfactory to the user. The completed concept sheet can be saved on the device or printed depending on the purpose of use. The created concept sheet can also be downloaded in PDF or image format.

[0716] For example, if a user enters "Design concept for a new smartphone" as the theme and presses the generate button, the device sends this information to the server. The server then calls DALL-E's API, generates image data based on the theme, and sends it back to the user. The user can then review the generated image, make any necessary adjustments, and finally save or download it.

[0717] Example prompt sentence:

[0718] "Generate modern and futuristic design images based on new smartphone design concepts."

[0719] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0720] Step 1:

[0721] Users access the system's homepage through a web browser. The homepage displays a form containing fields for inputting ideas or themes to be researched and a "Generate" button. When users input a theme and click the "Generate" button, this information becomes input for the next step.

[0722] Step 2:

[0723] The device sends the theme entered by the user to the server as an HTTP request. The input data includes the user ID and the theme entered. When this request is processed, the input data is parsed.

[0724] Step 3:

[0725] The server analyzes the received HTTP request and extracts the input theme or idea. The analyzed data becomes input data to be passed to the generative AI model. Here, the content of the input text is checked and prepared to be passed to the generative AI model as appropriate parameters.

[0726] Step 4:

[0727] The server calls the API of a generative AI model (e.g., DALL-E) to request image generation based on the input theme. It sends an API request containing a prompt and necessary parameters. The input includes analyzed data (themes and ideas), and image data is generated as output.

[0728] Step 5:

[0729] The generative AI model receives an API request from the server and generates relevant images based on the input theme. This process generates image data and sends it back to the server in a certain format (e.g. PNG, JPEG). The generated image is the main output here.

[0730] Step 6:

[0731] The server receives the image data returned from the generative AI model and temporarily stores it. It then sends the stored image data to the user's device as an HTTP response. At this point, the HTTP response contains the image data and metadata (e.g., generation date and time, related information, etc.).

[0732] Step 7:

[0733] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image along with operation buttons such as regenerate, save, and download. The user can check the generated image. Here, the screen display is based on the input image data.

[0734] Step 8:

[0735] The user can check the displayed images and add captions or adjust the layout as necessary. The user can then check the generated images and complete the adjusted concept sheet to their satisfaction. The adjusted concept sheet is obtained as the output.

[0736] Step 9:

[0737] Users can save the completed concept sheet to their device or download it in PDF or image format. Users can perform the respective action by clicking the save or download button, which will result in a file saved as the final output.

[0738] (Application example 1)

[0739] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0740] While image generation technology using artificial intelligence is becoming more widespread, there is still a lack of systems that can quickly create effective advertising visuals. Furthermore, creating advertising materials often requires a lot of time and effort, so there is a growing demand for systems that can handle not only concept images but also editing, saving, and downloading all in one place. Conventional systems have made it difficult for users to efficiently generate and edit visual data for use as advertising materials.

[0741] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0742] In this invention, the server includes means for providing an interface for inputting concepts or themes to be investigated, means for requesting a generative AI model to generate visual data based on the concepts or themes, means for receiving the visual data returned from the generative AI model, means for providing a user interface for displaying the visual data, and means for saving the visual data in a state that allows the user to edit it and for downloading it for use as advertising material, thereby enabling users to quickly and efficiently create advertising visuals and then edit, save, and download them on the spot.

[0743] A "concept" refers to an abstract idea or subject that a user explores.

[0744] "Subject" refers to the specific theme or topic that is the subject of research or image generation.

[0745] "Interface" refers to the UI components that allow users to input concepts and themes, and the method of operating them.

[0746] A "generative artificial intelligence model" refers to an artificial intelligence system that automatically generates relevant visual data based on input concepts or topics.

[0747] "Visual data" refers to images and images generated by generative artificial intelligence models.

[0748] "User interface" refers to UI components that display the generated visual data and allow the user to operate and edit it.

[0749] "Editable" refers to a state in which a user can process the generated visual data, such as by changing the text or layout.

[0750] "Saving" refers to storing the user-edited visual data in a storage device for later reuse.

[0751] "Download" refers to transferring the generated and edited visual data to a terminal and making it available for use.

[0752] This invention is a system for quickly and efficiently creating advertising visuals, which can generate, edit, save, and download visual data using a generative artificial intelligence model based on concepts and themes entered by the user.

[0753] The server has a means for providing an interface for inputting concepts or themes to be researched, a means for requesting the generation of visual data using a generative artificial intelligence model, a means for receiving the returned visual data, and a means for displaying the visual data to the user. In this system, users can generate, edit, and save advertising materials through a smartphone application.

[0754] The hardware required is a smartphone and a server, and the software required includes:

[0755] React Native: Building user interfaces.

[0756] Redux: Application state management.

[0757] Node.js: Server program development.

[0758] Express: API routing.

[0759] DALL-E API: Image generation using generative artificial intelligence models.

[0760] When a user launches the app, a React Native interface is displayed with a field to input a "concept" or "subject." For example, the user enters "advertising campaign for new product launch" and presses the "Generate" button, which sends the information to a Node.js server via an HTTP request.

[0761] The server uses Express to analyze the received data and calls the DALL-E API to generate visual data. The generated visual data is sent back to the server in a specific format (e.g., PNG, JPEG). The server then resends it to the user's smartphone and displays it in a React Native user interface.

[0762] The user interface provides functions for reviewing and editing the generated visual data. For example, users can add ad copy or adjust the layout. After that, users can save the generated ad material to their smartphone by pressing the "Save" or "Download" button and download it as needed.

[0763] For example, the following prompts are used:

[0764] "Generate images for a new smartphone-themed advertising campaign, highlighting the OLED display as a distinctive feature."

[0765] "Generate a visual to advertise a fall fashion sale. Use casual clothing and a background of autumn leaves."

[0766] This system allows users to efficiently generate advertising visuals and edit and save them as needed, significantly reducing the effort and time required to create advertisements.

[0767] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0768] Step 1:

[0769] The user launches the smartphone application.

[0770] Input: None

[0771] Output: Application home screen

[0772] Specific behavior: An application created using React Native starts and displays the user interface.

[0773] Step 2:

[0774] The user inputs the concept or subject to be investigated.

[0775] Input: Concept or subject (e.g., "advertising campaign for new product launch")

[0776] Output: The input data

[0777] What it does: A user enters a concept or subject to research into a React Native form field.

[0778] Step 3:

[0779] The user presses the "Generate" button.

[0780] Input: Concepts and topics typed, button clicks

[0781] Output: HTTP request

[0782] Specific behavior: When the button is clicked, the entered data is sent to the Node.js server as an HTTP request.

[0783] Step 4:

[0784] The server receives the HTTP request and parses the data.

[0785] Input: HTTP request (concept or subject)

[0786] Output: Analyzed data (concepts and themes)

[0787] What it does: Receives requests and parses data using Node.js and Express.

[0788] Step 5:

[0789] The server calls the DALL-E API to request the generation of visual data.

[0790] Input: Analyzed data (concepts and themes)

[0791] Output: Visual data (images)

[0792] Specific operation: The parsed data is sent to the DALL-E API as a prompt, and related visual data (images) are generated.

[0793] Step 6:

[0794] The server receives the generated visual data and returns it to the user's terminal.

[0795] Input: Visual data (images)

[0796] Output: HTTP response (visual data)

[0797] Specific operation: The server receives the visual data returned from the DALL-E API and sends it to the user's device as an HTTP response.

[0798] Step 7:

[0799] The device displays the visual data received.

[0800] Input: HTTP response (visual data)

[0801] Output: The image to be displayed

[0802] What it does: Visual data is displayed in a React Native user interface.

[0803] Step 8:

[0804] The user edits the visual data as needed.

[0805] Input: Editing content (text, layout, etc.)

[0806] Output: Edited visual data

[0807] Specific behavior: The user edits the visual data through the provided user interface.

[0808] Step 9:

[0809] The user saves or downloads the visual data.

[0810] Input: Click the "Save" or "Download" button

[0811] Output: Saved or downloaded file

[0812] Specific behavior: When the user presses the "Save" or "Download" button, the edited visual data is saved or downloaded to the smartphone.

[0813] Through the above processing steps, the user can efficiently generate advertising visuals, and edit, save, and download them as needed.

[0814] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0815] The present invention is a system that dynamically adjusts the interface display content and the style of the generated image according to the user's emotion by combining a system that generates images of concept sheets used in market research, etc. with an emotion engine that recognizes the user's emotion. A description will be given of an embodiment of this system with specific examples.

[0816] First, users access the system's homepage through a web browser and are presented with a form for entering ideas or themes to be researched, which includes input fields and a "Generate" button.

[0817] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters (e.g., user ID, etc.).

[0818] The server analyzes the received data and extracts themes and ideas. The system also incorporates an emotion engine that analyzes the user's emotions and recognizes them from their facial expressions, tone of voice, and input. For example, if the user is using a device with a camera, emotions can be extracted from facial expression analysis and voice input.

[0819] The emotion engine recognizes the user's emotions (e.g., joy, seriousness, impatience, etc.) and adjusts the interface display content and image generation parameters accordingly. For example, if the user is feeling joyful, it can instruct the engine to generate a bright, positive style design.

[0820] The server generates an image by calling the API of the generative AI model based on the analyzed data and data from the emotion engine. At this time, parameters including the user's emotion data along with the theme and idea are sent to the API.

[0821] The generative AI model generates relevant images based on the received theme, idea, and emotional data, and returns them to the server in the specified format (e.g., PNG, JPEG).

[0822] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. If necessary, image metadata (such as the creation date and time, related information, etc.) is also sent.

[0823] The device analyzes the received image data and displays it in the web browser. At this time, the image display is accompanied by operation buttons (regenerate, save, download, etc.) so that the user can check the image.

[0824] The user checks the displayed images and, if necessary, adds captions or adjusts the layout to create the final concept sheet. The emotion engine generates images that match the user's mood, allowing for more satisfying document creation.

[0825] Users can save the concept sheets they create, print them, or download them in PDF or image format. This system allows users to create more sensible and efficient concept sheets for market research. In particular, the combination of a generative AI model and an emotion engine makes it possible to quickly and easily obtain images that reflect the user's emotions, significantly reducing the amount of work required.

[0826] The processing flow will be explained below.

[0827] This invention is a system that dynamically adjusts the interface display content and the style of the generated image according to the user's emotions by combining an emotion engine that recognizes the user's emotions with a system that generates images of concept sheets used in market research, etc. An embodiment of this system will be described below, broken down into processing steps and with specific examples.

[0828] Step 1:

[0829] Users open a web browser, access the system's home page, and are presented with a form to input ideas or topics to be researched.

[0830] Step 2:

[0831] The user enters the idea or theme to be researched (e.g., "Design concept for a new smartphone") into the input field and presses the "Generate" button.

[0832] Step 3:

[0833] The device sends the input data (ideas and themes) to the server as an HTTP request, along with any necessary parameters (e.g., input content, user ID, etc.).

[0834] Step 4:

[0835] The server analyzes the received HTTP request and extracts the input themes and ideas.

[0836] Step 5:

[0837] Using the device's camera and microphone, an emotion engine is activated to recognize emotions from the user's facial expressions, tone of voice, input content, etc.

[0838] Step 6:

[0839] The emotion engine analyzes the recognized emotion data and generates emotion results, for example, recognizing that the user seems to be having fun.

[0840] Step 7:

[0841] The server combines the extracted themes and idea data with the emotion results from the emotion engine, calls the API of the generative AI model, and requests image generation. At this time, the emotion results are also included in the parameters sent.

[0842] Step 8:

[0843] The generative AI model generates relevant images based on the received theme, idea, and emotional data, and returns them to the server in the specified format (e.g., PNG, JPEG).

[0844] Step 9:

[0845] The server receives the image data returned by the generative AI model and temporarily stores it. At this stage, metadata is added as needed.

[0846] Step 10:

[0847] The server sends the saved image data to the device as an HTTP response, along with image metadata (e.g., creation date and time, related information, etc.) if necessary.

[0848] Step 11:

[0849] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image along with necessary operation buttons (regenerate, save, download, etc.).

[0850] Step 12:

[0851] The user checks the displayed images, adds captions as necessary, adjusts the layout, and creates the final concept sheet. The emotion engine generates images that match the user's mood, increasing user satisfaction.

[0852] Step 13:

[0853] Users can save their concept sheets, print them or download them in PDF or image format.

[0854] This system allows users to efficiently create concept sheets for market research. In particular, the combination of a generative AI model and an emotion engine allows users to quickly and easily generate images that reflect their emotions, significantly reducing the amount of work required.

[0855] Example 2

[0856] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0857] Conventional concept sheet creation systems for market research do not take user emotions into account, making it difficult to provide interface display content and generated image styles that match those emotions. This can lead to a decrease in user satisfaction and the completeness of the sheet. Furthermore, conventional systems require manual adjustments to reflect user emotions, which is inefficient.

[0858] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for inputting a theme or idea to be researched, means for requesting a generative AI model to generate an image based on the theme or idea, means for receiving image data returned from the generative AI model, means including an emotion analysis engine for recognizing a user's emotion, means for adjusting the display content of the interface and the style of the generated image based on the emotion data recognized by the emotion analysis engine, and means for saving the image data in a state that can be edited by the user. This makes it possible to dynamically adjust the display content of the interface and the style of the generated image according to the user's emotion, improve user satisfaction, and efficiently create concept sheets for market research.

[0859] "Themes or ideas under investigation" refers to subjects or concepts that are addressed for purposes such as market research or product development.

[0860] "Interface" refers to the screen and input means that users use to access and operate the system.

[0861] A "generative artificial intelligence model" is a type of artificial intelligence that has the ability to generate images from given text or other data.

[0862] "Means for requesting image generation" refers to a method or mechanism for sending specific instructions to an artificial intelligence model to generate an image.

[0863] An "emotion analysis engine" is a software engine that recognizes emotions from a user's facial expressions, voice, input content, etc.

[0864] "Emotion Data" refers to information regarding a user's emotions as recognized by the emotion analysis engine.

[0865] "User interface" refers to the visual and operational elements through which a user interacts with a system.

[0866] "Means for saving in an editable state" refers to a method for saving generated images and concept sheets so that the user can change or modify them later.

[0867] The present invention relates to a system for generating images based on a theme or idea of ​​a research subject, while taking into consideration the emotions of a user. Specific embodiments will be described below.

[0868] This system mainly uses the user's terminal, a server, a generative AI model, and an emotion analysis engine. Each component of the system operates as follows:

[0869] First, the user accesses the system's homepage using a web browser. The homepage provides an interface for entering research topics and ideas. The interface includes a text input field and a "Generate" button. When the user enters a topic, such as "Design concepts for new smartphones," and presses the "Generate" button, the device sends this data to the server. The transmission is made via an HTTP request, and includes the input data along with required parameters (e.g., user ID, etc.).

[0870] The server analyzes the received data and extracts themes and ideas. This analysis includes extracting keywords based on themes and analyzing text. The server also incorporates a sentiment analysis engine that recognizes emotions from the user's facial expressions, tone of voice, and input. If the user is using a device with a camera or microphone, the sentiment analysis engine extracts emotions from input from these devices.

[0871] When the emotion analysis engine recognizes the user's emotions (e.g., joy, seriousness, impatience, etc.), it dynamically adjusts the interface display content and image generation parameters based on the results. For example, if the user seems to be having fun, the emotion analysis engine can instruct the generation of a bright, positive style design.

[0872] The server combines the emotion data obtained from the emotion analysis engine with the analyzed theme data and calls the API of a generative AI model. Examples of the use of generative AI models include OpenAI's DALL-E and MidJourney. When the API is called, a prompt containing emotion data in addition to the theme or idea is sent. For example, a prompt such as "Generate a design concept for a new smartphone in a bright and fun style" is used.

[0873] The generative AI model generates a related image based on the received theme, idea, and emotional data, and returns it to the server in the specified format (e.g., PNG, JPEG). The server receives the generated image data and temporarily stores it. It then sends it to the user's device as an HTTP response. If necessary, image metadata (such as the date and time of creation, related information, etc.) is also sent.

[0874] The device analyzes the received image data and displays it in the user's web browser. At this time, operation buttons (regenerate, save, download, etc.) are also displayed so that the user can check the image. The user can check the displayed image, add captions as needed, adjust the layout, and create the final concept sheet. The emotion analysis engine generates images that match the user's mood, making it possible to create documents that are more satisfying.

[0875] Users can save the concept sheets they create, print them, or download them in PDF or image format. This system enables users to create concept sheets for market research that are tailored to their emotions. In particular, the combination of a generative AI model and an emotion analysis engine makes it possible to quickly and easily obtain images that reflect the user's emotions, significantly reducing the amount of work required.

[0876] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0877] Step 1:

[0878] The user accesses the system's homepage using a web browser. The homepage displays an interface for inputting research topics and ideas. When the user enters "design concept for a new smartphone" in the text input field and presses the "Generate" button, the input data is sent from the device to the server.

[0879] (Input) User input data including themes

[0880] (Output) The HTTP request sent to the server

[0881] Step 2:

[0882] The device sends the submitted data to the server as an HTTP request. This request includes the theme data entered by the user and any necessary parameters (e.g., user ID, etc.). The device performs the appropriate encoding to maintain the integrity of the data.

[0883] (Input) User input data

[0884] (Output) HTTP request to the server

[0885] Step 3:

[0886] The server receives the HTTP request and begins analyzing the data. The analysis process includes extracting themes and ideas, sometimes using natural language processing (NLP) tools. The server extracts keywords based on the themes and stores them in data storage.

[0887] (Input) HTTP request data from the terminal

[0888] (Output) Extracted results of themes and ideas

[0889] Step 4:

[0890] The server uses an emotion analysis engine to recognize emotions from the user's facial expressions, tone of voice, input content, etc. For example, if the user is using a device with a camera, a library such as OpenCV is used to analyze facial expressions. The server saves the emotion analysis results in data storage.

[0891] (Input) Input data from camera and microphone

[0892] (Output) Emotion analysis results

[0893] Step 5:

[0894] The server combines the emotion data obtained from the emotion analysis engine with the analyzed theme data to generate an API request for the generative AI model. For example, the API request might include a prompt such as, "Generate a design for a new smartphone in a bright and fun style."

[0895] (Input) Emotion data and theme data

[0896] (Output) API requests to generative AI models

[0897] Step 6:

[0898] The server calls the API of the generative AI model and requests it to generate an image. After the API request is sent, the model generates an image based on the prompt.

[0899] (Input) API request

[0900] (Output) Generated image data

[0901] Step 7:

[0902] The generative AI model generates an image based on the requested prompt and sends it back to the server in the specified format (e.g. PNG, JPEG). The server receives this data.

[0903] (Input) API request

[0904] (Output) Generated image data

[0905] Step 8:

[0906] The server temporarily stores the generated image data, and then sends this data to the terminal as an HTTP response, including image metadata (such as the generation date and time, related information, etc.) as needed.

[0907] (Input) Generated image data

[0908] (Output) The HTTP response sent to the device

[0909] Step 9:

[0910] The device analyzes the received image data and displays the image in a web browser. The display includes a UI for checking the image (e.g., regenerate, save, download, etc.).

[0911] (Input) HTTP response from the server

[0912] (output) Images and UI displayed in the web browser

[0913] Step 10:

[0914] The user checks the displayed images, adds captions as necessary, and adjusts the layout to create the final concept sheet.

[0915] (Input) Displayed image

[0916] (Output) Concept sheet after user editing

[0917] Step 11:

[0918] Users can save their concept sheets, print them or download them in PDF or image format.

[0919] (Input) User edited concept sheet

[0920] (Output) Saved Concept Sheet file

[0921] (Application example 2)

[0922] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0923] Conventional advertising banner generation tools generate images based on themes entered by the user, but they are unable to provide designs that reflect the user's emotions, making it difficult to generate banners that are optimal for each individual user.In addition, the user interface is fixed, making it impossible to dynamically adjust according to the user's emotions or situation, making it difficult to improve the user experience.

[0924] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for inputting an idea or theme to be investigated, means for requesting image generation from a generative AI model based on the idea or theme and user emotion data, means for receiving image data returned from the generative AI model, means for displaying the image data in a dynamic interface according to the user emotion, and means for saving the image data in a state that can be edited by the user. This makes it possible to dynamically generate advertising banners that reflect the user emotion, improving the user experience.

[0925] The "idea or theme to be investigated" refers to the concept or idea that the user inputs to generate an image based on the generative artificial intelligence model.

[0926] An "interface" is a web page or application screen through which a user inputs an idea or theme and requests image generation.

[0927] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates images based on ideas, themes, and emotional data input by the user.

[0928] "Emotion data" is information about the user's emotional state analyzed from facial expressions, tone of voice, and the like.

[0929] A "dynamic interface" is a flexible user interface whose display content changes depending on the user's emotions and situation.

[0930] "Image data" refers to visual data generated as a result of a generative artificial intelligence model.

[0931] The "means for saving in an editable state" is a function for saving the generated image data in a form that allows the user to re-edit it.

[0932] A "webcam" is a camera device used to capture a user's facial expressions.

[0933] An "emotion recognition engine" is a technology that analyzes emotions from a user's facial expressions and tone of voice and outputs them as data.

[0934] The present invention relates to a system for recognizing a user's emotion and generating an advertising banner based on the emotion. In this embodiment, the system is constructed as follows.

[0935] 1. Provide an interface for users to input ideas and themes to be researched. The interface has a field where users can input themes and press the "Generate" button, as well as a camera and microphone for sentiment analysis.

[0936] 2. When the user inputs an idea or theme through the interface and presses the "Generate" button, the device sends the input data, including the theme and the user's emotional data, to the server.

[0937] 3. The server analyzes the emotion data using an emotion recognition engine that analyzes the user's facial expressions and tone of voice. The emotion recognition engine uses existing face recognition libraries such as OpenCV and Dlib.

[0938] 4. Based on the analyzed emotion data and the theme entered by the user, the server requests the generative AI model to generate an image. The server sends the theme, idea, and emotion data to the generative AI model, which then generates a related image.

[0939] 5. The generative AI model generates an image of the advertising banner based on the received data and sends it back to the server in the specified format (e.g. PNG, JPEG).

[0940] 6. The server sends the generated advertising banner image to the user's device and displays it on the interface so that the user can view it. At this time, a dynamic interface responding to the user's emotions is displayed, with bright colors and relevant operation buttons.

[0941] 7. Users can review the displayed advertising banner and regenerate, save, or edit it as needed. The final banner can be saved in the form desired by the user.

[0942] As a concrete example, suppose a user inputs the theme "Summer Sale Promotion" and smiles at the camera. The system detects a "positive emotion" from the smile and generates a bright, colorful style advertising banner based on this emotion. An example prompt sentence in this scenario is "Theme: Summer Sale Promotion, Emotion: Positive."

[0943] In this way, a system is realized that efficiently generates advertising banners that reflect the user's emotions and supports the creation of advertisements that provide high user satisfaction.

[0944] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0945] Step 1:

[0946] The user inputs the idea or theme to be researched into the interface, and a camera and microphone are used for emotion analysis. The device collects facial expressions captured by the camera and voice recorded by the microphone. The input at this stage is the idea or theme entered by the user, as well as image and audio data for emotion analysis.

[0947] Step 2:

[0948] The device sends the collected data (ideas, themes, facial images, and voice recordings) to the server via HTTP requests. The input data is used by the server for analysis, generating data packets that are sent to the server as output.

[0949] Step 3:

[0950] The server uses an emotion recognition engine (e.g., OpenCV, Dlib) to analyze the received facial expression data and voice data and identify the user's emotions. The input data is facial expression images and voice data, and data processing involves extracting features from facial expressions and analyzing voice tones. The output is emotion data that represents the analyzed user's emotions.

[0951] Step 4:

[0952] The server calls the API of the generative AI model based on the emotion data and the theme entered by the user, requesting image generation. In the API call, the generative AI model receives the theme and emotion data as input. In the data calculation, the model generates a banner image based on the input data. The generated advertising banner image is obtained as the output.

[0953] Step 5:

[0954] The server receives the advertising banner image returned from the generative AI model and sends it to the user's device. At this stage, the banner image is input to the server, and image data is output to the user's device as an HTTP response.

[0955] Step 6:

[0956] The terminal displays the advertising banner image received from the server on the interface, dynamically adjusting the color scheme and display elements of the interface according to the user's emotions. The input data is the received banner image, and the output is displayed on the interface.

[0957] Step 7:

[0958] The user can view the displayed ad banner and regenerate, save, or edit it as needed. During this operation, the user operates the interface to fine-tune the generated banner, and the final banner is saved. The input is the user's operation data, and the output is the final edited ad banner image.

[0959] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0960] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0961] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0962] [Fourth embodiment]

[0963] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0964] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0965] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0966] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0967] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0968] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0969] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0970] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0971] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0972] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0973] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0974] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0975] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0976] The present invention is a system for generating images of concept sheets used in market research, etc., in which a generative artificial intelligence model automatically generates images in response to user input. A description will be given of an embodiment of this system, including specific examples.

[0977] First, the user accesses the system's homepage through a web browser, which displays a form for inputting ideas or themes to be explored, including input fields and a "Generate" button.

[0978] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters (e.g., user ID, etc.).

[0979] The server analyzes the received data and extracts themes and ideas. Based on this extracted information, the server calls the API of a generative AI model (e.g., DALL-E) to request image generation. The API call requires parameters including the input theme and idea.

[0980] When the generative AI model receives instructions from the server, it generates relevant images based on the input data. The generated image data is sent back to the server in a fixed format (e.g. PNG, JPEG).

[0981] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. This response also includes the image data and other metadata (e.g., creation date and time, related information, etc.) to make it easier for the user to view and confirm the image.

[0982] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image and necessary operation buttons (e.g., regenerate, save, download, etc.), allowing the user to check the generated image.

[0983] The user can check the displayed images and add captions or adjust the layout as necessary. This allows the user to complete a concept sheet that is adjusted to their satisfaction. The completed concept sheet can be saved on the device or printed depending on the purpose of use. The created concept sheet can also be downloaded in PDF or image format.

[0984] This system allows users to efficiently create concept sheets for market research. In particular, by utilizing a generative AI model, appropriate images can be obtained quickly and easily, significantly reducing the amount of work required.

[0985] The processing flow will be explained below.

[0986] Step 1:

[0987] Users open a web browser, access the system's home page, and are presented with a form to input ideas or topics to be researched.

[0988] Step 2:

[0989] The user enters the idea or theme to be researched (e.g., "Design concept for a new smartphone") into the input field and presses the "Generate" button.

[0990] Step 3:

[0991] The device sends the input data (ideas and themes) to the server as an HTTP request, along with any necessary parameters (e.g., input content, user ID, etc.).

[0992] Step 4:

[0993] The server analyzes the received HTTP request and extracts the input themes and ideas.

[0994] Step 5:

[0995] The server calls the API of the generative AI model based on the analyzed data and requests image generation, sending parameters including the theme and idea to the API.

[0996] Step 6:

[0997] The generative AI model generates related images based on the received theme or idea and returns them to the server in the specified format (e.g. PNG, JPEG).

[0998] Step 7:

[0999] The server receives the image data returned from the generative artificial intelligence model and temporarily stores it.

[1000] Step 8:

[1001] The server sends the saved image data to the device as an HTTP response, along with image metadata (e.g., creation date and time, related information, etc.) if necessary.

[1002] Step 9:

[1003] The device analyzes the received image data and displays it in the web browser. At this time, the image display is accompanied by operation buttons (e.g., regenerate, save, download, etc.) so that the user can check the image.

[1004] Step 10:

[1005] The user checks the displayed images and, if necessary, adds captions or adjusts the layout to create the final concept sheet.

[1006] Step 11:

[1007] Users can save the concept sheets they create, print them, or download them in PDF or image format.

[1008] Example 1

[1009] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1010] In conventional market research, visualizing concept sheets and ideas requires a lot of time and effort. Furthermore, there are limitations to manually creating and editing images, making it difficult to quickly obtain appropriate images. This has led to a decrease in efficiency in the early stages of market research and product development.

[1011] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1012] In this invention, the server includes means for a user to input an idea or theme to be researched into an interface, means for transmitting the input idea or theme to the server as an HTTP request, means for the server to analyze the idea or theme and request an image generation from a generative AI model, means for receiving image data returned from the generative AI model, means for transmitting the image data to the user's terminal as an HTTP response, means for the user interface to display the image data, and means for saving the image data in a user-editable state, thereby enabling users to quickly and accurately create concept sheets for market research.

[1013] "User" refers to the entity that utilizes the system to input ideas and themes to be researched and to review and edit the generated images.

[1014] "Interface" refers to elements that include fields for users to enter ideas or themes to be explored and buttons to request image generation.

[1015] An "HTTP request" is a communication protocol for sending data from a user's terminal to a server.

[1016] "Server" refers to a computer system that receives and analyzes data sent by a user, requests image generation from a generative artificial intelligence model, and returns the generated data to the user.

[1017] A "generative artificial intelligence model" refers to an algorithm or computer program that generates relevant images based on an input idea or theme.

[1018] "Image data" refers to data in file format that contains visual information generated by a generative artificial intelligence model.

[1019] "HTTP response" is a communication protocol for sending data from a server to a user's terminal.

[1020] The term "user interface" refers to the screen display and operation means that allow the user to check and edit the generated image data.

[1021] "Saving" refers to the act of storing the generated image data in a state that allows the user to access it later.

[1022] The present invention is a system for generating images of concept sheets used in market research, etc., in which a generative artificial intelligence model automatically generates images in response to user input.

[1023] First, the user accesses the system's homepage through a web browser, which displays a form for inputting ideas or themes to be researched, including input fields and a "Generate" button.

[1024] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters such as the user ID.

[1025] The server analyzes the received data and extracts themes and ideas. Based on this extracted information, the server calls the API of a generative AI model (e.g., DALL-E) to request image generation. The API call requires parameters including the input theme and idea.

[1026] When the generative AI model receives instructions from the server, it generates relevant images based on the input data. The generated image data is sent back to the server in a fixed format (e.g. PNG, JPEG).

[1027] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. This response also includes the image data and other metadata (e.g., creation date and time, related information, etc.) to make it easier for the user to view and confirm the image.

[1028] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image and necessary operation buttons (e.g., regenerate, save, download, etc.), allowing the user to check the generated image.

[1029] The user can check the displayed images and add captions or adjust the layout as necessary. This allows the user to complete a concept sheet that is satisfactory to the user. The completed concept sheet can be saved on the device or printed depending on the purpose of use. The created concept sheet can also be downloaded in PDF or image format.

[1030] For example, if a user enters "Design concept for a new smartphone" as the theme and presses the generate button, the device sends this information to the server. The server then calls DALL-E's API, generates image data based on the theme, and sends it back to the user. The user can then review the generated image, make any necessary adjustments, and finally save or download it.

[1031] Example prompt sentence:

[1032] "Generate modern and futuristic design images based on new smartphone design concepts."

[1033] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1034] Step 1:

[1035] Users access the system's homepage through a web browser. The homepage displays a form containing fields for inputting ideas or themes to be researched and a "Generate" button. When users input a theme and click the "Generate" button, this information becomes input for the next step.

[1036] Step 2:

[1037] The device sends the theme entered by the user to the server as an HTTP request. The input data includes the user ID and the theme entered. When this request is processed, the input data is parsed.

[1038] Step 3:

[1039] The server analyzes the received HTTP request and extracts the input theme or idea. The analyzed data becomes input data to be passed to the generative AI model. Here, the content of the input text is checked and prepared to be passed to the generative AI model as appropriate parameters.

[1040] Step 4:

[1041] The server calls the API of a generative AI model (e.g., DALL-E) to request image generation based on the input theme. It sends an API request containing a prompt and necessary parameters. The input includes analyzed data (themes and ideas), and image data is generated as output.

[1042] Step 5:

[1043] The generative AI model receives an API request from the server and generates relevant images based on the input theme. This process generates image data and sends it back to the server in a certain format (e.g. PNG, JPEG). The generated image is the main output here.

[1044] Step 6:

[1045] The server receives the image data returned from the generative AI model and temporarily stores it. It then sends the stored image data to the user's device as an HTTP response. At this point, the HTTP response contains the image data and metadata (e.g., generation date and time, related information, etc.).

[1046] Step 7:

[1047] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image along with operation buttons such as regenerate, save, and download. The user can check the generated image. Here, the screen display is based on the input image data.

[1048] Step 8:

[1049] The user can check the displayed images and add captions or adjust the layout as necessary. The user can then check the generated images and complete the adjusted concept sheet to their satisfaction. The adjusted concept sheet is obtained as the output.

[1050] Step 9:

[1051] Users can save the completed concept sheet to their device or download it in PDF or image format. Users can perform the respective action by clicking the save or download button, which will result in a file saved as the final output.

[1052] (Application example 1)

[1053] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1054] While image generation technology using artificial intelligence is becoming more widespread, there is still a lack of systems that can quickly create effective advertising visuals. Furthermore, creating advertising materials often requires a lot of time and effort, so there is a growing demand for systems that can handle not only concept images but also editing, saving, and downloading all in one place. Conventional systems have made it difficult for users to efficiently generate and edit visual data for use as advertising materials.

[1055] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1056] In this invention, the server includes means for providing an interface for inputting concepts or themes to be investigated, means for requesting a generative AI model to generate visual data based on the concepts or themes, means for receiving the visual data returned from the generative AI model, means for providing a user interface for displaying the visual data, and means for saving the visual data in a state that allows the user to edit it and for downloading it for use as advertising material, thereby enabling users to quickly and efficiently create advertising visuals and then edit, save, and download them on the spot.

[1057] A "concept" refers to an abstract idea or subject that a user explores.

[1058] "Subject" refers to the specific theme or topic that is the subject of research or image generation.

[1059] "Interface" refers to the UI components that allow users to input concepts and themes, and the method of operating them.

[1060] A "generative artificial intelligence model" refers to an artificial intelligence system that automatically generates relevant visual data based on input concepts or topics.

[1061] "Visual data" refers to images and images generated by generative artificial intelligence models.

[1062] "User interface" refers to UI components that display the generated visual data and allow the user to operate and edit it.

[1063] "Editable" refers to a state in which a user can process the generated visual data, such as by changing the text or layout.

[1064] "Saving" refers to storing the user-edited visual data in a storage device for later reuse.

[1065] "Download" refers to transferring the generated and edited visual data to a terminal and making it available for use.

[1066] This invention is a system for quickly and efficiently creating advertising visuals, which can generate, edit, save, and download visual data using a generative artificial intelligence model based on concepts and themes entered by the user.

[1067] The server has a means for providing an interface for inputting concepts or themes to be researched, a means for requesting the generation of visual data using a generative artificial intelligence model, a means for receiving the returned visual data, and a means for displaying the visual data to the user. In this system, users can generate, edit, and save advertising materials through a smartphone application.

[1068] The hardware required is a smartphone and a server, and the software required includes:

[1069] React Native: Building user interfaces.

[1070] Redux: Application state management.

[1071] Node.js: Server program development.

[1072] Express: API routing.

[1073] DALL-E API: Image generation using generative artificial intelligence models.

[1074] When a user launches the app, a React Native interface is displayed with a field to input a "concept" or "subject." For example, the user enters "advertising campaign for new product launch" and presses the "Generate" button, which sends the information to a Node.js server via an HTTP request.

[1075] The server uses Express to analyze the received data and calls the DALL-E API to generate visual data. The generated visual data is sent back to the server in a specific format (e.g., PNG, JPEG). The server then resends it to the user's smartphone and displays it in a React Native user interface.

[1076] The user interface provides functions for reviewing and editing the generated visual data. For example, users can add ad copy or adjust the layout. After that, users can save the generated ad material to their smartphone by pressing the "Save" or "Download" button and download it as needed.

[1077] For example, the following prompts are used:

[1078] "Generate images for a new smartphone-themed advertising campaign, highlighting the OLED display as a distinctive feature."

[1079] "Generate a visual to advertise a fall fashion sale. Use casual clothing and a background of autumn leaves."

[1080] This system allows users to efficiently generate advertising visuals and edit and save them as needed, significantly reducing the effort and time required to create advertisements.

[1081] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1082] Step 1:

[1083] The user launches the smartphone application.

[1084] Input: None

[1085] Output: Application home screen

[1086] Specific behavior: An application created using React Native starts and displays the user interface.

[1087] Step 2:

[1088] The user inputs the concept or subject to be investigated.

[1089] Input: Concept or subject (e.g., "advertising campaign for new product launch")

[1090] Output: The input data

[1091] What it does: A user enters a concept or subject to research into a React Native form field.

[1092] Step 3:

[1093] The user presses the "Generate" button.

[1094] Input: Concepts and topics typed, button clicks

[1095] Output: HTTP request

[1096] Specific behavior: When the button is clicked, the entered data is sent to the Node.js server as an HTTP request.

[1097] Step 4:

[1098] The server receives the HTTP request and parses the data.

[1099] Input: HTTP request (concept or subject)

[1100] Output: Analyzed data (concepts and themes)

[1101] What it does: Receives requests and parses data using Node.js and Express.

[1102] Step 5:

[1103] The server calls the DALL-E API to request the generation of visual data.

[1104] Input: Analyzed data (concepts and themes)

[1105] Output: Visual data (images)

[1106] Specific operation: The parsed data is sent to the DALL-E API as a prompt, and related visual data (images) are generated.

[1107] Step 6:

[1108] The server receives the generated visual data and returns it to the user's terminal.

[1109] Input: Visual data (images)

[1110] Output: HTTP response (visual data)

[1111] Specific operation: The server receives the visual data returned from the DALL-E API and sends it to the user's device as an HTTP response.

[1112] Step 7:

[1113] The device displays the visual data received.

[1114] Input: HTTP response (visual data)

[1115] Output: The image to be displayed

[1116] What it does: Visual data is displayed in a React Native user interface.

[1117] Step 8:

[1118] The user edits the visual data as needed.

[1119] Input: Editing content (text, layout, etc.)

[1120] Output: Edited visual data

[1121] Specific behavior: The user edits the visual data through the provided user interface.

[1122] Step 9:

[1123] The user saves or downloads the visual data.

[1124] Input: Click the "Save" or "Download" button

[1125] Output: Saved or downloaded file

[1126] Specific behavior: When the user presses the "Save" or "Download" button, the edited visual data is saved or downloaded to the smartphone.

[1127] Through the above processing steps, the user can efficiently generate advertising visuals, and edit, save, and download them as needed.

[1128] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1129] The present invention is a system that dynamically adjusts the interface display content and the style of the generated image according to the user's emotion by combining a system that generates images of concept sheets used in market research, etc. with an emotion engine that recognizes the user's emotion. A description will be given of an embodiment of this system with specific examples.

[1130] First, users access the system's homepage through a web browser and are presented with a form for entering ideas or themes to be researched, which includes input fields and a "Generate" button.

[1131] When a user inputs a theme, such as "Design Concept for a New Smartphone," and presses the "Generate" button, the device sends this information to the server via an HTTP request, which includes the input data along with necessary parameters (e.g., user ID, etc.).

[1132] The server analyzes the received data and extracts themes and ideas. The system also incorporates an emotion engine that analyzes the user's emotions and recognizes them from their facial expressions, tone of voice, and input. For example, if the user is using a device with a camera, emotions can be extracted from facial expression analysis and voice input.

[1133] The emotion engine recognizes the user's emotions (e.g., joy, seriousness, impatience, etc.) and adjusts the interface display content and image generation parameters accordingly. For example, if the user is feeling joyful, it can instruct the engine to generate a bright, positive style design.

[1134] The server generates an image by calling the API of the generative AI model based on the analyzed data and data from the emotion engine. At this time, parameters including the user's emotion data along with the theme and idea are sent to the API.

[1135] The generative AI model generates relevant images based on the received theme, idea, and emotional data, and returns them to the server in the specified format (e.g., PNG, JPEG).

[1136] The server receives the image data returned from the generative AI model and temporarily stores it. At this point, the server sends the image data to the user's device as an HTTP response. If necessary, image metadata (such as the creation date and time, related information, etc.) is also sent.

[1137] The device analyzes the received image data and displays it in the web browser. At this time, the image display is accompanied by operation buttons (regenerate, save, download, etc.) so that the user can check the image.

[1138] The user checks the displayed images and, if necessary, adds captions or adjusts the layout to create the final concept sheet. The emotion engine generates images that match the user's mood, allowing for more satisfying document creation.

[1139] Users can save the concept sheets they create, print them, or download them in PDF or image format. This system allows users to create more sensible and efficient concept sheets for market research. In particular, the combination of a generative AI model and an emotion engine makes it possible to quickly and easily obtain images that reflect the user's emotions, significantly reducing the amount of work required.

[1140] The processing flow will be explained below.

[1141] This invention is a system that dynamically adjusts the interface display content and the style of the generated image according to the user's emotions by combining an emotion engine that recognizes the user's emotions with a system that generates images of concept sheets used in market research, etc. An embodiment of this system will be described below, broken down into processing steps and with specific examples.

[1142] Step 1:

[1143] Users open a web browser, access the system's home page, and are presented with a form to input ideas or topics to be researched.

[1144] Step 2:

[1145] The user enters the idea or theme to be researched (e.g., "Design concept for a new smartphone") into the input field and presses the "Generate" button.

[1146] Step 3:

[1147] The device sends the input data (ideas and themes) to the server as an HTTP request, along with any necessary parameters (e.g., input content, user ID, etc.).

[1148] Step 4:

[1149] The server analyzes the received HTTP request and extracts the input themes and ideas.

[1150] Step 5:

[1151] Using the device's camera and microphone, an emotion engine is activated to recognize emotions from the user's facial expressions, tone of voice, input content, etc.

[1152] Step 6:

[1153] The emotion engine analyzes the recognized emotion data and generates emotion results, for example, recognizing that the user seems to be having fun.

[1154] Step 7:

[1155] The server combines the extracted themes and idea data with the emotion results from the emotion engine, calls the API of the generative AI model, and requests image generation. At this time, the emotion results are also included in the parameters sent.

[1156] Step 8:

[1157] The generative AI model generates relevant images based on the received theme, idea, and emotional data, and returns them to the server in the specified format (e.g., PNG, JPEG).

[1158] Step 9:

[1159] The server receives the image data returned by the generative AI model and temporarily stores it. At this stage, metadata is added as needed.

[1160] Step 10:

[1161] The server sends the saved image data to the device as an HTTP response, along with image metadata (e.g., creation date and time, related information, etc.) if necessary.

[1162] Step 11:

[1163] The device analyzes the received image data and displays it in a web browser. The user interface displays the generated image along with necessary operation buttons (regenerate, save, download, etc.).

[1164] Step 12:

[1165] The user checks the displayed images, adds captions as necessary, adjusts the layout, and creates the final concept sheet. The emotion engine generates images that match the user's mood, increasing user satisfaction.

[1166] Step 13:

[1167] Users can save their concept sheets, print them or download them in PDF or image format.

[1168] This system allows users to efficiently create concept sheets for market research. In particular, the combination of a generative AI model and an emotion engine allows users to quickly and easily generate images that reflect their emotions, significantly reducing the amount of work required.

[1169] Example 2

[1170] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1171] Conventional concept sheet creation systems for market research do not take user emotions into account, making it difficult to provide interface display content and generated image styles that match those emotions. This can lead to a decrease in user satisfaction and the completeness of the sheet. Furthermore, conventional systems require manual adjustments to reflect user emotions, which is inefficient.

[1172] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for inputting a theme or idea to be researched, means for requesting a generative AI model to generate an image based on the theme or idea, means for receiving image data returned from the generative AI model, means including an emotion analysis engine for recognizing a user's emotion, means for adjusting the display content of the interface and the style of the generated image based on the emotion data recognized by the emotion analysis engine, and means for saving the image data in a state that can be edited by the user. This makes it possible to dynamically adjust the display content of the interface and the style of the generated image according to the user's emotion, improve user satisfaction, and efficiently create concept sheets for market research.

[1173] "Themes or ideas under investigation" refers to subjects or concepts that are addressed for purposes such as market research or product development.

[1174] "Interface" refers to the screen and input means that users use to access and operate the system.

[1175] A "generative artificial intelligence model" is a type of artificial intelligence that has the ability to generate images from given text or other data.

[1176] "Means for requesting image generation" refers to a method or mechanism for sending specific instructions to an artificial intelligence model to generate an image.

[1177] An "emotion analysis engine" is a software engine that recognizes emotions from a user's facial expressions, voice, input content, etc.

[1178] "Emotion Data" refers to information regarding a user's emotions as recognized by the emotion analysis engine.

[1179] "User interface" refers to the visual and operational elements through which a user interacts with a system.

[1180] "Means for saving in an editable state" refers to a method for saving generated images and concept sheets so that the user can change or modify them later.

[1181] The present invention relates to a system for generating images based on a theme or idea of ​​a research subject, while taking into consideration the emotions of a user. Specific embodiments will be described below.

[1182] This system mainly uses the user's terminal, a server, a generative AI model, and an emotion analysis engine. Each component of the system operates as follows:

[1183] First, the user accesses the system's homepage using a web browser. The homepage provides an interface for entering research topics and ideas. The interface includes a text input field and a "Generate" button. When the user enters a topic, such as "Design concepts for new smartphones," and presses the "Generate" button, the device sends this data to the server. The transmission is made via an HTTP request, and includes the input data along with required parameters (e.g., user ID, etc.).

[1184] The server analyzes the received data and extracts themes and ideas. This analysis includes extracting keywords based on themes and analyzing text. The server also incorporates a sentiment analysis engine that recognizes emotions from the user's facial expressions, tone of voice, and input. If the user is using a device with a camera or microphone, the sentiment analysis engine extracts emotions from input from these devices.

[1185] When the emotion analysis engine recognizes the user's emotions (e.g., joy, seriousness, impatience, etc.), it dynamically adjusts the interface display content and image generation parameters based on the results. For example, if the user seems to be having fun, the emotion analysis engine can instruct the generation of a bright, positive style design.

[1186] The server combines the emotion data obtained from the emotion analysis engine with the analyzed theme data and calls the API of a generative AI model. Examples of the use of generative AI models include OpenAI's DALL-E and MidJourney. When the API is called, a prompt containing emotion data in addition to the theme or idea is sent. For example, a prompt such as "Generate a design concept for a new smartphone in a bright and fun style" is used.

[1187] The generative AI model generates a related image based on the received theme, idea, and emotional data, and returns it to the server in the specified format (e.g., PNG, JPEG). The server receives the generated image data and temporarily stores it. It then sends it to the user's device as an HTTP response. If necessary, image metadata (such as the date and time of creation, related information, etc.) is also sent.

[1188] The device analyzes the received image data and displays it in the user's web browser. At this time, operation buttons (regenerate, save, download, etc.) are also displayed so that the user can check the image. The user can check the displayed image, add captions as needed, adjust the layout, and create the final concept sheet. The emotion analysis engine generates images that match the user's mood, making it possible to create documents that are more satisfying.

[1189] Users can save the concept sheets they create, print them, or download them in PDF or image format. This system enables users to create concept sheets for market research that are tailored to their emotions. In particular, the combination of a generative AI model and an emotion analysis engine makes it possible to quickly and easily obtain images that reflect the user's emotions, significantly reducing the amount of work required.

[1190] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1191] Step 1:

[1192] The user accesses the system's homepage using a web browser. The homepage displays an interface for inputting research topics and ideas. When the user enters "design concept for a new smartphone" in the text input field and presses the "Generate" button, the input data is sent from the device to the server.

[1193] (Input) User input data including themes

[1194] (Output) The HTTP request sent to the server

[1195] Step 2:

[1196] The device sends the submitted data to the server as an HTTP request. This request includes the theme data entered by the user and any necessary parameters (e.g., user ID, etc.). The device performs the appropriate encoding to maintain the integrity of the data.

[1197] (Input) User input data

[1198] (Output) HTTP request to the server

[1199] Step 3:

[1200] The server receives the HTTP request and begins analyzing the data. The analysis process includes extracting themes and ideas, sometimes using natural language processing (NLP) tools. The server extracts keywords based on the themes and stores them in data storage.

[1201] (Input) HTTP request data from the terminal

[1202] (Output) Extracted results of themes and ideas

[1203] Step 4:

[1204] The server uses an emotion analysis engine to recognize emotions from the user's facial expressions, tone of voice, input content, etc. For example, if the user is using a device with a camera, a library such as OpenCV is used to analyze facial expressions. The server saves the emotion analysis results in data storage.

[1205] (Input) Input data from camera and microphone

[1206] (Output) Emotion analysis results

[1207] Step 5:

[1208] The server combines the emotion data obtained from the emotion analysis engine with the analyzed theme data to generate an API request for the generative AI model. For example, the API request might include a prompt such as, "Generate a design for a new smartphone in a bright and fun style."

[1209] (Input) Emotion data and theme data

[1210] (Output) API requests to generative AI models

[1211] Step 6:

[1212] The server calls the API of the generative AI model and requests it to generate an image. After the API request is sent, the model generates an image based on the prompt.

[1213] (Input) API request

[1214] (Output) Generated image data

[1215] Step 7:

[1216] The generative AI model generates an image based on the requested prompt and sends it back to the server in the specified format (e.g. PNG, JPEG). The server receives this data.

[1217] (Input) API request

[1218] (Output) Generated image data

[1219] Step 8:

[1220] The server temporarily stores the generated image data, and then sends this data to the terminal as an HTTP response, including image metadata (such as the generation date and time, related information, etc.) as needed.

[1221] (Input) Generated image data

[1222] (Output) The HTTP response sent to the device

[1223] Step 9:

[1224] The device analyzes the received image data and displays the image in a web browser. The display includes a UI for checking the image (e.g., regenerate, save, download, etc.).

[1225] (Input) HTTP response from the server

[1226] (output) Images and UI displayed in the web browser

[1227] Step 10:

[1228] The user checks the displayed images, adds captions as necessary, and adjusts the layout to create the final concept sheet.

[1229] (Input) Displayed image

[1230] (Output) Concept sheet after user editing

[1231] Step 11:

[1232] Users can save their concept sheets, print them or download them in PDF or image format.

[1233] (Input) User edited concept sheet

[1234] (Output) Saved Concept Sheet file

[1235] (Application example 2)

[1236] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1237] Conventional advertising banner generation tools generate images based on themes entered by the user, but they are unable to provide designs that reflect the user's emotions, making it difficult to generate banners that are optimal for each individual user.In addition, the user interface is fixed, making it impossible to dynamically adjust according to the user's emotions or situation, making it difficult to improve the user experience.

[1238] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for inputting an idea or theme to be investigated, means for requesting image generation from a generative AI model based on the idea or theme and user emotion data, means for receiving image data returned from the generative AI model, means for displaying the image data in a dynamic interface according to the user emotion, and means for saving the image data in a state that can be edited by the user. This makes it possible to dynamically generate advertising banners that reflect the user emotion, improving the user experience.

[1239] The "idea or theme to be investigated" refers to the concept or idea that the user inputs to generate an image based on the generative artificial intelligence model.

[1240] An "interface" is a web page or application screen through which a user inputs an idea or theme and requests image generation.

[1241] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates images based on ideas, themes, and emotional data input by the user.

[1242] "Emotion data" is information about the user's emotional state analyzed from facial expressions, tone of voice, and the like.

[1243] A "dynamic interface" is a flexible user interface whose display content changes depending on the user's emotions and situation.

[1244] "Image data" refers to visual data generated as a result of a generative artificial intelligence model.

[1245] The "means for saving in an editable state" is a function for saving the generated image data in a form that allows the user to re-edit it.

[1246] A "webcam" is a camera device used to capture a user's facial expressions.

[1247] An "emotion recognition engine" is a technology that analyzes emotions from a user's facial expressions and tone of voice and outputs them as data.

[1248] The present invention relates to a system for recognizing a user's emotion and generating an advertising banner based on the emotion. In this embodiment, the system is constructed as follows.

[1249] 1. Provide an interface for users to input ideas and themes to be researched. The interface has a field where users can input themes and press the "Generate" button, as well as a camera and microphone for sentiment analysis.

[1250] 2. When the user inputs an idea or theme through the interface and presses the "Generate" button, the device sends the input data, including the theme and the user's emotional data, to the server.

[1251] 3. The server analyzes the emotion data using an emotion recognition engine that analyzes the user's facial expressions and tone of voice. The emotion recognition engine uses existing face recognition libraries such as OpenCV and Dlib.

[1252] 4. Based on the analyzed emotion data and the theme entered by the user, the server requests the generative AI model to generate an image. The server sends the theme, idea, and emotion data to the generative AI model, which then generates a related image.

[1253] 5. The generative AI model generates an image of the advertising banner based on the received data and sends it back to the server in the specified format (e.g. PNG, JPEG).

[1254] 6. The server sends the generated advertising banner image to the user's device and displays it on the interface so that the user can view it. At this time, a dynamic interface responding to the user's emotions is displayed, with bright colors and relevant operation buttons.

[1255] 7. Users can review the displayed advertising banner and regenerate, save, or edit it as needed. The final banner can be saved in the form desired by the user.

[1256] As a concrete example, suppose a user inputs the theme "Summer Sale Promotion" and smiles at the camera. The system detects a "positive emotion" from the smile and generates a bright, colorful style advertising banner based on this emotion. An example prompt sentence in this scenario is "Theme: Summer Sale Promotion, Emotion: Positive."

[1257] In this way, a system is realized that efficiently generates advertising banners that reflect the user's emotions and supports the creation of advertisements that provide high user satisfaction.

[1258] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1259] Step 1:

[1260] The user inputs the idea or theme to be researched into the interface, and a camera and microphone are used for emotion analysis. The device collects facial expressions captured by the camera and voice recorded by the microphone. The input at this stage is the idea or theme entered by the user, as well as image and audio data for emotion analysis.

[1261] Step 2:

[1262] The device sends the collected data (ideas, themes, facial images, and voice recordings) to the server via HTTP requests. The input data is used by the server for analysis, generating data packets that are sent to the server as output.

[1263] Step 3:

[1264] The server uses an emotion recognition engine (e.g., OpenCV, Dlib) to analyze the received facial expression data and voice data and identify the user's emotions. The input data is facial expression images and voice data, and data processing involves extracting features from facial expressions and analyzing voice tones. The output is emotion data that represents the analyzed user's emotions.

[1265] Step 4:

[1266] The server calls the API of the generative AI model based on the emotion data and the theme entered by the user, requesting image generation. In the API call, the generative AI model receives the theme and emotion data as input. In the data calculation, the model generates a banner image based on the input data. The generated advertising banner image is obtained as the output.

[1267] Step 5:

[1268] The server receives the advertising banner image returned from the generative AI model and sends it to the user's device. At this stage, the banner image is input to the server, and image data is output to the user's device as an HTTP response.

[1269] Step 6:

[1270] The terminal displays the advertising banner image received from the server on the interface, dynamically adjusting the color scheme and display elements of the interface according to the user's emotions. The input data is the received banner image, and the output is displayed on the interface.

[1271] Step 7:

[1272] The user can view the displayed ad banner and regenerate, save, or edit it as needed. During this operation, the user operates the interface to fine-tune the generated banner, and the final banner is saved. The input is the user's operation data, and the output is the final edited ad banner image.

[1273] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1274] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1275] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1276] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1277] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1278] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1279] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1280] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1281] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1282] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1283] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1284] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1285] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1286] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1287] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1288] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1289] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1290] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1291] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1292] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1293] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1294] The following is further disclosed regarding the above embodiment.

[1295] (Claim 1)

[1296] a means for providing an interface for inputting ideas or themes to be investigated;

[1297] A means for requesting a generative artificial intelligence model to generate an image based on the idea or theme;

[1298] means for receiving image data returned from the generative artificial intelligence model;

[1299] means for providing a user interface for displaying the image data;

[1300] means for saving the image data in a state that the image data can be edited by a user;

[1301] A system including:

[1302] (Claim 2)

[1303] 2. The system of claim 1, wherein the interface includes a field for inputting the idea or theme and a button for requesting the image generation.

[1304] (Claim 3)

[1305] 2. The system according to claim 1, wherein the generative artificial intelligence model generates the most appropriate image based on an input idea or theme.

[1306] "Example 1"

[1307] (Claim 1)

[1308] a means for a user to input ideas or topics to be investigated into the interface;

[1309] means for transmitting the input idea or theme to a server as an HTTP request;

[1310] A server analyzes the idea or theme and requests a generative artificial intelligence model to generate an image;

[1311] means for receiving image data returned from the generative artificial intelligence model;

[1312] means for transmitting the image data to a user terminal as an HTTP response;

[1313] means for a user interface to display said image data;

[1314] means for saving the image data in a state that the image data can be edited by a user;

[1315] A system including:

[1316] (Claim 2)

[1317] 2. The system of claim 1, wherein the interface includes a field for inputting the idea or theme and a button for requesting the image generation.

[1318] (Claim 3)

[1319] 2. The system according to claim 1, wherein the generative artificial intelligence model generates the most appropriate image based on an input idea or theme.

[1320] "Application Example 1"

[1321] (Claim 1)

[1322] a means for providing an interface for inputting concepts or subjects to be investigated;

[1323] A means for requesting a generative artificial intelligence model to generate visual data based on the concept or theme;

[1324] means for receiving visual data returned from said generative artificial intelligence model;

[1325] means for providing a user interface for displaying said visual data;

[1326] means for saving the visual data in a state that allows the user to edit it;

[1327] A system including:

[1328] (Claim 2)

[1329] 2. The system of claim 1, wherein the interface includes a field for inputting the concept or subject and a button for requesting the generation of the visual data.

[1330] (Claim 3)

[1331] 2. The system of claim 1, wherein the generative artificial intelligence model generates the most appropriate visual data based on an input concept or subject.

[1332] (Claim 4)

[1333] 2. The system according to claim 1, wherein after the visual data is generated, the user can edit, save, and download it for use as advertising material.

[1334] "Example 2: Combining Emotion Engines"

[1335] (Claim 1)

[1336] a means for providing an interface for inputting themes and ideas to be investigated;

[1337] A means for requesting a generative artificial intelligence model to generate an image based on the theme or idea;

[1338] means for receiving image data returned from the generative artificial intelligence model;

[1339] means for providing a user interface for displaying the image data;

[1340] means including a sentiment analysis engine for recognizing a user's sentiment;

[1341] means for adjusting the display content of the interface and the generated image style based on the emotion data recognized by the emotion analysis engine;

[1342] means for saving the image data in a state that the image data can be edited by a user;

[1343] A system including:

[1344] (Claim 2)

[1345] 2. The system according to claim 1, wherein the interface comprises a field for inputting the theme or idea and a button for requesting the image generation.

[1346] (Claim 3)

[1347] 2. The system of claim 1, wherein the generative artificial intelligence model generates the most appropriate image based on an input theme or idea and recognized emotional data.

[1348] "Application example 2 when combining emotion engines"

[1349] (Claim 1)

[1350] a means for providing an interface for inputting ideas or themes to be investigated;

[1351] a means for requesting a generative artificial intelligence model to generate an image based on the idea, theme, and user emotion data;

[1352] means for receiving image data returned from the generative artificial intelligence model;

[1353] means for displaying the image data in a dynamic interface according to the user's emotions;

[1354] means for saving the image data in a state that the image data can be edited by a user;

[1355] A system including:

[1356] (Claim 2)

[1357] The system according to claim 1, characterized in that the interface has a field for inputting the idea or theme and a button for requesting image generation, and further has a function for analyzing emotions from the user's facial expressions and tone of voice.

[1358] (Claim 3)

[1359] 2. The system according to claim 1, wherein the generative artificial intelligence model generates the most appropriate image based on an input idea or theme and user emotional data. [Explanation of symbols]

[1360] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for providing an interface for inputting ideas or themes to be investigated; A means for requesting a generative artificial intelligence model to generate an image based on the idea or theme; means for receiving image data returned from the generative artificial intelligence model; means for providing a user interface for displaying the image data; means for saving the image data in a state that the image data can be edited by a user; A system including:

2. 2. The system according to claim 1, wherein the interface comprises a field for inputting the idea or theme and a button for requesting the image generation.

3. 2. The system according to claim 1, wherein the generative artificial intelligence model generates the most appropriate image based on an input idea or theme.

Images