System

Abstract

A system is provided.SOLUTION: A system for generating a specific design image using a plurality of reference images by a non-professional includes means for uploading the reference images, means for selecting specific portions of the uploaded reference images, adjustment means for blending the plurality of selected image portions, image generation means for generating a blended image, and means for storing and sharing the generated image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When non-experts communicate specific ideas about a design to designers, they often have difficulty successfully realizing the image they envision, resulting in numerous retakes. This situation reduces the efficiency of the design process and increases time and costs. Therefore, there is a need for a way for non-experts to visualize their ideas specifically and share them with designers. [Means for solving the problem]

[0005] To solve this problem, the present invention provides a system as follows: First, a non-expert uploads multiple reference images to the tool. Next, a tool is used to select specific parts from the uploaded images, using a rectangular selection tool or lasso tool to select the specific parts. The blending degree of each selected part is then adjusted using a slider, and the image generation AI blends the images based on the adjustment value to generate a new image. Finally, the system provides a function to save and share the generated images. This allows non-experts to specifically visualize their own ideas and efficiently communicate them to designers.

[0006] "Non-experts" refers to general users who do not have specialized knowledge or experience in design.

[0007] "Reference images" refers to multiple images that serve as a basis for generating a specific design image.

[0008] "Means for uploading" refers to a function that allows a user to transfer reference images stored in local storage to the system.

[0009] "Selection means" refers to tools or functions for selecting specific portions of an uploaded reference image.

[0010] "Adjustment means" refers to an interface such as a slider for manipulating the degree of blending of multiple selected image portions.

[0011] "Image generation means" refers to an image generation algorithm or artificial intelligence for generating a new image based on the blending conditions set by the adjustment means.

[0012] "Means for saving and sharing" refers to the functionality for saving generated images long-term and sharing them with other users and designers.

[0013] The "rectangle selection tool" refers to a selection tool for cutting out a rectangular area from a reference image.

[0014] The "lasso tool" refers to a tool for freely selecting irregularly shaped areas.

[0015] "Image generation artificial intelligence" refers to machine learning models and algorithms that generate new images based on given reference images and blending settings. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention provides a system for non-experts to create specific design images using reference images. The system is operated by a user, a terminal, and a server. The program processing will be described in detail below.

[0038] Program processing

[0039] Uploading a reference image

[0040] User

[0041] The user selects multiple reference images from their local storage and uploads them through the tool's interface, for example, selecting three different image files.

[0042] Terminal

[0043] The terminal temporarily stores these image files according to the user's selection and sends an upload request to the server.

[0044] server

[0045] The server receives the uploaded image files, assigns a unique identifier to each image, and stores it in a specified directory. It also records the location and identifier of each image in a database.

[0046] Selecting and arranging images

[0047] User

[0048] Users can use the rectangular selection tool or lasso tool to select the part of an uploaded reference image they want to use, such as a specific logo or color.

[0049] Terminal

[0050] The terminal records the coordinate information of the part selected by the user and transmits it to the server.

[0051] server

[0052] The server then clips out the selected portion based on the received coordinate information, saves the newly generated thumbnail image, and updates the information about the selected portion in the database.

[0053] Blending Adjustment

[0054] User

[0055] The user adjusts the blending ratio of the selected image by using the sliders, for example, 50% of image A, 30% of image B, and 20% of image C.

[0056] Terminal

[0057] The device sends the slider setting value to the server in real time.

[0058] server

[0059] The server generates a new blended image based on the received slider settings using image generation AI, which integrates the features of each image based on the settings.

[0060] Check and correct the generated image

[0061] User

[0062] The user reviews the resulting image and adjusts the sliders again if necessary, repeating the process until satisfied.

[0063] Terminal

[0064] The terminal retransmits the user's readjusted settings and displays the latest generated image.

[0065] server

[0066] The server then performs the blending process again based on the readjusted settings to generate a new image.

[0067] Saving and sharing generated images

[0068] server

[0069] Once the final image is finalized, the server stores the generated image and generates a download link that the user can access.

[0070] Terminal

[0071] The device will display this download link to the user, who can click the link to download it or use the sharing tool to share it with other designers.

[0072] User

[0073] The user can send the final, concrete design image to the designer, allowing the designer to proceed with the detailed design based on the user's concrete image.

[0074] Example: Product logo design

[0075] scenario

[0076] Suppose a user wants to create a logo design for a new product.

[0077] User

[0078] Users upload three reference images of different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[0079] Server and Device

[0080] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[0081] Final production and sharing

[0082] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[0083] The above is a specific embodiment for carrying out the present invention.

[0084] The processing flow will be explained below.

[0085] Step 1:

[0086] User

[0087] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[0088] Step 2:

[0089] Terminal

[0090] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[0091] Step 3:

[0092] server

[0093] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, "image1.jpg" is saved to " / uploads / 12345.jpg."

[0094] Step 4:

[0095] server

[0096] The server records the storage location and identifier of each image in a database, which allows it to be referenced in subsequent processing.

[0097] Step 5:

[0098] User

[0099] The user uses the system's interface to view a list of reference images they have uploaded, each displayed as a thumbnail.

[0100] Step 6:

[0101] User

[0102] The user uses the rectangular selection tool or lasso tool to select a specific portion of the reference image, for example, a rectangular selection of a specific logo from "image1.jpg."

[0103] Step 7:

[0104] Terminal

[0105] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[0106] Step 8:

[0107] server

[0108] The server then cuts out a specific portion of the reference image based on the received coordinate information. The cut-out portion is saved as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[0109] Step 9:

[0110] User

[0111] The user uses sliders to adjust the blending of the cropped image parts, adjusting the transparency and blending strength of each reference image part.

[0112] Step 10:

[0113] Terminal

[0114] The terminal transmits the slider value set by the user to the server in real time.

[0115] Step 11:

[0116] server

[0117] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[0118] Step 12:

[0119] server

[0120] The new image generated is saved on the server, for example as "blend_result.jpg".

[0121] Step 13:

[0122] User

[0123] The user reviews the resulting image and further adjusts the sliders as needed, and this process is repeated as many times as necessary until the user is satisfied.

[0124] Step 14:

[0125] Terminal

[0126] The terminal transmits the readjusted slider setting value to the server again.

[0127] Step 15:

[0128] server

[0129] The server then uses the image generation AI to perform blending again based on the new slider settings, and a new image is generated.

[0130] Step 16:

[0131] server

[0132] Once the final generated image is confirmed, the server stores it and generates a download link that the user can access.

[0133] Step 17:

[0134] Terminal

[0135] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[0136] Step 18:

[0137] User

[0138] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[0139] The above are the processing steps of the system that allows a non-expert to generate a specific design image using multiple reference images.

[0140] Example 1

[0141] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0142] In modern design work, when non-experts try to create a concrete design based on their own ideas, they need specialized knowledge and skills, which takes a lot of time. Furthermore, sharing a specific design image requires a lot of trial and error, so an efficient design process is required. Given this background, there is a need for a system that allows non-experts to easily generate and share high-quality design images.

[0143] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0144] In this invention, the server includes a means for uploading reference images, a means for selecting specific portions of the uploaded reference images, a means for recording and transmitting coordinate information of the selected reference image portions to the server, an adjustment means for blending the multiple selected image portions, an image generation means using a generative AI model to generate a blended image, and a means for saving and sharing the generated image. This allows even non-experts to easily generate specific design images using multiple reference images and share them with others.

[0145] A "reference image" is an image file that a user uploads to be used as a design element.

[0146] "Non-professionals" refers to general users who do not have specialized design knowledge or skills.

[0147] "Uploading means" is a function that allows a user to send an image file from local storage to the server.

[0148] The "means for selecting a specific portion" is a tool that allows the user to specify and extract a specific portion of the reference image.

[0149] "Coordinate information" is X, Y position data that indicates the position of a specific part of the reference image.

[0150] The "transmitting means" is a communication means for transmitting data from the terminal to the server.

[0151] An "adjuster" is a device or interface for setting the blending ratio of multiple image portions.

[0152] A "generative AI model" is an algorithm or system for generating design images using artificial intelligence technology.

[0153] An "image generator" is a process or device for integrating portions of reference images to generate a new image.

[0154] The "means for saving" is a function for storing the generated image as data.

[0155] "Means for sharing" is a function for sharing the generated image with other users and designers.

[0156] The present invention provides a system for non-experts to generate specific design images using reference images. This system is operated by a user, a terminal, and a server. The specific configuration and operation of the system are described below.

[0157] System Configuration

[0158] User

[0159] The user accesses the system through an interface to upload a reference image, select specific parts, and adjust the blending. The user selects an image from local storage and uploads it to the system as a reference image. The user's operations are transmitted to the terminal via the interface.

[0160] Terminal

[0161] The terminal receives instructions from the user and temporarily stores data such as image files and coordinate information. The terminal also transmits information selected by the user to the server in real time and displays the results. The terminal acts as an interface for image processing.

[0162] server

[0163] The server processes the data sent from the devices and plays a central role in generating images. The server has the following functions:

[0164] Uploading and managing reference images: The server receives reference images uploaded by users, assigns a unique identifier, and stores them. It also records the storage location and identifier of each image in a database. Specifically, OpenCV is used as the image processing library.

[0165] Selection of a specific part of the reference image: The coordinate information of the specific part selected by the user is received, and the selected part is cropped to generate a new thumbnail image. This information is also recorded in the database.

[0166] Blending: Based on user input, an image-generating artificial intelligence (AI) model is used to combine features from selected images to generate a new blended image. This process uses a deep learning framework (e.g., TENSORFLOW® or PyTorch).

[0167] Saving and sharing the generated image: Once the final image is confirmed, the server saves it and creates a download link that users can access. They can use this link to download the image or share it with others using the sharing tool.

[0168] Specific examples

[0169] For example, if a user wants to create a logo design for a new product, they would follow these steps:

[0170] 1. The user uploads reference images of three different logo designs for a new product into the tool.

[0171] 2. Next, select specific parts of each image (e.g. font style, color, icon) using the rectangular selection tool.

[0172] 3. The device sends the coordinate information of the selected area to the server, and the server generates a new image based on that.

[0173] 4. The user uses the slider to adjust the blending of the images to achieve the desired design image.

[0174] 5. Review the final logo image, save it, and share the download link with your designer.

[0175] Prompt Sentence Examples

[0176] "I want to create a logo design for a new product. Upload reference images, select distinctive features from each image, and generate a new design."

[0177] The above is a specific embodiment for carrying out the present invention. This system enables even non-experts to easily create high-quality design images and share them efficiently.

[0178] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0179] Step 1: Upload a reference image

[0180] User

[0181] Users select reference images for their design from their local storage and upload them using the system interface. For example, they can choose three different images for a new product logo design.

[0182] Input: Image file in local storage

[0183] Output: Image file uploaded to the system

[0184] Terminal

[0185] The device temporarily stores the image file selected by the user and sends an upload request to the server, along with image metadata (such as file name, file size, and image format).

[0186] Input: Image file uploaded by the user

[0187] Output: Upload request to server

[0188] server

[0189] The server assigns a unique identifier (such as a UUID) to the received image file and stores it in the specified directory. It also records the storage path, identifier, and metadata of each image in a database.

[0190] Input: Image file and metadata sent from the device

[0191] Output: Image file information recorded in the database

[0192] ---

[0193] Step 2: Select and place an image

[0194] User

[0195] Users can use the rectangular selection or lasso tools to select the portion of an uploaded image they want to use, such as a specific logo, color pattern, or font style.

[0196] Input: Reference image uploaded to the system

[0197] Output: Coordinate information of the selected image part

[0198] Terminal

[0199] The terminal records the coordinate information (X and Y coordinates of the start and end points) of the area selected by the user and sends this coordinate information and the corresponding image identifier to the server.

[0200] Input: Coordinate information selected by the user

[0201] Output: Send coordinates and identifier to server

[0202] server

[0203] The server then crops the selected part based on the received coordinates, using an image processing library (e.g., OpenCV), saves the cropped image part as a newly generated thumbnail, and updates the database.

[0204] Input: Coordinate information and image identifier sent from the device

[0205] Output: Cropped thumbnail images stored in a database

[0206] ---

[0207] Step 3: Blending adjustments

[0208] User

[0209] The user adjusts the blending ratio of the selected image parts by manipulating the sliders, for example, 50% of image A, 30% of image B, and 20% of image C.

[0210] Input: Information about the selected image part

[0211] Output: Slider setting (blending ratio for each image part)

[0212] Terminal

[0213] The device sends the slider setting value specified by the user to the server in real time using a real-time protocol such as WebSocket.

[0214] Input: The slider setting adjusted by the user

[0215] Output: Send slider setting value to server

[0216] server

[0217] The server generates a new image based on the slider settings received using a generative AI model for blending the selected areas, using a deep learning framework (e.g., TensorFlow, PyTorch).

[0218] Input: Slider setting value and selection information

[0219] Output: The resulting blended image

[0220] ---

[0221] Step 4: Check and correct the generated image

[0222] User

[0223] The user reviews the generated image and adjusts the sliders again if necessary, repeating the process until they reach a design they are happy with.

[0224] Input: The generated blended image

[0225] Output: Rescaled slider value

[0226] Terminal

[0227] The terminal retransmits the user's new settings to the server and displays the latest generated image.

[0228] Input: The slider setting that the user has readjusted

[0229] Output: Sends the reset values ​​to the server and displays the latest image

[0230] server

[0231] The server then performs the blending process again based on the resubmitted settings and generates a new image, again using the generative AI model.

[0232] Input: Resent slider setting value

[0233] Output: Regenerated blended image

[0234] ---

[0235] Step 5: Save and share the generated image

[0236] server

[0237] Once the final image is confirmed, the server stores the generated image and creates a download link that users can access using a temporary token for security reasons.

[0238] Input: Final generated image

[0239] Output: Download link

[0240] Terminal

[0241] The device will display this download link to the user, who can click the link to download the image or use the share feature to share it with others.

[0242] Input: Download link sent by the server

[0243] Output: Display link to user

[0244] User

[0245] The user can send the final, concrete design image to the designer, allowing the designer to proceed with the detailed design based on the user's concrete image.

[0246] Input: Generated image obtained from the download link

[0247] Output: Share the design image with the designer

[0248] The above is a tentative concrete flow of the program processing of this system.

[0249] (Application example 1)

[0250] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0251] Creating advertising materials easily often requires specialized design knowledge and advanced techniques, making it difficult for non-experts to create them. Furthermore, the process of effectively utilizing multiple reference images held by users to generate specific design images is inefficient and time-consuming. Furthermore, there is a lack of methods for appropriately blending multiple reference images to generate new designs. There is a need for a system that can solve these issues and enable anyone to easily generate high-quality advertising materials.

[0252] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0253] In this invention, the server includes a means for uploading reference images, a means for selecting specific portions of the uploaded reference images, and an adjustment means for blending the selected image portions. This allows even non-experts to easily generate specific design images using multiple reference images and use them as advertising materials. Furthermore, the server includes a means for providing prompts during the image generation process, allowing users to easily select and adjust designs.

[0254] "Reference Image" means an image file used to form part or all of a design.

[0255] "Non-professionals" refer to general users who do not have specialized knowledge or advanced skills in design or image editing.

[0256] A "specific design image" is the final visual expression that the user is aiming for.

[0257] "Upload" refers to the act of sending data from a user's terminal to a server.

[0258] A "selection tool" is a tool or interface for specifying and cutting out a particular portion of a reference image.

[0259] The "blending adjustment means" is an interface for setting how to appropriately combine multiple selected image portions.

[0260] "Image generation method" refers to a process or technique that generates a new image based on selected image portions and configured blending adjustments.

[0261] "Prompts" are instructions or hints provided to the user to assist in generating advertising material.

[0262] "Advertising Materials" means visual, textual or other form of design elements used for advertising purposes.

[0263] "Storage and sharing means" refers to an interface or function for long-term storage of generated images and sharing with other users and designers.

[0264] This invention provides a system that allows non-experts to create specific design images using multiple reference images. This system is operated by a user, a terminal, and a server. The operation of the program and its processing are described in detail below.

[0265] Uploading a reference image

[0266] A user selects multiple reference images from their local storage and uploads them to the server via their device. Specifically, the user uses an interface for uploading reference images from a smartphone or tablet and sends the data to the server. The device then sends an upload request to the server, and the server stores the received image files with a unique identifier.

[0267] Selecting and arranging images

[0268] The user selects a specific part of the uploaded reference image using a rectangular selection tool or lasso tool. For example, a specific logo or color can be selected. The device records the coordinate information of the selected part and sends it to the server. The server then crops the selected part based on the received coordinate information and saves a newly generated thumbnail image.

[0269] Blending Adjustment

[0270] The user adjusts the blending ratio of the selected image parts by operating the sliders. For example, they can set 50% of image A, 30% of image B, and 20% of image C. The device sends the slider settings to the server in real time. Based on the received slider settings, the server generates a new blended image using an image generation AI model. This AI model specifically integrates the features of multiple reference images to generate a new design image.

[0271] Checking and correcting generated images

[0272] The user reviews the generated image and adjusts the sliders again if necessary. This process can be repeated until the user is satisfied. The device resends the user's adjusted settings and displays the latest generated image. The server re-performs the blending process based on the adjusted settings and generates a new image.

[0273] Saving and sharing generated images

[0274] Once the final design image is confirmed, the server saves the generated image and generates a download link that users can access and download, or share with others using the sharing tool.

[0275] Hardware and software used

[0276] Hardware: User's smartphone, tablet, server

[0277] Software: PIL library for image processing, requests library for HTTP communication, image generation model

[0278] Specific examples

[0279] If a user wants to create advertising materials for a new product, they can proceed using the following prompt:

[0280] "Please upload three images:

[0281] 1. Logo image

[0282] 2. Color-Scheme Images

[0283] 3. Sample images of the design

[0284] Select the part of each image you want to use:

[0285] Logo: 10, 10, 100, 100

[0286] Color scheme: 20, 20, 120, 120

[0287] Design samples: 30, 30, 140, 140

[0288] Set the blend ratio:

[0289] Logo: 50%

[0290] Color Scheme: 30%

[0291] Design sample: 20%

[0292] Review the generated ad assets, adjust them again if necessary, and save or share the final ad asset you're happy with.

[0293] In this way, a system is provided that enables even non-experts to easily generate high-quality advertising materials.

[0294] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0295] Step 1: Upload an image

[0296] Input: Reference images stored in the user's local storage.

[0297] How it works: The user selects a reference image from their smartphone or tablet and uploads it to the server using the application's interface.

[0298] Data processing and calculation: The terminal temporarily saves the selected image file and sends the image data to the server via an HTTP request.

[0299] Output: Reference image and unique identifier stored on the server.

[0300] Step 2: Select an image area

[0301] Input: Reference images uploaded to the server.

[0302] What it does: The user uses the in-app rectangular selection tool or lasso tool to select a specific part of the reference image.

[0303] Data processing and calculation: The terminal records the coordinate information of the part selected by the user and sends it to the server.

[0304] Output: Coordinate information recorded on the server and thumbnail images of the cropped areas.

[0305] Step 3: Blending adjustments

[0306] Input: Coordinate information and image data of the selected area recorded on the server.

[0307] What it does: The user manipulates an in-app slider to set the blend ratio of multiple selections.

[0308] Data processing and calculation: The device sends the slider values ​​to the server in real time. The server generates a new blended image based on these settings using an image generation AI model.

[0309] Output: The new image data generated.

[0310] Step 4: Check and adjust the generated image

[0311] Input: Generated image sent from the server to the device.

[0312] Specific operation: The user can check the generated image within the app and adjust the sliders again to correct the blending ratio if necessary.

[0313] Data processing and calculation: The device sends the adjusted slider values ​​to the server, and the server performs blending again based on the new settings.

[0314] Output: The new generated image data after correction.

[0315] Step 5: Save and share the resulting image

[0316] Input: The final generated image data.

[0317] Specific operation: Once the user finalizes the design image, the server saves the image and generates a download link, which the user can use to download the image or share it with other users.

[0318] Output: Download link and final generated image data.

[0319] By following the steps above, even non-experts can easily generate advertising materials and efficiently create the desired design.

[0320] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0321] The present invention is a system that allows non-experts to create specific design images using reference images, and by combining it with an emotion engine that recognizes the user's emotions, it makes the design process smoother and more personalized. This system is executed by the user, a terminal, and a server. The program processing is described in detail below.

[0322] Program processing

[0323] Uploading a reference image

[0324] User

[0325] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[0326] Terminal

[0327] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[0328] server

[0329] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the saved location and identifier in the database.

[0330] Selecting and arranging images

[0331] User

[0332] The user uses the system interface to view a list of uploaded reference images. Each image is displayed as a thumbnail. The user can then use the rectangular selection tool or lasso tool to select specific portions of the reference images. For example, select a specific logo from "image1.jpg."

[0333] Terminal

[0334] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[0335] server

[0336] The server then cuts out a specific portion of the reference image based on the received coordinate information, and saves the cut-out portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[0337] Blending Adjustment

[0338] User

[0339] The user adjusts the blending strength of the selected image portion by manipulating the sliders, which adjust the transparency and blending strength of each reference image portion.

[0340] Terminal

[0341] The terminal transmits the slider value set by the user to the server in real time.

[0342] server

[0343] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[0344] server

[0345] The new image generated is saved on the server, for example as "blend_result.jpg".

[0346] emotion recognition

[0347] User

[0348] To recognize the user's emotions in real time, the emotion engine analyzes the user's facial expressions and vocal tone. Once the emotion data is acquired, the user's current emotional state is determined.

[0349] Terminal

[0350] The device receives emotion data from the emotion engine and transmits it to the server. For example, emotions such as "happiness," "surprise," and "sadness" are sent as data.

[0351] server

[0352] The server analyzes this emotion data and suggests the selection of reference images and automatic adjustment of blending conditions. It also automatically adjusts the characteristics of the generated image (hue, brightness, contrast, etc.) based on the emotion data.

[0353] Check and correct the generated image

[0354] User

[0355] The user reviews the blended image adjusted based on the emotion engine's suggestions, and further adjusts the sliders as needed. This process is repeated as many times as necessary until the user is satisfied.

[0356] Terminal

[0357] The terminal transmits the readjusted slider setting value to the server again.

[0358] server

[0359] The server then uses the image generation AI to perform the blending process again based on the new slider settings, and a new image is generated.

[0360] Saving and sharing generated images

[0361] server

[0362] Once the final generated image is confirmed, the server stores it and generates a download link that the user can access.

[0363] Terminal

[0364] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[0365] User

[0366] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[0367] Example: Product logo design

[0368] scenario

[0369] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[0370] User

[0371] Users upload three reference images of different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[0372] Server and Device

[0373] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[0374] emotion recognition

[0375] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[0376] Final production and sharing

[0377] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[0378] The above is a specific embodiment for carrying out the present invention.

[0379] The processing flow will be explained below.

[0380] Program processing

[0381] Step 1:

[0382] User

[0383] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[0384] Step 2:

[0385] Terminal

[0386] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[0387] Step 3:

[0388] server

[0389] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the saved location and identifier in the database.

[0390] Step 4:

[0391] User

[0392] The user uses the system's interface to view a list of reference images they have uploaded, each displayed as a thumbnail.

[0393] Step 5:

[0394] User

[0395] The user uses the rectangular selection tool or lasso tool to select a specific portion of the reference image, for example, a specific logo portion from "image1.jpg."

[0396] Step 6:

[0397] Terminal

[0398] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[0399] Step 7:

[0400] server

[0401] The server then cuts out a specific portion of the reference image based on the received coordinate information, and saves the cut-out portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[0402] Step 8:

[0403] User

[0404] The user uses sliders to adjust the blending of selected image parts, adjusting the transparency and overlay strength of each reference image part.

[0405] Step 9:

[0406] Terminal

[0407] The terminal transmits the slider value set by the user to the server in real time.

[0408] Step 10:

[0409] server

[0410] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[0411] Step 11:

[0412] server

[0413] The new image generated is saved on the server, for example as "blend_result.jpg".

[0414] Step 12:

[0415] User

[0416] To recognize the user's emotions in real time, the emotion engine analyzes the user's facial expressions and vocal tone. Based on the acquired emotion data, the user's current emotional state is determined.

[0417] Step 13:

[0418] Terminal

[0419] The device receives emotion data from the emotion engine and transmits it to the server. For example, emotions such as "joy" or "surprise" are sent as data.

[0420] Step 14:

[0421] server

[0422] The server analyzes this emotion data and suggests reference image selection and automatic blending adjustments. For example, if the user expresses a "sad" emotion, the system suggests a gentler color tone.

[0423] Step 15:

[0424] User

[0425] The user reviews the blended image adjusted based on the emotion engine's suggestions, further adjusts the sliders as needed, and repeats this process until the user is satisfied.

[0426] Step 16:

[0427] Terminal

[0428] The terminal transmits the readjusted slider setting value to the server again.

[0429] Step 17:

[0430] server

[0431] The server then uses the image generation AI to perform the blending process again based on the new slider settings, and a new image is generated.

[0432] Step 18:

[0433] server

[0434] Once the final image is confirmed, the server saves it and generates a download link for the user to access, for example, " / downloads / final_image123.jpg".

[0435] Step 19:

[0436] Terminal

[0437] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[0438] Step 20:

[0439] User

[0440] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[0441] Example: Product logo design

[0442] scenario

[0443] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[0444] User

[0445] Users upload three reference images of different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[0446] Server and Device

[0447] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[0448] emotion recognition

[0449] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[0450] Final production and sharing

[0451] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[0452] The above is a specific embodiment for carrying out the present invention.

[0453] Example 2

[0454] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0455] Conventional design systems have the problem that it is difficult for non-experts to generate concrete design images, and they also have the problem that they cannot incorporate user emotions into the design process, making it difficult to provide a personalized user experience.

[0456] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for uploading a reference image, a means for selecting a specific portion of the uploaded reference image, an adjustment means for blending the selected image portions, an image generation means for generating a blended image, a means for recognizing a user's emotions in real time and suggesting adjustments to the generated image, and a means for saving and sharing the generated image. This allows even a non-expert to easily generate specific design images and propose individualized designs based on the user's emotions.

[0457] A "reference image" is an image file that a user uses when generating a design image.

[0458] "Means for uploading" is a function that allows a user to send a reference image from local storage to the system.

[0459] The "means for selecting a specific portion" is a tool that allows the user to select a portion of interest from within the reference image.

[0460] The "adjustment means for blending" is a function that adjusts the transparency and overlay strength of the image portion selected by the user.

[0461] The "image generation means" is a function that integrates the blended image parts to create a new design image.

[0462] "Means for recognizing emotions in real time" is a function that analyzes the user's facial expressions and vocal tone to determine their current emotional state.

[0463] The "means for saving and sharing the generated image" is a function for saving the final generated design image and sharing it with other users.

[0464] The present invention is a system that allows non-experts to generate specific design images using multiple reference images. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the design process becomes smoother and more personalized. Specific embodiments for implementing the present invention are described below.

[0465] Uploading a reference image

[0466] User

[0467] The user selects multiple reference images from their local storage and uploads them using the system interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[0468] Terminal

[0469] The terminal stores the image file selected by the user in a temporary storage and transmits an upload request to the server. For example, the terminal stores the image in a temporary folder in the local storage and acquires the file path.

[0470] server

[0471] The server receives image files uploaded from the device, assigns a unique identifier to each file, and saves them in a specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the location and identifier of each image in a database.

[0472] Selecting and arranging images

[0473] User

[0474] The user uses the system interface to view a list of uploaded reference images. Each image is displayed as a thumbnail. The user can then use the rectangular selection tool or lasso tool to select specific portions of the reference images. For example, select a specific logo from "image1.jpg."

[0475] Terminal

[0476] The device records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server. Specifically, the coordinate data of the selected range is sent to the server in JSON format.

[0477] server

[0478] The server then extracts a specific portion of the reference image based on the received coordinate information, saves the extracted portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg," and records the metadata of the new image in the database.

[0479] Blending Adjustment

[0480] User

[0481] Users can adjust the blending of selected image parts by adjusting the slider: moving it to the right increases transparency, moving it to the left increases opacity, and also adjust the strength of the blending.

[0482] Terminal

[0483] The device sends the slider values ​​(transparency and overlay strength) set by the user to the server in real time. Specifically, the values ​​are updated sequentially according to the slider operation and sent to the server.

[0484] server

[0485] The server then sends blending instructions to the image generation AI model based on the slider settings received. The AI ​​model then uses an algorithm to combine the characteristics of each reference image part and generate a new image based on the user's settings. The new image is saved as "blend_result.jpg."

[0486] emotion recognition

[0487] User

[0488] To recognize a user's emotions in real time, the emotion engine analyzes facial expressions and vocal tone, for example, by capturing a picture of the user's face with a camera and recording the user's voice with a microphone.

[0489] Terminal

[0490] The device receives emotion data obtained from the emotion engine and sends it to the server. For example, emotion data such as "happiness," "surprise," and "sadness" is sent in JSON format.

[0491] server

[0492] The server analyzes this emotion data and suggests the selection of reference images and automatic adjustment of blending. It automatically adjusts the characteristics of the generated image (hue, brightness, contrast, etc.) based on the emotion data to provide the optimal design for the user.

[0493] Check and correct the generated image

[0494] User

[0495] The user can then review the blended image adjusted based on the emotion engine's suggestions and, if necessary, further adjust the sliders, for example, fine-tuning the hue, brightness, and contrast.

[0496] Terminal

[0497] The terminal transmits the readjusted slider setting value to the server again, specifically, the latest setting value according to the user's readjustment.

[0498] server

[0499] The server then uses the image generation AI model to blend the new slider settings and generate a new image. This process is repeated until the user is satisfied.

[0500] Saving and sharing generated images

[0501] server

[0502] Once the final image is confirmed, the server saves it and generates a download link that the user can access, for example, "final_design.jpg" and creates a download link.

[0503] Terminal

[0504] The device will display the generated download link to the user, who can click the link to download the image or use the sharing tool to share it with other designers.

[0505] User

[0506] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with detailed design work based on the user's concrete image.

[0507] Example: Product logo design

[0508] scenario

[0509] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[0510] User

[0511] Users upload three different reference images of logo designs to the tool and then select specific parts of each image (e.g., font style, color, icon) using the rectangular selection tool.

[0512] Server and Device

[0513] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[0514] emotion recognition

[0515] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[0516] Final production and sharing

[0517] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[0518] Example prompts for generative AI models

[0519] For example, the prompt to generate a new logo design might look like this:

[0520] "Generate a modern and elegant logo. Integrate font styles, colors, and icons from reference images to add a bright and energetic element."

[0521] The above is a specific embodiment for carrying out the present invention.

[0522] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0523] Step 1: Upload a reference image

[0524] User

[0525] The user selects multiple reference images from local storage using the system interface and clicks the upload button. As input, the user provides the image files (e.g., "image1.jpg", "image2.png", "image3.gif").

[0526] Terminal

[0527] The device temporarily saves the input image file in local storage and obtains the file path. Then, it sends an upload request to the server. As an output, it generates the image file path.

[0528] server

[0529] The server receives image files sent from the device, assigns a unique identifier to each, and saves them in a specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the storage location and identifier of each image in a database. As output, it generates the saved image file path and identifier.

[0530] Step 2: Select and place an image

[0531] User

[0532] The user sees a list of reference images uploaded on the system interface. Each image is displayed as a thumbnail, and the user can select specific parts using the rectangular selection tool or lasso tool. The thumbnail of the reference image is provided as input. The user selects a specific logo part from "image1.jpg".

[0533] Terminal

[0534] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server, which generates the coordinate data of the selected part as output.

[0535] server

[0536] The server then clips the specific part of the reference image based on the received coordinate information, saves it as a new image file, for example, " / uploads / 12345_thumbnail.jpg", records the metadata of the new image in the database, and generates the clipped image file path as output.

[0537] Step 3: Blending adjustments

[0538] User

[0539] The user operates sliders to adjust the blending of selected image parts, setting the transparency and overlay strength by moving the sliders. The slider position is provided as input.

[0540] Terminal

[0541] The device sends the slider values ​​(transparency and overlay strength) set by the user to the server in real time, and generates the setting value data as output.

[0542] server

[0543] The server instructs the image generation AI model to perform blending based on the received slider settings. The image generation AI model generates a new image by integrating the characteristics of each reference image part. The generated new image is saved as "blend_result.jpg". As output, it generates the generated image file.

[0544] Step 4: Emotion Recognition

[0545] User

[0546] The emotion engine analyzes the user's facial expressions and voice tone in real time. The inputs are the user's facial images and voice data captured by the camera and microphone.

[0547] Terminal

[0548] The terminal receives emotion data obtained from the emotion engine and transmits it to the server. For example, emotion data such as "happiness," "surprise," and "sadness" is transmitted. Emotion data is generated as output.

[0549] server

[0550] The server analyzes this emotion data, suggests reference image selection and automatic blending adjustment, and automatically adjusts the characteristics (hue, brightness, contrast, etc.) of the generated image based on the emotion data. The adjusted properties are generated as output.

[0551] Step 5: Check and correct the generated image

[0552] User

[0553] The user reviews the blended image based on the adjustments suggested by the system and readjusts the sliders as needed, providing the adjusted image as input, e.g., fine-tuning hue, brightness, and contrast.

[0554] Terminal

[0555] The device sends the readjusted slider setting value back to the server, which generates the new setting value as output.

[0556] server

[0557] The server then applies the blending process again using the image generation AI model based on the new slider settings to generate a new image. This process is repeated until the user is satisfied. The output is a regenerated image file.

[0558] Step 6: Save and share the generated image

[0559] server

[0560] Once the final image is confirmed, the server saves it and generates a download link that the user can access. For example, it generates "final_design.jpg" and creates a download link. The final design image is provided as input. The download link is generated as output.

[0561] Terminal

[0562] The terminal displays the generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool. The download link is displayed as an output.

[0563] User

[0564] The user downloads the final, concrete design image and sends it to the designer, allowing the designer to proceed with detailed design work based on the user's concrete image. The downloaded image file is provided as output.

[0565] (Application example 2)

[0566] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0567] Current design generation systems have difficulty providing interfaces that non-experts can use effectively, and do not adequately customize based on the emotional state of individual users. Furthermore, the content provided in autonomous vehicles is not optimized according to the emotional state of passengers, which prevents passenger satisfaction from being fully achieved.

[0568] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for uploading a reference image, means for selecting specific portions of the uploaded reference image, means for adjusting the blending of the selected image portions, means for generating an image using a generative AI model for generating a blended image, means for saving and sharing the generated image, means for recognizing passenger emotions in real time, and means for adjusting the generated content based on the emotion data. This enables even non-experts to easily generate specific design images and optimize the content in an autonomous vehicle according to the passenger's emotional state.

[0569] "Reference images" are image files used for design and content generation.

[0570] "Means for uploading" refers to a function that allows a user to transfer data from a local environment to a server.

[0571] The "means for selecting a specific portion" refers to a tool or software function for selecting a specific area of ​​a reference image.

[0572] "Adjustment means" refers to a function for adjusting the blending of selected image portions.

[0573] A "generative AI model" refers to a model that uses artificial intelligence to generate new images and content.

[0574] "Image generation means" refers to a function for generating images using a generative AI model.

[0575] "Means for storing and sharing" refers to the functionality for storing generated content and sharing it with other users and systems.

[0576] "Means of recognizing emotions in real time" refers to a function that analyzes passengers' facial expressions and voice to determine their emotional state at that time.

[0577] "Emotional Data" refers to emotional state information obtained by means of real-time emotion recognition.

[0578] "Means for adjusting content" refers to functions for optimizing the advertisements and entertainment content displayed based on the acquired emotional data.

[0579] The present invention is a system that makes it easy for even non-experts to generate specific design images and provides content based on the emotions of passengers in autonomous vehicles. Specific embodiments are described below.

[0580] System configuration

[0581] This system consists of users, terminals, and a server. To realize the system, the following hardware and software are required:

[0582] 1. Hardware

[0583] Camera (for facial recognition inside the car)

[0584] In-vehicle computer

[0585] High-Performance Server

[0586] 2. Software

[0587] Facial expression recognition API (e.g. facial expression recognition API)

[0588] Image generation AI (e.g. TensorFlow)

[0589] Database system (e.g. MySQL (registered trademark))

[0590] Interface tools (e.g., Marquee tool, Lasso tool)

[0591] Program processing flow

[0592] 1. Acquiring emotion recognition data

[0593] User: A passenger enters the vehicle.

[0594] Device: An in-car camera captures passengers' faces and analyzes them in real time using an emotion recognition API.

[0595] Server: The analysis results (emotion data) are sent to the server.

[0596] 2. Emotion-Based Design Generation

[0597] Server: Receives emotion data and initiates the design generation process. Users upload reference images and use the interface to select specific parts of the image. The selected image parts are stored in a database.

[0598] User: Use the slider to adjust the blending amount.

[0599] Server: The image generation AI generates new images based on the user's settings. This time, the generative AI model is used.

[0600] 3. Adjust your content

[0601] Server: Automatically adjusts the generated blended images and content based on emotion data. If a passenger expresses joy, bright colors and energetic content will be generated.

[0602] 4. Final review and feedback

[0603] User: Review the generated content and adjust the sliders as needed. The adjusted data is resubmitted to the server and new content is generated. This process is repeated until the user is satisfied.

[0604] Example scenario

[0605] The specific processing flow is illustrated based on the following scenario:

[0606] Scenario: A passenger is perceived to have a depressed expression.

[0607] Emotion Recognition: The emotion recognition module detects "sadness."

[0608] Content generation: The system generates light-hearted comedy videos and refreshing landscape images to turn sadness into joy.

[0609] Feedback: The passenger's expression becomes happy, and the system records this change and reflects it next time.

[0610] Prompt Sentence Examples

[0611] An example of a prompt might be:

[0612] 1. The camera captures passengers' facial expression data and analyzes it in real time.

[0613] 2. If the emotion is recognized as "sadness," the AI ​​model is used to generate ads with upbeat text and images, as well as comedy videos.

[0614] 3. The generated content is displayed on the in-car display, and passenger feedback is retrieved using a slider and reflected in the system.

[0615] The above is a specific embodiment for carrying out the present invention.

[0616] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0617] Step 1: Obtaining emotion data

[0618] Subject: Terminal

[0619] Specific explanation: The in-car camera detects the passenger's face and analyzes the emotional data using a facial expression recognition API. The analyzed data is output as an emotional status such as "happiness," "surprise," or "sadness" and sent to the server.

[0620] Input: Passenger's face image

[0621] Output: Emotion data

[0622] How it works: The device's camera captures facial images, and the facial recognition API extracts facial features from the images and quantifies the emotional state.

[0623] Step 2: Upload a reference image

[0624] Subject: User

[0625] Specific Description: The user selects multiple reference images from the device and uploads them through the system interface. The uploaded images are temporarily stored on the device.

[0626] Input: User selected reference image file

[0627] Output: Reference images uploaded to the server

[0628] Operation details: When the user selects a reference image from local storage and clicks the upload button, the selected image is saved on the device and an upload request is sent to the server.

[0629] Step 3: Select an image area

[0630] Subject: User

[0631] Specific explanation: The user uses the interface to select a specific part of the uploaded reference image using the rectangular selection tool or lasso tool. The coordinate information of the selected part is recorded.

[0632] Input: Reference image and selection tool operation information

[0633] Output: Coordinate data of the selected image part

[0634] Operation details: The user selects an image portion on the interface, and the coordinate information of the selected area is recorded and transmitted by the device.

[0635] Step 4: Blending image parts

[0636] Subject: User

[0637] What it means: The user uses sliders in the interface to adjust the transparency and blending strength of multiple selected image parts, setting the blending effect.

[0638] Input: Coordinate data of the selected image part and user slider setting value

[0639] Output: Blending instruction data based on the set value

[0640] How it works: The user adjusts the slider and the setting is sent to the server in real time.

[0641] Step 5: Image generation

[0642] Subject: Server

[0643] Specific explanation: The server generates a new image using the image generation AI model based on the received slider setting value. The generated image is saved on the server.

[0644] Input: Blending instruction data

[0645] Output: The new image generated

[0646] Operation details: The server inputs instruction data into the generative AI model, and the model outputs a new image, which is saved in a specific directory on the server.

[0647] Step 6: Tailor content based on sentiment data

[0648] Subject: Server

[0649] Specifically, the server automatically adjusts the color, brightness, and content type of the generated images and content based on the emotional data.

[0650] Input: Generated images and emotion data

[0651] Output: Emotionally optimized content

[0652] How it works: The server analyzes the emotion data and adjusts the parameter settings of the image generation AI model to recreate new images and content.

[0653] Step 7: Final review and feedback

[0654] Subject: User

[0655] Specifically: The user can check the generated content and make further adjustments using the sliders. The adjusted settings are sent to the server and the content is generated again.

[0656] Input: Generated content and user feedback

[0657] Output: The final generated content

[0658] How it works: The user reviews the content, makes any necessary adjustments, and the results are sent to the server, which regenerates the content.

[0659] Step 8: Final saving and sharing

[0660] Subject: Server

[0661] Specific description: The server stores the final generated content and generates a download link that can be accessed by the user.

[0662] Input: Finalized content

[0663] Output: Download link

[0664] What it does: The server stores the generated content, generates a link in a format that the user can access, and displays it to the user.

[0665] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0666] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0667] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0668] [Second embodiment]

[0669] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0670] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0671] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0672] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0673] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0674] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0675] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0676] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0677] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0678] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0679] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0680] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0681] The present invention provides a system for non-experts to create specific design images using reference images. The system is operated by a user, a terminal, and a server. The program processing will be described in detail below.

[0682] Program processing

[0683] Uploading a reference image

[0684] User

[0685] The user selects multiple reference images from their local storage and uploads them through the tool's interface, for example, selecting three different image files.

[0686] Terminal

[0687] The terminal temporarily stores these image files according to the user's selection and sends an upload request to the server.

[0688] server

[0689] The server receives the uploaded image files, assigns a unique identifier to each image, and stores it in a specified directory. It also records the location and identifier of each image in a database.

[0690] Selecting and arranging images

[0691] User

[0692] Users can use the rectangular selection tool or lasso tool to select the part of an uploaded reference image they want to use, such as a specific logo or color.

[0693] Terminal

[0694] The terminal records the coordinate information of the part selected by the user and transmits it to the server.

[0695] server

[0696] The server then clips out the selected portion based on the received coordinate information, saves the newly generated thumbnail image, and updates the information about the selected portion in the database.

[0697] Blending Adjustment

[0698] User

[0699] The user adjusts the blending ratio of the selected image by using sliders, for example, 50% of image A, 30% of image B, and 20% of image C.

[0700] Terminal

[0701] The device sends the slider setting value to the server in real time.

[0702] server

[0703] The server generates a new blended image based on the received slider settings using image generation AI, which integrates the features of each image based on the settings.

[0704] Checking and correcting generated images

[0705] User

[0706] The user reviews the resulting image and adjusts the sliders again if necessary, repeating the process until satisfied.

[0707] Terminal

[0708] The terminal retransmits the user's readjusted settings and displays the latest generated image.

[0709] server

[0710] The server then performs the blending process again based on the readjusted settings to generate a new image.

[0711] Saving and sharing generated images

[0712] server

[0713] Once the final image is finalized, the server stores the generated image and generates a download link that the user can access.

[0714] Terminal

[0715] The device will display this download link to the user, who can click the link to download it or use the sharing tool to share it with other designers.

[0716] User

[0717] The user can send the final, concrete design image to the designer, allowing the designer to proceed with the detailed design based on the user's concrete image.

[0718] Example: Product logo design

[0719] scenario

[0720] Suppose a user wants to create a logo design for a new product.

[0721] User

[0722] Users upload three reference images of different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[0723] Server and Device

[0724] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[0725] Final production and sharing

[0726] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[0727] The above is a specific embodiment for carrying out the present invention.

[0728] The processing flow will be explained below.

[0729] Step 1:

[0730] User

[0731] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[0732] Step 2:

[0733] Terminal

[0734] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[0735] Step 3:

[0736] server

[0737] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, "image1.jpg" is saved to " / uploads / 12345.jpg."

[0738] Step 4:

[0739] server

[0740] The server records the storage location and identifier of each image in a database, which allows it to be referenced in subsequent processing.

[0741] Step 5:

[0742] User

[0743] The user uses the system's interface to view a list of reference images they have uploaded, each displayed as a thumbnail.

[0744] Step 6:

[0745] User

[0746] The user uses the rectangular selection tool or lasso tool to select a specific portion of the reference image, for example, a rectangular selection of a specific logo from "image1.jpg."

[0747] Step 7:

[0748] Terminal

[0749] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[0750] Step 8:

[0751] server

[0752] The server then cuts out a specific portion of the reference image based on the received coordinate information. The cut-out portion is saved as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[0753] Step 9:

[0754] User

[0755] The user uses sliders to adjust the blending of the cropped image parts, adjusting the transparency and blending strength of each reference image part.

[0756] Step 10:

[0757] Terminal

[0758] The terminal transmits the slider value set by the user to the server in real time.

[0759] Step 11:

[0760] server

[0761] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[0762] Step 12:

[0763] server

[0764] The new image generated is saved on the server, for example as "blend_result.jpg".

[0765] Step 13:

[0766] User

[0767] The user reviews the resulting image and further adjusts the sliders as needed, and this process is repeated as many times as necessary until the user is satisfied.

[0768] Step 14:

[0769] Terminal

[0770] The terminal transmits the readjusted slider setting value to the server again.

[0771] Step 15:

[0772] server

[0773] The server then uses the image generation AI to perform blending again based on the new slider settings, and a new image is generated.

[0774] Step 16:

[0775] server

[0776] Once the final generated image is confirmed, the server stores it and generates a download link that the user can access.

[0777] Step 17:

[0778] Terminal

[0779] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[0780] Step 18:

[0781] User

[0782] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[0783] The above are the processing steps of the system that allows a non-expert to generate a specific design image using multiple reference images.

[0784] Example 1

[0785] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0786] In modern design work, when non-experts try to create a concrete design based on their own ideas, they need specialized knowledge and skills, which takes a lot of time. Furthermore, sharing a specific design image requires a lot of trial and error, so an efficient design process is required. Given this background, there is a need for a system that allows non-experts to easily generate and share high-quality design images.

[0787] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0788] In this invention, the server includes a means for uploading reference images, a means for selecting specific portions of the uploaded reference images, a means for recording and transmitting coordinate information of the selected reference image portions to the server, an adjustment means for blending the multiple selected image portions, an image generation means using a generative AI model to generate a blended image, and a means for saving and sharing the generated image. This allows even non-experts to easily generate specific design images using multiple reference images and share them with others.

[0789] A "reference image" is an image file that a user uploads to be used as a design element.

[0790] "Non-professionals" refers to general users who do not have specialized design knowledge or skills.

[0791] "Uploading means" is a function that allows a user to send an image file from local storage to the server.

[0792] The "means for selecting a specific portion" is a tool that allows the user to specify and extract a specific portion of the reference image.

[0793] "Coordinate information" is X, Y position data that indicates the position of a specific part of the reference image.

[0794] The "transmitting means" is a communication means for transmitting data from the terminal to the server.

[0795] An "adjuster" is a device or interface for setting the blending ratio of multiple image portions.

[0796] A "generative AI model" is an algorithm or system for generating design images using artificial intelligence technology.

[0797] An "image generator" is a process or device for integrating portions of reference images to generate a new image.

[0798] The "means for saving" is a function for storing the generated image as data.

[0799] "Means for sharing" is a function for sharing the generated image with other users and designers.

[0800] The present invention provides a system for non-experts to generate specific design images using reference images. This system is operated by a user, a terminal, and a server. The specific configuration and operation of the system are described below.

[0801] System Configuration

[0802] User

[0803] The user accesses the system through an interface to upload a reference image, select specific parts, and adjust the blending. The user selects an image from local storage and uploads it to the system as a reference image. The user's operations are transmitted to the terminal via the interface.

[0804] Terminal

[0805] The terminal receives instructions from the user and temporarily stores data such as image files and coordinate information. The terminal also transmits information selected by the user to the server in real time and displays the results. The terminal acts as an interface for image processing.

[0806] server

[0807] The server processes the data sent from the devices and plays a central role in generating images. The server has the following functions:

[0808] Uploading and managing reference images: The server receives reference images uploaded by users, assigns a unique identifier, and stores them. It also records the storage location and identifier of each image in a database. Specifically, OpenCV is used as the image processing library.

[0809] Selection of a specific part of the reference image: The coordinate information of the specific part selected by the user is received, and the selected part is cropped to generate a new thumbnail image. This information is also recorded in the database.

[0810] Blending: Based on user input, an image-generating artificial intelligence (AI) model is used to combine features from selected images to generate a new blended image, using a deep learning framework (e.g., TensorFlow or PyTorch).

[0811] Saving and sharing the generated image: Once the final image is confirmed, the server saves it and creates a download link that users can access. They can use this link to download the image or share it with others using the sharing tool.

[0812] Specific examples

[0813] For example, if a user wants to create a logo design for a new product, they would follow these steps:

[0814] 1. The user uploads reference images of three different logo designs for a new product into the tool.

[0815] 2. Next, select specific parts of each image (e.g. font style, color, icon) using the rectangular selection tool.

[0816] 3. The device sends the coordinate information of the selected area to the server, and the server generates a new image based on that.

[0817] 4. The user uses the slider to adjust the blending of the images to achieve the desired design image.

[0818] 5. Review the final logo image, save it, and share the download link with your designer.

[0819] Prompt Sentence Examples

[0820] "I want to create a logo design for a new product. Upload reference images, select distinctive features from each image, and generate a new design."

[0821] The above is a specific embodiment for carrying out the present invention. This system enables even non-experts to easily create high-quality design images and share them efficiently.

[0822] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0823] Step 1: Upload a reference image

[0824] User

[0825] Users select reference images for their design from their local storage and upload them using the system interface. For example, they can choose three different images for a new product logo design.

[0826] Input: Image file in local storage

[0827] Output: Image file uploaded to the system

[0828] Terminal

[0829] The device temporarily stores the image file selected by the user and sends an upload request to the server, along with image metadata (such as file name, file size, and image format).

[0830] Input: Image file uploaded by the user

[0831] Output: Upload request to server

[0832] server

[0833] The server assigns a unique identifier (such as a UUID) to the received image file and stores it in the specified directory. It also records the storage path, identifier, and metadata of each image in a database.

[0834] Input: Image file and metadata sent from the device

[0835] Output: Image file information recorded in the database

[0836] ---

[0837] Step 2: Select and place an image

[0838] User

[0839] Users can use the rectangular selection or lasso tools to select the portion of an uploaded image they want to use, such as a specific logo, color pattern, or font style.

[0840] Input: Reference image uploaded to the system

[0841] Output: Coordinate information of the selected image part

[0842] Terminal

[0843] The terminal records the coordinate information (X and Y coordinates of the start and end points) of the area selected by the user and sends this coordinate information and the corresponding image identifier to the server.

[0844] Input: Coordinate information selected by the user

[0845] Output: Send coordinates and identifier to server

[0846] server

[0847] The server then crops the selected part based on the received coordinates, using an image processing library (e.g., OpenCV), saves the cropped image part as a newly generated thumbnail, and updates the database.

[0848] Input: Coordinate information and image identifier sent from the device

[0849] Output: Cropped thumbnail images stored in a database

[0850] ---

[0851] Step 3: Blending adjustments

[0852] User

[0853] The user adjusts the blending ratio of the selected image parts by manipulating the sliders, for example, 50% of image A, 30% of image B, and 20% of image C.

[0854] Input: Information about the selected image part

[0855] Output: Slider setting (blending ratio for each image part)

[0856] Terminal

[0857] The device sends the slider setting value specified by the user to the server in real time using a real-time protocol such as WebSocket.

[0858] Input: The slider setting adjusted by the user

[0859] Output: Send slider setting value to server

[0860] server

[0861] The server generates a new image based on the slider settings received using a generative AI model for blending the selected areas, using a deep learning framework (e.g., TensorFlow, PyTorch).

[0862] Input: Slider setting value and selection information

[0863] Output: The resulting blended image

[0864] ---

[0865] Step 4: Check and correct the generated image

[0866] User

[0867] The user reviews the generated image and adjusts the sliders again if necessary, repeating the process until they reach a design they are happy with.

[0868] Input: The generated blended image

[0869] Output: Rescaled slider value

[0870] Terminal

[0871] The terminal retransmits the user's new settings to the server and displays the latest generated image.

[0872] Input: The slider setting that the user has readjusted

[0873] Output: Sends the reset values ​​to the server and displays the latest image

[0874] server

[0875] The server then performs the blending process again based on the resubmitted settings and generates a new image, again using the generative AI model.

[0876] Input: Resent slider setting value

[0877] Output: Regenerated blended image

[0878] ---

[0879] Step 5: Save and share the generated image

[0880] server

[0881] Once the final image is confirmed, the server stores the generated image and creates a download link that users can access using a temporary token for security reasons.

[0882] Input: Final generated image

[0883] Output: Download link

[0884] Terminal

[0885] The device will display this download link to the user, who can click the link to download the image or use the share feature to share it with others.

[0886] Input: Download link sent by the server

[0887] Output: Display link to user

[0888] User

[0889] The user can send the final, concrete design image to the designer, allowing the designer to proceed with the detailed design based on the user's concrete image.

[0890] Input: Generated image obtained from the download link

[0891] Output: Share the design image with the designer

[0892] The above is a tentative concrete flow of the program processing of this system.

[0893] (Application example 1)

[0894] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0895] Creating advertising materials easily often requires specialized design knowledge and advanced techniques, making it difficult for non-experts to create them. Furthermore, the process of effectively utilizing multiple reference images held by users to generate specific design images is inefficient and time-consuming. Furthermore, there is a lack of methods for appropriately blending multiple reference images to generate new designs. There is a need for a system that can solve these issues and enable anyone to easily generate high-quality advertising materials.

[0896] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0897] In this invention, the server includes a means for uploading reference images, a means for selecting specific portions of the uploaded reference images, and an adjustment means for blending the selected image portions. This allows even non-experts to easily generate specific design images using multiple reference images and use them as advertising materials. Furthermore, the server includes a means for providing prompts during the image generation process, allowing users to easily select and adjust designs.

[0898] "Reference Image" means an image file used to form part or all of a design.

[0899] "Non-professionals" refer to general users who do not have specialized knowledge or advanced skills in design or image editing.

[0900] A "specific design image" is the final visual expression that the user is aiming for.

[0901] "Upload" refers to the act of sending data from a user's terminal to a server.

[0902] A "selection tool" is a tool or interface for specifying and cutting out a particular portion of a reference image.

[0903] The "blending adjustment means" is an interface for setting how to appropriately combine multiple selected image portions.

[0904] "Image generation method" refers to a process or technique that generates a new image based on selected image portions and configured blending adjustments.

[0905] "Prompts" are instructions or hints provided to the user to assist in generating advertising material.

[0906] "Advertising Materials" means visual, textual or other form of design elements used for advertising purposes.

[0907] "Storage and sharing means" refers to an interface or function for long-term storage of generated images and sharing with other users and designers.

[0908] This invention provides a system that allows non-experts to create specific design images using multiple reference images. This system is operated by a user, a terminal, and a server. The operation of the program and its processing are described in detail below.

[0909] Uploading a reference image

[0910] A user selects multiple reference images from their local storage and uploads them to the server via their device. Specifically, the user uses an interface for uploading reference images from a smartphone or tablet and sends the data to the server. The device then sends an upload request to the server, and the server stores the received image files with a unique identifier.

[0911] Selecting and arranging images

[0912] The user selects a specific part of the uploaded reference image using a rectangular selection tool or lasso tool. For example, a specific logo or color can be selected. The device records the coordinate information of the selected part and sends it to the server. The server then crops the selected part based on the received coordinate information and saves a newly generated thumbnail image.

[0913] Blending Adjustment

[0914] The user adjusts the blending ratio of the selected image parts by operating the sliders. For example, they can set 50% of image A, 30% of image B, and 20% of image C. The device sends the slider settings to the server in real time. Based on the received slider settings, the server generates a new blended image using an image generation AI model. This AI model specifically integrates the features of multiple reference images to generate a new design image.

[0915] Checking and correcting generated images

[0916] The user reviews the generated image and adjusts the sliders again if necessary. This process can be repeated until the user is satisfied. The device resends the user's adjusted settings and displays the latest generated image. The server re-performs the blending process based on the adjusted settings and generates a new image.

[0917] Saving and sharing generated images

[0918] Once the final design image is confirmed, the server saves the generated image and generates a download link that users can access and download, or share with others using the sharing tool.

[0919] Hardware and software used

[0920] Hardware: User's smartphone, tablet, server

[0921] Software: PIL library for image processing, requests library for HTTP communication, image generation model

[0922] Specific examples

[0923] If a user wants to create advertising materials for a new product, they can proceed using the following prompt:

[0924] "Please upload three images:

[0925] 1. Logo image

[0926] 2. Color-Scheme Images

[0927] 3. Sample images of the design

[0928] Select the part of each image you want to use:

[0929] Logo: 10, 10, 100, 100

[0930] Color scheme: 20, 20, 120, 120

[0931] Design samples: 30, 30, 140, 140

[0932] Set the blend ratio:

[0933] Logo: 50%

[0934] Color Scheme: 30%

[0935] Design sample: 20%

[0936] Review the generated ad assets, adjust them again if necessary, and save or share the final ad asset you're happy with.

[0937] In this way, a system is provided that enables even non-experts to easily generate high-quality advertising materials.

[0938] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0939] Step 1: Upload an image

[0940] Input: Reference images stored in the user's local storage.

[0941] How it works: The user selects a reference image from their smartphone or tablet and uploads it to the server using the application's interface.

[0942] Data processing and calculation: The terminal temporarily saves the selected image file and sends the image data to the server via an HTTP request.

[0943] Output: Reference image and unique identifier stored on the server.

[0944] Step 2: Select an image area

[0945] Input: Reference images uploaded to the server.

[0946] What it does: The user uses the in-app rectangular selection tool or lasso tool to select a specific part of the reference image.

[0947] Data processing and calculation: The terminal records the coordinate information of the part selected by the user and sends it to the server.

[0948] Output: Coordinate information recorded on the server and thumbnail images of the cropped areas.

[0949] Step 3: Blending adjustments

[0950] Input: Coordinate information and image data of the selected area recorded on the server.

[0951] What it does: The user manipulates an in-app slider to set the blend ratio of multiple selections.

[0952] Data processing and calculation: The device sends the slider values ​​to the server in real time. The server generates a new blended image based on these settings using an image generation AI model.

[0953] Output: The new image data generated.

[0954] Step 4: Check and adjust the generated image

[0955] Input: Generated image sent from the server to the device.

[0956] Specific operation: The user can check the generated image within the app and adjust the sliders again to correct the blending ratio if necessary.

[0957] Data processing and calculation: The device sends the adjusted slider values ​​to the server, and the server performs blending again based on the new settings.

[0958] Output: The new generated image data after correction.

[0959] Step 5: Save and share the resulting image

[0960] Input: The final generated image data.

[0961] Specific operation: Once the user finalizes the design image, the server saves the image and generates a download link, which the user can use to download the image or share it with other users.

[0962] Output: Download link and final generated image data.

[0963] By following the steps above, even non-experts can easily generate advertising materials and efficiently create the desired design.

[0964] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0965] The present invention is a system that allows non-experts to create specific design images using reference images, and by combining it with an emotion engine that recognizes the user's emotions, it makes the design process smoother and more personalized. This system is executed by the user, a terminal, and a server. The program processing is described in detail below.

[0966] Program processing

[0967] Uploading a reference image

[0968] User

[0969] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[0970] Terminal

[0971] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[0972] server

[0973] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the saved location and identifier in the database.

[0974] Selecting and arranging images

[0975] User

[0976] The user uses the system interface to view a list of uploaded reference images. Each image is displayed as a thumbnail. The user can then use the rectangular selection tool or lasso tool to select specific portions of the reference images. For example, select a specific logo from "image1.jpg."

[0977] Terminal

[0978] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[0979] server

[0980] The server then cuts out a specific portion of the reference image based on the received coordinate information, and saves the cut-out portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[0981] Blending Adjustment

[0982] User

[0983] The user adjusts the blending strength of the selected image portion by manipulating the sliders, which adjust the transparency and blending strength of each reference image portion.

[0984] Terminal

[0985] The terminal transmits the slider value set by the user to the server in real time.

[0986] server

[0987] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[0988] server

[0989] The new image generated is saved on the server, for example as "blend_result.jpg".

[0990] emotion recognition

[0991] User

[0992] To recognize the user's emotions in real time, the emotion engine analyzes the user's facial expressions and vocal tone. Once the emotion data is acquired, the user's current emotional state is determined.

[0993] Terminal

[0994] The device receives emotion data from the emotion engine and transmits it to the server. For example, emotions such as "happiness," "surprise," and "sadness" are sent as data.

[0995] server

[0996] The server analyzes this emotion data and suggests the selection of reference images and automatic adjustment of blending conditions. It also automatically adjusts the characteristics of the generated image (hue, brightness, contrast, etc.) based on the emotion data.

[0997] Checking and correcting generated images

[0998] User

[0999] The user reviews the blended image adjusted based on the emotion engine's suggestions, and further adjusts the sliders as needed. This process is repeated as many times as necessary until the user is satisfied.

[1000] Terminal

[1001] The terminal transmits the readjusted slider setting value to the server again.

[1002] server

[1003] The server then uses the image generation AI to perform the blending process again based on the new slider settings, and a new image is generated.

[1004] Saving and sharing generated images

[1005] server

[1006] Once the final generated image is confirmed, the server stores it and generates a download link that the user can access.

[1007] Terminal

[1008] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[1009] User

[1010] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[1011] Example: Product logo design

[1012] scenario

[1013] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[1014] User

[1015] Users upload three reference images of different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[1016] Server and Device

[1017] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[1018] emotion recognition

[1019] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[1020] Final production and sharing

[1021] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[1022] The above is a specific embodiment for carrying out the present invention.

[1023] The processing flow will be explained below.

[1024] Program processing

[1025] Step 1:

[1026] User

[1027] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[1028] Step 2:

[1029] Terminal

[1030] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[1031] Step 3:

[1032] server

[1033] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the saved location and identifier in the database.

[1034] Step 4:

[1035] User

[1036] The user uses the system's interface to view a list of reference images they have uploaded, each displayed as a thumbnail.

[1037] Step 5:

[1038] User

[1039] The user uses the rectangular selection tool or lasso tool to select a specific portion of the reference image, for example, a specific logo portion from "image1.jpg."

[1040] Step 6:

[1041] Terminal

[1042] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[1043] Step 7:

[1044] server

[1045] The server then cuts out a specific portion of the reference image based on the received coordinate information, and saves the cut-out portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[1046] Step 8:

[1047] User

[1048] The user uses sliders to adjust the blending of selected image parts, adjusting the transparency and overlay strength of each reference image part.

[1049] Step 9:

[1050] Terminal

[1051] The terminal transmits the slider value set by the user to the server in real time.

[1052] Step 10:

[1053] server

[1054] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[1055] Step 11:

[1056] server

[1057] The new image generated is saved on the server, for example as "blend_result.jpg".

[1058] Step 12:

[1059] User

[1060] To recognize the user's emotions in real time, the emotion engine analyzes the user's facial expressions and vocal tone. Based on the acquired emotion data, the user's current emotional state is determined.

[1061] Step 13:

[1062] Terminal

[1063] The device receives emotion data from the emotion engine and transmits it to the server. For example, emotions such as "joy" or "surprise" are sent as data.

[1064] Step 14:

[1065] server

[1066] The server analyzes this emotion data and suggests reference image selection and automatic blending adjustments. For example, if the user expresses a "sad" emotion, the system suggests a gentler color tone.

[1067] Step 15:

[1068] User

[1069] The user reviews the blended image adjusted based on the emotion engine's suggestions, further adjusts the sliders as needed, and repeats this process until the user is satisfied.

[1070] Step 16:

[1071] Terminal

[1072] The terminal transmits the readjusted slider setting value to the server again.

[1073] Step 17:

[1074] server

[1075] The server then uses the image generation AI to perform the blending process again based on the new slider settings, and a new image is generated.

[1076] Step 18:

[1077] server

[1078] Once the final image is confirmed, the server saves it and generates a download link for the user to access, for example, " / downloads / final_image123.jpg".

[1079] Step 19:

[1080] Terminal

[1081] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[1082] Step 20:

[1083] User

[1084] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[1085] Example: Product logo design

[1086] scenario

[1087] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[1088] User

[1089] Users upload three reference images of different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[1090] Server and Device

[1091] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[1092] emotion recognition

[1093] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[1094] Final production and sharing

[1095] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[1096] The above is a specific embodiment for carrying out the present invention.

[1097] Example 2

[1098] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1099] Conventional design systems have the problem that it is difficult for non-experts to generate concrete design images, and they also have the problem that they cannot incorporate user emotions into the design process, making it difficult to provide a personalized user experience.

[1100] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for uploading a reference image, a means for selecting a specific portion of the uploaded reference image, an adjustment means for blending the selected image portions, an image generation means for generating a blended image, a means for recognizing a user's emotions in real time and suggesting adjustments to the generated image, and a means for saving and sharing the generated image. This allows even a non-expert to easily generate specific design images and propose individualized designs based on the user's emotions.

[1101] A "reference image" is an image file that a user uses when generating a design image.

[1102] "Means for uploading" is a function that allows a user to send a reference image from local storage to the system.

[1103] The "means for selecting a specific portion" is a tool that allows the user to select a portion of interest from within the reference image.

[1104] The "adjustment means for blending" is a function that adjusts the transparency and overlay strength of the image portion selected by the user.

[1105] The "image generation means" is a function that integrates the blended image parts to create a new design image.

[1106] "Means for recognizing emotions in real time" is a function that analyzes the user's facial expressions and vocal tone to determine their current emotional state.

[1107] The "means for saving and sharing the generated image" is a function for saving the final generated design image and sharing it with other users.

[1108] The present invention is a system that allows non-experts to generate specific design images using multiple reference images. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the design process becomes smoother and more personalized. Specific embodiments for implementing the present invention are described below.

[1109] Uploading a reference image

[1110] User

[1111] The user selects multiple reference images from their local storage and uploads them using the system interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[1112] Terminal

[1113] The terminal stores the image file selected by the user in a temporary storage and transmits an upload request to the server. For example, the terminal stores the image in a temporary folder in the local storage and acquires the file path.

[1114] server

[1115] The server receives image files uploaded from the device, assigns a unique identifier to each file, and saves them in a specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the location and identifier of each image in a database.

[1116] Selecting and arranging images

[1117] User

[1118] The user uses the system interface to view a list of uploaded reference images. Each image is displayed as a thumbnail. The user can then use the rectangular selection tool or lasso tool to select specific portions of the reference images. For example, select a specific logo from "image1.jpg."

[1119] Terminal

[1120] The device records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server. Specifically, the coordinate data of the selected range is sent to the server in JSON format.

[1121] server

[1122] The server then extracts a specific portion of the reference image based on the received coordinate information, saves the extracted portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg," and records the metadata of the new image in the database.

[1123] Blending Adjustment

[1124] User

[1125] Users can adjust the blending of selected image parts by adjusting the slider: moving it to the right increases transparency, moving it to the left increases opacity, and also adjust the strength of the blending.

[1126] Terminal

[1127] The device sends the slider values ​​(transparency and overlay strength) set by the user to the server in real time. Specifically, the values ​​are updated sequentially according to the slider operation and sent to the server.

[1128] server

[1129] The server then sends blending instructions to the image generation AI model based on the slider settings received. The AI ​​model then uses an algorithm to combine the characteristics of each reference image part and generate a new image based on the user's settings. The new image is saved as "blend_result.jpg."

[1130] emotion recognition

[1131] User

[1132] To recognize a user's emotions in real time, the emotion engine analyzes facial expressions and vocal tone, for example, by capturing a picture of the user's face with a camera and recording the user's voice with a microphone.

[1133] Terminal

[1134] The device receives emotion data obtained from the emotion engine and sends it to the server. For example, emotion data such as "happiness," "surprise," and "sadness" is sent in JSON format.

[1135] server

[1136] The server analyzes this emotion data and suggests the selection of reference images and automatic adjustment of blending. It automatically adjusts the characteristics of the generated image (hue, brightness, contrast, etc.) based on the emotion data to provide the optimal design for the user.

[1137] Check and correct the generated image

[1138] User

[1139] The user can then review the blended image adjusted based on the emotion engine's suggestions and, if necessary, further adjust the sliders, for example, fine-tuning the hue, brightness, and contrast.

[1140] Terminal

[1141] The terminal transmits the readjusted slider setting value to the server again, specifically, the latest setting value according to the user's readjustment.

[1142] server

[1143] The server then uses the image generation AI model to blend the new slider settings and generate a new image. This process is repeated until the user is satisfied.

[1144] Saving and sharing generated images

[1145] server

[1146] Once the final image is confirmed, the server saves it and generates a download link that the user can access, for example, "final_design.jpg" and creates a download link.

[1147] Terminal

[1148] The device will display the generated download link to the user, who can click the link to download the image or use the sharing tool to share it with other designers.

[1149] User

[1150] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with detailed design work based on the user's concrete image.

[1151] Example: Product logo design

[1152] scenario

[1153] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[1154] User

[1155] Users upload three different reference images of logo designs to the tool and then select specific parts of each image (e.g., font style, color, icon) using the rectangular selection tool.

[1156] Server and Device

[1157] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[1158] emotion recognition

[1159] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[1160] Final production and sharing

[1161] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[1162] Example prompts for generative AI models

[1163] For example, the prompt to generate a new logo design might look like this:

[1164] "Generate a modern and elegant logo. Integrate font styles, colors, and icons from reference images to add a bright and energetic element."

[1165] The above is a specific embodiment for carrying out the present invention.

[1166] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1167] Step 1: Upload a reference image

[1168] User

[1169] The user selects multiple reference images from local storage using the system interface and clicks the upload button. As input, the user provides the image files (e.g., "image1.jpg", "image2.png", "image3.gif").

[1170] Terminal

[1171] The device temporarily saves the input image file in local storage and obtains the file path. Then, it sends an upload request to the server. As an output, it generates the image file path.

[1172] server

[1173] The server receives image files sent from the device, assigns a unique identifier to each, and saves them in a specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the storage location and identifier of each image in a database. As output, it generates the saved image file path and identifier.

[1174] Step 2: Select and place an image

[1175] User

[1176] The user sees a list of reference images uploaded on the system interface. Each image is displayed as a thumbnail, and the user can select specific parts using the rectangular selection tool or lasso tool. The thumbnail of the reference image is provided as input. The user selects a specific logo part from "image1.jpg".

[1177] Terminal

[1178] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server, which generates the coordinate data of the selected part as output.

[1179] server

[1180] The server then clips the specific part of the reference image based on the received coordinate information, saves it as a new image file, for example, " / uploads / 12345_thumbnail.jpg", records the metadata of the new image in the database, and generates the clipped image file path as output.

[1181] Step 3: Blending

[1182] User

[1183] The user operates sliders to adjust the blending of selected image parts, setting the transparency and overlay strength by moving the sliders. The slider position is provided as input.

[1184] Terminal

[1185] The device sends the slider values ​​(transparency and overlay strength) set by the user to the server in real time, and generates the setting value data as output.

[1186] server

[1187] The server instructs the image generation AI model to perform blending based on the received slider settings. The image generation AI model generates a new image by integrating the characteristics of each reference image part. The generated new image is saved as "blend_result.jpg". As output, it generates the generated image file.

[1188] Step 4: Emotion Recognition

[1189] User

[1190] The emotion engine analyzes the user's facial expressions and voice tone in real time. The inputs are the user's facial images and voice data captured by the camera and microphone.

[1191] Terminal

[1192] The terminal receives emotion data obtained from the emotion engine and transmits it to the server. For example, emotion data such as "happiness," "surprise," and "sadness" is transmitted. Emotion data is generated as output.

[1193] server

[1194] The server analyzes this emotion data, suggests reference image selection and automatic blending adjustment, and automatically adjusts the characteristics (hue, brightness, contrast, etc.) of the generated image based on the emotion data. The adjusted properties are generated as output.

[1195] Step 5: Check and correct the generated image

[1196] User

[1197] The user reviews the blended image based on the adjustments suggested by the system and readjusts the sliders as needed, providing the adjusted image as input, e.g., fine-tuning hue, brightness, and contrast.

[1198] Terminal

[1199] The device sends the readjusted slider setting value back to the server, which generates the new setting value as output.

[1200] server

[1201] The server then applies the blending process again using the image generation AI model based on the new slider settings to generate a new image. This process is repeated until the user is satisfied. The output is a regenerated image file.

[1202] Step 6: Save and share the generated image

[1203] server

[1204] Once the final image is confirmed, the server saves it and generates a download link that the user can access. For example, it generates "final_design.jpg" and creates a download link. The final design image is provided as input. The download link is generated as output.

[1205] Terminal

[1206] The terminal displays the generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool. The download link is displayed as an output.

[1207] User

[1208] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with detailed design work based on the user's concrete image. The downloaded image file is provided as output.

[1209] (Application example 2)

[1210] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1211] Current design generation systems have difficulty providing interfaces that non-experts can use effectively, and do not adequately customize based on the emotional state of individual users. Furthermore, the content provided in autonomous vehicles is not optimized according to the emotional state of passengers, which prevents passenger satisfaction from being fully achieved.

[1212] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for uploading a reference image, means for selecting specific portions of the uploaded reference image, means for adjusting the blending of the selected image portions, means for generating an image using a generative AI model for generating a blended image, means for saving and sharing the generated image, means for recognizing passenger emotions in real time, and means for adjusting the generated content based on the emotion data. This enables even non-experts to easily generate specific design images and optimize the content in an autonomous vehicle according to the passenger's emotional state.

[1213] "Reference images" are image files used for design and content generation.

[1214] "Means for uploading" refers to a function that allows a user to transfer data from a local environment to a server.

[1215] The "means for selecting a specific portion" refers to a tool or software function for selecting a specific area of ​​a reference image.

[1216] "Adjustment means" refers to a function for adjusting the blending of selected image portions.

[1217] A "generative AI model" refers to a model that uses artificial intelligence to generate new images and content.

[1218] "Image generation means" refers to a function for generating images using a generative AI model.

[1219] "Means for storing and sharing" refers to the functionality for storing generated content and sharing it with other users and systems.

[1220] "Means of recognizing emotions in real time" refers to a function that analyzes passengers' facial expressions and voice to determine their emotional state at that time.

[1221] "Emotional Data" refers to emotional state information obtained by means of real-time emotion recognition.

[1222] "Means for adjusting content" refers to functions for optimizing the advertisements and entertainment content displayed based on the acquired emotional data.

[1223] The present invention is a system that makes it easy for even non-experts to generate specific design images and provides content based on the emotions of passengers in autonomous vehicles. Specific embodiments are described below.

[1224] System configuration

[1225] This system consists of users, terminals, and a server. To realize the system, the following hardware and software are required:

[1226] 1. Hardware

[1227] Camera (for facial recognition inside the car)

[1228] In-vehicle computer

[1229] High-Performance Server

[1230] 2. Software

[1231] Facial expression recognition API (e.g. facial expression recognition API)

[1232] Image generation AI (e.g. TensorFlow)

[1233] Database systems (e.g. MySQL)

[1234] Interface tools (e.g., Marquee tool, Lasso tool)

[1235] Program processing flow

[1236] 1. Acquiring emotion recognition data

[1237] User: A passenger enters the vehicle.

[1238] Device: An in-car camera captures passengers' faces and analyzes them in real time using an emotion recognition API.

[1239] Server: The analysis results (emotion data) are sent to the server.

[1240] 2. Emotion-Based Design Generation

[1241] Server: Receives emotion data and initiates the design generation process. Users upload reference images and use the interface to select specific parts of the image. The selected image parts are stored in a database.

[1242] User: Use the slider to adjust the blending amount.

[1243] Server: The image generation AI generates new images based on the user's settings. This time, the generative AI model is used.

[1244] 3. Adjust your content

[1245] Server: Automatically adjusts the generated blended images and content based on emotion data. If the passenger is expressing joy, bright colors and energetic content will be generated.

[1246] 4. Final review and feedback

[1247] User: Review the generated content and adjust the sliders as needed. The adjusted data is resubmitted to the server and new content is generated. This process is repeated until the user is satisfied.

[1248] Example scenario

[1249] The specific processing flow is illustrated based on the following scenario:

[1250] Scenario: A passenger is perceived to have a depressed expression.

[1251] Emotion Recognition: The emotion recognition module detects "sadness."

[1252] Content generation: The system generates light-hearted comedy videos and refreshing landscape images to turn sadness into joy.

[1253] Feedback: The passenger's expression becomes happy, and the system records this change and reflects it next time.

[1254] Prompt Sentence Examples

[1255] An example of a prompt might be:

[1256] 1. The camera captures passengers' facial expression data and analyzes it in real time.

[1257] 2. If the emotion is recognized as "sadness," the AI ​​model is used to generate ads with upbeat text and images, as well as comedy videos.

[1258] 3. The generated content is displayed on the in-car display, and passenger feedback is retrieved using a slider and reflected in the system.

[1259] The above is a specific embodiment for carrying out the present invention.

[1260] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1261] Step 1: Obtaining emotion data

[1262] Subject: Terminal

[1263] Specific explanation: The in-car camera detects the passenger's face and analyzes the emotional data using a facial expression recognition API. The analyzed data is output as an emotional status such as "happiness," "surprise," or "sadness" and sent to the server.

[1264] Input: Passenger's face image

[1265] Output: Emotion data

[1266] How it works: The device's camera captures facial images, and the facial recognition API extracts facial features from the images and quantifies the emotional state.

[1267] Step 2: Upload a reference image

[1268] Subject: User

[1269] Specific Description: The user selects multiple reference images from the device and uploads them through the system interface. The uploaded images are temporarily stored on the device.

[1270] Input: User selected reference image file

[1271] Output: Reference images uploaded to the server

[1272] Operation details: When the user selects a reference image from local storage and clicks the upload button, the selected image is saved on the device and an upload request is sent to the server.

[1273] Step 3: Select an image area

[1274] Subject: User

[1275] Specific explanation: The user uses the interface to select a specific part of the uploaded reference image using the rectangular selection tool or lasso tool. The coordinate information of the selected part is recorded.

[1276] Input: Reference image and selection tool operation information

[1277] Output: Coordinate data of the selected image part

[1278] Operation details: The user selects an image portion on the interface, and the coordinate information of the selected area is recorded and transmitted by the device.

[1279] Step 4: Blending image parts

[1280] Subject: User

[1281] What it means: The user uses sliders in the interface to adjust the transparency and blending strength of multiple selected image parts, setting the blending effect.

[1282] Input: Coordinate data of the selected image part and user slider setting value

[1283] Output: Blending instruction data based on the set value

[1284] How it works: The user adjusts the slider and the setting is sent to the server in real time.

[1285] Step 5: Image generation

[1286] Subject: Server

[1287] Specific explanation: The server generates a new image using the image generation AI model based on the received slider setting value. The generated image is saved on the server.

[1288] Input: Blending instruction data

[1289] Output: The new image generated

[1290] Operation details: The server inputs instruction data into the generative AI model, and the model outputs a new image, which is saved in a specific directory on the server.

[1291] Step 6: Tailor content based on sentiment data

[1292] Subject: Server

[1293] Specifically, the server automatically adjusts the color, brightness, and content type of the generated images and content based on the emotional data.

[1294] Input: Generated images and emotion data

[1295] Output: Emotionally optimized content

[1296] How it works: The server analyzes the emotion data and adjusts the parameter settings of the image generation AI model to recreate new images and content.

[1297] Step 7: Final review and feedback

[1298] Subject: User

[1299] Specifically: The user can check the generated content and make further adjustments using the sliders. The adjusted settings are sent to the server and the content is generated again.

[1300] Input: Generated content and user feedback

[1301] Output: The final generated content

[1302] How it works: The user reviews the content, makes any necessary adjustments, and the results are sent to the server, which regenerates the content.

[1303] Step 8: Final saving and sharing

[1304] Subject: Server

[1305] Specific description: The server stores the final generated content and generates a download link that can be accessed by the user.

[1306] Input: Finalized content

[1307] Output: Download link

[1308] What it does: The server stores the generated content, generates a link in a format that the user can access, and displays it to the user.

[1309] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1310] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1311] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1312] [Third embodiment]

[1313] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1314] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1315] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1316] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1317] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1318] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1319] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1320] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1321] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1322] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1323] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1324] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1325] The present invention provides a system for non-experts to create specific design images using reference images. The system is operated by a user, a terminal, and a server. The program processing will be described in detail below.

[1326] Program processing

[1327] Uploading a reference image

[1328] User

[1329] The user selects multiple reference images from their local storage and uploads them through the tool's interface, for example, selecting three different image files.

[1330] Terminal

[1331] The terminal temporarily stores these image files according to the user's selection and sends an upload request to the server.

[1332] server

[1333] The server receives the uploaded image files, assigns a unique identifier to each image, and stores it in a specified directory. It also records the location and identifier of each image in a database.

[1334] Selecting and arranging images

[1335] User

[1336] Users can use the rectangular selection tool or lasso tool to select the part of an uploaded reference image they want to use, such as a specific logo or color.

[1337] Terminal

[1338] The terminal records the coordinate information of the part selected by the user and transmits it to the server.

[1339] server

[1340] The server then clips out the selected portion based on the received coordinate information, saves the newly generated thumbnail image, and updates the information about the selected portion in the database.

[1341] Blending Adjustment

[1342] User

[1343] The user adjusts the blending ratio of the selected image by using the sliders, for example, 50% of image A, 30% of image B, and 20% of image C.

[1344] Terminal

[1345] The device sends the slider setting value to the server in real time.

[1346] server

[1347] The server generates a new blended image based on the received slider settings using image generation AI, which integrates the features of each image based on the settings.

[1348] Check and correct the generated image

[1349] User

[1350] The user reviews the resulting image and adjusts the sliders again if necessary, repeating the process until satisfied.

[1351] Terminal

[1352] The terminal retransmits the user's readjusted settings and displays the latest generated image.

[1353] server

[1354] The server then performs the blending process again based on the readjusted settings to generate a new image.

[1355] Saving and sharing generated images

[1356] server

[1357] Once the final image is finalized, the server stores the generated image and generates a download link that the user can access.

[1358] Terminal

[1359] The device will display this download link to the user, who can click the link to download it or use the sharing tool to share it with other designers.

[1360] User

[1361] The user can send the final, concrete design image to the designer, allowing the designer to proceed with the detailed design based on the user's concrete image.

[1362] Example: Product logo design

[1363] scenario

[1364] Suppose a user wants to create a logo design for a new product.

[1365] User

[1366] Users upload reference images of three different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[1367] Server and Device

[1368] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[1369] Final production and sharing

[1370] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[1371] The above is a specific embodiment for carrying out the present invention.

[1372] The processing flow will be explained below.

[1373] Step 1:

[1374] User

[1375] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[1376] Step 2:

[1377] Terminal

[1378] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[1379] Step 3:

[1380] server

[1381] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, "image1.jpg" is saved to " / uploads / 12345.jpg."

[1382] Step 4:

[1383] server

[1384] The server records the storage location and identifier of each image in a database, which allows it to be referenced in subsequent processing.

[1385] Step 5:

[1386] User

[1387] The user uses the system's interface to view a list of reference images they have uploaded, each displayed as a thumbnail.

[1388] Step 6:

[1389] User

[1390] The user uses the rectangular selection tool or lasso tool to select a specific portion of the reference image, for example, a rectangular selection of a specific logo from "image1.jpg."

[1391] Step 7:

[1392] Terminal

[1393] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[1394] Step 8:

[1395] server

[1396] The server then cuts out a specific portion of the reference image based on the received coordinate information. The cut-out portion is saved as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[1397] Step 9:

[1398] User

[1399] The user uses sliders to adjust the blending of the cropped image parts, adjusting the transparency and blending strength of each reference image part.

[1400] Step 10:

[1401] Terminal

[1402] The terminal transmits the slider value set by the user to the server in real time.

[1403] Step 11:

[1404] server

[1405] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[1406] Step 12:

[1407] server

[1408] The new image generated is saved on the server, for example as "blend_result.jpg".

[1409] Step 13:

[1410] User

[1411] The user reviews the resulting image and further adjusts the sliders as needed, and this process is repeated as many times as necessary until the user is satisfied.

[1412] Step 14:

[1413] Terminal

[1414] The terminal transmits the readjusted slider setting value to the server again.

[1415] Step 15:

[1416] server

[1417] The server then uses the image generation AI to perform blending again based on the new slider settings, and a new image is generated.

[1418] Step 16:

[1419] server

[1420] Once the final generated image is confirmed, the server stores it and generates a download link that the user can access.

[1421] Step 17:

[1422] Terminal

[1423] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[1424] Step 18:

[1425] User

[1426] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[1427] The above are the processing steps of the system that allows a non-expert to generate a specific design image using multiple reference images.

[1428] Example 1

[1429] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1430] In modern design work, when non-experts try to create a concrete design based on their own ideas, they need specialized knowledge and skills, which takes a lot of time. Furthermore, sharing a specific design image requires a lot of trial and error, so an efficient design process is required. Given this background, there is a need for a system that allows non-experts to easily generate and share high-quality design images.

[1431] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1432] In this invention, the server includes a means for uploading reference images, a means for selecting specific portions of the uploaded reference images, a means for recording and transmitting coordinate information of the selected reference image portions to the server, an adjustment means for blending the multiple selected image portions, an image generation means using a generative AI model to generate a blended image, and a means for saving and sharing the generated image. This allows even non-experts to easily generate specific design images using multiple reference images and share them with others.

[1433] A "reference image" is an image file that a user uploads to be used as a design element.

[1434] "Non-professionals" refers to general users who do not have specialized design knowledge or skills.

[1435] "Uploading means" is a function that allows a user to send an image file from local storage to the server.

[1436] The "means for selecting a specific portion" is a tool that allows the user to specify and extract a specific portion of the reference image.

[1437] "Coordinate information" is X, Y position data that indicates the position of a specific part of the reference image.

[1438] The "transmitting means" is a communication means for transmitting data from the terminal to the server.

[1439] An "adjuster" is a device or interface for setting the blending ratio of multiple image portions.

[1440] A "generative AI model" is an algorithm or system for generating design images using artificial intelligence technology.

[1441] An "image generator" is a process or device for integrating portions of reference images to generate a new image.

[1442] The "means for saving" is a function for storing the generated image as data.

[1443] "Means for sharing" is a function for sharing the generated image with other users and designers.

[1444] The present invention provides a system for non-experts to generate specific design images using reference images. This system is operated by a user, a terminal, and a server. The specific configuration and operation of the system are described below.

[1445] System Configuration

[1446] User

[1447] The user accesses the system through an interface to upload a reference image, select specific parts, and adjust the blending. The user selects an image from local storage and uploads it to the system as a reference image. The user's operations are transmitted to the terminal via the interface.

[1448] Terminal

[1449] The terminal receives instructions from the user and temporarily stores data such as image files and coordinate information. The terminal also transmits information selected by the user to the server in real time and displays the results. The terminal acts as an interface for image processing.

[1450] server

[1451] The server processes the data sent from the devices and plays a central role in generating images. The server has the following functions:

[1452] Uploading and managing reference images: The server receives reference images uploaded by users, assigns a unique identifier, and stores them. It also records the storage location and identifier of each image in a database. Specifically, OpenCV is used as the image processing library.

[1453] Selection of a specific part of the reference image: The coordinate information of the specific part selected by the user is received, and the selected part is cropped to generate a new thumbnail image. This information is also recorded in the database.

[1454] Blending: Based on user input, an image-generating artificial intelligence (AI) model is used to combine features from selected images to generate a new blended image, using a deep learning framework (e.g., TensorFlow or PyTorch).

[1455] Saving and sharing the generated image: Once the final image is confirmed, the server saves it and creates a download link that users can access. They can use this link to download the image or share it with others using the sharing tool.

[1456] Specific examples

[1457] For example, if a user wants to create a logo design for a new product, they would follow these steps:

[1458] 1. The user uploads reference images of three different logo designs for a new product into the tool.

[1459] 2. Next, select specific parts of each image (e.g. font style, color, icon) using the rectangular selection tool.

[1460] 3. The device sends the coordinate information of the selected area to the server, and the server generates a new image based on that.

[1461] 4. The user uses the slider to adjust the blending of the images to achieve the desired design image.

[1462] 5. Review the final logo image, save it, and share the download link with your designer.

[1463] Prompt Sentence Examples

[1464] "I want to create a logo design for a new product. Upload reference images, select distinctive features from each image, and generate a new design."

[1465] The above is a specific embodiment for carrying out the present invention. This system enables even non-experts to easily create high-quality design images and share them efficiently.

[1466] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1467] Step 1: Upload a reference image

[1468] User

[1469] Users select reference images for their design from their local storage and upload them using the system interface. For example, they can choose three different images for a new product logo design.

[1470] Input: Image file in local storage

[1471] Output: Image file uploaded to the system

[1472] Terminal

[1473] The device temporarily stores the image file selected by the user and sends an upload request to the server, along with image metadata (such as file name, file size, and image format).

[1474] Input: Image file uploaded by the user

[1475] Output: Upload request to server

[1476] server

[1477] The server assigns a unique identifier (such as a UUID) to the received image file and stores it in the specified directory. It also records the storage path, identifier, and metadata of each image in a database.

[1478] Input: Image file and metadata sent from the device

[1479] Output: Image file information recorded in the database

[1480] ---

[1481] Step 2: Select and place an image

[1482] User

[1483] Users can use the rectangular selection or lasso tools to select the portion of an uploaded image they want to use, such as a specific logo, color pattern, or font style.

[1484] Input: Reference image uploaded to the system

[1485] Output: Coordinate information of the selected image part

[1486] Terminal

[1487] The terminal records the coordinate information (X and Y coordinates of the start and end points) of the area selected by the user and sends this coordinate information and the corresponding image identifier to the server.

[1488] Input: Coordinate information selected by the user

[1489] Output: Send coordinates and identifier to server

[1490] server

[1491] The server then crops the selected part based on the received coordinates, using an image processing library (e.g., OpenCV), saves the cropped image part as a newly generated thumbnail, and updates the database.

[1492] Input: Coordinate information and image identifier sent from the device

[1493] Output: Cropped thumbnail images stored in a database

[1494] ---

[1495] Step 3: Blending

[1496] User

[1497] The user adjusts the blending ratio of the selected image parts by manipulating the sliders, for example, 50% of image A, 30% of image B, and 20% of image C.

[1498] Input: Information about the selected image part

[1499] Output: Slider setting (blending ratio for each image part)

[1500] Terminal

[1501] The device sends the slider setting value specified by the user to the server in real time using a real-time protocol such as WebSocket.

[1502] Input: The slider setting adjusted by the user

[1503] Output: Send slider setting value to server

[1504] server

[1505] The server generates a new image based on the slider settings received using a generative AI model for blending the selected areas, using a deep learning framework (e.g., TensorFlow, PyTorch).

[1506] Input: Slider setting value and selection information

[1507] Output: The resulting blended image

[1508] ---

[1509] Step 4: Check and correct the generated image

[1510] User

[1511] The user reviews the generated image and adjusts the sliders again if necessary, repeating the process until they reach a design they are happy with.

[1512] Input: The generated blended image

[1513] Output: Rescaled slider value

[1514] Terminal

[1515] The terminal retransmits the user's new settings to the server and displays the latest generated image.

[1516] Input: The slider setting that the user has readjusted

[1517] Output: Sends the reset values ​​to the server and displays the latest image

[1518] server

[1519] The server then performs the blending process again based on the resubmitted settings and generates a new image, again using the generative AI model.

[1520] Input: Resent slider setting value

[1521] Output: Regenerated blended image

[1522] ---

[1523] Step 5: Save and share the generated image

[1524] server

[1525] Once the final image is confirmed, the server stores the generated image and creates a download link that users can access using a temporary token for security reasons.

[1526] Input: Final generated image

[1527] Output: Download link

[1528] Terminal

[1529] The device will display this download link to the user, who can click the link to download the image or use the share feature to share it with others.

[1530] Input: Download link sent by the server

[1531] Output: Display link to user

[1532] User

[1533] The user can send the final, concrete design image to the designer, allowing the designer to proceed with the detailed design based on the user's concrete image.

[1534] Input: Generated image obtained from the download link

[1535] Output: Share the design image with the designer

[1536] The above is a tentative concrete flow of the program processing of this system.

[1537] (Application example 1)

[1538] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1539] Creating advertising materials easily often requires specialized design knowledge and advanced techniques, making it difficult for non-experts to create them. Furthermore, the process of effectively utilizing multiple reference images held by users to generate specific design images is inefficient and time-consuming. Furthermore, there is a lack of methods for appropriately blending multiple reference images to generate new designs. There is a need for a system that can solve these issues and enable anyone to easily generate high-quality advertising materials.

[1540] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1541] In this invention, the server includes a means for uploading reference images, a means for selecting specific portions of the uploaded reference images, and an adjustment means for blending the selected image portions. This allows even non-experts to easily generate specific design images using multiple reference images and use them as advertising materials. Furthermore, the server includes a means for providing prompts during the image generation process, allowing users to easily select and adjust designs.

[1542] "Reference Image" means an image file used to form part or all of a design.

[1543] "Non-professionals" refer to general users who do not have specialized knowledge or advanced skills in design or image editing.

[1544] A "specific design image" is the final visual expression that the user is aiming for.

[1545] "Upload" refers to the act of sending data from a user's terminal to a server.

[1546] A "selection tool" is a tool or interface for specifying and cutting out a particular portion of a reference image.

[1547] The "blending adjustment means" is an interface for setting how to appropriately combine multiple selected image portions.

[1548] "Image generation method" refers to a process or technique that generates a new image based on selected image portions and configured blending adjustments.

[1549] "Prompts" are instructions or hints provided to the user to assist in generating advertising material.

[1550] "Advertising Materials" means visual, textual or other form of design elements used for advertising purposes.

[1551] "Storage and sharing means" refers to an interface or function for long-term storage of generated images and sharing with other users and designers.

[1552] This invention provides a system that allows non-experts to create specific design images using multiple reference images. This system is operated by a user, a terminal, and a server. The operation of the program and its processing are described in detail below.

[1553] Uploading a reference image

[1554] A user selects multiple reference images from their local storage and uploads them to the server via their device. Specifically, the user uses an interface for uploading reference images from a smartphone or tablet and sends the data to the server. The device then sends an upload request to the server, and the server stores the received image files with a unique identifier.

[1555] Selecting and arranging images

[1556] The user selects a specific part of the uploaded reference image using a rectangular selection tool or lasso tool. For example, a specific logo or color can be selected. The device records the coordinate information of the selected part and sends it to the server. The server then crops the selected part based on the received coordinate information and saves a newly generated thumbnail image.

[1557] Blending Adjustment

[1558] The user adjusts the blending ratio of the selected image parts by operating the sliders. For example, they can set 50% of image A, 30% of image B, and 20% of image C. The device sends the slider settings to the server in real time. Based on the received slider settings, the server generates a new blended image using an image generation AI model. This AI model specifically integrates the features of multiple reference images to generate a new design image.

[1559] Check and correct the generated image

[1560] The user reviews the generated image and adjusts the sliders again if necessary. This process can be repeated until the user is satisfied. The device resends the user's adjusted settings and displays the latest generated image. The server re-performs the blending process based on the adjusted settings and generates a new image.

[1561] Saving and sharing generated images

[1562] Once the final design image is confirmed, the server saves the generated image and generates a download link that users can access and download, or share with others using the sharing tool.

[1563] Hardware and software used

[1564] Hardware: User's smartphone, tablet, server

[1565] Software: PIL library for image processing, requests library for HTTP communication, image generation model

[1566] Specific examples

[1567] If a user wants to create advertising materials for a new product, they can proceed using the following prompt:

[1568] "Please upload three images:

[1569] 1. Logo image

[1570] 2. Color-Scheme Images

[1571] 3. Sample images of the design

[1572] Select the part of each image you want to use:

[1573] Logo: 10, 10, 100, 100

[1574] Color scheme: 20, 20, 120, 120

[1575] Design samples: 30, 30, 140, 140

[1576] Set the blend ratio:

[1577] Logo: 50%

[1578] Color Scheme: 30%

[1579] Design sample: 20%

[1580] Review the generated ad assets, adjust them again if necessary, and save or share the final ad asset you're happy with.

[1581] In this way, a system is provided that enables even non-experts to easily generate high-quality advertising materials.

[1582] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1583] Step 1: Upload an image

[1584] Input: Reference images stored in the user's local storage.

[1585] How it works: The user selects a reference image from their smartphone or tablet and uploads it to the server using the application's interface.

[1586] Data processing and calculation: The terminal temporarily saves the selected image file and sends the image data to the server via an HTTP request.

[1587] Output: Reference image and unique identifier stored on the server.

[1588] Step 2: Select an image area

[1589] Input: Reference images uploaded to the server.

[1590] What it does: The user uses the in-app rectangular selection tool or lasso tool to select a specific part of the reference image.

[1591] Data processing and calculation: The terminal records the coordinate information of the part selected by the user and sends it to the server.

[1592] Output: Coordinate information recorded on the server and thumbnail images of the cropped areas.

[1593] Step 3: Blending adjustments

[1594] Input: Coordinate information and image data of the selected area recorded on the server.

[1595] What it does: The user manipulates an in-app slider to set the blend ratio of multiple selections.

[1596] Data processing and calculation: The device sends the slider values ​​to the server in real time. The server generates a new blended image based on these settings using an image generation AI model.

[1597] Output: The new image data generated.

[1598] Step 4: Check and adjust the generated image

[1599] Input: Generated image sent from the server to the device.

[1600] Specific operation: The user can check the generated image within the app and adjust the sliders again to correct the blending ratio if necessary.

[1601] Data processing and calculation: The device sends the adjusted slider values ​​to the server, and the server performs blending again based on the new settings.

[1602] Output: The new generated image data after correction.

[1603] Step 5: Save and share the resulting image

[1604] Input: The final generated image data.

[1605] Specific operation: Once the user finalizes the design image, the server saves the image and generates a download link, which the user can use to download the image or share it with other users.

[1606] Output: Download link and final generated image data.

[1607] By following the steps above, even non-experts can easily generate advertising materials and efficiently create the desired design.

[1608] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1609] The present invention is a system that allows non-experts to create specific design images using reference images, and by combining it with an emotion engine that recognizes the user's emotions, it makes the design process smoother and more personalized. This system is executed by the user, a terminal, and a server. The program processing is described in detail below.

[1610] Program processing

[1611] Uploading a reference image

[1612] User

[1613] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[1614] Terminal

[1615] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[1616] server

[1617] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the saved location and identifier in the database.

[1618] Selecting and arranging images

[1619] User

[1620] The user uses the system interface to view a list of uploaded reference images. Each image is displayed as a thumbnail. The user can then use the rectangular selection tool or lasso tool to select specific portions of the reference images. For example, select a specific logo from "image1.jpg."

[1621] Terminal

[1622] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[1623] server

[1624] The server then cuts out a specific portion of the reference image based on the received coordinate information, and saves the cut-out portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[1625] Blending Adjustment

[1626] User

[1627] The user adjusts the blending strength of the selected image portion by manipulating the sliders, which adjust the transparency and blending strength of each reference image portion.

[1628] Terminal

[1629] The terminal transmits the slider value set by the user to the server in real time.

[1630] server

[1631] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[1632] server

[1633] The new image generated is saved on the server, for example as "blend_result.jpg".

[1634] emotion recognition

[1635] User

[1636] To recognize the user's emotions in real time, the emotion engine analyzes the user's facial expressions and vocal tone. Once the emotion data is acquired, the user's current emotional state is determined.

[1637] Terminal

[1638] The device receives emotion data from the emotion engine and transmits it to the server. For example, emotions such as "happiness," "surprise," and "sadness" are sent as data.

[1639] server

[1640] The server analyzes this emotion data and suggests the selection of reference images and automatic adjustment of blending conditions. It also automatically adjusts the characteristics of the generated image (hue, brightness, contrast, etc.) based on the emotion data.

[1641] Check and correct the generated image

[1642] User

[1643] The user reviews the blended image adjusted based on the emotion engine's suggestions, and further adjusts the sliders as needed. This process is repeated as many times as necessary until the user is satisfied.

[1644] Terminal

[1645] The terminal transmits the readjusted slider setting value to the server again.

[1646] server

[1647] The server then uses the image generation AI to perform the blending process again based on the new slider settings, and a new image is generated.

[1648] Saving and sharing generated images

[1649] server

[1650] Once the final generated image is confirmed, the server stores it and generates a download link that the user can access.

[1651] Terminal

[1652] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[1653] User

[1654] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[1655] Example: Product logo design

[1656] scenario

[1657] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[1658] User

[1659] Users upload three reference images of different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[1660] Server and Device

[1661] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[1662] emotion recognition

[1663] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[1664] Final production and sharing

[1665] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[1666] The above is a specific embodiment for carrying out the present invention.

[1667] The processing flow will be explained below.

[1668] Program processing

[1669] Step 1:

[1670] User

[1671] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[1672] Step 2:

[1673] Terminal

[1674] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[1675] Step 3:

[1676] server

[1677] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the saved location and identifier in the database.

[1678] Step 4:

[1679] User

[1680] The user uses the system's interface to view a list of reference images they have uploaded, each displayed as a thumbnail.

[1681] Step 5:

[1682] User

[1683] The user uses the rectangular selection tool or lasso tool to select a specific portion of the reference image, for example, a specific logo portion from "image1.jpg."

[1684] Step 6:

[1685] Terminal

[1686] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[1687] Step 7:

[1688] server

[1689] The server then cuts out a specific portion of the reference image based on the received coordinate information, and saves the cut-out portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[1690] Step 8:

[1691] User

[1692] The user uses sliders to adjust the blending of selected image parts, adjusting the transparency and overlay strength of each reference image part.

[1693] Step 9:

[1694] Terminal

[1695] The terminal transmits the slider value set by the user to the server in real time.

[1696] Step 10:

[1697] server

[1698] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[1699] Step 11:

[1700] server

[1701] The new image generated is saved on the server, for example as "blend_result.jpg".

[1702] Step 12:

[1703] User

[1704] To recognize the user's emotions in real time, the emotion engine analyzes the user's facial expressions and vocal tone. Based on the acquired emotion data, the user's current emotional state is determined.

[1705] Step 13:

[1706] Terminal

[1707] The device receives emotion data from the emotion engine and transmits it to the server. For example, emotions such as "joy" or "surprise" are sent as data.

[1708] Step 14:

[1709] server

[1710] The server analyzes this emotion data and suggests reference image selection and automatic blending adjustments. For example, if the user expresses a "sad" emotion, the system suggests a gentler color tone.

[1711] Step 15:

[1712] User

[1713] The user reviews the blended image adjusted based on the emotion engine's suggestions, further adjusts the sliders as needed, and repeats this process until the user is satisfied.

[1714] Step 16:

[1715] Terminal

[1716] The terminal transmits the readjusted slider setting value to the server again.

[1717] Step 17:

[1718] server

[1719] The server then uses the image generation AI to perform the blending process again based on the new slider settings, and a new image is generated.

[1720] Step 18:

[1721] server

[1722] Once the final image is confirmed, the server saves it and generates a download link for the user to access, for example, " / downloads / final_image123.jpg".

[1723] Step 19:

[1724] Terminal

[1725] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[1726] Step 20:

[1727] User

[1728] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[1729] Example: Product logo design

[1730] scenario

[1731] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[1732] User

[1733] Users upload three reference images of different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[1734] Server and Device

[1735] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[1736] emotion recognition

[1737] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[1738] Final production and sharing

[1739] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[1740] The above is a specific embodiment for carrying out the present invention.

[1741] Example 2

[1742] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1743] Conventional design systems have the problem that it is difficult for non-experts to generate concrete design images, and they also have the problem that they cannot incorporate user emotions into the design process, making it difficult to provide a personalized user experience.

[1744] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for uploading a reference image, a means for selecting a specific portion of the uploaded reference image, an adjustment means for blending the selected image portions, an image generation means for generating a blended image, a means for recognizing a user's emotions in real time and suggesting adjustments to the generated image, and a means for saving and sharing the generated image. This allows even a non-expert to easily generate specific design images and propose individualized designs based on the user's emotions.

[1745] A "reference image" is an image file that a user uses when generating a design image.

[1746] "Means for uploading" is a function that allows a user to send a reference image from local storage to the system.

[1747] The "means for selecting a specific portion" is a tool that allows the user to select a portion of interest from within the reference image.

[1748] The "adjustment means for blending" is a function that adjusts the transparency and overlay strength of the image portion selected by the user.

[1749] The "image generation means" is a function that integrates the blended image parts to create a new design image.

[1750] "Means for recognizing emotions in real time" is a function that analyzes the user's facial expressions and vocal tone to determine their current emotional state.

[1751] The "means for saving and sharing the generated image" is a function for saving the final generated design image and sharing it with other users.

[1752] The present invention is a system that allows non-experts to generate specific design images using multiple reference images. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the design process becomes smoother and more personalized. Specific embodiments for implementing the present invention are described below.

[1753] Uploading a reference image

[1754] User

[1755] The user selects multiple reference images from their local storage and uploads them using the system interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[1756] Terminal

[1757] The terminal stores the image file selected by the user in a temporary storage and transmits an upload request to the server. For example, the terminal stores the image in a temporary folder in the local storage and acquires the file path.

[1758] server

[1759] The server receives image files uploaded from the device, assigns a unique identifier to each file, and saves them in a specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the location and identifier of each image in a database.

[1760] Selecting and arranging images

[1761] User

[1762] The user uses the system interface to view a list of uploaded reference images. Each image is displayed as a thumbnail. The user can then use the rectangular selection tool or lasso tool to select specific portions of the reference images. For example, select a specific logo from "image1.jpg."

[1763] Terminal

[1764] The device records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server. Specifically, the coordinate data of the selected range is sent to the server in JSON format.

[1765] server

[1766] The server then extracts a specific portion of the reference image based on the received coordinate information, saves the extracted portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg," and records the metadata of the new image in the database.

[1767] Blending Adjustment

[1768] User

[1769] Users can adjust the blending of selected image parts with a slider: moving the slider to the right increases transparency, moving it to the left increases opacity, and also adjust the strength of the blending.

[1770] Terminal

[1771] The device sends the slider values ​​(transparency and overlay strength) set by the user to the server in real time. Specifically, the values ​​are updated sequentially according to the slider operation and sent to the server.

[1772] server

[1773] The server then sends blending instructions to the image generation AI model based on the slider settings received. The AI ​​model then uses an algorithm to combine the characteristics of each reference image part and generate a new image based on the user's settings. The new image is saved as "blend_result.jpg."

[1774] emotion recognition

[1775] User

[1776] To recognize a user's emotions in real time, the emotion engine analyzes facial expressions and vocal tone, for example, by capturing a picture of the user's face with a camera and recording the user's voice with a microphone.

[1777] Terminal

[1778] The device receives emotion data obtained from the emotion engine and sends it to the server. For example, emotion data such as "happiness," "surprise," and "sadness" is sent in JSON format.

[1779] server

[1780] The server analyzes this emotion data and suggests the selection of reference images and automatic adjustment of blending. It automatically adjusts the characteristics of the generated image (hue, brightness, contrast, etc.) based on the emotion data to provide the optimal design for the user.

[1781] Check and correct the generated image

[1782] User

[1783] The user can then review the blended image adjusted based on the emotion engine's suggestions and, if necessary, further adjust the sliders, for example, fine-tuning the hue, brightness, and contrast.

[1784] Terminal

[1785] The terminal transmits the readjusted slider setting value to the server again, specifically, the latest setting value according to the user's readjustment.

[1786] server

[1787] The server then uses the image generation AI model to blend the new slider settings and generate a new image. This process is repeated until the user is satisfied.

[1788] Saving and sharing generated images

[1789] server

[1790] Once the final image is confirmed, the server saves it and generates a download link that the user can access, for example, "final_design.jpg" and creates a download link.

[1791] Terminal

[1792] The device will display the generated download link to the user, who can click the link to download the image or use the sharing tool to share it with other designers.

[1793] User

[1794] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with detailed design work based on the user's concrete image.

[1795] Example: Product logo design

[1796] scenario

[1797] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[1798] User

[1799] Users upload three different reference images of logo designs to the tool and then select specific parts of each image (e.g., font style, color, icon) using the rectangular selection tool.

[1800] Server and Device

[1801] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[1802] emotion recognition

[1803] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[1804] Final production and sharing

[1805] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[1806] Example prompts for generative AI models

[1807] For example, the prompt to generate a new logo design might look like this:

[1808] "Generate a modern and elegant logo. Integrate font styles, colors, and icons from reference images to add a bright and energetic element."

[1809] The above is a specific embodiment for carrying out the present invention.

[1810] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1811] Step 1: Upload a reference image

[1812] User

[1813] The user selects multiple reference images from local storage using the system interface and clicks the upload button. As input, the user provides the image files (e.g., "image1.jpg", "image2.png", "image3.gif").

[1814] Terminal

[1815] The device temporarily saves the input image file in local storage and obtains the file path. Then, it sends an upload request to the server. As an output, it generates the image file path.

[1816] server

[1817] The server receives image files sent from the device, assigns a unique identifier to each, and saves them in a specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the storage location and identifier of each image in a database. As output, it generates the saved image file path and identifier.

[1818] Step 2: Select and place an image

[1819] User

[1820] The user sees a list of reference images uploaded on the system interface. Each image is displayed as a thumbnail, and the user can select specific parts using the rectangular selection tool or lasso tool. The thumbnail of the reference image is provided as input. The user selects a specific logo part from "image1.jpg".

[1821] Terminal

[1822] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server, which generates the coordinate data of the selected part as output.

[1823] server

[1824] The server then clips the specific part of the reference image based on the received coordinate information, saves it as a new image file, for example, " / uploads / 12345_thumbnail.jpg", records the metadata of the new image in the database, and generates the clipped image file path as output.

[1825] Step 3: Blending

[1826] User

[1827] The user operates sliders to adjust the blending of selected image parts, setting the transparency and overlay strength by moving the sliders. The slider position is provided as input.

[1828] Terminal

[1829] The device sends the slider values ​​(transparency and overlay strength) set by the user to the server in real time, and generates the setting value data as output.

[1830] server

[1831] The server instructs the image generation AI model to perform blending based on the received slider settings. The image generation AI model generates a new image by integrating the characteristics of each reference image part. The generated new image is saved as "blend_result.jpg". As output, it generates the generated image file.

[1832] Step 4: Emotion Recognition

[1833] User

[1834] The emotion engine analyzes the user's facial expressions and voice tone in real time. The inputs are the user's facial images and voice data captured by the camera and microphone.

[1835] Terminal

[1836] The terminal receives emotion data obtained from the emotion engine and transmits it to the server. For example, emotion data such as "happiness," "surprise," and "sadness" is transmitted. Emotion data is generated as output.

[1837] server

[1838] The server analyzes this emotion data, suggests reference image selection and automatic blending adjustment, and automatically adjusts the characteristics (hue, brightness, contrast, etc.) of the generated image based on the emotion data. The adjusted properties are generated as output.

[1839] Step 5: Check and correct the generated image

[1840] User

[1841] The user reviews the blended image based on the adjustments suggested by the system and readjusts the sliders as needed, providing the adjusted image as input, e.g., fine-tuning hue, brightness, and contrast.

[1842] Terminal

[1843] The device sends the readjusted slider setting value back to the server, which generates the new setting value as output.

[1844] server

[1845] The server then applies the blending process again using the image generation AI model based on the new slider settings to generate a new image. This process is repeated until the user is satisfied. The output is a regenerated image file.

[1846] Step 6: Save and share the generated image

[1847] server

[1848] Once the final image is confirmed, the server saves it and generates a download link that the user can access. For example, it generates "final_design.jpg" and creates a download link. The final design image is provided as input. The download link is generated as output.

[1849] Terminal

[1850] The terminal displays the generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool. The download link is displayed as an output.

[1851] User

[1852] The user downloads the final, concrete design image and sends it to the designer, allowing the designer to proceed with detailed design work based on the user's concrete image. The downloaded image file is provided as output.

[1853] (Application example 2)

[1854] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1855] Current design generation systems have difficulty providing interfaces that non-experts can use effectively, and do not adequately customize based on the emotional state of individual users. Furthermore, the content provided in autonomous vehicles is not optimized according to the emotional state of passengers, which prevents passenger satisfaction from being fully achieved.

[1856] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for uploading a reference image, means for selecting specific portions of the uploaded reference image, means for adjusting the blending of the selected image portions, means for generating an image using a generative AI model for generating a blended image, means for saving and sharing the generated image, means for recognizing passenger emotions in real time, and means for adjusting the generated content based on the emotion data. This enables even non-experts to easily generate specific design images and optimize the content in an autonomous vehicle according to the passenger's emotional state.

[1857] "Reference images" are image files used for design and content generation.

[1858] "Means for uploading" refers to a function that allows a user to transfer data from a local environment to a server.

[1859] The "means for selecting a specific portion" refers to a tool or software function for selecting a specific area of ​​a reference image.

[1860] "Adjustment means" refers to a function for adjusting the blending of selected image portions.

[1861] A "generative AI model" refers to a model that uses artificial intelligence to generate new images and content.

[1862] "Image generation means" refers to a function for generating images using a generative AI model.

[1863] "Means for storing and sharing" refers to the functionality for storing generated content and sharing it with other users and systems.

[1864] "Means of recognizing emotions in real time" refers to a function that analyzes passengers' facial expressions and voice to determine their emotional state at that time.

[1865] "Emotional Data" refers to emotional state information obtained by means of real-time emotion recognition.

[1866] "Means for adjusting content" refers to functions for optimizing the advertisements and entertainment content displayed based on the acquired emotional data.

[1867] The present invention is a system that makes it easy for even non-experts to generate specific design images and provides content based on the emotions of passengers in autonomous vehicles. Specific embodiments are described below.

[1868] System configuration

[1869] This system consists of users, terminals, and a server. To realize the system, the following hardware and software are required:

[1870] 1. Hardware

[1871] Camera (for facial recognition inside the car)

[1872] In-vehicle computer

[1873] High-Performance Server

[1874] 2. Software

[1875] Facial expression recognition API (e.g. facial expression recognition API)

[1876] Image generation AI (e.g. TensorFlow)

[1877] Database systems (e.g. MySQL)

[1878] Interface tools (e.g., Marquee tool, Lasso tool)

[1879] Program processing flow

[1880] 1. Acquiring emotion recognition data

[1881] User: A passenger enters the vehicle.

[1882] Device: An in-car camera captures passengers' faces and analyzes them in real time using an emotion recognition API.

[1883] Server: The analysis results (emotion data) are sent to the server.

[1884] 2. Emotion-Based Design Generation

[1885] Server: Receives emotion data and initiates the design generation process. Users upload reference images and use the interface to select specific parts of the image. The selected image parts are stored in a database.

[1886] User: Use the slider to adjust the blending amount.

[1887] Server: The image generation AI generates new images based on the user's settings. This time, the generative AI model is used.

[1888] 3. Adjust your content

[1889] Server: Automatically adjusts the generated blended images and content based on emotion data. If a passenger expresses joy, bright colors and energetic content will be generated.

[1890] 4. Final review and feedback

[1891] User: Review the generated content and adjust the sliders as needed. The adjusted data is resubmitted to the server and new content is generated. This process is repeated until the user is satisfied.

[1892] Example scenario

[1893] The specific processing flow is illustrated based on the following scenario:

[1894] Scenario: A passenger is perceived to have a depressed expression.

[1895] Emotion Recognition: The emotion recognition module detects "sadness."

[1896] Content generation: The system generates light-hearted comedy videos and refreshing landscape images to turn sadness into joy.

[1897] Feedback: The passenger's expression becomes happy, and the system records this change and reflects it next time.

[1898] Prompt Sentence Examples

[1899] An example of a prompt might be:

[1900] 1. The camera captures passengers' facial expression data and analyzes it in real time.

[1901] 2. If the emotion is recognized as "sadness," the AI ​​model is used to generate ads with upbeat text and images, as well as comedy videos.

[1902] 3. The generated content is displayed on the in-car display, and passenger feedback is retrieved using a slider and reflected in the system.

[1903] The above is a specific embodiment for carrying out the present invention.

[1904] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1905] Step 1: Obtaining emotion data

[1906] Subject: Terminal

[1907] Specific explanation: The in-car camera detects the passenger's face and analyzes the emotional data using a facial expression recognition API. The analyzed data is output as an emotional status such as "happiness," "surprise," or "sadness" and sent to the server.

[1908] Input: Passenger's face image

[1909] Output: Emotion data

[1910] How it works: The device's camera captures facial images, and the facial recognition API extracts facial features from the images and quantifies the emotional state.

[1911] Step 2: Upload a reference image

[1912] Subject: User

[1913] Specific Description: The user selects multiple reference images from the device and uploads them through the system interface. The uploaded images are temporarily stored on the device.

[1914] Input: User selected reference image file

[1915] Output: Reference images uploaded to the server

[1916] Operation details: When the user selects a reference image from local storage and clicks the upload button, the selected image is saved on the device and an upload request is sent to the server.

[1917] Step 3: Select an image area

[1918] Subject: User

[1919] Specific explanation: The user uses the interface to select a specific part of the uploaded reference image using the rectangular selection tool or lasso tool. The coordinate information of the selected part is recorded.

[1920] Input: Reference image and selection tool operation information

[1921] Output: Coordinate data of the selected image part

[1922] Operation details: The user selects an image portion on the interface, and the coordinate information of the selected area is recorded and transmitted by the device.

[1923] Step 4: Blending image parts

[1924] Subject: User

[1925] What it means: The user uses sliders in the interface to adjust the transparency and blending strength of multiple selected image parts, setting the blending effect.

[1926] Input: Coordinate data of the selected image part and user slider setting value

[1927] Output: Blending instruction data based on the set value

[1928] How it works: The user adjusts the slider and the setting is sent to the server in real time.

[1929] Step 5: Image generation

[1930] Subject: Server

[1931] Specific explanation: The server generates a new image using the image generation AI model based on the received slider setting value. The generated image is saved on the server.

[1932] Input: Blending instruction data

[1933] Output: The new image generated

[1934] Operation details: The server inputs instruction data into the generative AI model, and the model outputs a new image, which is saved in a specific directory on the server.

[1935] Step 6: Tailor content based on sentiment data

[1936] Subject: Server

[1937] Specifically, the server automatically adjusts the color, brightness, and content type of the generated images and content based on the emotional data.

[1938] Input: Generated images and emotion data

[1939] Output: Emotionally optimized content

[1940] How it works: The server analyzes the emotion data and adjusts the parameter settings of the image generation AI model to recreate new images and content.

[1941] Step 7: Final review and feedback

[1942] Subject: User

[1943] Specifically: The user can check the generated content and make further adjustments using the sliders. The adjusted settings are sent to the server and the content is generated again.

[1944] Input: Generated content and user feedback

[1945] Output: The final generated content

[1946] How it works: The user reviews the content, makes any necessary adjustments, and the results are sent to the server, which regenerates the content.

[1947] Step 8: Final saving and sharing

[1948] Subject: Server

[1949] Specific description: The server stores the final generated content and generates a download link that can be accessed by the user.

[1950] Input: Finalized content

[1951] Output: Download link

[1952] What it does: The server stores the generated content, generates a link in a format that the user can access, and displays it to the user.

[1953] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1954] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1955] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1956] [Fourth embodiment]

[1957] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1958] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1959] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1960] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1961] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1962] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1963] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1964] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1965] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1966] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1967] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1968] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1969] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1970] The present invention provides a system for non-experts to create specific design images using reference images. The system is operated by a user, a terminal, and a server. The program processing will be described in detail below.

[1971] Program processing

[1972] Uploading a reference image

[1973] User

[1974] The user selects multiple reference images from their local storage and uploads them through the tool's interface, for example, selecting three different image files.

[1975] Terminal

[1976] The terminal temporarily stores these image files according to the user's selection and sends an upload request to the server.

[1977] server

[1978] The server receives the uploaded image files, assigns a unique identifier to each image, and stores it in a specified directory. It also records the location and identifier of each image in a database.

[1979] Selecting and arranging images

[1980] User

[1981] Users can use the rectangular selection tool or lasso tool to select the part of an uploaded reference image they want to use, such as a specific logo or color.

[1982] Terminal

[1983] The terminal records the coordinate information of the part selected by the user and transmits it to the server.

[1984] server

[1985] The server then clips out the selected portion based on the received coordinate information, saves the newly generated thumbnail image, and updates the information about the selected portion in the database.

[1986] Blending Adjustment

[1987] User

[1988] The user adjusts the blending ratio of the selected image by using the sliders, for example, 50% of image A, 30% of image B, and 20% of image C.

[1989] Terminal

[1990] The device sends the slider setting value to the server in real time.

[1991] server

[1992] The server generates a new blended image based on the received slider settings using image generation AI, which integrates the features of each image based on the settings.

[1993] Check and correct the generated image

[1994] User

[1995] The user reviews the resulting image and adjusts the sliders again if necessary, repeating the process until satisfied.

[1996] Terminal

[1997] The terminal retransmits the user's readjusted settings and displays the latest generated image.

[1998] server

[1999] The server then performs the blending process again based on the readjusted settings to generate a new image.

[2000] Saving and sharing generated images

[2001] server

[2002] Once the final image is finalized, the server stores the generated image and generates a download link that the user can access.

[2003] Terminal

[2004] The device will display this download link to the user, who can click the link to download it or use the sharing tool to share it with other designers.

[2005] User

[2006] The user can send the final, concrete design image to the designer, allowing the designer to proceed with the detailed design based on the user's concrete image.

[2007] Example: Product logo design

[2008] scenario

[2009] Suppose a user wants to create a logo design for a new product.

[2010] User

[2011] Users upload three reference images of different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[2012] Server and Device

[2013] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[2014] Final production and sharing

[2015] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[2016] The above is a specific embodiment for carrying out the present invention.

[2017] The processing flow will be explained below.

[2018] Step 1:

[2019] User

[2020] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[2021] Step 2:

[2022] Terminal

[2023] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[2024] Step 3:

[2025] server

[2026] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, "image1.jpg" is saved to " / uploads / 12345.jpg."

[2027] Step 4:

[2028] server

[2029] The server records the storage location and identifier of each image in a database, which allows it to be referenced in subsequent processing.

[2030] Step 5:

[2031] User

[2032] The user uses the system's interface to view a list of reference images they have uploaded, each displayed as a thumbnail.

[2033] Step 6:

[2034] User

[2035] The user uses the rectangular selection tool or lasso tool to select a specific portion of the reference image, for example, a rectangular selection of a specific logo from "image1.jpg."

[2036] Step 7:

[2037] Terminal

[2038] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[2039] Step 8:

[2040] server

[2041] The server then cuts out a specific portion of the reference image based on the received coordinate information. The cut-out portion is saved as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[2042] Step 9:

[2043] User

[2044] The user uses sliders to adjust the blending of the cropped image parts, adjusting the transparency and blending strength of each reference image part.

[2045] Step 10:

[2046] Terminal

[2047] The terminal transmits the slider value set by the user to the server in real time.

[2048] Step 11:

[2049] server

[2050] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[2051] Step 12:

[2052] server

[2053] The new image generated is saved on the server, for example as "blend_result.jpg".

[2054] Step 13:

[2055] User

[2056] The user reviews the resulting image and further adjusts the sliders as needed, and this process is repeated as many times as necessary until the user is satisfied.

[2057] Step 14:

[2058] Terminal

[2059] The terminal transmits the readjusted slider setting value to the server again.

[2060] Step 15:

[2061] server

[2062] The server then uses the image generation AI to perform blending again based on the new slider settings, and a new image is generated.

[2063] Step 16:

[2064] server

[2065] Once the final generated image is confirmed, the server stores it and generates a download link that the user can access.

[2066] Step 17:

[2067] Terminal

[2068] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[2069] Step 18:

[2070] User

[2071] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[2072] The above are the processing steps of the system that allows a non-expert to generate a specific design image using multiple reference images.

[2073] Example 1

[2074] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2075] In modern design work, when non-experts try to create a concrete design based on their own ideas, they need specialized knowledge and skills, which takes a lot of time. Furthermore, sharing a specific design image requires a lot of trial and error, so an efficient design process is required. Given this background, there is a need for a system that allows non-experts to easily generate and share high-quality design images.

[2076] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2077] In this invention, the server includes a means for uploading reference images, a means for selecting specific portions of the uploaded reference images, a means for recording and transmitting coordinate information of the selected reference image portions to the server, an adjustment means for blending the multiple selected image portions, an image generation means using a generative AI model to generate a blended image, and a means for saving and sharing the generated image. This allows even non-experts to easily generate specific design images using multiple reference images and share them with others.

[2078] A "reference image" is an image file that a user uploads to be used as a design element.

[2079] "Non-professionals" refers to general users who do not have specialized design knowledge or skills.

[2080] "Uploading means" is a function that allows a user to send an image file from local storage to the server.

[2081] The "means for selecting a specific portion" is a tool that allows the user to specify and extract a specific portion of the reference image.

[2082] "Coordinate information" is X, Y position data that indicates the position of a specific part of the reference image.

[2083] The "transmitting means" is a communication means for transmitting data from the terminal to the server.

[2084] An "adjuster" is a device or interface for setting the blending ratio of multiple image portions.

[2085] A "generative AI model" is an algorithm or system for generating design images using artificial intelligence technology.

[2086] An "image generator" is a process or device for integrating portions of reference images to generate a new image.

[2087] The "means for saving" is a function for storing the generated image as data.

[2088] "Means for sharing" is a function for sharing the generated image with other users and designers.

[2089] The present invention provides a system for non-experts to generate specific design images using reference images. This system is operated by a user, a terminal, and a server. The specific configuration and operation of the system are described below.

[2090] System Configuration

[2091] User

[2092] The user accesses the system through an interface to upload a reference image, select specific parts, and adjust the blending. The user selects an image from local storage and uploads it to the system as a reference image. The user's operations are transmitted to the terminal via the interface.

[2093] Terminal

[2094] The terminal receives instructions from the user and temporarily stores data such as image files and coordinate information. The terminal also transmits information selected by the user to the server in real time and displays the results. The terminal acts as an interface for image processing.

[2095] server

[2096] The server processes the data sent from the devices and plays a central role in generating images. The server has the following functions:

[2097] Uploading and managing reference images: The server receives reference images uploaded by users, assigns a unique identifier, and stores them. It also records the storage location and identifier of each image in a database. Specifically, OpenCV is used as the image processing library.

[2098] Selection of a specific part of the reference image: The coordinate information of the specific part selected by the user is received, and the selected part is cropped to generate a new thumbnail image. This information is also recorded in the database.

[2099] Blending: Based on user input, an image-generating artificial intelligence (AI) model is used to combine features from selected images to generate a new blended image, using a deep learning framework (e.g., TensorFlow or PyTorch).

[2100] Saving and sharing the generated image: Once the final image is confirmed, the server saves it and creates a download link that users can access. They can use this link to download the image or share it with others using the sharing tool.

[2101] Specific examples

[2102] For example, if a user wants to create a logo design for a new product, they would follow these steps:

[2103] 1. The user uploads reference images of three different logo designs for a new product into the tool.

[2104] 2. Next, select specific parts of each image (e.g. font style, color, icon) using the rectangular selection tool.

[2105] 3. The device sends the coordinate information of the selected area to the server, and the server generates a new image based on that.

[2106] 4. The user uses the slider to adjust the blending of the images to achieve the desired design image.

[2107] 5. Review the final logo image, save it, and share the download link with your designer.

[2108] Prompt Sentence Examples

[2109] "I want to create a logo design for a new product. Upload reference images, select distinctive features from each image, and generate a new design."

[2110] The above is a specific embodiment for carrying out the present invention. This system enables even non-experts to easily create high-quality design images and share them efficiently.

[2111] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2112] Step 1: Upload a reference image

[2113] User

[2114] Users select reference images for their design from their local storage and upload them using the system interface. For example, they can choose three different images for a new product logo design.

[2115] Input: Image file in local storage

[2116] Output: Image file uploaded to the system

[2117] Terminal

[2118] The device temporarily stores the image file selected by the user and sends an upload request to the server, along with image metadata (such as file name, file size, and image format).

[2119] Input: Image file uploaded by the user

[2120] Output: Upload request to server

[2121] server

[2122] The server assigns a unique identifier (such as a UUID) to the received image file and stores it in the specified directory. It also records the storage path, identifier, and metadata of each image in a database.

[2123] Input: Image file and metadata sent from the device

[2124] Output: Image file information recorded in the database

[2125] ---

[2126] Step 2: Select and place an image

[2127] User

[2128] Users can use the rectangular selection or lasso tools to select the portion of an uploaded image they want to use, such as a specific logo, color pattern, or font style.

[2129] Input: Reference image uploaded to the system

[2130] Output: Coordinate information of the selected image part

[2131] Terminal

[2132] The terminal records the coordinate information (X and Y coordinates of the start and end points) of the area selected by the user and sends this coordinate information and the corresponding image identifier to the server.

[2133] Input: Coordinate information selected by the user

[2134] Output: Send coordinates and identifier to server

[2135] server

[2136] The server then crops the selected part based on the received coordinates, using an image processing library (e.g., OpenCV), saves the cropped image part as a newly generated thumbnail, and updates the database.

[2137] Input: Coordinate information and image identifier sent from the device

[2138] Output: Cropped thumbnail images stored in a database

[2139] ---

[2140] Step 3: Blending

[2141] User

[2142] The user adjusts the blending ratio of the selected image parts by manipulating the sliders, for example, 50% of image A, 30% of image B, and 20% of image C.

[2143] Input: Information about the selected image part

[2144] Output: Slider setting (blending ratio for each image part)

[2145] Terminal

[2146] The device sends the slider setting value specified by the user to the server in real time using a real-time protocol such as WebSocket.

[2147] Input: The slider setting adjusted by the user

[2148] Output: Send slider setting value to server

[2149] server

[2150] The server generates a new image based on the slider settings received using a generative AI model for blending the selected areas, using a deep learning framework (e.g., TensorFlow, PyTorch).

[2151] Input: Slider setting value and selection information

[2152] Output: The resulting blended image

[2153] ---

[2154] Step 4: Check and correct the generated image

[2155] User

[2156] The user reviews the generated image and adjusts the sliders again if necessary, repeating the process until they reach a design they are happy with.

[2157] Input: The generated blended image

[2158] Output: Rescaled slider value

[2159] Terminal

[2160] The terminal retransmits the user's new settings to the server and displays the latest generated image.

[2161] Input: The slider setting that the user has readjusted

[2162] Output: Sends the reset values ​​to the server and displays the latest image

[2163] server

[2164] The server then performs the blending process again based on the resubmitted settings and generates a new image, again using the generative AI model.

[2165] Input: Resent slider setting value

[2166] Output: Regenerated blended image

[2167] ---

[2168] Step 5: Save and share the generated image

[2169] server

[2170] Once the final image is confirmed, the server stores the generated image and creates a download link that users can access using a temporary token for security reasons.

[2171] Input: Final generated image

[2172] Output: Download link

[2173] Terminal

[2174] The device will display this download link to the user, who can click the link to download the image or use the share feature to share it with others.

[2175] Input: Download link sent by the server

[2176] Output: Display link to user

[2177] User

[2178] The user can send the final, concrete design image to the designer, allowing the designer to proceed with the detailed design based on the user's concrete image.

[2179] Input: Generated image obtained from the download link

[2180] Output: Share the design image with the designer

[2181] The above is a tentative concrete flow of the program processing of this system.

[2182] (Application example 1)

[2183] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2184] Creating advertising materials easily often requires specialized design knowledge and advanced techniques, making it difficult for non-experts to create them. Furthermore, the process of effectively utilizing multiple reference images held by users to generate specific design images is inefficient and time-consuming. Furthermore, there is a lack of methods for appropriately blending multiple reference images to generate new designs. There is a need for a system that can solve these issues and enable anyone to easily generate high-quality advertising materials.

[2185] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2186] In this invention, the server includes a means for uploading reference images, a means for selecting specific portions of the uploaded reference images, and an adjustment means for blending the selected image portions. This allows even non-experts to easily generate specific design images using multiple reference images and use them as advertising materials. Furthermore, the server includes a means for providing prompts during the image generation process, allowing users to easily select and adjust designs.

[2187] "Reference Image" means an image file used to form part or all of a design.

[2188] "Non-professionals" refer to general users who do not have specialized knowledge or advanced skills in design or image editing.

[2189] A "specific design image" is the final visual expression that the user is aiming for.

[2190] "Upload" refers to the act of sending data from a user's terminal to a server.

[2191] A "selection tool" is a tool or interface for specifying and cutting out a particular portion of a reference image.

[2192] The "blending adjustment means" is an interface for setting how to appropriately combine multiple selected image portions.

[2193] "Image generation method" refers to a process or technique that generates a new image based on selected image portions and configured blending adjustments.

[2194] "Prompts" are instructions or hints provided to the user to assist in generating advertising material.

[2195] "Advertising Materials" means visual, textual or other form of design elements used for advertising purposes.

[2196] "Storage and sharing means" refers to an interface or function for long-term storage of generated images and sharing with other users and designers.

[2197] This invention provides a system that allows non-experts to create specific design images using multiple reference images. This system is operated by a user, a terminal, and a server. The operation of the program and its processing are described in detail below.

[2198] Uploading a reference image

[2199] A user selects multiple reference images from their local storage and uploads them to the server via their device. Specifically, the user uses an interface for uploading reference images from a smartphone or tablet and sends the data to the server. The device then sends an upload request to the server, and the server stores the received image files with a unique identifier.

[2200] Selecting and arranging images

[2201] The user selects a specific part of the uploaded reference image using a rectangular selection tool or lasso tool. For example, a specific logo or color can be selected. The device records the coordinate information of the selected part and sends it to the server. The server then crops the selected part based on the received coordinate information and saves a newly generated thumbnail image.

[2202] Blending Adjustment

[2203] The user adjusts the blending ratio of the selected image parts by operating the sliders. For example, they can set 50% of image A, 30% of image B, and 20% of image C. The device sends the slider settings to the server in real time. Based on the received slider settings, the server generates a new blended image using an image generation AI model. This AI model specifically integrates the features of multiple reference images to generate a new design image.

[2204] Checking and correcting generated images

[2205] The user reviews the generated image and adjusts the sliders again if necessary. This process can be repeated until the user is satisfied. The device resends the user's adjusted settings and displays the latest generated image. The server re-performs the blending process based on the adjusted settings and generates a new image.

[2206] Saving and sharing generated images

[2207] Once the final design image is confirmed, the server saves the generated image and generates a download link that users can access and download, or share with others using the sharing tool.

[2208] Hardware and software used

[2209] Hardware: User's smartphone, tablet, server

[2210] Software: PIL library for image processing, requests library for HTTP communication, image generation model

[2211] Specific examples

[2212] If a user wants to create advertising materials for a new product, they can proceed using the following prompt:

[2213] "Please upload three images:

[2214] 1. Logo image

[2215] 2. Color-Scheme Images

[2216] 3. Sample images of the design

[2217] Select the part of each image you want to use:

[2218] Logo: 10, 10, 100, 100

[2219] Color scheme: 20, 20, 120, 120

[2220] Design samples: 30, 30, 140, 140

[2221] Set the blend ratio:

[2222] Logo: 50%

[2223] Color Scheme: 30%

[2224] Design sample: 20%

[2225] Review the generated ad assets, adjust them again if necessary, and save or share the final ad asset you're happy with.

[2226] In this way, a system is provided that enables even non-experts to easily generate high-quality advertising materials.

[2227] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2228] Step 1: Upload an image

[2229] Input: Reference images stored in the user's local storage.

[2230] How it works: The user selects a reference image from their smartphone or tablet and uploads it to the server using the application's interface.

[2231] Data processing and calculation: The terminal temporarily saves the selected image file and sends the image data to the server via an HTTP request.

[2232] Output: Reference image and unique identifier stored on the server.

[2233] Step 2: Select an image area

[2234] Input: Reference images uploaded to the server.

[2235] What it does: The user uses the in-app rectangular selection tool or lasso tool to select a specific part of the reference image.

[2236] Data processing and calculation: The terminal records the coordinate information of the part selected by the user and sends it to the server.

[2237] Output: Coordinate information recorded on the server and thumbnail images of the cropped areas.

[2238] Step 3: Blending

[2239] Input: Coordinate information and image data of the selected area recorded on the server.

[2240] What it does: The user manipulates an in-app slider to set the blend ratio of multiple selections.

[2241] Data processing and calculation: The device sends the slider values ​​to the server in real time. The server generates a new blended image based on these settings using an image generation AI model.

[2242] Output: The new image data generated.

[2243] Step 4: Check and adjust the generated image

[2244] Input: Generated image sent from the server to the device.

[2245] Specific operation: The user can check the generated image within the app and adjust the sliders again to correct the blending ratio if necessary.

[2246] Data processing and calculation: The device sends the adjusted slider values ​​to the server, and the server performs blending again based on the new settings.

[2247] Output: The new generated image data after correction.

[2248] Step 5: Save and share the resulting image

[2249] Input: The final generated image data.

[2250] Specific operation: Once the user finalizes the design image, the server saves the image and generates a download link, which the user can use to download the image or share it with other users.

[2251] Output: Download link and final generated image data.

[2252] By following the steps above, even non-experts can easily generate advertising materials and efficiently create the desired design.

[2253] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2254] The present invention is a system that allows non-experts to create specific design images using reference images, and by combining it with an emotion engine that recognizes the user's emotions, it makes the design process smoother and more personalized. This system is executed by the user, a terminal, and a server. The program processing is described in detail below.

[2255] Program processing

[2256] Uploading a reference image

[2257] User

[2258] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[2259] Terminal

[2260] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[2261] server

[2262] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the saved location and identifier in the database.

[2263] Selecting and arranging images

[2264] User

[2265] The user uses the system interface to view a list of uploaded reference images. Each image is displayed as a thumbnail. The user can then use the rectangular selection tool or lasso tool to select specific portions of the reference images. For example, select a specific logo from "image1.jpg."

[2266] Terminal

[2267] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[2268] server

[2269] The server then cuts out a specific portion of the reference image based on the received coordinate information, and saves the cut-out portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[2270] Blending Adjustment

[2271] User

[2272] The user adjusts the blending strength of the selected image portion by manipulating the sliders, which adjust the transparency and blending strength of each reference image portion.

[2273] Terminal

[2274] The terminal transmits the slider value set by the user to the server in real time.

[2275] server

[2276] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[2277] server

[2278] The new image generated is saved on the server, for example as "blend_result.jpg".

[2279] emotion recognition

[2280] User

[2281] To recognize the user's emotions in real time, the emotion engine analyzes the user's facial expressions and vocal tone. Once the emotion data is acquired, the user's current emotional state is determined.

[2282] Terminal

[2283] The device receives emotion data from the emotion engine and transmits it to the server. For example, emotions such as "happiness," "surprise," and "sadness" are sent as data.

[2284] server

[2285] The server analyzes this emotion data and suggests the selection of reference images and automatic adjustment of blending conditions. It also automatically adjusts the characteristics of the generated image (hue, brightness, contrast, etc.) based on the emotion data.

[2286] Checking and correcting generated images

[2287] User

[2288] The user reviews the blended image adjusted based on the emotion engine's suggestions, and further adjusts the sliders as needed. This process is repeated as many times as necessary until the user is satisfied.

[2289] Terminal

[2290] The terminal transmits the readjusted slider setting value to the server again.

[2291] server

[2292] The server then uses the image generation AI to perform the blending process again based on the new slider settings, and a new image is generated.

[2293] Saving and sharing generated images

[2294] server

[2295] Once the final generated image is confirmed, the server stores it and generates a download link that the user can access.

[2296] Terminal

[2297] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[2298] User

[2299] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[2300] Example: Product logo design

[2301] scenario

[2302] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[2303] User

[2304] Users upload three reference images of different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[2305] Server and Device

[2306] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[2307] emotion recognition

[2308] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[2309] Final production and sharing

[2310] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[2311] The above is a specific embodiment for carrying out the present invention.

[2312] The processing flow will be explained below.

[2313] Program processing

[2314] Step 1:

[2315] User

[2316] The user selects multiple reference images from their local storage and uploads them using the tool's interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[2317] Step 2:

[2318] Terminal

[2319] The terminal stores these image files in a temporary storage according to the user's selection, and sends an upload request to the server.

[2320] Step 3:

[2321] server

[2322] The server receives the uploaded image file, assigns a unique identifier, and saves it in the specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the saved location and identifier in the database.

[2323] Step 4:

[2324] User

[2325] The user uses the system's interface to view a list of reference images they have uploaded, each displayed as a thumbnail.

[2326] Step 5:

[2327] User

[2328] The user uses the rectangular selection tool or lasso tool to select a specific portion of the reference image, for example, a specific logo portion from "image1.jpg."

[2329] Step 6:

[2330] Terminal

[2331] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server.

[2332] Step 7:

[2333] server

[2334] The server then cuts out a specific portion of the reference image based on the received coordinate information, and saves the cut-out portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg."

[2335] Step 8:

[2336] User

[2337] The user uses sliders to adjust the blending of selected image parts, adjusting the transparency and overlay strength of each reference image part.

[2338] Step 9:

[2339] Terminal

[2340] The terminal transmits the slider value set by the user to the server in real time.

[2341] Step 10:

[2342] server

[2343] The server then instructs the image generation AI to blend based on the slider settings received. The AI ​​model then integrates the characteristics of each reference image part and generates a new image based on the user's settings.

[2344] Step 11:

[2345] server

[2346] The new image generated is saved on the server, for example as "blend_result.jpg".

[2347] Step 12:

[2348] User

[2349] To recognize the user's emotions in real time, the emotion engine analyzes the user's facial expressions and vocal tone. Based on the acquired emotion data, the user's current emotional state is determined.

[2350] Step 13:

[2351] Terminal

[2352] The device receives emotion data from the emotion engine and transmits it to the server. For example, emotions such as "joy" or "surprise" are sent as data.

[2353] Step 14:

[2354] server

[2355] The server analyzes this emotion data and suggests reference image selection and automatic blending adjustments. For example, if the user expresses a "sad" emotion, the system suggests a gentler color tone.

[2356] Step 15:

[2357] User

[2358] The user reviews the blended image adjusted based on the emotion engine's suggestions, further adjusts the sliders as needed, and repeats this process until the user is satisfied.

[2359] Step 16:

[2360] Terminal

[2361] The terminal transmits the readjusted slider setting value to the server again.

[2362] Step 17:

[2363] server

[2364] The server then uses the image generation AI to perform the blending process again based on the new slider settings, and a new image is generated.

[2365] Step 18:

[2366] server

[2367] Once the final image is confirmed, the server saves it and generates a download link for the user to access, for example, " / downloads / final_image123.jpg".

[2368] Step 19:

[2369] Terminal

[2370] The device will then display a generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool.

[2371] Step 20:

[2372] User

[2373] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with the detailed design based on the user's concrete image.

[2374] Example: Product logo design

[2375] scenario

[2376] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[2377] User

[2378] Users upload three reference images of different logo designs to the tool, then select specific elements from each image (e.g., font style, color, icon) using the rectangular selection tool.

[2379] Server and Device

[2380] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[2381] emotion recognition

[2382] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[2383] Final production and sharing

[2384] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[2385] The above is a specific embodiment for carrying out the present invention.

[2386] Example 2

[2387] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2388] Conventional design systems have the problem that it is difficult for non-experts to generate concrete design images, and they also have the problem that they cannot incorporate user emotions into the design process, making it difficult to provide a personalized user experience.

[2389] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for uploading a reference image, a means for selecting a specific portion of the uploaded reference image, an adjustment means for blending the selected image portions, an image generation means for generating a blended image, a means for recognizing a user's emotions in real time and suggesting adjustments to the generated image, and a means for saving and sharing the generated image. This allows even a non-expert to easily generate specific design images and propose individualized designs based on the user's emotions.

[2390] A "reference image" is an image file that a user uses when generating a design image.

[2391] "Means for uploading" is a function that allows a user to send a reference image from local storage to the system.

[2392] The "means for selecting a specific portion" is a tool that allows the user to select a portion of interest from within the reference image.

[2393] The "adjustment means for blending" is a function that adjusts the transparency and overlay strength of the image portion selected by the user.

[2394] The "image generation means" is a function that integrates the blended image parts to create a new design image.

[2395] "Means for recognizing emotions in real time" is a function that analyzes the user's facial expressions and vocal tone to determine their current emotional state.

[2396] The "means for saving and sharing the generated image" is a function for saving the final generated design image and sharing it with other users.

[2397] The present invention is a system that allows non-experts to generate specific design images using multiple reference images. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the design process becomes smoother and more personalized. Specific embodiments for implementing the present invention are described below.

[2398] Uploading a reference image

[2399] User

[2400] The user selects multiple reference images from their local storage and uploads them using the system interface. For example, select three images from the desktop: "image1.jpg", "image2.png", and "image3.gif".

[2401] Terminal

[2402] The terminal stores the image file selected by the user in a temporary storage and transmits an upload request to the server. For example, the terminal stores the image in a temporary folder in the local storage and acquires the file path.

[2403] server

[2404] The server receives image files uploaded from the device, assigns a unique identifier to each file, and saves them in a specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the location and identifier of each image in a database.

[2405] Selecting and arranging images

[2406] User

[2407] The user uses the system interface to view a list of uploaded reference images. Each image is displayed as a thumbnail. The user can then use the rectangular selection tool or lasso tool to select specific portions of the reference images. For example, select a specific logo from "image1.jpg."

[2408] Terminal

[2409] The device records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server. Specifically, the coordinate data of the selected range is sent to the server in JSON format.

[2410] server

[2411] The server then extracts a specific portion of the reference image based on the received coordinate information, saves the extracted portion as a new image file, for example, " / uploads / 12345_thumbnail.jpg," and records the metadata of the new image in the database.

[2412] Blending Adjustment

[2413] User

[2414] Users can adjust the blending of selected image parts by adjusting the slider: moving it to the right increases transparency, moving it to the left increases opacity, and also adjust the strength of the blending.

[2415] Terminal

[2416] The device sends the slider values ​​(transparency and overlay strength) set by the user to the server in real time. Specifically, the values ​​are updated sequentially according to the slider operation and sent to the server.

[2417] server

[2418] The server then sends blending instructions to the image generation AI model based on the slider settings received. The AI ​​model then uses an algorithm to combine the characteristics of each reference image part and generate a new image based on the user's settings. The new image is saved as "blend_result.jpg."

[2419] emotion recognition

[2420] User

[2421] To recognize a user's emotions in real time, the emotion engine analyzes facial expressions and vocal tone, for example, by capturing a picture of the user's face with a camera and recording the user's voice with a microphone.

[2422] Terminal

[2423] The device receives emotion data obtained from the emotion engine and sends it to the server. For example, emotion data such as "happiness," "surprise," and "sadness" is sent in JSON format.

[2424] server

[2425] The server analyzes this emotion data and suggests the selection of reference images and automatic adjustment of blending. It automatically adjusts the characteristics of the generated image (hue, brightness, contrast, etc.) based on the emotion data to provide the optimal design for the user.

[2426] Check and correct the generated image

[2427] User

[2428] The user can then review the blended image adjusted based on the emotion engine's suggestions and further adjust the sliders as needed, for example, fine-tuning the hue, brightness, and contrast.

[2429] Terminal

[2430] The terminal transmits the readjusted slider setting value to the server again, specifically, the latest setting value according to the user's readjustment.

[2431] server

[2432] The server then uses the image generation AI model to blend the new slider settings and generate a new image. This process is repeated until the user is satisfied.

[2433] Saving and sharing generated images

[2434] server

[2435] Once the final image is confirmed, the server saves it and generates a download link that the user can access, for example, "final_design.jpg" and creates a download link.

[2436] Terminal

[2437] The device will display the generated download link to the user, who can click the link to download the image or use the sharing tool to share it with other designers.

[2438] User

[2439] The user downloads the final, concrete design image and sends it to the designer, who can then proceed with detailed design work based on the user's concrete image.

[2440] Example: Product logo design

[2441] scenario

[2442] Imagine a user wants to create a logo design for a new product. Their goal is to create a modern, cool logo.

[2443] User

[2444] Users upload three different reference images of logo designs to the tool and then select specific parts of each image (e.g., font style, color, icon) using the rectangular selection tool.

[2445] Server and Device

[2446] The device sends coordinate information to the server, which then generates a new image based on that information. The user then adjusts the blending using sliders, repeating the process until the desired design image is realized.

[2447] emotion recognition

[2448] The emotion engine recognizes the user's emotions, and if the emotion "joy" is recognized, for example, the system suggests bright colors and energetic elements.

[2449] Final production and sharing

[2450] The generated logo image can be viewed, saved, and the download link shared with the designer, allowing the user to clearly communicate their image to the designer and ensuring an efficient design process.

[2451] Example prompts for generative AI models

[2452] For example, the prompt to generate a new logo design might look like this:

[2453] "Generate a modern and elegant logo. Integrate font styles, colors, and icons from reference images to add a bright and energetic element."

[2454] The above is a specific embodiment for carrying out the present invention.

[2455] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2456] Step 1: Upload a reference image

[2457] User

[2458] The user selects multiple reference images from local storage using the system interface and clicks the upload button. As input, the user provides the image files (e.g., "image1.jpg", "image2.png", "image3.gif").

[2459] Terminal

[2460] The device temporarily saves the input image file in local storage and obtains the file path. Then, it sends an upload request to the server. As an output, it generates the image file path.

[2461] server

[2462] The server receives image files sent from the device, assigns a unique identifier to each, and saves them in a specified directory. For example, it saves "image1.jpg" to " / uploads / 12345.jpg." It also records the storage location and identifier of each image in a database. As output, it generates the saved image file path and identifier.

[2463] Step 2: Select and place an image

[2464] User

[2465] The user sees a list of reference images uploaded on the system interface. Each image is displayed as a thumbnail, and the user can select specific parts using the rectangular selection tool or lasso tool. The thumbnail of the reference image is provided as input. The user selects a specific logo part from "image1.jpg".

[2466] Terminal

[2467] The terminal records the coordinate information (x, y, width, height) of the part selected by the user and sends that information to the server, which generates the coordinate data of the selected part as output.

[2468] server

[2469] The server then clips the specific part of the reference image based on the received coordinate information, saves it as a new image file, for example, " / uploads / 12345_thumbnail.jpg", records the metadata of the new image in the database, and generates the clipped image file path as output.

[2470] Step 3: Blending

[2471] User

[2472] The user operates sliders to adjust the blending of selected image parts, setting the transparency and overlay strength by moving the sliders. The slider position is provided as input.

[2473] Terminal

[2474] The device sends the slider values ​​(transparency and overlay strength) set by the user to the server in real time, and generates the setting value data as output.

[2475] server

[2476] The server instructs the image generation AI model to perform blending based on the received slider settings. The image generation AI model generates a new image by integrating the characteristics of each reference image part. The generated new image is saved as "blend_result.jpg". As output, it generates the generated image file.

[2477] Step 4: Emotion Recognition

[2478] User

[2479] The emotion engine analyzes the user's facial expressions and voice tone in real time. The inputs are the user's facial images and voice data captured by the camera and microphone.

[2480] Terminal

[2481] The terminal receives emotion data obtained from the emotion engine and transmits it to the server. For example, emotion data such as "happiness," "surprise," and "sadness" is transmitted. Emotion data is generated as output.

[2482] server

[2483] The server analyzes this emotion data, suggests reference image selection and automatic blending adjustment, and automatically adjusts the characteristics (hue, brightness, contrast, etc.) of the generated image based on the emotion data. The adjusted properties are generated as output.

[2484] Step 5: Check and correct the generated image

[2485] User

[2486] The user reviews the blended image based on the adjustments suggested by the system and readjusts the sliders as needed, providing the adjusted image as input, e.g., fine-tuning hue, brightness, and contrast.

[2487] Terminal

[2488] The device sends the readjusted slider setting value back to the server, which generates the new setting value as output.

[2489] server

[2490] The server then applies the blending process again using the image generation AI model based on the new slider settings to generate a new image. This process is repeated until the user is satisfied. The output is a regenerated image file.

[2491] Step 6: Save and share the generated image

[2492] server

[2493] Once the final image is confirmed, the server saves it and generates a download link that the user can access. For example, it generates "final_design.jpg" and creates a download link. The final design image is provided as input. The download link is generated as output.

[2494] Terminal

[2495] The terminal displays the generated download link to the user, who can click the link to download the image or share it with other designers using the sharing tool. The download link is displayed as an output.

[2496] User

[2497] The user downloads the final, concrete design image and sends it to the designer, allowing the designer to proceed with detailed design work based on the user's concrete image. The downloaded image file is provided as output.

[2498] (Application example 2)

[2499] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2500] Current design generation systems have difficulty providing interfaces that non-experts can use effectively, and do not adequately customize based on the emotional state of individual users. Furthermore, the content provided in autonomous vehicles is not optimized according to the emotional state of passengers, which prevents passenger satisfaction from being fully achieved.

[2501] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for uploading a reference image, means for selecting specific portions of the uploaded reference image, means for adjusting the blending of the selected image portions, means for generating an image using a generative AI model for generating a blended image, means for saving and sharing the generated image, means for recognizing passenger emotions in real time, and means for adjusting the generated content based on the emotion data. This enables even non-experts to easily generate specific design images and optimize the content in an autonomous vehicle according to the passenger's emotional state.

[2502] "Reference images" are image files used for design and content generation.

[2503] "Means for uploading" refers to a function that allows a user to transfer data from a local environment to a server.

[2504] The "means for selecting a specific portion" refers to a tool or software function for selecting a specific area of ​​a reference image.

[2505] "Adjustment means" refers to a function for adjusting the blending of selected image portions.

[2506] A "generative AI model" refers to a model that uses artificial intelligence to generate new images and content.

[2507] "Image generation means" refers to a function for generating images using a generative AI model.

[2508] "Means for storing and sharing" refers to the functionality for storing generated content and sharing it with other users and systems.

[2509] "Means of recognizing emotions in real time" refers to a function that analyzes passengers' facial expressions and voice to determine their emotional state at that time.

[2510] "Emotional Data" refers to emotional state information obtained by means of real-time emotion recognition.

[2511] "Means for adjusting content" refers to functions for optimizing the advertisements and entertainment content displayed based on the acquired emotional data.

[2512] The present invention is a system that makes it easy for even non-experts to generate specific design images and provides content based on the emotions of passengers in autonomous vehicles. Specific embodiments are described below.

[2513] System configuration

[2514] This system consists of users, terminals, and a server. To realize the system, the following hardware and software are required:

[2515] 1. Hardware

[2516] Camera (for facial recognition inside the car)

[2517] In-vehicle computer

[2518] High-Performance Server

[2519] 2. Software

[2520] Facial expression recognition API (e.g. facial expression recognition API)

[2521] Image generation AI (e.g. TensorFlow)

[2522] Database systems (e.g. MySQL)

[2523] Interface tools (e.g., Marquee tool, Lasso tool)

[2524] Program processing flow

[2525] 1. Acquiring emotion recognition data

[2526] User: A passenger enters the vehicle.

[2527] Device: An in-car camera captures passengers' faces and analyzes them in real time using an emotion recognition API.

[2528] Server: The analysis results (emotion data) are sent to the server.

[2529] 2. Emotion-Based Design Generation

[2530] Server: Receives emotion data and initiates the design generation process. Users upload reference images and use the interface to select specific parts of the image. The selected image parts are stored in a database.

[2531] User: Use the slider to adjust the blending amount.

[2532] Server: The image generation AI generates new images based on the user's settings. This time, the generative AI model is used.

[2533] 3. Adjust your content

[2534] Server: Automatically adjusts the generated blended images and content based on emotion data. If the passenger is expressing joy, bright colors and energetic content will be generated.

[2535] 4. Final review and feedback

[2536] User: Review the generated content and adjust the sliders as needed. The adjusted data is resubmitted to the server and new content is generated. This process is repeated until the user is satisfied.

[2537] Example scenario

[2538] The specific processing flow is illustrated based on the following scenario:

[2539] Scenario: A passenger is perceived to have a depressed expression.

[2540] Emotion Recognition: The emotion recognition module detects "sadness."

[2541] Content generation: The system generates light-hearted comedy videos and refreshing landscape images to turn sadness into joy.

[2542] Feedback: The passenger's expression becomes happy, and the system records this change and reflects it next time.

[2543] Prompt Sentence Examples

[2544] An example of a prompt might be:

[2545] 1. The camera captures passengers' facial expression data and analyzes it in real time.

[2546] 2. If the emotion is recognized as "sadness," the AI ​​model is used to generate ads with upbeat text and images, as well as comedy videos.

[2547] 3. The generated content is displayed on the in-car display, and passenger feedback is retrieved using a slider and reflected in the system.

[2548] The above is a specific embodiment for carrying out the present invention.

[2549] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2550] Step 1: Obtaining emotion data

[2551] Subject: Terminal

[2552] Specific explanation: The in-car camera detects the passenger's face and analyzes the emotional data using a facial expression recognition API. The analyzed data is output as an emotional status such as "happiness," "surprise," or "sadness" and sent to the server.

[2553] Input: Passenger's face image

[2554] Output: Emotion data

[2555] How it works: The device's camera captures facial images, and the facial recognition API extracts facial features from the images and quantifies the emotional state.

[2556] Step 2: Upload a reference image

[2557] Subject: User

[2558] Specific Description: The user selects multiple reference images from the device and uploads them through the system interface. The uploaded images are temporarily stored on the device.

[2559] Input: User selected reference image file

[2560] Output: Reference images uploaded to the server

[2561] Operation details: When the user selects a reference image from local storage and clicks the upload button, the selected image is saved on the device and an upload request is sent to the server.

[2562] Step 3: Select an image area

[2563] Subject: User

[2564] Specific explanation: The user uses the interface to select a specific part of the uploaded reference image using the rectangular selection tool or lasso tool. The coordinate information of the selected part is recorded.

[2565] Input: Reference image and selection tool operation information

[2566] Output: Coordinate data of the selected image part

[2567] Operation details: The user selects an image portion on the interface, and the coordinate information of the selected area is recorded and transmitted by the device.

[2568] Step 4: Blending image parts

[2569] Subject: User

[2570] What it means: The user uses sliders in the interface to adjust the transparency and blending strength of multiple selected image parts, setting the blending effect.

[2571] Input: Coordinate data of the selected image part and user slider setting value

[2572] Output: Blending instruction data based on the set value

[2573] How it works: The user adjusts the slider and the setting is sent to the server in real time.

[2574] Step 5: Image generation

[2575] Subject: Server

[2576] Specific explanation: The server generates a new image using the image generation AI model based on the received slider setting value. The generated image is saved on the server.

[2577] Input: Blending instruction data

[2578] Output: The new image generated

[2579] Operation details: The server inputs instruction data into the generative AI model, and the model outputs a new image, which is saved in a specific directory on the server.

[2580] Step 6: Tailor content based on sentiment data

[2581] Subject: Server

[2582] Specifically, the server automatically adjusts the color, brightness, and content type of the generated images and content based on the emotional data.

[2583] Input: Generated images and emotion data

[2584] Output: Emotionally optimized content

[2585] How it works: The server analyzes the emotion data and adjusts the parameter settings of the image generation AI model to recreate new images and content.

[2586] Step 7: Final review and feedback

[2587] Subject: User

[2588] Specifically: The user can check the generated content and make further adjustments using the sliders. The adjusted settings are sent to the server and the content is generated again.

[2589] Input: Generated content and user feedback

[2590] Output: The final generated content

[2591] How it works: The user reviews the content, makes any necessary adjustments, and the results are sent to the server, which regenerates the content.

[2592] Step 8: Final saving and sharing

[2593] Subject: Server

[2594] Specific description: The server stores the final generated content and generates a download link that can be accessed by the user.

[2595] Input: Finalized content

[2596] Output: Download link

[2597] What it does: The server stores the generated content, generates a link in a format that the user can access, and displays it to the user.

[2598] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2599] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2600] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2601] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2602] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2603] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2604] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2605] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2606] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2607] The emoti...

Claims

1. A system for non-experts to generate specific design images using multiple reference images, a means for uploading reference images; means for selecting a particular portion of the uploaded reference image; an adjustment means for blending a plurality of selected image portions; image generation means for generating a blended image; a means for storing and sharing the generated images; A system including:

2. 2. The system of claim 1, wherein the means for selecting a particular portion of the reference image uses a rectangular selection tool or a lasso tool.

3. 10. The system of claim 1, wherein the image generating means for generating the blended image uses image generating artificial intelligence.

Images