system

Abstract

To provide a system that automatically makes an illustration drawn by a user beautiful using advanced image processing techniques.SOLUTION: A system comprises: means for acquiring image data representing an illustration drawn by a user; means for automatically adjusting visual characteristics of the image data; means for recognizing the user's feeling using a feeling engine; and means for adjusting the visual characteristics of the image data by using a prompt sentence that instructs adjustment of the visual characteristics of the image data and generative AI on the basis of the user's feeling.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Current image editing tools require advanced technology and are difficult for ordinary users to use freely. Also, there is no means to easily beautify the illustrations drawn by oneself.

Means for Solving the Problems

[0005] The present invention provides a system that automatically beautifies illustrations drawn by users using advanced image processing technology. Furthermore, it provides an intuitive user interface and means for users to adjust beautification parameters, realizing an environment where users can freely beautify illustrations.

Brief Description of the Drawings

[0006] [Figure 1]This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1 of Embodiment 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2 of Embodiment 2. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2. [Figure 15] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 3 of Example 3. [Figure 16] This is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Form Example 3. [Figure 17] It is a sequence diagram showing the processing flow of the data processing system in Example 1 of Form Example 1 when combined with an emotion engine. [Figure 18] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. [Figure 19] It is a sequence diagram showing the processing flow of the data processing system in Example 2 of Form Example 2 when combined with an emotion engine. [Figure 20] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2 when combined with an emotion engine. [Figure 21] It is a sequence diagram showing the processing flow of the data processing system in Example 3 of Form Example 3 when combined with an emotion engine. [Figure 22] It is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Form Example 3 when combined with an emotion engine.

Embodiments for Carrying Out the Invention

[0007] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (TENSOR PROCESSING UNIT (registered trademark)), etc.

[0010] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0011] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disk (e.g., hard disk), or magnetic tape, etc.

[0012] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0014] [First Embodiment]

[0015] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0016] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0019] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0022] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0026] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0027] "Example of form 1"

[0028] One embodiment of the present invention provides a system that automatically beautifies illustrations drawn by a user. This system has the function of automatically beautifying illustrations drawn by a user using advanced image processing technology. Specifically, it receives an illustration drawn by a user as input, automatically adjusts the visual characteristics of the illustration such as color tone, brightness, and contrast, and outputs a beautified illustration.

[0029] "Example of form 2"

[0030] Furthermore, as one embodiment of the present invention, an intuitively operable user interface is provided. This user interface includes intuitively operable UI elements such as sliders and dials. Specifically, the user can adjust the beautification parameters by operating the sliders and dials. This allows the user to adjust the degree of beautification of the illustration to their liking.

[0031] "Example of form 3"

[0032] Furthermore, in one embodiment of the present invention, a means is provided that allows the user to adjust the beautification parameters. Specifically, the user can freely adjust the degree of beautification of the illustration by directly inputting the beautification parameters. For example, the user can beautify the illustration to their liking by adjusting parameters such as "color intensity" and "brightness adjustment".

[0033] The following describes the processing flow for each example of the form.

[0034] "Example of form 1"

[0035] Step 1: Input the illustration drawn by the user into the system. This input can be done via file upload, drag and drop, etc.

[0036] Step 2: The system analyzes the input illustration and automatically adjusts its visual characteristics such as hue, brightness, and contrast. This adjustment is performed using a pre-trained machine learning model.

[0037] Step 3: The system outputs the adjusted illustration. The output can be displayed directly on the screen or downloaded as a file.

[0038] "Example of form 2"

[0039] Step 1: The user operates the system's user interface. Specifically, they adjust the beautification parameters by operating sliders and dials.

[0040] Step 2: The system adjusts the beautification parameters according to the user's input and beautifies the illustration.

[0041] Step 3: The system outputs the beautified illustration. The output can be displayed directly on the screen or downloaded as a file.

[0042] "Example of form 3"

[0043] Step 1: The user directly inputs the beautification parameters. Specifically, parameters such as "color intensity" and "brightness adjustment" are specified using numerical values ​​or sliders.

[0044] Step 2: The system beautifies the illustration based on the beautification parameters entered by the user.

[0045] Step 3: The system outputs the beautified illustration. The output can be displayed directly on the screen or downloaded as a file.

[0046] (Example 1)

[0047] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0048] Traditional illustration beautification systems required users to manually adjust color tone, brightness, and contrast, resulting in a cumbersome operation. Furthermore, they lacked an intuitive user interface, making the beautification process difficult for users. Additionally, the lack of real-time progress displays and completion notifications resulted in a poor user experience.

[0049] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes means for the user to upload an illustration, means for the server to receive the illustration, means for the server to adjust the color tone, means for the server to adjust the brightness, means for the server to adjust the contrast, means for the server to generate a beautified illustration, and means for the server to provide the beautified illustration to the user. This makes it possible for the user to easily beautify an illustration through an intuitive interface. In addition, the user experience is improved by displaying the progress in real time and notifying the user when the processing is complete.

[0050] "Advanced image processing technology" refers to technology that automatically adjusts the visual characteristics of an image, such as its hue, brightness, and contrast.

[0051] A "user interface" is an interface that includes screens and input devices for a user to operate a system.

[0052] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[0053] "Means for users to upload illustrations" refers to a function that allows users to send illustration files from their devices to the system.

[0054] "The means by which the server receives illustrations" refers to the function that allows the server to receive and save illustration files sent by users.

[0055] "Means for the server to adjust the color tone" refers to a function that allows the server to automatically change the hue and saturation of an illustration.

[0056] "A means for the server to adjust brightness" refers to a function that allows the server to automatically change the brightness of the illustration.

[0057] "A means for the server to adjust contrast" refers to a function that allows the server to automatically change the difference in brightness between light and dark areas in an illustration.

[0058] "Means for the server to generate beautified illustrations" refers to a function that allows the server to create new illustrations after adjusting the color tone, brightness, and contrast.

[0059] "Means by which the server provides beautified illustrations to users" refers to a function that provides users with beautified illustrations generated by the server in a downloadable format.

[0060] This invention relates to a system that automatically beautifies illustrations drawn by a user. The system provides an intuitive user interface and allows the user to adjust beautification parameters for adjusting visual characteristics such as hue, brightness, and contrast.

[0061] Hardware and software to be used

[0062] The server is a computer system for running Python programs, using OpenCV and Pillow as image processing libraries. Users access the system using internet-connected devices (such as PCs, smartphones, and tablets).

[0063] Program Processing Description

[0064] The server receives the illustration uploaded by the user. Next, the server adjusts the color tone of the image using OpenCV. Specifically, it converts the image to HSV (Hue, Saturation, Value) space, adjusts the hue appropriately, and then converts it back to RGB space.

[0065] The server then uses Pillow to adjust the image brightness. Specifically, it adjusts the brightness using Pillow's ImageEnhance.Brightness class. Next, it adjusts the contrast using Pillow's ImageEnhance.Contrast class.

[0066] After these processes, an enhanced illustration is generated and provided to the user. The user can obtain the enhanced illustration by clicking the download link on their device.

[0067] Specific example

[0068] As a concrete example, let's consider the case of beautifying an illustration "example_illustration.png" drawn by a user.

[0069] 1. User actions:

[0070] The user uploads "example_illustration.png" from their device to the system.

[0071] 2. Server processing:

[0072] The server receives the uploaded image and adjusts its color tone, brightness, and contrast.

[0073] Save the adjusted image as "beautified_illustration.png".

[0074] 3. Output to the user:

[0075] The server provides the user with a beautified illustration, "beautified_illustration.png".

[0076] An example of a prompt message is as follows:

[0077] Please upload your user-created illustration, "example_illustration.png". The system will automatically adjust the color tone, brightness, and contrast to provide an enhanced illustration. You will receive a notification when processing is complete.

[0078] In this way, the system works to automatically beautify illustrations drawn by users.

[0079] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0080] Step 1:

[0081] The user uploads an illustration.

[0082] Users upload their own illustration files (e.g., "example_illustration.png") from their terminal via the system's web interface. The input is the user's illustration file, and the output is the illustration file sent to the server.

[0083] Step 2:

[0084] The server receives the illustration.

[0085] The server receives illustration files uploaded by users and saves them to a specified directory. The input is the illustration file sent by the user, and the output is the illustration file saved on the server. Specifically, the server checks the file format and size and verifies that it is in an appropriate format.

[0086] Step 3:

[0087] The server adjusts the color tone.

[0088] The server loads the illustration using the OpenCV cv2.imread function. Next, it converts the image to HSV (Hue, Saturation, Value) space using the cv2.cvtColor function. It then adjusts the hue appropriately and converts it back to RGB space. The input is the illustration file stored on the server, and the output is the color-adjusted illustration. Specifically, the hue is shifted by 10 degrees.

[0089] Step 4:

[0090] The server adjusts the brightness.

[0091] The server uses Pillow's ImageEnhance.Brightness class to adjust the brightness of the image. Specifically, it increases the brightness by 1.2 times. The input is an illustration with adjusted color tone, and the output is an illustration with adjusted brightness.

[0092] Step 5:

[0093] The server adjusts the contrast.

[0094] The server uses Pillow's ImageEnhance.Contrast class to adjust the contrast of the image. Specifically, it increases the contrast by 1.3 times. The input is an illustration with adjusted brightness, and the output is an illustration with adjusted contrast.

[0095] Step 6:

[0096] The server generates an aesthetically pleasing illustration.

[0097] The server saves the adjusted image as "beautified_illustration.png". The input is an illustration with adjusted contrast, and the output is a beautified illustration file.

[0098] Step 7:

[0099] The server provides users with an enhanced version of the illustration.

[0100] The server provides the user with a beautified illustration file, "beautified_illustration.png". The user can obtain the beautified illustration by clicking the download link from their device. The input is the beautified illustration file, and the output is the download link provided to the user. Specifically, the server displays the progress in real time and sends a notification to the user when the process is complete.

[0101] (Application Example 1)

[0102] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0103] Traditional illustration beautification systems offer the functionality to automatically beautify user-submitted illustrations, but they lack a convenient way to share these beautified illustrations with content distribution services. Furthermore, the customization options for the beautification process are limited, making it difficult for users to adjust the process to their own preferences.

[0104] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0105] In this invention, the server includes means for automatically beautifying user-drawn illustrations using advanced image processing technology, means for providing an intuitive user interface, means for allowing the user to adjust beautification parameters, means for directly sharing the beautified illustrations with content distribution services, and means for providing customization options for the beautification process. This makes it possible for users to easily share beautified illustrations and further customize the beautification process to their liking.

[0106] "Advanced image processing technology" refers to technology that automatically adjusts the visual characteristics of an image, such as its hue, brightness, and contrast.

[0107] "User-generated illustrations" refer to paintings or drawings created by users, either by hand or using digital tools.

[0108] "Methods for automatically beautifying" refer to algorithms and software that improve the visual characteristics of illustrations without user intervention.

[0109] A "user interface that users can operate intuitively" is an interface designed to allow users to operate it easily and intuitively.

[0110] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[0111] "User-adjustable means" refers to a feature that allows users to manually change beautification parameters.

[0112] A "content distribution service" is a service that distributes digital content such as images and videos over the internet.

[0113] "Shareable means" refers to features that allow users to share the beautified illustrations with other users or platforms.

[0114] "Customization options" are settings and features that allow users to adjust the beautification process to their own preferences.

[0115] The following system can be constructed as an embodiment of this invention.

[0116] System Overview

[0117] This system automatically beautifies user-submitted illustrations and shares those beautified illustrations with content distribution services. The system includes a smartphone, a server, and software utilizing advanced image processing technology.

[0118] Hardware and software to be used

[0119] Hardware: Smartphones (iOS or Android®), Servers

[0120] Software: Python, OpenCV (image processing library)

[0121] Data processing and data calculation

[0122] 1. Loading images

[0123] Users take photos of or upload illustrations they have drawn on their smartphones. This utilizes the smartphone's camera function and file upload function.

[0124] 2. Adjusting the color tone

[0125] The server converts the image to the HSV color space and increases its saturation. This makes the illustration's colors more vibrant.

[0126] 3. Adjusting Brightness

[0127] The server performs a scaling transformation to increase the brightness of the image. This makes the illustration brighter.

[0128] 4. Adjust the contrast

[0129] The server converts the image to the LAB color space and performs histogram equalization of the L channel to improve contrast. This enhances the visual characteristics of the illustration.

[0130] 5. Saving and sharing images

[0131] The beautified images are stored on the server, and users can share them directly to content distribution services via smartphone applications.

[0132] Specific example

[0133] Users take a photo of an illustration they drew on their smartphone and upload it to the app, whereupon the server automatically beautifies the illustration. The beautified illustration can then be adjusted in terms of color tone, brightness, and contrast to suit the user's preferences. Finally, the beautified illustration can be shared on social media and other content distribution services.

[0134] Example of a prompt

[0135] "Please develop an app that beautifies user-submitted illustrations. The app should automatically adjust the color tone, brightness, and contrast of the illustrations, and output the beautified versions."

[0136] In this way, a system can be implemented that allows users to easily create and share beautified illustrations.

[0137] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0138] Step 1:

[0139] The user takes a picture of an illustration they drew on their smartphone or uploads it. The input is the illustration image drawn by the user, and the output is the image file saved on the smartphone. Specifically, the user takes a picture of the illustration using the smartphone's camera function or uploads an existing image file to the application.

[0140] Step 2:

[0141] The device sends captured or uploaded illustration images to the server. The input is an image file stored on the smartphone, and the output is the image data transferred to the server. Specifically, the smartphone application calls an API to send the image file to the server.

[0142] Step 3:

[0143] The server reads the received illustration image. The input is the image data transferred to the server, and the output is the image object loaded into memory. Specifically, the server uses Python and OpenCV to read the image file.

[0144] Step 4:

[0145] The server adjusts the color tone of the image. The input is an image object loaded into memory, and the output is an image object with adjusted color tone. Specifically, the server converts the image to the HSV color space and increases its saturation.

[0146] Step 5:

[0147] The server adjusts the brightness of the image. The input is a color-adjusted image object, and the output is a brightness-adjusted image object. Specifically, the server performs a scaling transformation to increase the brightness of the image.

[0148] Step 6:

[0149] The server adjusts the contrast of the image. The input is an image object with adjusted brightness, and the output is an image object with adjusted contrast. Specifically, the server converts the image to the LAB color space and performs histogram equalization of the L channel.

[0150] Step 7:

[0151] The server saves the beautified image. The input is an image object with adjusted contrast, and the output is a beautified image file saved on the server. Specifically, the server saves the beautified image as a file.

[0152] Step 8:

[0153] The device retrieves a beautified image from the server. The input is a beautified image file stored on the server, and the output is a beautified image file downloaded to the smartphone. Specifically, the smartphone application calls an API to download the beautified image from the server.

[0154] Step 9:

[0155] Users share beautified illustrations to content distribution services. The input is a beautified image file downloaded to a smartphone, and the output is a beautified image uploaded to a content distribution service. Specifically, users share beautified illustrations to social media and other content distribution services using a smartphone application.

[0156] (Example 2)

[0157] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0158] Conventional image processing systems required specialized knowledge and complex operations to beautify user-generated illustrations, lacking an intuitive user interface. Furthermore, adjusting beautification parameters was difficult, making it challenging for users to customize their illustrations to their preferences. Additionally, the beautification process using generative AI models was not performed in real time, resulting in delayed feedback from user input.

[0159] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0160] In this invention, the server includes means for converting beautification parameters operated by the user into prompt sentences, means for inputting the prompt sentences into a generation AI model to beautify the illustration, and means for displaying the beautified illustration to the user. This allows the user to adjust the beautification parameters and view the beautified illustration in real time through an intuitively operable user interface.

[0161] "Advanced image processing technology" is a general term for advanced algorithms and techniques used to automatically beautify illustrations drawn by users.

[0162] "User interface" is a general term for the screens and input devices that users use to operate a system, and includes intuitively operable UI elements.

[0163] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[0164] A "prompt message" is a text-based instruction used to input the beautification parameters set by the user into the generating AI model.

[0165] A "generative AI model" is an artificial intelligence model used to beautify illustrations based on input prompt text, and includes, for example, "Stable Diffusion" and "DALL-E".

[0166] A "slider" is a UI element that allows users to adjust values ​​by dragging.

[0167] A "dial" is a UI element that allows users to adjust values ​​by rotating it.

[0168] A "server" is a computer system that receives prompt messages, inputs them into a generation AI model, generates an aesthetically pleasing illustration, and sends it back to the terminal.

[0169] A "terminal" is a device operated by a user, which provides a user interface, retrieves beautification parameters, generates prompt messages, and sends them to the server.

[0170] This invention is a system that automatically beautifies illustrations drawn by users, and provides a user interface that allows users to operate it intuitively. The system converts the beautification parameters operated by the user into prompt sentences, inputs them into a generating AI model, and beautifies the illustration. A specific embodiment of this system is described below.

[0171] Hardware and software to be used

[0172] hardware

[0173] Terminal: A device operated by the user, providing a user interface, acquiring beautification parameters, and generating and sending prompt messages to the server. Specifically, this includes personal computers, tablets, and smartphones.

[0174] Server: A computer system that receives prompt messages, inputs them into a generation AI model, generates an aesthetically pleasing illustration, and sends it back to the terminal.

[0175] software

[0176] User Interface: Includes UI elements that allow users to operate intuitively. Specifically, this includes sliders, dials, and similar elements.

[0177] Generative AI Model: This is an artificial intelligence model used to beautify illustrations based on prompt text. Specifically, models such as "Stable Diffusion" and "DALL-E" are used.

[0178] Data processing and data calculation

[0179] User actions

[0180] The user interacts with the user interface on the device. Specifically, they adjust beautification parameters using sliders and dials. For example, if the user moves the slider to the right, the beautification parameter increases.

[0181] Terminal processing

[0182] The terminal acquires real-time position information of sliders and dials operated by the user. This position information is interpreted as a beautification parameter. For example, if the slider is set to its maximum value, the beautification parameter will be "100". The terminal generates a prompt message based on the acquired beautification parameter. For example, if the beautification parameter is "100", it will generate a prompt message that says "Beautification parameter: 100".

[0183] Server Processing

[0184] The server receives the prompt message sent from the terminal. It inputs the received prompt message into the generating AI model. The generating AI model beautifies the illustration based on the input prompt message. For example, if the prompt message is "Beautification parameter: 100", it processes the illustration with the degree of beautification set to the maximum. The server receives the beautified illustration from the generating AI model and sends it back to the terminal.

[0185] Terminal display

[0186] The device receives the beautified illustration sent back from the server and displays it to the user. This allows the user to see the illustration based on the beautification parameters they have set.

[0187] Specific example

[0188] Example of a prompt

[0189] "Beautification parameter: 100"

[0190] By inputting this prompt into the AI ​​model, you can set the level of illustration beautification to the maximum.

[0191] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0192] Step 1:

[0193] The user interacts with UI elements.

[0194] The user operates the user interface on the device, adjusting beautification parameters using sliders and dials. For example, moving the slider to the right increases the beautification parameter. The input is the user's actions, and the output is the adjusted beautification parameter.

[0195] Step 2:

[0196] The device retrieves the beautification parameters.

[0197] The terminal acquires real-time position information of sliders and dials operated by the user. This position information is interpreted as a beautification parameter. For example, if the slider is set to its maximum value, the beautification parameter will be "100". The input is the user's operation position information, and the output is the beautification parameter.

[0198] Step 3:

[0199] The terminal generates the prompt message.

[0200] The terminal generates a prompt message based on the acquired beautification parameter. For example, if the beautification parameter is "100", it will generate the prompt message "Beautification parameter: 100". The input is the beautification parameter, and the output is the prompt message.

[0201] Step 4:

[0202] The terminal sends a prompt message to the server.

[0203] The terminal sends the generated prompt message to the server. This transmission is performed using a communication protocol such as an HTTP request. The input is the prompt message, and the output is a notification to the server that the transmission has been completed.

[0204] Step 5:

[0205] The server receives the prompt message.

[0206] The server receives a prompt message sent from the terminal. The received prompt message is used in the next processing step. The input is the prompt message, and the output is an acknowledgment.

[0207] Step 6:

[0208] The server inputs prompt messages into the generated AI model.

[0209] The server inputs the received prompt message into the generating AI model. For example, it can use generating AI models such as "Stable Diffusion" or "DALL-E". The input is the prompt message, and the output is a notification to the generating AI model that input is complete.

[0210] Step 7:

[0211] The generative AI model beautifies the illustration.

[0212] The generative AI model beautifies illustrations based on the input prompt text. For example, if the prompt text "Beautification parameter: 100" is input, the model will set the illustration's beautification level to the maximum and process it. The input is the prompt text, and the output is the beautified illustration.

[0213] Step 8:

[0214] The server sends the beautified illustration back to the terminal.

[0215] The server receives the beautified illustration generated by the AI ​​model and sends it back to the terminal. This return is also done using a communication protocol such as an HTTP response. The input is the beautified illustration, and the output is a notification to the terminal that the transmission is complete.

[0216] Step 9:

[0217] The device displays an aesthetically pleasing illustration.

[0218] The terminal receives the beautified illustration sent back from the server and displays it to the user. This allows the user to see the illustration based on the beautification parameters they have set. The input is the beautified illustration, and the output is the display to the user.

[0219] (Application Example 2)

[0220] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0221] Conventional image beautification systems lacked an intuitive user interface for beautifying user-generated illustrations and photographs, making real-time preview and saving / sharing of beautified images difficult. Furthermore, users struggled to fine-tune visual characteristics when adjusting beautification parameters.

[0222] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0223] In this invention, the server includes means for automatically beautifying user-drawn illustrations using advanced image processing technology, means for providing an intuitive user interface, means for allowing the user to adjust beautification parameters, means for previewing user-uploaded images in real time, and means for saving or sharing the beautified images. This allows the user to intuitively operate the server, view the beautified image in real time, adjust the visual characteristics to their liking, and save or share the final image.

[0224] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing, transforming, and correcting images.

[0225] "User-created illustrations" refers to drawings and diagrams that users have manually created.

[0226] "Automatic beautification methods" refer to technologies and devices that improve the quality and appearance of images without user intervention.

[0227] A "user interface that users can operate intuitively" refers to an interface designed to allow users to operate it easily and naturally.

[0228] "Beautification parameters" refer to settings used to adjust the visual characteristics of an image, such as color tone, brightness, and contrast.

[0229] "User-adjustable means" refers to features that allow users to change settings and parameters to suit their preferences.

[0230] "Real-time preview" refers to a function that displays the results immediately when a user performs an action.

[0231] "Means for saving or sharing beautified images" refers to functions for saving processed images to a device or sharing them with other users.

[0232] The system for carrying out this invention automatically beautifies illustrations and photographs drawn by the user and provides an intuitive user interface. Specific embodiments of this system are described below.

[0233] System Configuration

[0234] hardware

[0235] Smartphone: A device used by users to upload and manipulate images.

[0236] Server: Provides computing resources for image processing.

[0237] software

[0238] OpenCV: An image processing library. It performs image analysis, transformation, and modification.

[0239] Tkinter: A GUI library used to build user interfaces.

[0240] Data processing and data calculation

[0241] Image loading

[0242] The user uploads an image using their smartphone. The server reads this image using OpenCV.

[0243] Adjusting the beautification parameters

[0244] Users adjust beautification parameters (smoothness, brightness, contrast) by operating sliders and dials displayed on their smartphone screen. This allows users to see the changes in the image in real time.

[0245] Real-time preview

[0246] The server processes the image in real time based on the beautification parameters adjusted by the user and sends a preview to the smartphone. This allows the user to see the results immediately.

[0247] Saving and sharing images

[0248] Users can save images they are satisfied with to their smartphones. They can also share them with other users via social media or email.

[0249] Specific example

[0250] Users open the app on their smartphones and upload photos. They use sliders to adjust smoothness, brightness, and contrast, and preview the beautified photos in real time. Once satisfied, they save the photos or share them on social media.

[0251] Example of a prompt

[0252] "Create an app that lets users upload photos from their smartphones, adjust smoothness, brightness, and contrast using sliders, preview the beautified photos in real time, and then save or share them."

[0253] In this way, the invention allows users to intuitively operate the system, view beautified images in real time, adjust visual characteristics to their liking, and save or share the final image.

[0254] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0255] Step 1:

[0256] Upload image

[0257] The user uses their smartphone to launch the application, select an image, and upload it.

[0258] Input: The image file selected by the user.

[0259] Output: Image data sent to the server.

[0260] Specific operation: The user selects an image from their smartphone's gallery and presses the "Upload" button. The image data is sent to the server.

[0261] Step 2:

[0262] Image loading

[0263] The server reads the received image data using OpenCV.

[0264] Input: Image data sent to the server.

[0265] Output: Image object loaded with OpenCV.

[0266] Specific operation: The server receives the image data and loads the image into memory using the OpenCV cv2.imread function.

[0267] Step 3:

[0268] Initial settings for beautification parameters

[0269] The server sets the initial beautification parameters (smoothness, brightness, contrast).

[0270] Input: None (initial setting value).

[0271] Output: Initially set beautification parameters.

[0272] Specific operation: The server sets the initial values of smoothness, brightness, and contrast and holds these values.

[0273] Step 4:

[0274] Display of user interface

[0275] The terminal displays the user interface and provides sliders and dials.

[0276] Input: Initially set beautification parameters.

[0277] Output: Display of user interface.

[0278] Specific operation: The terminal uses Tkinter to display a user interface including sliders and dials.

[0279] Step 5:

[0280] Adjustment of beautification parameters

[0281] The user operates the sliders and dials to adjust the beautification parameters.

[0282] Input: Values of sliders and dials by user operation.

[0283] Output: Adjusted beautification parameters.

[0284] Specific operation: The user operates the sliders and dials and sends new parameter values to the server.

[0285] Step 6:

[0286] Image beautification processing

[0287] The server processes the image based on the adjusted beautification parameters.

[0288] Input: Adjusted beautification parameters, loaded image object.

[0289] Output: A beautified image object.

[0290] Specific operation: The server uses OpenCV functions to adjust smoothness, brightness, and contrast to beautify the image.

[0291] Step 7:

[0292] Displaying real-time preview

[0293] The device displays a real-time preview of the beautified image.

[0294] Input: A beautified image object.

[0295] Output: The beautified image displayed on the device.

[0296] Specific operation: The server sends the beautified image to the terminal, and the terminal displays it on the screen.

[0297] Step 8:

[0298] Saving and sharing images

[0299] Users save or share the beautified images.

[0300] Input: A beautified image object.

[0301] Output: Saved image file or shared image link.

[0302] Specific operation: When the user presses the "Save" or "Share" button, the terminal saves the image to the local storage or shares it via SNS or email.

[0303] (Example 3)

[0304] Next, Example 3 of Form Example 3 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart device 14 is referred to as a "terminal".

[0305] In a conventional illustration beautification system, it is difficult for users to intuitively adjust beautification parameters, and often the generated illustrations do not meet the users' expectations. Furthermore, since there was a lack of a process for appropriately analyzing the parameters input by the user and generating a prompt sentence suitable for the generation AI model, the degree of beautification of the illustrations was unstable.

[0306] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0307] In this invention, the server includes means for automatically beautifying an illustration drawn by a user, means for providing a user interface that can be intuitively operated by the user, means for allowing the user to adjust beautification parameters, means for transmitting the beautification parameters input by the user to the server, means for analyzing the beautification parameters received by the server and generating a prompt sentence suitable for the generation AI model, means for the generation AI model to beautify the illustration based on the prompt sentence, and means for the server to transmit the beautified illustration to the user's terminal. Thereby, it becomes possible for the user to intuitively adjust the beautification parameters and obtain a stably beautified illustration using the generation AI model.

[0308] The "advanced image processing technology" is a technology for analyzing, modifying, and converting images using a computer, and particularly realizes highly accurate and complex processing.

[0309] A "user interface" is a means for a user to interact with a system, and it includes both visual and manipulative elements.

[0310] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, and include factors such as hue, brightness, and contrast.

[0311] A "generative AI model" is an algorithm or system that uses artificial intelligence to generate or transform data, and is particularly specialized for image generation.

[0312] A "prompt message" is text-based input data used to give instructions to a generative AI model.

[0313] A "server" is a computer system that provides services to clients over a network.

[0314] A "device" is a device that a user directly operates, and includes computers, smartphones, tablets, and other similar devices.

[0315] "Analysis" is the process of examining data in detail and understanding its structure and meaning.

[0316] "Transmission" is the act of moving data from one location to another.

[0317] An "illustration" is an image intended for visual expression, created using hand-drawn or digital tools.

[0318] Modes for carrying out the invention

[0319] This invention is a system that automatically beautifies illustrations drawn by a user, provides an intuitive user interface, and includes means for the user to adjust the beautification parameters. Specific embodiments of this system are described below.

[0320] Hardware and software to use

[0321] This system uses the following hardware and software:

[0322] Server: A computer system that provides services to clients over a network.

[0323] Device: A device that a user directly interacts with (e.g., computer, smartphone, tablet).

[0324] Generative AI models: Algorithms and systems that use artificial intelligence to generate or transform data (e.g., Stable Diffusion, DALL-E).

[0325] System Processing Overview

[0326] Users input beautification parameters using their devices. For example, users set parameters such as adjusting color intensity and brightness using a dedicated application on their device or a web interface. Specifically, users use sliders or text boxes to set "color intensity" to 50 and "brightness adjustment" to 70.

[0327] The terminal sends the beautification parameters entered by the user to the server. This transmission is done using communication protocols such as HTTP requests or WebSockets. The server analyzes the received beautification parameters and generates prompts suitable for the AI ​​model. For example, the server might generate a prompt such as, "Generate an illustration with a hue intensity of 50 and a brightness adjustment of 70."

[0328] The server inputs prompt text into the generation AI model, requesting it to beautify an illustration. In doing so, the server calls the generation AI model's API. The generation AI model beautifies the illustration based on the prompt text and returns the result to the server. The returned data is an image file of the beautified illustration.

[0329] The server sends the beautified illustration received from the generating AI model to the terminal. This transmission also uses communication protocols such as HTTP responses or WebSockets. The user then views the beautified illustration on the terminal. The terminal's application or web interface has the functionality to display the received illustration.

[0330] Specific example

[0331] If the user uses their device to set "Color Intensity" to 50 and "Brightness Adjustment" to 70, the server will input the following prompt message into the AI ​​model:

[0332] Example of a prompt:

[0333] "Please generate an illustration with a hue intensity of 50 and a brightness adjustment of 70."

[0334] The generative AI model receives this prompt and beautifies the illustration based on the specified parameters. As a result, a beautified illustration tailored to the user's preferences is generated.

[0335] In this way, users can intuitively adjust the beautification parameters and obtain stable, beautified illustrations using the generation AI model. The flow of the specific processing in Example 3 will be explained with reference to Figure 15.

[0336] Step 1:

[0337] The user enters beautification parameters using a terminal.

[0338] Input: Users set parameters such as color intensity and brightness using a dedicated application or web interface on their device.

[0339] Specific operation: The user uses a slider or text box to set the "Tone Intensity" to 50 and the "Brightness Adjustment" to 70.

[0340] Output: Beautification parameters set on the terminal.

[0341] Step 2:

[0342] The terminal sends the entered beautification parameters to the server.

[0343] Input: Beautification parameters set on the device.

[0344] Specific operation: The terminal creates an HTTP POST request and sends the beautification parameters in JSON format to the server endpoint.

[0345] Output: Beautification parameters sent to the server.

[0346] Step 3:

[0347] The server analyzes the beautification parameters it receives and generates prompt sentences suitable for the AI ​​model.

[0348] Input: Beautification parameters sent to the server.

[0349] Specific operation: The server parses the JSON data it receives and performs string manipulation to generate a prompt message. For example, it generates a prompt message such as "Generate an illustration with a hue intensity of 50 and a brightness adjustment of 70."

[0350] Output: The generated prompt message.

[0351] Step 4:

[0352] The server inputs prompt text into the generated AI model and requests that it beautify the illustration.

[0353] Input: The generated prompt message.

[0354] Specific operation: The server sends an HTTP POST request to the API endpoint of the generated AI model, sending data including a prompt message.

[0355] Output: The prompt message sent to the generating AI model.

[0356] Step 5:

[0357] The AI ​​model generates an aesthetically pleasing illustration and sends it back to the server.

[0358] Input: The prompt text sent to the generating AI model.

[0359] Specific operation: The generation AI model internally processes the illustration for aesthetics and then returns the generated illustration to the server as binary data.

[0360] Output: Binary data of the beautified illustration sent back to the server.

[0361] Step 6:

[0362] The server sends the beautified illustration to the terminal.

[0363] Input: Binary data of the beautified illustration sent back to the server.

[0364] Specific operation: The server sends the received binary data to the terminal as an HTTP response.

[0365] Output: A beautified illustration sent to the terminal.

[0366] Step 7:

[0367] The user views the beautified illustration on their device.

[0368] Input: A beautified illustration sent to the terminal.

[0369] Specific operation: The application displays the illustration received by the device, and the user confirms the illustration.

[0370] Output: User-confirmed, beautified illustration.

[0371] (Application Example 3)

[0372] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0373] Conventional image beautification systems often lack an intuitive user interface when automatically beautifying user-generated illustrations. Furthermore, they often lack features such as real-time preview of the adjusted illustration or the ability to save adjustments and share them with other content distribution services. This presents a challenge for users, as it makes it difficult to beautify illustrations to their liking and easily share them.

[0374] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0375] In this invention, the server includes means for automatically beautifying illustrations drawn by the user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for the user to adjust beautification parameters, means for previewing the adjusted illustration in real time, and means for saving the adjustments and sharing them with other content distribution services. This makes it possible for users to beautify illustrations to their liking and easily share them.

[0376] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing and transforming images.

[0377] "User-created illustrations" refers to drawings and diagrams that users have manually created.

[0378] "Automatic beautification methods" refer to technologies that improve the quality and appearance of images without user intervention.

[0379] A "user interface that users can operate intuitively" refers to an interface that users can easily understand and operate.

[0380] "Beautification parameters" refer to settings used to adjust the visual characteristics of an image, such as color tone, brightness, and contrast.

[0381] "A means of previewing the adjusted illustration in real time" refers to a technology that instantly displays the results when a user changes parameters.

[0382] "Means of saving adjustments and sharing them with other content distribution services" refers to technology that records image adjustments made by users and shares them with other online platforms.

[0383] The system for implementing this invention automatically beautifies illustrations drawn by the user and provides a user interface that the user can operate intuitively. The system is implemented as an application that runs on devices such as smartphones and tablets.

[0384] Hardware and software to use

[0385] Hardware: Smartphones, tablets

[0386] Software: Python, OpenCV, Tkinter

[0387] System Configuration

[0388] 1. Image Loading: Users upload photos and illustrations taken with their devices to the application.

[0389] 2. Adjusting beautification parameters: Users can adjust beautification parameters such as "color intensity" and "brightness adjustment" using sliders and dials within the application.

[0390] 3. Real-time preview: Image beautification is performed in real time based on the adjusted parameters, and the results are displayed instantly.

[0391] 4. Saving and sharing adjustments: Users can save their satisfactory adjustment results and share them on social media and other content distribution services.

[0392] Processing flow

[0393] 1. Image Loading: Select an image from your device's camera or gallery and upload it to the application. Load the image using OpenCV and start processing.

[0394] 2. Adjusting Beautification Parameters: Using Tkinter, provide sliders and dials that users can intuitively operate. Users will adjust the beautification parameters by manipulating these UI elements.

[0395] 3. Real-time preview: Every time the user changes a parameter, the brightness and contrast of the image are adjusted using OpenCV's convertScaleAbs function, and the results are displayed in real time.

[0396] 4. Saving and sharing adjustments: Provides a function to save the adjusted image to the device and share it on social media or other content distribution services.

[0397] Specific example

[0398] For example, a user can upload a landscape photo taken with their smartphone to the application, enhance the colors, slightly reduce the brightness, and then share it on Instagram.

[0399] Example of a prompt

[0400] "Upload a landscape photo, enhance the colors, slightly reduce the brightness, and share it on Instagram."

[0401] In this way, users can beautify illustrations to their liking and easily share them.

[0402] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[0403] Step 1:

[0404] Users upload photos and illustrations taken with their devices to the application.

[0405] Input: An image file selected from the device's camera or gallery.

[0406] Output: Image data loaded into the application.

[0407] Specific operation: The user launches the application, presses the image upload button, and selects an image from the device's camera roll or gallery. The selected image is loaded into the application.

[0408] Step 2:

[0409] The image data read by the device is processed using OpenCV.

[0410] Input: Loaded image data.

[0411] Output: An image object that can be processed by OpenCV.

[0412] Specific operation: The application uses the OpenCV library to convert the loaded image data into an internal format, making it ready for processing.

[0413] Step 3:

[0414] The user operates sliders and dials to adjust the beautification parameters.

[0415] Input: The value of a slider or dial as determined by the user's actions.

[0416] Output: Adjusted beautification parameters.

[0417] Specific operation: The user operates sliders and dials within the application to set parameters such as "color intensity" and "brightness adjustment." The set values ​​are reflected in the application in real time.

[0418] Step 4:

[0419] The device previews the image in real time based on the adjusted beautification parameters.

[0420] Input: Adjusted beautification parameters and original image data.

[0421] Output: The adjusted image is previewed in real time.

[0422] Specific operation: The application uses the OpenCV convertScaleAbs function to adjust the brightness and contrast of an image based on the beautification parameters set by the user, and displays the results in real time.

[0423] Step 5:

[0424] Users save the adjusted image and share it with other content distribution services.

[0425] Input: Adjusted image data.

[0426] Output: Saved image files and shared images.

[0427] Specific actions: The user presses the save button to save the adjusted image to their device. They then press the share button to upload the image to Instagram or other social media platforms.

[0428] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0429] "Example of form 1"

[0430] One embodiment of the present invention is an illustration beautification system incorporating an emotion engine. This system includes a function to automatically beautify illustrations drawn by the user, a function to provide an intuitive user interface, a function to allow the user to adjust beautification parameters, and an emotion engine that recognizes the user's emotions. The emotion engine recognizes the user's emotions from the user's facial expressions, tone of voice, text input, etc., and adjusts the beautification parameters according to those emotions. Specifically, when the user is feeling happy, the color tone of the illustration is adjusted to be brighter, and when the user is feeling sad, the color tone is adjusted to be darker.

[0431] This allows for the beautification of illustrations in accordance with the user's emotions.

[0432] "Example of form 2"

[0433] Furthermore, the emotion engine has the ability to recognize changes in the user's emotions in real time and dynamically adjust the beautification parameters accordingly. For example, if a user starts drawing feeling joy but then becomes sad midway through, the emotion engine will capture this change in real time and immediately adjust the beautification parameters. This makes it possible to beautify illustrations in response to changes in the user's emotions.

[0434] "Example of form 3"

[0435] Furthermore, the emotion engine also has a learning function to recognize the user's emotions. Specifically, it collects data such as the user's facial expressions, voice tone, and text input, and uses machine learning algorithms to train an emotion recognition model. Through this training, the emotion engine becomes able to recognize the user's emotions more accurately, enabling the beautification of illustrations that are more adapted to the user's emotions.

[0436] The following describes the processing flow for each example form.

[0437] "Example of form 1"

[0438] Step 1: Input the illustration drawn by the user into the system.

[0439] Step 2: The system uses an emotion engine to recognize the user's emotions based on their facial expressions, tone of voice, text input, etc.

[0440] Step 3: The emotion engine adjusts the beautification parameters according to the recognized emotion.

[0441] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[0442] "Example of form 2"

[0443] Step 1: Input the illustration drawn by the user into the system.

[0444] Step 2: The system uses an emotion engine to recognize the user's emotions in real time based on their facial expressions, tone of voice, text input, etc.

[0445] Step 3: The emotion engine dynamically adjusts the beautification parameters in response to the perceived change in emotion.

[0446] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[0447] "Example of form 3"

[0448] Step 1: Collect data such as the user's facial expressions, voice tone, and text input.

[0449] Step 2: Using the collected data, the emotion engine trains an emotion recognition model using a machine learning algorithm.

[0450] Step 3: Use the trained emotion recognition model to recognize the user's emotions and adjust the beautification parameters accordingly.

[0451] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[0452] (Example 1)

[0453] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0454] Conventional illustration beautification systems failed to consider the user's emotions when automatically beautifying user-submitted illustrations. Therefore, the beautification process did not reflect the user's feelings, making it difficult to increase user satisfaction. Furthermore, if the user interface was not intuitive, it could be complex and difficult to use.

[0455] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes means for acquiring image data representing an illustration drawn by the user, means for automatically adjusting the visual characteristics of the image data, means for recognizing the user's emotions using an emotion engine, a prompt statement instructing the adjustment of the visual characteristics of the image data based on the user's emotions, and means for adjusting the visual characteristics of the image data using a generative AI. This makes it possible to beautify the illustration in accordance with the user's emotions and increase user satisfaction. The server also further includes means for adjusting the visual characteristics of the image data via a user interface that can be operated by the user. The automatic adjustment means adjusts the hue, brightness, and contrast of the image data.

[0456] "Advanced image processing techniques" refer to techniques for adjusting the visual characteristics of an image, such as color tone, brightness, and contrast, and utilize libraries such as OpenCV and TENSORFLOW®.

[0457] "User-generated illustrations" refer to image data created by users using hand-drawn methods or digital tools.

[0458] "Automatic beautification methods" refer to algorithms and programs that automatically adjust the visual characteristics of user-drawn illustrations to improve their appearance.

[0459] A "user interface that users can operate intuitively" is an interface designed to be easily understood and operated by users, and includes UI elements such as sliders and dials.

[0460] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[0461] "Means of recognizing user emotions" refer to algorithms and programs that analyze a user's facial expressions, tone of voice, text input, etc., to determine the user's emotions.

[0462] "Means for adjusting beautification parameters" refer to algorithms or programs that change beautification parameters according to the user's emotions and adjust the visual characteristics of the illustration.

[0463] "Means of returning beautified illustrations to users" refers to communication and display means for providing users with adjusted illustrations.

[0464] This invention relates to a system that automatically beautifies illustrations drawn by users. Specific embodiments of this system are described below.

[0465] System Overview

[0466] This system receives user-drawn illustrations as input, automatically adjusts their visual characteristics such as color tone, brightness, and contrast, and outputs a beautified illustration. Furthermore, it has a function to recognize the user's emotions and adjust the beautification parameters accordingly.

[0467] Hardware and software to use

[0468] Server: A server with high-performance computing capabilities will be used. The server will receive illustration data from users and perform image processing and emotion recognition.

[0469] Device: A device used by the user, such as a smartphone or PC. The device is used for uploading illustrations and displaying the beautified illustrations.

[0470] software:

[0471] OpenCV: An image processing library. Used to adjust the color tone, brightness, and contrast of illustrations.

[0472] TensorFlow: A machine learning library. Used to build emotion recognition engines.

[0473] Program processing

[0474] The server receives the illustration uploaded by the user. Next, the server automatically adjusts the visual characteristics of the illustration, such as color tone, brightness, and contrast, using OpenCV. Specifically, it performs the following processes:

[0475] Color adjustment: Use the OpenCV cv2.cvtColor function to convert the color space of the illustration and adjust the color tone.

[0476] Brightness adjustment: Use the cv2.convertScaleAbs function to adjust the brightness of the illustration.

[0477] Adjusting contrast: Use the cv2.equalizeHist function to adjust the contrast of the illustration.

[0478] Furthermore, in systems incorporating an emotion engine, the server analyzes the user's facial expressions, tone of voice, and text input to recognize the user's emotions. The emotion engine uses a neural network model built using TensorFlow. If the user is happy, the server adjusts the color tone of the illustration to be brighter; if the user is sad, it adjusts the color tone to be darker.

[0479] Specific example

[0480] As a concrete example, we will show the procedure for uploading an illustration drawn by a user and having the system beautify that illustration.

[0481] User uploads illustration: The user opens the dedicated app on their smartphone, selects the illustration they have drawn, and taps the "Upload" button.

[0482] The server receives the illustrations: The server receives the uploaded illustration files and saves them to the / uploads directory.

[0483] The server adjusts the visual characteristics of the illustration: The server reads the saved illustration file and uses OpenCV to adjust the hue, brightness, and contrast.

[0484] The server uses an emotion engine to recognize the user's emotions: The server analyzes the user's facial expression image and determines that the user is happy.

[0485] The server uses generative AI to beautify illustrations according to emotion: The server uses generative AI and OpenCV to adjust the color tone of the illustrations to be brighter and generate beautified illustrations.

[0486] The server returns the beautified illustration to the user: The server sends the beautified illustration as an HTTP response, which is then displayed on the user's smartphone.

[0487] Example of a prompt

[0488] Users are delighted. Use OpenCV to adjust the color tone, brightness, and contrast of user-drawn illustrations according to their emotions.

[0489] In this way, a system is realized that automatically beautifies illustrations drawn by users and makes adjustments according to the user's emotions.

[0490] The flow of the specific processing in Example 1 will be explained using Figure 17.

[0491] Step 1:

[0492] Users upload illustrations. Users upload their drawings to the system using their devices (smartphones or PCs). Specifically, they select the illustration file via a web browser or dedicated app and click the upload button. The input is the illustration file selected by the user, and the output is the illustration data sent to the server.

[0493] Step 2:

[0494] The server receives the illustration. The server receives the illustration file uploaded by the user. The server temporarily stores the illustration data sent via HTTP request. The input is the illustration data sent by the user, and the output is the illustration file stored on the server.

[0495] Step 3:

[0496] The server adjusts the visual characteristics of the illustration. The server uses the OpenCV library to automatically adjust the illustration's color tone, brightness, contrast, and other visual properties. The specific process is as follows:

[0497] Color Adjustment: The cv2.cvtColor function is used to convert the color space of the illustration and adjust its color tone. The input is an illustration file stored on the server, and the output is the color-adjusted illustration data.

[0498] Brightness adjustment: Use the cv2.convertScaleAbs function to adjust the brightness of the illustration. The input is the color-adjusted illustration data, and the output is the brightness-adjusted illustration data.

[0499] Contrast adjustment: Use the cv2.equalizeHist function to adjust the contrast of the illustration. The input is illustration data with brightness adjusted, and the output is illustration data with contrast adjusted.

[0500] Step 4:

[0501] The server recognizes the user's emotions using an emotion engine. The emotion engine uses a neural network model built using TensorFlow. It analyzes the user's facial expressions, audio data, and text input to determine the user's emotions. The input is the user's facial expressions, audio data, and text input, and the output is the user's emotion data.

[0502] Step 5:

[0503] The server uses generative AI and OpenCV to beautify illustrations according to emotions. The server beautifies the illustrations according to the emotions it recognizes of the user. For example, if the user is happy, the server adjusts the color tone of the illustration to be brighter. Specifically, it changes the color tone adjustment parameters and processes the illustration again using OpenCV. The input is the user's emotion data and the illustration data with adjusted contrast, and the output is the illustration data beautified according to the emotion.

[0504] Step 6:

[0505] The server returns an aesthetically pleasing illustration to the user. Specifically, it sends the illustration data as an HTTP response and displays it on the user's device. The input is an aesthetically pleasing illustration data according to the emotion, and the output is the aesthetically pleasing illustration displayed on the user's device. The server then provides a user interface that the user can operate, specifically an interface for adjusting the visual characteristics of the image data. This user interface includes intuitively operable UI elements such as sliders and dials. This allows the user to make final adjustments to the visual characteristics.

[0506] (Application Example 1)

[0507] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0508] Conventional illustration beautification systems offer the functionality to automatically beautify user-generated illustrations, but they lack the ability to adjust beautification parameters according to the user's emotions, making it difficult to obtain optimal beautification results that match the user's feelings. Furthermore, there is a need to provide a user interface that can be intuitively operated on mobile devices such as smartphones. Therefore, the challenge is to provide a system that allows users to obtain beautification results that are more satisfying.

[0509] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0510] In this invention, the server includes means for automatically beautifying illustrations drawn by the user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for the user to adjust beautification parameters, means for recognizing the user's emotions and adjusting the beautification parameters accordingly, and means for operating as an application installed on a smartphone. This makes it possible to obtain optimal beautification results that correspond to the user's emotions.

[0511] "Advanced image processing technology" refers to technology that automatically adjusts the visual characteristics of an image, such as its hue, brightness, and contrast.

[0512] "User-generated illustrations" refer to images created by users using hand-drawn methods or digital tools.

[0513] "Automatic beautification methods" refer to technologies that improve the visual characteristics of illustrations without the user's intervention.

[0514] A "user interface that users can operate intuitively" is an interface that users can easily understand and operate.

[0515] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[0516] "User-adjustable means" refers to technology that allows users to change beautification parameters according to their preferences.

[0517] "Means of recognizing user emotions" refers to technologies that detect emotions from the user's facial expressions, tone of voice, and other similar factors.

[0518] "Means of adjusting beautification parameters according to emotions" refers to technology that has the function of automatically changing beautification parameters based on the recognized emotions of the user.

[0519] An "application installed on a smartphone" is a software program that runs on a smartphone.

[0520] One embodiment of this invention involves installing the EmotionArt application on a smartphone. The EmotionArt application has the function of automatically beautifying illustrations drawn by the user and further adjusting the visual characteristics of the illustrations according to the user's emotions.

[0521] The server includes the following measures:

[0522] 1. A method for automatically beautifying user-drawn illustrations using advanced image processing technology.

[0523] 2. A means of providing a user interface that users can operate intuitively.

[0524] 3. A means by which the user can adjust the beautification parameters.

[0525] 4. A means of recognizing the user's emotions and adjusting the beautification parameters accordingly.

[0526] 5. A means of operating as an application installed on a smartphone.

[0527] This makes it possible to obtain the optimal beautification result that responds to the user's emotions.

[0528] Hardware and software to use

[0529] Hardware:

[0530] Smartphone (camera, microphone, display)

[0531] software:

[0532] OpenCV (image processing library)

[0533] EmotionEngine (Emotion Recognition Engine)

[0534] ImageBeautifier (image beautification engine)

[0535] Processing flow

[0536] 1. Image Loading: The user takes a picture of their illustration with their smartphone camera and saves it as an image file.

[0537] 2. Emotion Recognition: Emotions are recognized from the user's facial expressions and tone of voice using EmotionEngine.

[0538] 3. Adjusting Beautification Parameters: Based on the perceived emotion, ImageBeautifier adjusts the visual characteristics of the illustration, such as hue, brightness, and contrast.

[0539] 4. Saving the beautified image: Save the beautified illustration as an image file and provide it to the user.

[0540] Specific example

[0541] The user draws an illustration on their smartphone and uploads it to the app. The app captures the user's facial expression with the camera and recognizes their emotion. If the recognized emotion is "joy," the app adjusts the illustration's color tone to be brighter. The beautified illustration is then displayed to the user and saved.

[0542] Example of a prompt

[0543] We are developing an application that beautifies user-generated illustrations. Please generate a program for an application that adjusts the color tone, brightness, and contrast of the illustration according to the user's emotions. The user's emotions will be recognized from facial expressions and tone of voice. The libraries to be used are OpenCV, EmotionEngine, and ImageBeautifier.

[0544] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[0545] Step 1:

[0546] The user draws an illustration on their smartphone and then takes a picture of the illustration with the camera.

[0547] Input: User-drawn illustration

[0548] Output: Captured illustration image file

[0549] Specific actions: The user launches the smartphone's camera app and takes a picture of the illustration they drew. The captured image is saved to the smartphone's storage.

[0550] Step 2:

[0551] The device reads the captured illustration image file.

[0552] Input: Captured illustration image file

[0553] Output: Image data

[0554] Specific operation: The EmotionArt application on the smartphone reads the image file and stores it in memory as image data.

[0555] Step 3:

[0556] The device uses EmotionEngine to recognize the user's emotions.

[0557] Input: User's facial expressions and tone of voice

[0558] Output: Recognized emotion data

[0559] Specific operation: The smartphone's camera and microphone are used to capture the user's facial expressions and voice tone, and EmotionEngine analyzes this data to recognize the user's emotions.

[0560] Step 4:

[0561] The device adjusts the beautification parameters based on the emotions it recognizes.

[0562] Input: Recognized emotion data

[0563] Output: Adjusted beautification parameters

[0564] Specific operation: The EmotionArt application automatically sets the beautification parameters (hue, brightness, contrast, etc.) of ImageBeautifier based on the recognized emotion data.

[0565] Step 5:

[0566] The device uses ImageBeautifier to beautify the illustration.

[0567] Input: Image data, adjusted beautification parameters

[0568] Output: Beautified illustration image data

[0569] Specific operation: ImageBeautifier uses image data and adjusted beautification parameters to adjust the color tone, brightness, contrast, etc. of the illustration and generate a beautified illustration.

[0570] Step 6:

[0571] The device saves the beautified illustration and displays it to the user.

[0572] Input: Beautified illustration image data

[0573] Output: Saved beautified illustration image file, beautified illustration displayed to the user

[0574] Specific operation: The beautified illustration image data is saved to the smartphone's storage and displayed on the application screen. The user can view the beautified illustration and make further adjustments as needed.

[0575] (Example 2)

[0576] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0577] Conventional illustration beautification systems required users to manually adjust beautification parameters, making the operation cumbersome. Furthermore, they lacked the functionality to dynamically adjust the degree of illustration beautification in response to changes in the user's emotions, making it difficult to beautify illustrations in a way that reflected the user's feelings. This resulted in a decrease in user satisfaction.

[0578] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0579] In this invention, the server includes means for automatically beautifying illustrations drawn by the user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for the user to adjust beautification parameters, and means for recognizing changes in the user's emotions in real time and dynamically adjusting the beautification parameters in accordance with those changes. As a result, the user can beautify illustrations in accordance with their emotions while operating intuitively.

[0580] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing, correcting, and transforming images.

[0581] "User interface" refers to the screens and means of operation that users use to interact with a system.

[0582] "Beautification parameters" refer to settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[0583] A "slider" refers to a UI element that allows users to adjust values ​​by dragging.

[0584] A "dial" refers to a UI element that allows users to adjust values ​​by rotating it.

[0585] "Emotional changes" refers to how a user's emotional state changes over time.

[0586] "Real-time recognition" refers to the instant detection and processing of changes in the user's emotions.

[0587] "Dynamic adjustment" refers to automatically changing the settings according to the situation.

[0588] Modes for carrying out the invention

[0589] This invention is a system that automatically beautifies illustrations drawn by users and dynamically adjusts the beautification parameters according to changes in the user's emotions. This system consists of three main elements: a server, a terminal, and a user.

[0590] server

[0591] The server performs the following functions:

[0592] 1. Providing a user interface

[0593] The server generates a user interface (UI) and sends it to the terminal. This UI includes intuitively operable elements such as sliders and dials.

[0594] Software used: UI libraries such as React and Vue.js.

[0595] 2. Receiving user actions

[0596] The server receives user operation data transmitted from the terminal. This includes things like the slider position and dial angle.

[0597] Software used: WebSocket, HTTP requests.

[0598] 3. Calculation of beautification parameters

[0599] The server calculates beautification parameters based on the received operation data. This determines the degree of beautification of the illustration.

[0600] Software to be used: Programming languages ​​such as Python and Node.js.

[0601] 4. Analysis of emotional data

[0602] The server receives and analyzes user emotion data transmitted from the terminal. It uses an emotion engine to recognize changes in the user's emotions in real time.

[0603] Software used: Emotion analysis engines such as Microsoft® Azure® Emotion API.

[0604] 5. Dynamic adjustment of beautification parameters

[0605] The server dynamically adjusts the beautification parameters based on the analyzed emotion data. This makes it possible to beautify illustrations in accordance with the user's emotions.

[0606] terminal

[0607] The device performs the following functions:

[0608] 1. Display of the user interface

[0609] The terminal displays the user interface received from the server. This includes UI elements such as sliders and dials.

[0610] Hardware to use: PC, tablet, smartphone.

[0611] Software used: Browser, HTML, JavaScript (registered trademark).

[0612] 2. Sending user actions

[0613] The device sends operation data to the server when the user operates the slider or dial.

[0614] Software used: JavaScript event listeners, WebSocket, HTTP requests.

[0615] 3. Collection of emotional data

[0616] The device uses sensors such as cameras and microphones to collect user emotion data.

[0617] Software used: WebRTC, MediaStream API.

[0618] User

[0619] The user performs the following actions:

[0620] 1. User Interface Operation

[0621] Users adjust the illustration's beautification parameters by manipulating sliders and dials on their device.

[0622] Specific actions: The user uses a mouse or touchscreen to drag sliders or rotate dials.

[0623] 2. Changes in emotions

[0624] The user's emotions change during the process of creating an illustration. These emotional changes are collected by the device and sent to the server.

[0625] Specific actions: Emotional data changes when the user smiles or frowns.

[0626] Specific example

[0627] Hardware and software to use

[0628] Hardware: PCs, tablets, smartphones

[0629] Software: User interface (UI) libraries (e.g., React, Vue.js), sentiment analysis engines (e.g., Microsoft Azure Emotion API)

[0630] Example of a prompt

[0631] "Please describe a system where the user adjusts the illustration's beautification parameters using sliders. Also, please include a feature that dynamically adjusts the beautification parameters in response to changes in the user's emotions."

[0632] By inputting this prompt into the generating AI model, a detailed explanation of the system can be obtained.

[0633] The flow of the specific processing in Example 2 will be explained using Figure 19.

[0634] Step 1:

[0635] Providing a user interface

[0636] The server generates a user interface (UI) and sends it to the device. It uses a UI library (e.g., React, Vue.js) as input and generates HTML and JavaScript code as output. Specifically, the server uses the UI library to build a UI containing intuitively operable elements such as sliders and dials, and then sends this to the device.

[0637] Step 2:

[0638] Display of user interface

[0639] The device displays the user interface received from the server. It receives HTML and JavaScript code from the server as input and renders the UI in the browser as output. Specifically, the device uses the browser to parse the received code and display the UI, including sliders and dials.

[0640] Step 3:

[0641] Sending User Actions

[0642] The device sends operation data to the server when the user operates a slider or dial. It receives user operation events as input and sends operation data to the server as output. Specifically, the device uses JavaScript event listeners to capture user operations and sends them to the server via WebSocket or HTTP requests.

[0643] Step 4:

[0644] Receiving user actions

[0645] The server receives user interaction data sent from the terminal. It receives interaction data from the terminal as input and uses it as output to calculate beautification parameters. Specifically, the server receives user interaction data in real time via WebSocket or HTTP requests.

[0646] Step 5:

[0647] Calculation of beautification parameters

[0648] The server calculates beautification parameters based on the received operation data. It takes user operation data as input and generates beautification parameters as output. Specifically, the server uses programming languages ​​such as Python or Node.js to execute the beautification algorithm and calculate the parameters.

[0649] Step 6:

[0650] Collection of emotional data

[0651] The device collects user emotion data using sensors such as cameras and microphones. It receives sensor data from the camera and microphone as input and sends emotion data to the server as output. Specifically, the device uses WebRTC or MediaStream APIs to collect data from the camera and microphone and send it to the server.

[0652] Step 7:

[0653] Analysis of emotional data

[0654] The server receives and analyzes user emotion data transmitted from the terminal. It takes emotion data from the terminal as input and generates analysis results as output. Specifically, the server uses an emotion analysis engine, such as the Microsoft Azure Emotion API, to analyze the user's emotion data.

[0655] Step 8:

[0656] Dynamic adjustment of beautification parameters

[0657] The server dynamically adjusts the beautification parameters based on the analyzed sentiment data. It receives the results of the sentiment analysis as input and generates updated beautification parameters as output. Specifically, the server recalculates and updates the parameters of the beautification algorithm based on the sentiment analysis results.

[0658] Step 9:

[0659] Illustration update

[0660] The terminal updates the illustration in real time based on updated beautification parameters received from the server. It receives updated beautification parameters from the server as input and displays the updated illustration as output. Specifically, the terminal uses a browser to redraw the illustration based on the received beautification parameters.

[0661] (Application Example 2)

[0662] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0663] Conventional illustration beautification systems require users to manually adjust beautification parameters, making it difficult to dynamically beautify images in response to changes in the user's emotions. Furthermore, they lacked intuitive user interfaces, hindering improvements in the user experience.

[0664] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0665] In this invention, the server includes means for automatically beautifying illustrations drawn by the user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for the user to adjust beautification parameters, and means for recognizing the user's emotions in real time and dynamically adjusting the beautification parameters according to those emotions. This enables dynamic illustration beautification in response to changes in the user's emotions, and improves the user experience through intuitive operation.

[0666] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing, transforming, and correcting images.

[0667] "User-generated illustrations" refers to visual works created by users, either manually or using digital tools.

[0668] "Automatic beautification methods" refer to technologies and algorithms that improve the visual characteristics of illustrations without user intervention.

[0669] A "user interface that users can operate intuitively" refers to an interface designed to allow users to operate it easily and naturally.

[0670] "Beautification parameters" refer to settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[0671] "User-adjustable means" refers to features or interfaces that allow users to change settings according to their preferences.

[0672] "Means of recognizing user emotions in real time" refers to technologies and systems that analyze a user's facial expressions and actions to instantly determine their emotions at any given moment.

[0673] "Means of dynamically adjusting beautification parameters in response to emotions" refers to technologies and algorithms that automatically change beautification parameters based on the recognized emotions of the user.

[0674] The following system configuration will be described as an embodiment for carrying out this invention.

[0675] System Configuration

[0676] This system recognizes emotions in real time as users draw illustrations using their smartphones and dynamically adjusts the illustration's beautification parameters accordingly. The system consists of the following main components:

[0677] 1. Smartphone

[0678] Camera: Captures the user's face and provides data for emotion recognition.

[0679] Touchscreen: Provides an interface for users to draw illustrations.

[0680] Processor: Performs image processing, emotion recognition, and UI operations.

[0681] 2. Software Components

[0682] OpenCV: An image processing library. Used for beautifying illustrations.

[0683] EmotionRecognizer: An emotion recognition library. It recognizes the user's emotions in real time.

[0684] UI Elements: A library that provides intuitively operable UI elements such as sliders and dials.

[0685] Program processing

[0686] The server implements the system using the following methods.

[0687] 1. A means of automatically beautifying user-drawn illustrations using advanced image processing technology.

[0688] Using OpenCV, we will perform beautification processing on user-submitted illustrations. Specifically, we will apply filters such as Gaussian blur to improve the visual characteristics of the illustrations.

[0689] 2. Means for providing a user interface that users can operate intuitively.

[0690] The UI Elements library provides intuitively operable UI elements such as sliders and dials. This allows users to easily adjust aesthetic parameters.

[0691] 3. Means that allow users to adjust beautification parameters.

[0692] The system allows users to adjust beautification parameters to their liking using sliders or dials. For example, parameters such as hue, brightness, and contrast can be adjusted.

[0693] 4. A means of recognizing user emotions in real time and dynamically adjusting beautification parameters according to those emotions.

[0694] EmotionRecognizer is used to capture the user's face and recognize their emotions in real time. Based on the recognized emotions, beautification parameters are dynamically adjusted. For example, if the user is feeling "joy," the beautification parameters are set high, and if they are feeling "sadness," they are set low.

[0695] Specific example

[0696] For example, when a user is drawing an illustration on their smartphone, if the camera captures the user's face and the emotion recognition library detects "joy," the beautification parameter will be set high, making the illustration more vibrant and brighter. Conversely, if "sadness" is detected, the beautification parameter will be set low, making the illustration slightly darker and more subdued.

[0697] Example of a prompt

[0698] When a user is drawing an illustration on their smartphone, if the camera captures the user's face and the emotion recognition library detects "joy," the beautification parameter is set high, making the illustration more vibrant and brighter. Conversely, if "sadness" is detected, the beautification parameter is set low, making the illustration slightly darker and more subdued.

[0699] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[0700] Step 1:

[0701] The device uses a camera to capture the user's face. The input is the user's face image, and the output is the captured face image data. This face image data is used as preprocessing for emotion recognition.

[0702] Step 2:

[0703] The device uses an emotion recognition library (EmotionRecognizer) to recognize the user's emotions in real time from captured facial image data. The input is facial image data, and the output is the recognized emotion (e.g., joy, sadness, surprise, etc.). This emotion data is used to adjust subsequent beautification parameters.

[0704] Step 3:

[0705] The device allows users to draw illustrations using a touchscreen. Input is the user's touch operation, and output is the drawn illustration data. This illustration data is subject to beautification processing.

[0706] Step 4:

[0707] The device uses a user interface (UI elements) to display sliders and dials, allowing the user to adjust beautification parameters. Input is the user's manipulation of the sliders and dials, and output is the adjusted beautification parameters. These parameters are used to beautify the illustration.

[0708] Step 5:

[0709] The device dynamically adjusts the beautification parameters based on recognized emotion data. The input is the emotion data and the beautification parameters adjusted by the user, and the output is the final adjusted beautification parameters. For example, if the user is feeling "joy," the beautification parameters are set high, and if they are feeling "sadness," they are set low.

[0710] Step 6:

[0711] The device uses OpenCV to perform illustration beautification based on the final adjusted beautification parameters. The input is the illustration data and the adjusted beautification parameters, and the output is the beautified illustration data. Specifically, it applies filters such as Gaussian blur to improve the visual characteristics of the illustration.

[0712] Step 7:

[0713] The device displays an enhanced illustration to the user. The input is enhanced illustration data, and the output is an enhanced illustration that the user can visually confirm. This allows the user to see the enhanced illustration in real time.

[0714] (Example 3)

[0715] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0716] Conventional illustration beautification systems allow users to adjust beautification parameters, but they cannot beautify illustrations in accordance with the user's emotions. Therefore, it was difficult to achieve illustration beautification that adapted to the user's feelings, making it challenging to increase user satisfaction. Furthermore, while systems with emotion recognition capabilities exist, the technology to apply this to illustration beautification had not yet been established.

[0717] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0718] In this invention, the server includes means for collecting user emotion data, means for analyzing the collected emotion data and training an emotion recognition model, and means for beautifying illustrations according to the user's emotions. This makes it possible to beautify illustrations that are adapted to the user's emotions.

[0719] "Advanced image processing technology" refers to techniques for adjusting the visual characteristics of an image, such as its hue, brightness, and contrast.

[0720] A "user interface" is the interface through which a user operates a system, and includes intuitively operable UI elements such as sliders and dials.

[0721] "Beautification parameters" are parameters used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[0722] "User emotional data" refers to emotional data collected from the user's facial expressions, tone of voice, text input, and other sources.

[0723] An "emotion recognition model" is a model trained using machine learning algorithms to recognize a user's emotions.

[0724] "Illustration beautification" is the process of making user-created illustrations more appealing by adjusting their visual characteristics such as color tone, brightness, and contrast.

[0725] Modes for carrying out the invention

[0726] This invention is a system that automatically beautifies user-drawn illustrations using advanced image processing technology and further adjusts the beautification according to the user's emotions. A specific embodiment of this system is described below.

[0727] Hardware and software to use

[0728] Hardware: User's device (PC, smartphone, etc.), camera, microphone

[0729] Software: Web browsers, dedicated applications, image processing libraries (such as OpenCV), machine learning libraries (such as TensorFlow and PyTorch)

[0730] Program processing

[0731] Adjusting the beautification parameters

[0732] The user inputs beautification parameters such as "color intensity" and "brightness adjustment" through the device's interface. For example, using a smartphone app, the user might move a slider to set the color intensity to 50. The device sends these parameters to the server, which then performs beautification processing on the illustration based on the received parameters.

[0733] Collection and analysis of emotional data

[0734] The user's device uses its camera and microphone to collect the user's facial expressions and voice tone. For example, the camera captures the user's face to obtain facial expression data. The device sends the collected emotion data to a server, which uses a machine learning algorithm to train an emotion recognition model. The trained model recognizes the user's emotions and beautifies the illustration accordingly.

[0735] Beautification and display of illustrations

[0736] The server uses a trained emotion recognition model to beautify the illustration according to the user's emotions. For example, if the user is smiling, warmer tones are added to the illustration. Finally, the beautified illustration is sent to the device and displayed to the user.

[0737] Specific example

[0738] The user opens the app on their smartphone and sets the color intensity to 50 and the brightness adjustment to 20. The device sends these parameters to the server, which then beautifies the illustration based on the received parameters. Next, if the user is smiling through the camera, the device sends that facial expression data to the server. The server learns an emotion recognition model and adds warm tones to the illustration according to the user's emotion. Finally, the beautified illustration is sent to the device and displayed to the user.

[0739] Example of a prompt

[0740] "Please describe a system where the user sets parameters for beautifying an illustration, and an emotion engine recognizes the user's emotions to beautify the illustration. Please include the names of specific hardware and software."

[0741] Thus, this invention provides a function that beautifies illustrations based on user input and emotions. The flow of the specific processing in Example 3 will be explained with reference to Figure 21.

[0742] Step 1:

[0743] The user enters beautification parameters.

[0744] Users input beautification parameters such as "color intensity" and "brightness adjustment" through the device's interface. For example, they might use a smartphone app to move a slider to set the color intensity to 50. The entered parameters are saved on the device.

[0745] Step 2:

[0746] The device sends beautification parameters to the server.

[0747] The terminal sends the beautification parameters entered by the user to the server using an HTTP request. Specifically, parameters such as a color intensity of 50 and a brightness adjustment of 20 are sent. The server receives this and proceeds to the next step.

[0748] Step 3:

[0749] The server receives the beautification parameters and beautifies the illustration.

[0750] The server uses an image processing library (such as OpenCV) to beautify the illustration based on the received beautification parameters. For example, it might set the color intensity to 50 and adjust the brightness to 20. The beautified illustration is then saved to the server.

[0751] Step 4:

[0752] Collect user sentiment data.

[0753] The user's device uses its camera and microphone to collect data on their facial expressions and voice tone. For example, the camera can capture the user's face to obtain facial expression data. The collected emotional data is stored on the device.

[0754] Step 5:

[0755] The device sends emotional data to the server.

[0756] The device sends the collected emotional data to the server using an HTTP request. Specifically, it sends the user's facial expression data and voice tone data. The server receives this data and proceeds to the next processing step.

[0757] Step 6:

[0758] The server analyzes the emotion data and trains an emotion recognition model.

[0759] The server analyzes the received emotion data and trains an emotion recognition model using machine learning algorithms (such as TensorFlow or PyTorch). For example, it can determine whether the user is smiling based on facial expression data. The trained model is stored on the server.

[0760] Step 7:

[0761] The server beautifies the illustrations according to the user's emotions.

[0762] The server uses a trained emotion recognition model to beautify illustrations according to the user's emotions. For example, if the user is smiling, warmer tones are added to the illustration. The beautified illustration is then saved on the server.

[0763] Step 8:

[0764] The server sends the beautified illustration to the terminal.

[0765] The server sends the beautified illustration to the terminal using an HTTP response. Specifically, the beautified illustration data is sent. The terminal receives this and proceeds to the next step.

[0766] Step 9:

[0767] The device displays an aesthetically pleasing illustration to the user.

[0768] The device displays the received, beautified illustration to the user. For example, the illustration is displayed in a smartphone app. The user can then see the beautified illustration.

[0769] (Application Example 3)

[0770] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0771] Conventional image beautification systems had the problem that it was difficult for users to beautify illustrations according to their own emotions and preferences. Furthermore, the lack of an intuitive interface for adjusting beautification parameters often made the process cumbersome. In addition, the absence of technology to recognize and beautify illustrations based on user emotions meant that optimal beautification tailored to the user's feelings could not be achieved.

[0772] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0773] In this invention, the server includes means for automatically beautifying illustrations drawn by the user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for allowing the user to adjust beautification parameters, means including an emotion recognition engine for recognizing the user's emotions, and means for beautifying the illustration based on the recognized emotions. This allows the user to beautify illustrations according to their emotions and preferences, and to easily adjust the beautification parameters through an intuitive interface. Furthermore, by using an emotion recognition engine, optimal beautification according to the user's emotions can be achieved.

[0774] "Advanced image processing technology" refers to sophisticated algorithms and techniques for beautifying images by adjusting their visual characteristics, such as hue, brightness, and contrast.

[0775] "User interface" is a general term for the screens and input devices that users use to operate a system, and includes UI elements such as sliders and dials that can be operated intuitively.

[0776] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast, and can be freely adjusted by the user.

[0777] An "emotion recognition engine" is a system that collects data such as the user's facial expressions, voice tone, and text input, and uses machine learning algorithms to recognize the user's emotions.

[0778] "A method for beautifying illustrations based on recognized emotions" is a technique that automatically adjusts the illustration's beautification parameters according to the user's emotions recognized by an emotion recognition engine, thereby achieving optimal beautification.

[0779] A system for carrying out this invention includes advanced image processing technology for automatically beautifying illustrations drawn by a user, a user interface that can be operated intuitively by the user, means for the user to adjust beautification parameters, an emotion recognition engine for recognizing the user's emotions, and means for beautifying the illustration based on the recognized emotions.

[0780] System Configuration

[0781] Hardware: Smartphone

[0782] Software: Python, OpenCV, Keras, emotion recognition library

[0783] Program Processing Description

[0784] 1. Image Loading: Load images taken by the user from the smartphone's camera or gallery. Image data is obtained using OpenCV.

[0785] 2. Emotion Recognition: Using an emotion recognition model trained on Keras, we recognize emotions from the user's facial expressions, tone of voice, text input, etc. We collect and analyze user emotion data using an emotion recognition library.

[0786] 3. Adjusting Beautification Parameters: Provide intuitive UI elements such as sliders and dials for adjusting the intensity and brightness of the color tones specified by the user. Users can freely adjust the beautification parameters by manipulating these UI elements.

[0787] 4. Emotion-Based Beautification: The image beautification parameters are automatically adjusted based on the recognized emotion. For example, if the user is perceived as "happy," the image will be beautified by setting the color intensity to 1.2 and the brightness to 30.

[0788] 5. Saving the results: Save the beautified image to the smartphone's gallery. Use OpenCV to save the processed image data.

[0789] Specific example

[0790] Input image: Landscape photo taken by the user

[0791] Emotion: When the user is perceived as "happy"

[0792] Beautification parameters: Saturation 1.2, Brightness 30

[0793] Example of a prompt

[0794] When a landscape photo taken by the user is used as the input image, and the emotion engine recognizes it as "happy," please beautify it by setting the color intensity to 1.2 and the brightness to 30.

[0795] In this way, users can beautify illustrations according to their emotions and preferences, and easily adjust the beautification parameters through an intuitive interface. Furthermore, by using an emotion recognition engine, optimal beautification tailored to the user's emotions can be achieved.

[0796] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[0797] Step 1:

[0798] The device reads images taken by the user from the smartphone's camera or gallery. Specifically, it uses OpenCV to acquire image data. The input is an image file selected by the user, and the output is an array of image data.

[0799] Step 2:

[0800] The server uses an emotion recognition model trained with Keras to recognize emotions from the user's facial expressions, tone of voice, text input, etc. Specifically, it uses an emotion recognition library to collect and analyze the user's emotion data. The input is an array of image data, and the output is the label of the recognized emotion.

[0801] Step 3:

[0802] The device provides intuitive UI elements such as sliders and dials for adjusting the intensity and brightness of the color tones specified by the user. By manipulating these UI elements, the user can freely adjust the beautification parameters. The input is the user's actions, and the output is the adjusted beautification parameters.

[0803] Step 4:

[0804] The server automatically adjusts image beautification parameters based on the recognized emotion. Specifically, it sets the color intensity and brightness according to the emotion recognized by the emotion recognition engine. The input is the recognized emotion label and the user-adjusted beautification parameters, and the output is the optimized beautification parameters.

[0805] Step 5:

[0806] The device beautifies images based on beautification parameters. Specifically, it uses OpenCV to adjust the intensity and brightness of the colors and process the images. The input is an array of optimized beautification parameters and image data, and the output is the beautified image data.

[0807] Step 6:

[0808] The device saves the beautified image to the smartphone's gallery. Specifically, it uses OpenCV to save the processed image data. The input is the beautified image data, and the output is the saved image file.

[0809] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0810] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0811] Other examples of generative AI include Gemini® (registered trademark) (Internet search). <url: https: gemini.google.com ?hl="ja">) are some examples.

[0812] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0813] [Second Embodiment]

[0814] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0815] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0816] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0817] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0818] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0819] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0820] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0821] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0822] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0823] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0824] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0825] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0826] "Example of form 1"

[0827] One embodiment of the present invention provides a system that automatically beautifies illustrations drawn by a user. This system has the function of automatically beautifying illustrations drawn by a user using advanced image processing technology. Specifically, it receives an illustration drawn by a user as input, automatically adjusts the visual characteristics of the illustration such as color tone, brightness, and contrast, and outputs a beautified illustration.

[0828] "Example of form 2"

[0829] Furthermore, as one embodiment of the present invention, an intuitively operable user interface is provided. This user interface includes intuitively operable UI elements such as sliders and dials. Specifically, the user can adjust the beautification parameters by operating the sliders and dials. This allows the user to adjust the degree of beautification of the illustration to their liking.

[0830] "Example of form 3"

[0831] Furthermore, in one embodiment of the present invention, a means is provided that allows the user to adjust the beautification parameters. Specifically, the user can freely adjust the degree of beautification of the illustration by directly inputting the beautification parameters. For example, the user can beautify the illustration to their liking by adjusting parameters such as "color intensity" and "brightness adjustment".

[0832] The following describes the processing flow for each example form.

[0833] "Example of form 1"

[0834] Step 1: Input the illustration drawn by the user into the system. This input can be done via file upload, drag and drop, etc.

[0835] Step 2: The system analyzes the input illustration and automatically adjusts its visual characteristics such as hue, brightness, and contrast. This adjustment is performed using a pre-trained machine learning model.

[0836] Step 3: The system outputs the adjusted illustration. The output can be displayed directly on the screen or downloaded as a file.

[0837] "Example of form 2"

[0838] Step 1: The user operates the system's user interface. Specifically, they adjust the beautification parameters by manipulating sliders and dials.

[0839] Step 2: The system adjusts the beautification parameters according to the user's input and beautifies the illustration.

[0840] Step 3: The system outputs the beautified illustration. The output can be displayed directly on the screen or downloaded as a file.

[0841] "Example of form 3"

[0842] Step 1: The user directly inputs the beautification parameters. Specifically, parameters such as "color intensity" and "brightness adjustment" are specified using numerical values ​​or sliders.

[0843] Step 2: The system beautifies the illustration based on the beautification parameters entered by the user.

[0844] Step 3: The system outputs the beautified illustration. The output can be displayed directly on the screen or downloaded as a file.

[0845] (Example 1)

[0846] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0847] Traditional illustration beautification systems required users to manually adjust color tone, brightness, and contrast, resulting in a cumbersome operation. Furthermore, they lacked an intuitive user interface, making the beautification process difficult for users. Additionally, the lack of real-time progress displays and completion notifications resulted in a poor user experience.

[0848] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes means for the user to upload an illustration, means for the server to receive the illustration, means for the server to adjust the color tone, means for the server to adjust the brightness, means for the server to adjust the contrast, means for the server to generate a beautified illustration, and means for the server to provide the beautified illustration to the user. This makes it possible for the user to easily beautify an illustration through an intuitive interface. In addition, the user experience is improved by displaying the progress in real time and notifying the user when the processing is complete.

[0849] "Advanced image processing technology" refers to technology that automatically adjusts the visual characteristics of an image, such as its hue, brightness, and contrast.

[0850] A "user interface" is an interface that includes screens and input devices for a user to operate a system.

[0851] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[0852] "Means for users to upload illustrations" refers to a function that allows users to send illustration files from their devices to the system.

[0853] "The means by which the server receives illustrations" refers to the function that allows the server to receive and save illustration files sent by users.

[0854] "Means for the server to adjust the color tone" refers to a function that allows the server to automatically change the hue and saturation of an illustration.

[0855] "A means for the server to adjust brightness" refers to a function that allows the server to automatically change the brightness of the illustration.

[0856] "A means for the server to adjust contrast" refers to a function that allows the server to automatically change the difference in brightness between light and dark areas in an illustration.

[0857] "Means for the server to generate beautified illustrations" refers to a function that allows the server to create new illustrations after adjusting the color tone, brightness, and contrast.

[0858] "Means by which the server provides beautified illustrations to users" refers to a function that provides users with beautified illustrations generated by the server in a downloadable format.

[0859] This invention relates to a system that automatically beautifies illustrations drawn by a user. The system provides an intuitive user interface and allows the user to adjust beautification parameters for adjusting visual characteristics such as hue, brightness, and contrast.

[0860] Hardware and software to be used

[0861] The server is a computer system for running Python programs, using OpenCV and Pillow as image processing libraries. Users access the system using internet-connected devices (such as PCs, smartphones, and tablets).

[0862] Program Processing Description

[0863] The server receives the illustration uploaded by the user. Next, the server adjusts the color tone of the image using OpenCV. Specifically, it converts the image to HSV (Hue, Saturation, Value) space, adjusts the hue appropriately, and then converts it back to RGB space.

[0864] The server then uses Pillow to adjust the image brightness. Specifically, it adjusts the brightness using Pillow's ImageEnhance.Brightness class. Next, it adjusts the contrast using Pillow's ImageEnhance.Contrast class.

[0865] After these processes, an enhanced illustration is generated and provided to the user. The user can obtain the enhanced illustration by clicking the download link on their device.

[0866] Specific example

[0867] As a concrete example, let's consider the case of beautifying an illustration "example_illustration.png" drawn by a user.

[0868] 1. User actions:

[0869] The user uploads "example_illustration.png" from their device to the system.

[0870] 2. Server processing:

[0871] The server receives the uploaded image and adjusts its color tone, brightness, and contrast.

[0872] Save the adjusted image as "beautified_illustration.png".

[0873] 3. Output to the user:

[0874] The server provides the user with a beautified illustration, "beautified_illustration.png".

[0875] An example of a prompt message is as follows:

[0876] Please upload your user-created illustration, "example_illustration.png". The system will automatically adjust the color tone, brightness, and contrast to provide an enhanced illustration. You will receive a notification when processing is complete.

[0877] In this way, the system works to automatically beautify illustrations drawn by users.

[0878] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0879] Step 1:

[0880] The user uploads an illustration.

[0881] Users upload their own illustration files (e.g., "example_illustration.png") from their terminal via the system's web interface. The input is the user's illustration file, and the output is the illustration file sent to the server.

[0882] Step 2:

[0883] The server receives the illustration.

[0884] The server receives illustration files uploaded by users and saves them to a specified directory. The input is the illustration file sent by the user, and the output is the illustration file saved on the server. Specifically, the server checks the file format and size and verifies that it is in an appropriate format.

[0885] Step 3:

[0886] The server adjusts the color tone.

[0887] The server loads the illustration using the OpenCV cv2.imread function. Next, it converts the image to HSV (Hue, Saturation, Value) space using the cv2.cvtColor function. It then adjusts the hue appropriately and converts it back to RGB space. The input is the illustration file stored on the server, and the output is the color-adjusted illustration. Specifically, the hue is shifted by 10 degrees.

[0888] Step 4:

[0889] The server adjusts the brightness.

[0890] The server uses Pillow's ImageEnhance.Brightness class to adjust the brightness of the image. Specifically, it increases the brightness by 1.2 times. The input is an illustration with adjusted color tone, and the output is an illustration with adjusted brightness.

[0891] Step 5:

[0892] The server adjusts the contrast.

[0893] The server uses Pillow's ImageEnhance.Contrast class to adjust the contrast of the image. Specifically, it increases the contrast by 1.3 times. The input is an illustration with adjusted brightness, and the output is an illustration with adjusted contrast.

[0894] Step 6:

[0895] The server generates an aesthetically pleasing illustration.

[0896] The server saves the adjusted image as "beautified_illustration.png". The input is an illustration with adjusted contrast, and the output is a beautified illustration file.

[0897] Step 7:

[0898] The server provides users with an enhanced version of the illustration.

[0899] The server provides the user with a beautified illustration file, "beautified_illustration.png". The user can obtain the beautified illustration by clicking the download link from their device. The input is the beautified illustration file, and the output is the download link provided to the user. Specifically, the server displays the progress in real time and sends a notification to the user when the process is complete.

[0900] (Application Example 1)

[0901] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0902] Traditional illustration beautification systems offer the functionality to automatically beautify user-submitted illustrations, but they lack a convenient way to share these beautified illustrations with content distribution services. Furthermore, the customization options for the beautification process are limited, making it difficult for users to adjust the process to their own preferences.

[0903] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0904] In this invention, the server includes means for automatically beautifying user-drawn illustrations using advanced image processing technology, means for providing an intuitive user interface, means for allowing the user to adjust beautification parameters, means for directly sharing the beautified illustrations with content distribution services, and means for providing customization options for the beautification process. This makes it possible for users to easily share beautified illustrations and further customize the beautification process to their liking.

[0905] "Advanced image processing technology" refers to technology that automatically adjusts the visual characteristics of an image, such as its hue, brightness, and contrast.

[0906] "User-generated illustrations" refer to paintings or drawings created by users, either by hand or using digital tools.

[0907] "Methods for automatically beautifying" refer to algorithms and software that improve the visual characteristics of illustrations without user intervention.

[0908] A "user interface that users can operate intuitively" is an interface designed to allow users to operate it easily and intuitively.

[0909] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[0910] "User-adjustable means" refers to a feature that allows users to manually change beautification parameters.

[0911] A "content distribution service" is a service that distributes digital content such as images and videos over the internet.

[0912] "Shareable means" refers to features that allow users to share the beautified illustrations with other users or platforms.

[0913] "Customization options" are settings and features that allow users to adjust the beautification process to their own preferences.

[0914] The following system can be constructed as an embodiment of this invention.

[0915] System Overview

[0916] This system automatically beautifies user-generated illustrations and shares those beautified illustrations with content distribution services. The system includes a smartphone, a server, and software utilizing advanced image processing technology.

[0917] Hardware and software to be used

[0918] Hardware: Smartphone (iOS or Android), Server

[0919] Software: Python, OpenCV (image processing library)

[0920] Data processing and data calculation

[0921] 1. Loading images

[0922] Users take photos of or upload illustrations they have drawn on their smartphones. This utilizes the smartphone's camera function and file upload function.

[0923] 2. Adjusting the color tone

[0924] The server converts the image to the HSV color space and increases its saturation. This makes the illustration's colors more vibrant.

[0925] 3. Adjusting Brightness

[0926] The server performs a scaling transformation to increase the brightness of the image. This makes the illustration brighter.

[0927] 4. Adjust the contrast

[0928] The server converts the image to the LAB color space and performs histogram equalization of the L channel to improve contrast. This enhances the visual characteristics of the illustration.

[0929] 5. Saving and sharing images

[0930] The beautified images are stored on the server, and users can share them directly to content distribution services via smartphone applications.

[0931] Specific example

[0932] Users take a photo of an illustration they drew on their smartphone and upload it to the app, whereupon the server automatically beautifies the illustration. The beautified illustration can then be adjusted in terms of color tone, brightness, and contrast to suit the user's preferences. Finally, the beautified illustration can be shared on social media and other content distribution services.

[0933] Example of a prompt

[0934] "Please develop an app that beautifies user-submitted illustrations. The app should automatically adjust the color tone, brightness, and contrast of the illustrations, and output the beautified versions."

[0935] In this way, a system can be implemented that allows users to easily create and share beautified illustrations.

[0936] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0937] Step 1:

[0938] The user takes a picture of an illustration they drew on their smartphone or uploads it. The input is the illustration image drawn by the user, and the output is the image file saved on the smartphone. Specifically, the user takes a picture of the illustration using the smartphone's camera function or uploads an existing image file to the application.

[0939] Step 2:

[0940] The device sends captured or uploaded illustration images to the server. The input is an image file stored on the smartphone, and the output is the image data transferred to the server. Specifically, the smartphone application calls an API to send the image file to the server.

[0941] Step 3:

[0942] The server reads the received illustration image. The input is the image data transferred to the server, and the output is the image object loaded into memory. Specifically, the server reads the image file using Python and OpenCV.

[0943] Step 4:

[0944] The server adjusts the color tone of the image. The input is an image object loaded into memory, and the output is an image object with adjusted color tone. Specifically, the server converts the image to the HSV color space and increases its saturation.

[0945] Step 5:

[0946] The server adjusts the brightness of the image. The input is a color-adjusted image object, and the output is a brightness-adjusted image object. Specifically, the server performs a scaling transformation to increase the brightness of the image.

[0947] Step 6:

[0948] The server adjusts the contrast of the image. The input is an image object with adjusted brightness, and the output is an image object with adjusted contrast. Specifically, the server converts the image to the LAB color space and performs histogram equalization of the L channel.

[0949] Step 7:

[0950] The server saves the beautified image. The input is an image object with adjusted contrast, and the output is a beautified image file saved on the server. Specifically, the server saves the beautified image as a file.

[0951] Step 8:

[0952] The device retrieves a beautified image from the server. The input is a beautified image file stored on the server, and the output is a beautified image file downloaded to the smartphone. Specifically, the smartphone application calls an API to download the beautified image from the server.

[0953] Step 9:

[0954] Users share beautified illustrations to content distribution services. The input is a beautified image file downloaded to a smartphone, and the output is a beautified image uploaded to a content distribution service. Specifically, users share beautified illustrations to social media and other content distribution services using a smartphone application.

[0955] (Example 2)

[0956] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0957] Conventional image processing systems required specialized knowledge and complex operations to beautify user-generated illustrations, lacking an intuitive user interface. Furthermore, adjusting beautification parameters was difficult, making it challenging for users to customize their illustrations to their preferences. Additionally, the beautification process using generative AI models was not performed in real time, resulting in delayed feedback from user input.

[0958] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0959] In this invention, the server includes means for converting beautification parameters operated by the user into prompt sentences, means for inputting the prompt sentences into a generation AI model to beautify the illustration, and means for displaying the beautified illustration to the user. This allows the user to adjust the beautification parameters and view the beautified illustration in real time through an intuitively operable user interface.

[0960] "Advanced image processing technology" is a general term for advanced algorithms and techniques used to automatically beautify illustrations drawn by users.

[0961] "User interface" is a general term for the screens and input devices that users use to operate a system, and includes intuitively operable UI elements.

[0962] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[0963] A "prompt message" is a text-based instruction used to input the beautification parameters set by the user into the generating AI model.

[0964] A "generative AI model" is an artificial intelligence model used to beautify illustrations based on input prompt text, and includes, for example, "Stable Diffusion" and "DALL-E".

[0965] A "slider" is a UI element that allows users to adjust values ​​by dragging.

[0966] A "dial" is a UI element that allows users to adjust values ​​by rotating it.

[0967] A "server" is a computer system that receives prompt messages, inputs them into a generation AI model, generates an aesthetically pleasing illustration, and sends it back to the terminal.

[0968] A "terminal" is a device operated by a user, which provides a user interface, retrieves beautification parameters, generates prompt messages, and sends them to the server.

[0969] This invention is a system that automatically beautifies illustrations drawn by users, and provides a user interface that allows users to operate it intuitively. The system converts the beautification parameters operated by the user into prompt sentences, inputs them into a generating AI model, and beautifies the illustration. A specific embodiment of this system is described below.

[0970] Hardware and software to be used

[0971] hardware

[0972] Terminal: A device operated by the user, providing a user interface, obtaining beautification parameters, and generating and sending prompt messages to the server. Specifically, this includes personal computers, tablets, and smartphones.

[0973] Server: A computer system that receives prompt messages, inputs them into a generation AI model, generates an aesthetically pleasing illustration, and sends it back to the terminal.

[0974] software

[0975] User Interface: Includes UI elements that allow users to operate intuitively. Specifically, this includes sliders, dials, and similar elements.

[0976] Generative AI Model: This is an artificial intelligence model used to beautify illustrations based on prompt text. Specifically, models such as "Stable Diffusion" and "DALL-E" are used.

[0977] Data processing and data calculation

[0978] User actions

[0979] The user interacts with the user interface on the device. Specifically, they adjust beautification parameters using sliders and dials. For example, if the user moves the slider to the right, the beautification parameter increases.

[0980] Terminal processing

[0981] The terminal acquires real-time position information of sliders and dials operated by the user. This position information is interpreted as a beautification parameter. For example, if the slider is set to its maximum value, the beautification parameter will be "100". The terminal generates a prompt message based on the acquired beautification parameter. For example, if the beautification parameter is "100", it will generate a prompt message that says "Beautification parameter: 100".

[0982] Server Processing

[0983] The server receives the prompt message sent from the terminal. It inputs the received prompt message into the generating AI model. The generating AI model beautifies the illustration based on the input prompt message. For example, if the prompt message is "Beautification parameter: 100", it processes the illustration with the degree of beautification set to the maximum. The server receives the beautified illustration from the generating AI model and sends it back to the terminal.

[0984] Terminal display

[0985] The device receives the beautified illustration sent back from the server and displays it to the user. This allows the user to see the illustration based on the beautification parameters they have set.

[0986] Specific example

[0987] Example of a prompt

[0988] "Beautification parameter: 100"

[0989] By inputting this prompt into the AI ​​model, you can set the level of illustration beautification to the maximum.

[0990] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0991] Step 1:

[0992] The user interacts with UI elements.

[0993] The user operates the user interface on the device, adjusting beautification parameters using sliders and dials. For example, moving the slider to the right increases the beautification parameter. The input is the user's actions, and the output is the adjusted beautification parameter.

[0994] Step 2:

[0995] The device retrieves the beautification parameters.

[0996] The terminal acquires real-time position information of sliders and dials operated by the user. This position information is interpreted as a beautification parameter. For example, if the slider is set to its maximum value, the beautification parameter will be "100". The input is the user's operation position information, and the output is the beautification parameter.

[0997] Step 3:

[0998] The terminal generates the prompt message.

[0999] The terminal generates a prompt message based on the acquired beautification parameter. For example, if the beautification parameter is "100", it will generate the prompt message "Beautification parameter: 100". The input is the beautification parameter, and the output is the prompt message.

[1000] Step 4:

[1001] The terminal sends a prompt message to the server.

[1002] The terminal sends the generated prompt message to the server. This transmission is performed using a communication protocol such as an HTTP request. The input is the prompt message, and the output is a notification to the server that the transmission has been completed.

[1003] Step 5:

[1004] The server receives the prompt message.

[1005] The server receives a prompt message sent from the terminal. The received prompt message is used in the next processing step. The input is the prompt message, and the output is an acknowledgment.

[1006] Step 6:

[1007] The server inputs prompt messages into the generated AI model.

[1008] The server inputs the received prompt message into the generating AI model. For example, it can use generating AI models such as "Stable Diffusion" or "DALL-E". The input is the prompt message, and the output is a notification to the generating AI model that input is complete.

[1009] Step 7:

[1010] The generative AI model beautifies the illustration.

[1011] The generative AI model beautifies illustrations based on the input prompt text. For example, if the prompt text "Beautification parameter: 100" is input, the model will set the illustration's beautification level to the maximum and process it. The input is the prompt text, and the output is the beautified illustration.

[1012] Step 8:

[1013] The server sends the beautified illustration back to the terminal.

[1014] The server receives the beautified illustration generated by the AI ​​model and sends it back to the terminal. This return is also done using a communication protocol such as an HTTP response. The input is the beautified illustration, and the output is a notification to the terminal that the transmission is complete.

[1015] Step 9:

[1016] The device displays an aesthetically pleasing illustration.

[1017] The terminal receives the beautified illustration sent back from the server and displays it to the user. This allows the user to see the illustration based on the beautification parameters they have set. The input is the beautified illustration, and the output is the display to the user.

[1018] (Application Example 2)

[1019] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[1020] Conventional image beautification systems lacked an intuitive user interface for beautifying user-generated illustrations and photographs, making real-time preview and saving / sharing of beautified images difficult. Furthermore, users struggled to fine-tune visual characteristics when adjusting beautification parameters.

[1021] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1022] In this invention, the server includes means for automatically beautifying user-drawn illustrations using advanced image processing technology, means for providing an intuitive user interface, means for allowing the user to adjust beautification parameters, means for previewing user-uploaded images in real time, and means for saving or sharing the beautified images. This allows the user to intuitively operate the server, view the beautified image in real time, adjust the visual characteristics to their liking, and save or share the final image.

[1023] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing, transforming, and correcting images.

[1024] "User-created illustrations" refers to drawings and diagrams that users have manually created.

[1025] "Automatic beautification methods" refer to technologies and devices that improve the quality and appearance of images without user intervention.

[1026] A "user interface that users can operate intuitively" refers to an interface designed to allow users to operate it easily and naturally.

[1027] "Beautification parameters" refer to settings used to adjust the visual characteristics of an image, such as color tone, brightness, and contrast.

[1028] "User-adjustable means" refers to features that allow users to change settings and parameters to suit their preferences.

[1029] "Real-time preview" refers to a function that displays the results immediately when a user performs an action.

[1030] "Means for saving or sharing beautified images" refers to functions for saving processed images to a device or sharing them with other users.

[1031] The system for carrying out this invention automatically beautifies illustrations and photographs drawn by the user and provides an intuitive user interface. Specific embodiments of this system are described below.

[1032] System Configuration

[1033] hardware

[1034] Smartphone: A device used by users to upload and manipulate images.

[1035] Server: Provides computing resources for image processing.

[1036] software

[1037] OpenCV: An image processing library. It performs image analysis, transformation, and modification.

[1038] Tkinter: A GUI library used to build user interfaces.

[1039] Data processing and data calculation

[1040] Image loading

[1041] The user uploads an image using their smartphone. The server reads this image using OpenCV.

[1042] Adjusting the beautification parameters

[1043] Users adjust beautification parameters (smoothness, brightness, contrast) by operating sliders and dials displayed on their smartphone screen. This allows users to see the changes in the image in real time.

[1044] Real-time preview

[1045] The server processes the image in real time based on the beautification parameters adjusted by the user and sends a preview to the smartphone. This allows the user to see the results immediately.

[1046] Saving and sharing images

[1047] Users can save images they are satisfied with to their smartphones. They can also share them with other users via social media or email.

[1048] Specific example

[1049] Users open the app on their smartphones and upload photos. They use sliders to adjust smoothness, brightness, and contrast, and preview the beautified photos in real time. Once satisfied, they save the photos or share them on social media.

[1050] Example of a prompt

[1051] "Create an app that lets users upload photos from their smartphones, adjust smoothness, brightness, and contrast using sliders, preview the beautified photos in real time, and then save or share them."

[1052] In this way, the invention allows users to intuitively operate the system, view beautified images in real time, adjust visual characteristics to their liking, and save or share the final image.

[1053] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1054] Step 1:

[1055] Upload image

[1056] The user uses their smartphone to launch the application, select an image, and upload it.

[1057] Input: The image file selected by the user.

[1058] Output: Image data sent to the server.

[1059] Specific operation: The user selects an image from their smartphone's gallery and presses the "Upload" button. The image data is sent to the server.

[1060] Step 2:

[1061] Image loading

[1062] The server reads the received image data using OpenCV.

[1063] Input: Image data sent to the server.

[1064] Output: Image object loaded with OpenCV.

[1065] Specific operation: The server receives the image data and loads the image into memory using the OpenCV cv2.imread function.

[1066] Step 3:

[1067] Initial settings for beautification parameters

[1068] The server sets the initial beautification parameters (smoothness, brightness, contrast).

[1069] Input: None (default setting).

[1070] Output: Initialized beautification parameters.

[1071] Specific operation: The server sets initial values ​​for smoothness, brightness, and contrast, and retains these values.

[1072] Step 4:

[1073] Display of User Interface

[1074] The terminal displays a user interface and provides sliders and dials.

[1075] Input: Initial beautification parameters.

[1076] Output: Display of the user interface.

[1077] Specific operation: The terminal uses Tkinter to display a user interface including sliders and dials.

[1078] Step 5:

[1079] Adjusting the beautification parameters

[1080] The user adjusts the beautification parameters by operating sliders and dials.

[1081] Input: The value of a slider or dial as determined by the user.

[1082] Output: Adjusted beautification parameters.

[1083] Specific operation: The user operates a slider or dial and sends a new parameter value to the server.

[1084] Step 6:

[1085] Image beautification processing

[1086] The server processes the image based on the adjusted beautification parameters.

[1087] Input: Adjusted beautification parameters, loaded image object.

[1088] Output: A beautified image object.

[1089] Specific operation: The server uses OpenCV functions to adjust smoothness, brightness, and contrast to beautify the image.

[1090] Step 7:

[1091] Displaying real-time preview

[1092] The device displays a real-time preview of the beautified image.

[1093] Input: A beautified image object.

[1094] Output: The beautified image displayed on the device.

[1095] Specific operation: The server sends the beautified image to the terminal, and the terminal displays it on the screen.

[1096] Step 8:

[1097] Saving and sharing images

[1098] Users save or share the beautified images.

[1099] Input: A beautified image object.

[1100] Output: Saved image file or shared image link.

[1101] Specific action: The user presses the "Save" or "Share" button, and the device saves the image to local storage or shares it via social media or email.

[1102] (Example 3)

[1103] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1104] Conventional illustration beautification systems often made it difficult for users to intuitively adjust beautification parameters, and the generated illustrations frequently failed to meet user expectations. Furthermore, the lack of a process to properly analyze user-input parameters and generate prompts suitable for the AI ​​model resulted in inconsistent levels of illustration beautification.

[1105] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[1106] In this invention, the server includes means for automatically beautifying illustrations drawn by the user, means for providing a user interface that the user can operate intuitively, means for the user to adjust beautification parameters, means for sending the beautification parameters entered by the user to the server, means for the server to analyze the received beautification parameters and generate prompt sentences suitable for the generating AI model, means for the generating AI model to beautify the illustration based on the prompt sentences, and means for the server to send the beautified illustration to the user's terminal. This makes it possible for the user to intuitively adjust the beautification parameters and obtain a stable beautified illustration using the generating AI model.

[1107] "Advanced image processing technology" refers to the technology of using computers to analyze, correct, and transform images, and is particularly focused on achieving highly accurate and complex processing.

[1108] A "user interface" is a means for a user to interact with a system, and it includes both visual and manipulative elements.

[1109] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, and include factors such as hue, brightness, and contrast.

[1110] A "generative AI model" is an algorithm or system that uses artificial intelligence to generate or transform data, and is particularly specialized for image generation.

[1111] A "prompt message" is text-based input data used to give instructions to a generative AI model.

[1112] A "server" is a computer system that provides services to clients over a network.

[1113] A "device" is a device that a user directly operates, and includes computers, smartphones, tablets, and other similar devices.

[1114] "Analysis" is the process of examining data in detail and understanding its structure and meaning.

[1115] "Transmission" is the act of moving data from one location to another.

[1116] An "illustration" is an image intended for visual expression, created using hand-drawn or digital tools.

[1117] Modes for carrying out the invention

[1118] This invention is a system that automatically beautifies illustrations drawn by a user, provides an intuitive user interface, and includes means for the user to adjust the beautification parameters. Specific embodiments of this system are described below.

[1119] Hardware and software to be used

[1120] This system uses the following hardware and software:

[1121] Server: A computer system that provides services to clients over a network.

[1122] Device: A device that a user directly interacts with (e.g., computer, smartphone, tablet).

[1123] Generative AI models: Algorithms and systems that use artificial intelligence to generate or transform data (e.g., Stable Diffusion, DALL-E).

[1124] System Processing Overview

[1125] Users input beautification parameters using their devices. For example, users set parameters such as adjusting color intensity and brightness using a dedicated application on their device or a web interface. Specifically, users use sliders or text boxes to set "color intensity" to 50 and "brightness adjustment" to 70.

[1126] The terminal sends the beautification parameters entered by the user to the server. This transmission is done using communication protocols such as HTTP requests or WebSockets. The server analyzes the received beautification parameters and generates prompts suitable for the AI ​​model. For example, the server might generate a prompt such as, "Generate an illustration with a hue intensity of 50 and a brightness adjustment of 70."

[1127] The server inputs prompt text into the generation AI model, requesting it to beautify an illustration. In doing so, the server calls the generation AI model's API. The generation AI model beautifies the illustration based on the prompt text and returns the result to the server. The returned data is an image file of the beautified illustration.

[1128] The server sends the beautified illustration received from the generating AI model to the terminal. This transmission also uses communication protocols such as HTTP responses or WebSockets. The user then views the beautified illustration on the terminal. The terminal's application or web interface has the functionality to display the received illustration.

[1129] Specific example

[1130] If the user uses their device to set "Color Intensity" to 50 and "Brightness Adjustment" to 70, the server will input the following prompt message into the AI ​​model:

[1131] Example of a prompt:

[1132] "Please generate an illustration with a hue intensity of 50 and a brightness adjustment of 70."

[1133] The generative AI model receives this prompt and beautifies the illustration based on the specified parameters. As a result, a beautified illustration tailored to the user's preferences is generated.

[1134] In this way, users can intuitively adjust the beautification parameters and obtain stable, beautified illustrations using the generation AI model. The flow of the specific processing in Example 3 will be explained with reference to Figure 15.

[1135] Step 1:

[1136] The user enters beautification parameters using a terminal.

[1137] Input: Users set parameters such as color intensity and brightness using a dedicated application or web interface on their device.

[1138] Specific action: The user uses a slider or text box to set the "Tone Intensity" to 50 and the "Brightness Adjustment" to 70.

[1139] Output: Beautification parameters set on the terminal.

[1140] Step 2:

[1141] The terminal sends the entered beautification parameters to the server.

[1142] Input: Beautification parameters set on the device.

[1143] Specific operation: The terminal creates an HTTP POST request and sends the beautification parameters in JSON format to the server endpoint.

[1144] Output: Beautification parameters sent to the server.

[1145] Step 3:

[1146] The server analyzes the beautification parameters it receives and generates prompt sentences suitable for the AI ​​model.

[1147] Input: Beautification parameters sent to the server.

[1148] Specific operation: The server parses the JSON data it receives and performs string manipulation to generate a prompt message. For example, it generates a prompt message such as "Generate an illustration with a hue intensity of 50 and a brightness adjustment of 70."

[1149] Output: The generated prompt message.

[1150] Step 4:

[1151] The server inputs prompt messages into the generated AI model, requesting it to beautify the illustrations.

[1152] Input: The generated prompt message.

[1153] Specific operation: The server sends an HTTP POST request to the API endpoint of the generated AI model, sending data including a prompt message.

[1154] Output: The prompt message sent to the generating AI model.

[1155] Step 5:

[1156] The AI ​​model generates an aesthetically pleasing illustration and sends it back to the server.

[1157] Input: The prompt text sent to the generating AI model.

[1158] Specific operation: The generation AI model internally processes the illustration for aesthetics and then returns the generated illustration to the server as binary data.

[1159] Output: Binary data of the beautified illustration sent back to the server.

[1160] Step 6:

[1161] The server sends the beautified illustration to the terminal.

[1162] Input: Binary data of the beautified illustration sent back to the server.

[1163] Specific operation: The server sends the received binary data to the terminal as an HTTP response.

[1164] Output: A beautified illustration sent to the terminal.

[1165] Step 7:

[1166] The user views the beautified illustration on their device.

[1167] Input: A beautified illustration sent to the terminal.

[1168] Specific operation: The application displays the illustration received by the device, and the user confirms the illustration.

[1169] Output: User-confirmed, beautified illustration.

[1170] (Application Example 3)

[1171] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 as a "terminal".

[1172] Conventional image beautification systems often lack an intuitive user interface when automatically beautifying user-generated illustrations. Furthermore, they often lack features such as real-time preview of the adjusted illustration or the ability to save adjustments and share them with other content distribution services. This presents a challenge for users, as it makes it difficult to beautify illustrations to their liking and easily share them.

[1173] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1174] In this invention, the server includes means for automatically beautifying illustrations drawn by the user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for the user to adjust beautification parameters, means for previewing the adjusted illustration in real time, and means for saving the adjustments and sharing them with other content distribution services. This makes it possible for users to beautify illustrations to their liking and easily share them.

[1175] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing and transforming images.

[1176] "User-created illustrations" refers to drawings and diagrams that users have manually created.

[1177] "Automatic beautification methods" refer to technologies that improve the quality and appearance of images without user intervention.

[1178] A "user interface that users can operate intuitively" refers to an interface that users can easily understand and operate.

[1179] "Beautification parameters" refer to settings used to adjust the visual characteristics of an image, such as color tone, brightness, and contrast.

[1180] "A means of previewing the adjusted illustration in real time" refers to a technology that instantly displays the results when a user changes parameters.

[1181] "Means of saving adjustments and sharing them with other content distribution services" refers to technology that records image adjustments made by users and shares them with other online platforms.

[1182] The system for implementing this invention automatically beautifies illustrations drawn by the user and provides a user interface that the user can operate intuitively. The system is implemented as an application that runs on devices such as smartphones and tablets.

[1183] Hardware and software to be used

[1184] Hardware: Smartphones, tablets

[1185] Software: Python, OpenCV, Tkinter

[1186] System Configuration

[1187] 1. Image Loading: Users upload photos and illustrations taken with their devices to the application.

[1188] 2. Adjusting beautification parameters: Users can adjust beautification parameters such as "color intensity" and "brightness adjustment" using sliders and dials within the application.

[1189] 3. Real-time preview: Image beautification is performed in real time based on the adjusted parameters, and the results are displayed instantly.

[1190] 4. Saving and sharing adjustments: Users can save their satisfactory adjustment results and share them on social media and other content distribution services.

[1191] Processing flow

[1192] 1. Image Loading: Select an image from your device's camera or gallery and upload it to the application. Load the image using OpenCV and start processing.

[1193] 2. Adjusting Beautification Parameters: Using Tkinter, provide sliders and dials that users can intuitively operate. Users will adjust the beautification parameters by manipulating these UI elements.

[1194] 3. Real-time preview: Every time the user changes a parameter, the brightness and contrast of the image are adjusted using OpenCV's convertScaleAbs function, and the results are displayed in real time.

[1195] 4. Saving and sharing adjustments: Provides a function to save the adjusted image to the device and share it on social media or other content distribution services.

[1196] Specific example

[1197] For example, a user can upload a landscape photo taken with their smartphone to the application, enhance the colors, slightly reduce the brightness, and then share it on Instagram.

[1198] Example of a prompt

[1199] "Upload a landscape photo, enhance the colors, slightly reduce the brightness, and share it on Instagram."

[1200] In this way, users can beautify illustrations to their liking and easily share them.

[1201] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[1202] Step 1:

[1203] Users upload photos and illustrations taken with their devices to the application.

[1204] Input: An image file selected from the device's camera or gallery.

[1205] Output: Image data loaded into the application.

[1206] Specific operation: The user launches the application, presses the image upload button, and selects an image from the device's camera roll or gallery. The selected image is loaded into the application.

[1207] Step 2:

[1208] The image data read by the device is processed using OpenCV.

[1209] Input: Loaded image data.

[1210] Output: An image object that can be processed by OpenCV.

[1211] Specific operation: The application uses the OpenCV library to convert the loaded image data into an internal format, making it ready for processing.

[1212] Step 3:

[1213] The user operates sliders and dials to adjust the beautification parameters.

[1214] Input: The value of a slider or dial as determined by the user.

[1215] Output: Adjusted beautification parameters.

[1216] Specific operation: The user operates sliders and dials within the application to set parameters such as "color intensity" and "brightness adjustment." The set values ​​are reflected in the application in real time.

[1217] Step 4:

[1218] The device previews the image in real time based on the adjusted beautification parameters.

[1219] Input: Adjusted beautification parameters and original image data.

[1220] Output: The adjusted image is previewed in real time.

[1221] Specific operation: The application uses the OpenCV convertScaleAbs function to adjust the brightness and contrast of an image based on the beautification parameters set by the user, and displays the results in real time.

[1222] Step 5:

[1223] Users save the adjusted image and share it with other content distribution services.

[1224] Input: Adjusted image data.

[1225] Output: Saved image files and shared images.

[1226] Specific actions: The user presses the save button to save the adjusted image to their device. They then press the share button to upload the image to Instagram or other social media platforms.

[1227] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1228] "Example of form 1"

[1229] One embodiment of the present invention is an illustration beautification system incorporating an emotion engine. This system includes a function to automatically beautify illustrations drawn by the user, a function to provide an intuitive user interface, a function to allow the user to adjust beautification parameters, and an emotion engine that recognizes the user's emotions. The emotion engine recognizes the user's emotions from the user's facial expressions, tone of voice, text input, etc., and adjusts the beautification parameters according to those emotions. Specifically, it is possible to beautify illustrations according to the user's emotions, such as adjusting the color tone of the illustration to be brighter when the user is feeling happy, and adjusting the color tone to be darker when the user is feeling sad.

[1230] "Example of form 2"

[1231] Furthermore, the emotion engine has the ability to recognize changes in the user's emotions in real time and dynamically adjust the beautification parameters accordingly. For example, if a user starts drawing feeling joy but then becomes sad midway through, the emotion engine will capture this change in real time and immediately adjust the beautification parameters. This makes it possible to beautify illustrations in response to changes in the user's emotions.

[1232] "Example of form 3"

[1233] Furthermore, the emotion engine also has a learning function to recognize the user's emotions. Specifically, it collects data such as the user's facial expressions, voice tone, and text input, and uses machine learning algorithms to train an emotion recognition model. Through this training, the emotion engine becomes able to recognize the user's emotions more accurately, enabling the beautification of illustrations that are more adapted to the user's emotions.

[1234] The following describes the processing flow for each example of the form.

[1235] "Example of form 1"

[1236] Step 1: Input the illustration drawn by the user into the system.

[1237] Step 2: The system uses an emotion engine to recognize the user's emotions based on their facial expressions, tone of voice, text input, etc.

[1238] Step 3: The emotion engine adjusts the beautification parameters according to the recognized emotion.

[1239] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[1240] "Example of form 2"

[1241] Step 1: Input the illustration drawn by the user into the system.

[1242] Step 2: The system uses an emotion engine to recognize the user's emotions in real time based on their facial expressions, tone of voice, text input, etc.

[1243] Step 3: The emotion engine dynamically adjusts the beautification parameters in response to the perceived change in emotion.

[1244] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[1245] "Example of form 3"

[1246] Step 1: Collect data such as the user's facial expressions, voice tone, and text input.

[1247] Step 2: Using the collected data, the emotion engine trains an emotion recognition model using a machine learning algorithm.

[1248] Step 3: Use the trained emotion recognition model to recognize the user's emotions and adjust the beautification parameters accordingly.

[1249] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[1250] (Example 1)

[1251] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1252] Conventional illustration beautification systems failed to consider the user's emotions when automatically beautifying user-submitted illustrations. Therefore, the beautification process did not reflect the user's feelings, making it difficult to increase user satisfaction. Furthermore, if the user interface was not intuitive, it could be complex and difficult to use.

[1253] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes means for acquiring image data representing an illustration drawn by the user, means for automatically adjusting the visual characteristics of the image data, means for recognizing the user's emotions using an emotion engine, a prompt statement instructing the adjustment of the visual characteristics of the image data based on the user's emotions, and means for adjusting the visual characteristics of the image data using a generative AI. This makes it possible to beautify the illustration in accordance with the user's emotions and increase user satisfaction. The server also further includes means for adjusting the visual characteristics of the image data via a user interface that can be operated by the user. The automatic adjustment means adjusts the hue, brightness, and contrast of the image data.

[1254] "Advanced image processing techniques" refer to techniques for adjusting the visual characteristics of an image, such as color tone, brightness, and contrast, and utilize libraries such as OpenCV and TENSORFLOW®.

[1255] "User-generated illustrations" refer to image data created by users using hand-drawn methods or digital tools.

[1256] "Automatic beautification methods" refer to algorithms and programs that automatically adjust the visual characteristics of user-drawn illustrations to improve their appearance.

[1257] A "user interface that users can operate intuitively" is an interface designed to be easily understood and operated by users, and includes UI elements such as sliders and dials.

[1258] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[1259] "Means of recognizing user emotions" refer to algorithms and programs that analyze a user's facial expressions, tone of voice, text input, etc., to determine the user's emotions.

[1260] "Means for adjusting beautification parameters" refer to algorithms or programs that change beautification parameters according to the user's emotions and adjust the visual characteristics of the illustration.

[1261] "Means of returning beautified illustrations to users" refers to communication and display means for providing users with adjusted illustrations.

[1262] This invention relates to a system that automatically beautifies illustrations drawn by users. Specific embodiments of this system are described below.

[1263] System Overview

[1264] This system receives user-drawn illustrations as input, automatically adjusts their visual characteristics such as color tone, brightness, and contrast, and outputs a beautified illustration. Furthermore, it has a function to recognize the user's emotions and adjust the beautification parameters accordingly.

[1265] Hardware and software to be used

[1266] Server: A server with high-performance computing capabilities will be used. The server will receive illustration data from users and perform image processing and emotion recognition.

[1267] Device: A device used by the user, such as a smartphone or PC. The device is used for uploading illustrations and displaying the beautified illustrations.

[1268] software:

[1269] OpenCV: An image processing library. Used to adjust the color tone, brightness, and contrast of illustrations.

[1270] TensorFlow: A machine learning library. Used to build emotion recognition engines.

[1271] Program processing

[1272] The server receives the illustration uploaded by the user. Next, the server automatically adjusts the visual characteristics of the illustration, such as color tone, brightness, and contrast, using OpenCV. Specifically, it performs the following processes:

[1273] Color adjustment: Use the OpenCV cv2.cvtColor function to convert the color space of the illustration and adjust the color tone.

[1274] Brightness adjustment: Use the cv2.convertScaleAbs function to adjust the brightness of the illustration.

[1275] Adjusting contrast: Use the cv2.equalizeHist function to adjust the contrast of the illustration.

[1276] Furthermore, in systems incorporating an emotion engine, the server analyzes the user's facial expressions, tone of voice, and text input to recognize the user's emotions. The emotion engine uses a neural network model built using TensorFlow. If the user is happy, the server adjusts the color tone of the illustration to be brighter; if the user is sad, it adjusts the color tone to be darker.

[1277] Specific example

[1278] As a concrete example, we will show the procedure for uploading an illustration drawn by a user and having the system beautify that illustration.

[1279] User uploads illustration: The user opens the dedicated app on their smartphone, selects the illustration they have drawn, and taps the "Upload" button.

[1280] The server receives the illustrations: The server receives the uploaded illustration files and saves them to the / uploads directory.

[1281] The server adjusts the visual characteristics of the illustration: The server reads the saved illustration file and uses OpenCV to adjust the hue, brightness, and contrast.

[1282] The server uses an emotion engine to recognize the user's emotions: The server analyzes the user's facial expression image and determines that the user is happy.

[1283] The server uses generative AI to beautify illustrations according to emotion: The server uses generative AI and OpenCV to adjust the color tone of the illustrations to be brighter and generate beautified illustrations.

[1284] The server returns the beautified illustration to the user: The server sends the beautified illustration as an HTTP response, which is then displayed on the user's smartphone.

[1285] Example of a prompt

[1286] Users are delighted. Use OpenCV to adjust the color tone, brightness, and contrast of user-drawn illustrations according to their emotions.

[1287] In this way, a system is realized that automatically beautifies illustrations drawn by users and makes adjustments according to the user's emotions.

[1288] The flow of the specific processing in Example 1 will be explained using Figure 17.

[1289] Step 1:

[1290] Users upload illustrations. Users upload their drawings to the system using their devices (smartphones or PCs). Specifically, they select the illustration file via a web browser or dedicated app and click the upload button. The input is the illustration file selected by the user, and the output is the illustration data sent to the server.

[1291] Step 2:

[1292] The server receives the illustration. The server receives the illustration file uploaded by the user. The server temporarily stores the illustration data sent via HTTP request. The input is the illustration data sent by the user, and the output is the illustration file stored on the server.

[1293] Step 3:

[1294] The server adjusts the visual characteristics of the illustration. The server uses the OpenCV library to automatically adjust the illustration's color tone, brightness, contrast, and other visual properties. The specific process is as follows:

[1295] Color Adjustment: The cv2.cvtColor function is used to convert the color space of the illustration and adjust its color tone. The input is an illustration file stored on the server, and the output is the color-adjusted illustration data.

[1296] Brightness adjustment: Use the cv2.convertScaleAbs function to adjust the brightness of the illustration. The input is the color-adjusted illustration data, and the output is the brightness-adjusted illustration data.

[1297] Contrast adjustment: Use the cv2.equalizeHist function to adjust the contrast of the illustration. The input is illustration data with brightness adjusted, and the output is illustration data with contrast adjusted.

[1298] Step 4:

[1299] The server recognizes the user's emotions using an emotion engine. The emotion engine uses a neural network model built using TensorFlow. It analyzes the user's facial expressions, audio data, and text input to determine the user's emotions. The input is the user's facial expressions, audio data, and text input, and the output is the user's emotion data.

[1300] Step 5:

[1301] The server uses generative AI and OpenCV to beautify illustrations according to emotions. The server beautifies the illustrations according to the emotions it recognizes of the user. For example, if the user is happy, the server adjusts the color tone of the illustration to be brighter. Specifically, it changes the color tone adjustment parameters and processes the illustration again using OpenCV. The input is the user's emotion data and the illustration data with adjusted contrast, and the output is the illustration data beautified according to the emotion.

[1302] Step 6:

[1303] The server returns an aesthetically pleasing illustration to the user. Specifically, it sends the illustration data as an HTTP response and displays it on the user's device. The input is an aesthetically pleasing illustration data according to the emotion, and the output is the aesthetically pleasing illustration displayed on the user's device. The server then provides a user interface that the user can operate, specifically an interface for adjusting the visual characteristics of the image data. This user interface includes intuitively operable UI elements such as sliders and dials. This allows the user to make final adjustments to the visual characteristics.

[1304] (Application Example 1)

[1305] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1306] Conventional illustration beautification systems offer the functionality to automatically beautify user-generated illustrations, but they lack the ability to adjust beautification parameters according to the user's emotions, making it difficult to obtain optimal beautification results that match the user's feelings. Furthermore, there is a need to provide a user interface that can be intuitively operated on mobile devices such as smartphones. Therefore, the challenge is to provide a system that allows users to obtain beautification results that are more satisfying.

[1307] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1308] In this invention, the server includes means for automatically beautifying illustrations drawn by the user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for the user to adjust beautification parameters, means for recognizing the user's emotions and adjusting the beautification parameters accordingly, and means for operating as an application installed on a smartphone. This makes it possible to obtain optimal beautification results that correspond to the user's emotions.

[1309] "Advanced image processing technology" refers to technology that automatically adjusts the visual characteristics of an image, such as its hue, brightness, and contrast.

[1310] "User-generated illustrations" refer to images created by users using hand-drawn methods or digital tools.

[1311] "Automatic beautification methods" refer to technologies that improve the visual characteristics of illustrations without the user's intervention.

[1312] A "user interface that users can operate intuitively" is an interface that users can easily understand and operate.

[1313] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[1314] "User-adjustable means" refers to technology that allows users to change beautification parameters according to their preferences.

[1315] "Means of recognizing user emotions" refers to technologies that detect emotions from the user's facial expressions, tone of voice, and other similar factors.

[1316] "Means of adjusting beautification parameters according to emotions" refers to technology that has the function of automatically changing beautification parameters based on the recognized emotions of the user.

[1317] An "application installed on a smartphone" is a software program that runs on a smartphone.

[1318] One embodiment of this invention involves installing the EmotionArt application on a smartphone. The EmotionArt application has the function of automatically beautifying illustrations drawn by the user and further adjusting the visual characteristics of the illustrations according to the user's emotions.

[1319] The server includes the following measures:

[1320] 1. A method for automatically beautifying user-drawn illustrations using advanced image processing technology.

[1321] 2. A means of providing a user interface that users can operate intuitively.

[1322] 3. A means by which the user can adjust the beautification parameters.

[1323] 4. A means of recognizing the user's emotions and adjusting the beautification parameters accordingly.

[1324] 5. A means of operating as an application installed on a smartphone.

[1325] This makes it possible to obtain the optimal beautification result that responds to the user's emotions.

[1326] Hardware and software to be used

[1327] Hardware:

[1328] Smartphone (camera, microphone, display)

[1329] software:

[1330] OpenCV (image processing library)

[1331] EmotionEngine (Emotion Recognition Engine)

[1332] ImageBeautifier (image beautification engine)

[1333] Processing flow

[1334] 1. Image Loading: The user takes a picture of their illustration with their smartphone camera and saves it as an image file.

[1335] 2. Emotion Recognition: Emotions are recognized from the user's facial expressions and tone of voice using EmotionEngine.

[1336] 3. Adjusting Beautification Parameters: Based on the perceived emotion, ImageBeautifier adjusts the visual characteristics of the illustration, such as hue, brightness, and contrast.

[1337] 4. Saving the beautified image: Save the beautified illustration as an image file and provide it to the user.

[1338] Specific example

[1339] The user draws an illustration on their smartphone and uploads it to the app. The app captures the user's facial expression with the camera and recognizes their emotion. If the recognized emotion is "joy," the app adjusts the illustration's color tone to be brighter. The beautified illustration is then displayed to the user and saved.

[1340] Example of a prompt

[1341] We are developing an application that beautifies user-generated illustrations. Please generate a program for an application that adjusts the color tone, brightness, and contrast of the illustration according to the user's emotions. The user's emotions will be recognized from facial expressions and tone of voice. The libraries to be used are OpenCV, EmotionEngine, and ImageBeautifier.

[1342] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[1343] Step 1:

[1344] The user draws an illustration on their smartphone and then takes a picture of the illustration with the camera.

[1345] Input: User-drawn illustration

[1346] Output: Captured illustration image file

[1347] Specific actions: The user launches the smartphone's camera app and takes a picture of the illustration they drew. The captured image is saved to the smartphone's storage.

[1348] Step 2:

[1349] The device reads the captured illustration image file.

[1350] Input: Captured illustration image file

[1351] Output: Image data

[1352] Specific operation: The EmotionArt application on the smartphone reads the image file and stores it in memory as image data.

[1353] Step 3:

[1354] The device uses EmotionEngine to recognize the user's emotions.

[1355] Input: User's facial expressions and tone of voice

[1356] Output: Recognized emotion data

[1357] Specific operation: The smartphone's camera and microphone are used to capture the user's facial expressions and voice tone, and EmotionEngine analyzes this data to recognize the user's emotions.

[1358] Step 4:

[1359] The device adjusts the beautification parameters based on the emotions it recognizes.

[1360] Input: Recognized emotion data

[1361] Output: Adjusted beautification parameters

[1362] Specific operation: The EmotionArt application automatically sets the beautification parameters (hue, brightness, contrast, etc.) of ImageBeautifier based on the recognized emotion data.

[1363] Step 5:

[1364] The device uses ImageBeautifier to beautify the illustration.

[1365] Input: Image data, adjusted beautification parameters

[1366] Output: Beautified illustration image data

[1367] Specific operation: ImageBeautifier uses image data and adjusted beautification parameters to adjust the color tone, brightness, contrast, etc. of the illustration and generate a beautified illustration.

[1368] Step 6:

[1369] The device saves the beautified illustration and displays it to the user.

[1370] Input: Beautified illustration image data

[1371] Output: Saved beautified illustration image file, beautified illustration displayed to the user

[1372] Specific operation: The beautified illustration image data is saved to the smartphone's storage and displayed on the application screen. The user can view the beautified illustration and make further adjustments as needed.

[1373] (Example 2)

[1374] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1375] Conventional illustration beautification systems required users to manually adjust beautification parameters, making the operation cumbersome. Furthermore, they lacked the functionality to dynamically adjust the degree of illustration beautification in response to changes in the user's emotions, making it difficult to beautify illustrations in a way that reflected the user's feelings. This resulted in a decrease in user satisfaction.

[1376] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1377] In this invention, the server includes means for automatically beautifying illustrations drawn by the user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for the user to adjust beautification parameters, and means for recognizing changes in the user's emotions in real time and dynamically adjusting the beautification parameters in accordance with those changes. As a result, the user can beautify illustrations in accordance with their emotions while operating intuitively.

[1378] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing, correcting, and transforming images.

[1379] "User interface" refers to the screens and means of operation that users use to interact with a system.

[1380] "Beautification parameters" refer to settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[1381] A "slider" refers to a UI element that allows users to adjust values ​​by dragging.

[1382] A "dial" refers to a UI element that allows users to adjust values ​​by rotating it.

[1383] "Emotional changes" refers to how a user's emotional state changes over time.

[1384] "Real-time recognition" refers to the instant detection and processing of changes in the user's emotions.

[1385] "Dynamic adjustment" refers to automatically changing the settings according to the situation.

[1386] Modes for carrying out the invention

[1387] This invention is a system that automatically beautifies illustrations drawn by users and dynamically adjusts the beautification parameters according to changes in the user's emotions. This system consists of three main elements: a server, a terminal, and a user.

[1388] server

[1389] The server performs the following functions:

[1390] 1. Providing a user interface

[1391] The server generates a user interface (UI) and sends it to the terminal. This UI includes intuitively operable elements such as sliders and dials.

[1392] Software used: UI libraries such as React and Vue.js.

[1393] 2. Receiving user actions

[1394] The server receives user operation data transmitted from the terminal. This includes things like the slider position and dial angle.

[1395] Software used: WebSocket, HTTP requests.

[1396] 3. Calculation of beautification parameters

[1397] The server calculates beautification parameters based on the received operation data. This determines the degree of beautification of the illustration.

[1398] Software to be used: Programming languages ​​such as Python and Node.js.

[1399] 4. Analysis of emotional data

[1400] The server receives and analyzes user emotion data transmitted from the terminal. It uses an emotion engine to recognize changes in the user's emotions in real time.

[1401] Software used: Sentiment analysis engines such as Microsoft Azure Emotion API.

[1402] 5. Dynamic adjustment of beautification parameters

[1403] The server dynamically adjusts the beautification parameters based on the analyzed emotion data. This makes it possible to beautify illustrations in accordance with the user's emotions.

[1404] terminal

[1405] The device performs the following functions:

[1406] 1. Display of the user interface

[1407] The terminal displays the user interface received from the server. This includes UI elements such as sliders and dials.

[1408] Hardware to use: PC, tablet, smartphone.

[1409] Software used: Browser, HTML, JavaScript.

[1410] 2. Sending user actions

[1411] The device sends operation data to the server when the user operates the slider or dial.

[1412] Software used: JavaScript event listeners, WebSocket, HTTP requests.

[1413] 3. Collection of emotional data

[1414] The device uses sensors such as cameras and microphones to collect user emotion data.

[1415] Software used: WebRTC, MediaStream API.

[1416] User

[1417] The user performs the following actions:

[1418] 1. User Interface Operation

[1419] Users adjust the illustration's beautification parameters by manipulating sliders and dials on their device.

[1420] Specific actions: The user uses a mouse or touchscreen to drag sliders or rotate dials.

[1421] 2. Changes in emotions

[1422] The user's emotions change during the process of creating an illustration. These emotional changes are collected by the device and sent to the server.

[1423] Specific actions: Emotional data changes when the user smiles or frowns.

[1424] Specific example

[1425] Hardware and software to be used

[1426] Hardware: PCs, tablets, smartphones

[1427] Software: User interface (UI) libraries (e.g., React, Vue.js), sentiment analysis engines (e.g., Microsoft Azure Emotion API)

[1428] Example of a prompt

[1429] "Please describe a system where the user adjusts the illustration's beautification parameters using sliders. Also, please include a feature that dynamically adjusts the beautification parameters in response to changes in the user's emotions."

[1430] By inputting this prompt into the generating AI model, a detailed explanation of the system can be obtained.

[1431] The flow of the specific processing in Example 2 will be explained using Figure 19.

[1432] Step 1:

[1433] Providing a user interface

[1434] The server generates a user interface (UI) and sends it to the device. It uses a UI library (e.g., React, Vue.js) as input and generates HTML and JavaScript code as output. Specifically, the server uses the UI library to build a UI containing intuitively operable elements such as sliders and dials, and then sends this to the device.

[1435] Step 2:

[1436] Display of user interface

[1437] The device displays the user interface received from the server. It receives HTML and JavaScript code from the server as input and renders the UI in the browser as output. Specifically, the device uses the browser to parse the received code and display the UI, including sliders and dials.

[1438] Step 3:

[1439] Sending User Actions

[1440] The device sends operation data to the server when the user operates a slider or dial. It receives user operation events as input and sends operation data to the server as output. Specifically, the device uses JavaScript event listeners to capture user operations and sends them to the server via WebSocket or HTTP requests.

[1441] Step 4:

[1442] Receiving user actions

[1443] The server receives user interaction data sent from the terminal. It receives interaction data from the terminal as input and uses it as output to calculate beautification parameters. Specifically, the server receives user interaction data in real time via WebSocket or HTTP requests.

[1444] Step 5:

[1445] Calculation of beautification parameters

[1446] The server calculates beautification parameters based on the received operation data. It takes user operation data as input and generates beautification parameters as output. Specifically, the server uses programming languages ​​such as Python or Node.js to execute the beautification algorithm and calculate the parameters.

[1447] Step 6:

[1448] Collection of emotional data

[1449] The device collects user emotion data using sensors such as cameras and microphones. It receives sensor data from the camera and microphone as input and sends emotion data to the server as output. Specifically, the device uses WebRTC or MediaStream APIs to collect data from the camera and microphone and send it to the server.

[1450] Step 7:

[1451] Analysis of emotional data

[1452] The server receives and analyzes user emotion data transmitted from the terminal. It takes emotion data from the terminal as input and generates analysis results as output. Specifically, the server uses an emotion analysis engine, such as the Microsoft Azure Emotion API, to analyze the user's emotion data.

[1453] Step 8:

[1454] Dynamic adjustment of beautification parameters

[1455] The server dynamically adjusts the beautification parameters based on the analyzed sentiment data. It receives the results of the sentiment analysis as input and generates updated beautification parameters as output. Specifically, the server recalculates and updates the parameters of the beautification algorithm based on the sentiment analysis results.

[1456] Step 9:

[1457] Illustration update

[1458] The terminal updates the illustration in real time based on updated beautification parameters received from the server. It receives updated beautification parameters from the server as input and displays the updated illustration as output. Specifically, the terminal uses a browser to redraw the illustration based on the received beautification parameters.

[1459] (Application Example 2)

[1460] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[1461] Conventional illustration beautification systems require users to manually adjust beautification parameters, making it difficult to dynamically beautify images in response to changes in the user's emotions. Furthermore, they lacked intuitive user interfaces, hindering improvements in the user experience.

[1462] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1463] In this invention, the server includes means for automatically beautifying illustrations drawn by the user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for the user to adjust beautification parameters, and means for recognizing the user's emotions in real time and dynamically adjusting the beautification parameters according to those emotions. This enables dynamic illustration beautification in response to changes in the user's emotions, and improves the user experience through intuitive operation.

[1464] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing, transforming, and correcting images.

[1465] "User-generated illustrations" refer to visual works created by users manually or using digital tools.

[1466] "Automatic beautification methods" refer to technologies and algorithms that improve the visual characteristics of illustrations without user intervention.

[1467] A "user interface that users can operate intuitively" refers to an interface designed to allow users to operate it easily and naturally.

[1468] "Beautification parameters" refer to settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[1469] "User-adjustable means" refers to features or interfaces that allow users to change settings according to their preferences.

[1470] "Means of recognizing user emotions in real time" refers to technologies and systems that analyze a user's facial expressions and actions to instantly determine their emotions at any given moment.

[1471] "Means of dynamically adjusting beautification parameters in response to emotions" refers to technologies and algorithms that automatically change beautification parameters based on the recognized emotions of the user.

[1472] The following system configuration will be described as an embodiment for carrying out this invention.

[1473] System Configuration

[1474] This system recognizes emotions in real time as users draw illustrations using their smartphones and dynamically adjusts the illustration's beautification parameters accordingly. The system consists of the following main components:

[1475] 1. Smartphone

[1476] Camera: Captures the user's face and provides data for emotion recognition.

[1477] Touchscreen: Provides an interface for users to draw illustrations.

[1478] Processor: Performs image processing, emotion recognition, and UI operations.

[1479] 2. Software Components

[1480] OpenCV: An image processing library. Used for beautifying illustrations.

[1481] EmotionRecognizer: An emotion recognition library. It recognizes the user's emotions in real time.

[1482] UI Elements: A library that provides intuitively operable UI elements such as sliders and dials.

[1483] Program processing

[1484] The server implements the system using the following methods.

[1485] 1. A means of automatically beautifying user-drawn illustrations using advanced image processing technology.

[1486] Using OpenCV, we will perform beautification processing on user-submitted illustrations. Specifically, we will apply filters such as Gaussian blur to improve the visual characteristics of the illustrations.

[1487] 2. Means for providing a user interface that users can operate intuitively.

[1488] The UI Elements library is used to provide intuitively operable UI elements such as sliders and dials. This allows users to easily adjust aesthetic parameters.

[1489] 3. Means that allow users to adjust beautification parameters.

[1490] The system allows users to adjust beautification parameters to their liking using sliders or dials. For example, parameters such as hue, brightness, and contrast can be adjusted.

[1491] 4. A means of recognizing user emotions in real time and dynamically adjusting beautification parameters according to those emotions.

[1492] EmotionRecognizer is used to capture the user's face and recognize their emotions in real time. Based on the recognized emotions, beautification parameters are dynamically adjusted. For example, if the user is feeling "joy," the beautification parameters are set high, and if they are feeling "sadness," they are set low.

[1493] Specific example

[1494] For example, when a user is drawing an illustration on their smartphone, if the camera captures the user's face and the emotion recognition library detects "joy," the beautification parameter will be set high, making the illustration more vibrant and brighter. Conversely, if "sadness" is detected, the beautification parameter will be set low, making the illustration slightly darker and more subdued.

[1495] Example of a prompt

[1496] When a user is drawing an illustration on their smartphone, if the camera captures the user's face and the emotion recognition library detects "joy," the beautification parameter is set high, making the illustration more vibrant and brighter. Conversely, if "sadness" is detected, the beautification parameter is set low, making the illustration slightly darker and more subdued.

[1497] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[1498] Step 1:

[1499] The device uses a camera to capture the user's face. The input is the user's face image, and the output is the captured face image data. This face image data is used as preprocessing for emotion recognition.

[1500] Step 2:

[1501] The device uses an emotion recognition library (EmotionRecognizer) to recognize the user's emotions in real time from captured facial image data. The input is facial image data, and the output is the recognized emotion (e.g., joy, sadness, surprise, etc.). This emotion data is used to adjust subsequent beautification parameters.

[1502] Step 3:

[1503] The device allows users to draw illustrations using a touchscreen. Input is the user's touch operation, and output is the drawn illustration data. This illustration data is subject to beautification processing.

[1504] Step 4:

[1505] The device uses a user interface (UI elements) to display sliders and dials, allowing the user to adjust beautification parameters. Input is the user's manipulation of the sliders and dials, and output is the adjusted beautification parameters. These parameters are used to beautify the illustration.

[1506] Step 5:

[1507] The device dynamically adjusts the beautification parameters based on recognized emotion data. The input is the emotion data and the beautification parameters adjusted by the user, and the output is the final adjusted beautification parameters. For example, if the user is feeling "joy," the beautification parameters are set high, and if they are feeling "sadness," they are set low.

[1508] Step 6:

[1509] The device uses OpenCV to perform illustration beautification based on the final adjusted beautification parameters. The input is the illustration data and the adjusted beautification parameters, and the output is the beautified illustration data. Specifically, it applies filters such as Gaussian blur to improve the visual characteristics of the illustration.

[1510] Step 7:

[1511] The device displays an enhanced illustration to the user. The input is enhanced illustration data, and the output is an enhanced illustration that the user can visually confirm. This allows the user to see the enhanced illustration in real time.

[1512] (Example 3)

[1513] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1514] Conventional illustration beautification systems allow users to adjust beautification parameters, but they cannot beautify illustrations in accordance with the user's emotions. Therefore, it was difficult to achieve illustration beautification that adapted to the user's feelings, making it challenging to increase user satisfaction. Furthermore, while systems with emotion recognition capabilities exist, the technology to apply this to illustration beautification had not yet been established.

[1515] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[1516] In this invention, the server includes means for collecting user emotion data, means for analyzing the collected emotion data and training an emotion recognition model, and means for beautifying illustrations according to the user's emotions. This makes it possible to beautify illustrations that are adapted to the user's emotions.

[1517] "Advanced image processing technology" refers to techniques for adjusting the visual characteristics of an image, such as its hue, brightness, and contrast.

[1518] A "user interface" is the interface through which a user operates a system, and includes intuitively operable UI elements such as sliders and dials.

[1519] "Beautification parameters" are parameters used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[1520] "User emotional data" refers to emotional data collected from the user's facial expressions, tone of voice, text input, and other sources.

[1521] An "emotion recognition model" is a model trained using machine learning algorithms to recognize a user's emotions.

[1522] "Illustration beautification" is the process of making user-created illustrations more appealing by adjusting their visual characteristics such as color tone, brightness, and contrast.

[1523] Modes for carrying out the invention

[1524] This invention is a system that automatically beautifies user-drawn illustrations using advanced image processing technology and further adjusts the beautification according to the user's emotions. A specific embodiment of this system is described below.

[1525] Hardware and software to be used

[1526] Hardware: User's device (PC, smartphone, etc.), camera, microphone

[1527] Software: Web browsers, dedicated applications, image processing libraries (such as OpenCV), machine learning libraries (such as TensorFlow and PyTorch)

[1528] Program processing

[1529] Adjusting the beautification parameters

[1530] The user inputs beautification parameters such as "color intensity" and "brightness adjustment" through the device's interface. For example, using a smartphone app, the user might move a slider to set the color intensity to 50. The device sends these parameters to the server, which then performs beautification processing on the illustration based on the received parameters.

[1531] Collection and analysis of emotional data

[1532] The user's device uses its camera and microphone to collect the user's facial expressions and voice tone. For example, the camera captures the user's face to obtain facial expression data. The device sends the collected emotion data to a server, which uses a machine learning algorithm to train an emotion recognition model. The trained model recognizes the user's emotions and beautifies the illustration accordingly.

[1533] Beautification and display of illustrations

[1534] The server uses a trained emotion recognition model to beautify the illustration according to the user's emotions. For example, if the user is smiling, warmer tones are added to the illustration. Finally, the beautified illustration is sent to the device and displayed to the user.

[1535] Specific example

[1536] The user opens the app on their smartphone and sets the color intensity to 50 and the brightness adjustment to 20. The device sends these parameters to the server, which then beautifies the illustration based on the received parameters. Next, if the user is smiling through the camera, the device sends that facial expression data to the server. The server learns an emotion recognition model and adds warm tones to the illustration according to the user's emotion. Finally, the beautified illustration is sent to the device and displayed to the user.

[1537] Example of a prompt

[1538] "Please describe a system where the user sets parameters for beautifying an illustration, and an emotion engine recognizes the user's emotions to beautify the illustration. Please include the names of specific hardware and software."

[1539] Thus, this invention provides a function that beautifies illustrations based on user input and emotions. The flow of the specific processing in Example 3 will be explained with reference to Figure 21.

[1540] Step 1:

[1541] The user enters beautification parameters.

[1542] Users input beautification parameters such as "color intensity" and "brightness adjustment" through the device's interface. For example, they might use a smartphone app to move a slider to set the color intensity to 50. The entered parameters are saved on the device.

[1543] Step 2:

[1544] The device sends beautification parameters to the server.

[1545] The terminal sends the beautification parameters entered by the user to the server using an HTTP request. Specifically, parameters such as a color intensity of 50 and a brightness adjustment of 20 are sent. The server receives this and proceeds to the next step.

[1546] Step 3:

[1547] The server receives the beautification parameters and beautifies the illustration.

[1548] The server uses an image processing library (such as OpenCV) to beautify the illustration based on the received beautification parameters. For example, it might set the color intensity to 50 and adjust the brightness to 20. The beautified illustration is then saved to the server.

[1549] Step 4:

[1550] Collect user sentiment data.

[1551] The user's device uses its camera and microphone to collect data on their facial expressions and voice tone. For example, the camera can capture a photo of the user's face to obtain facial expression data. The collected emotional data is stored on the device.

[1552] Step 5:

[1553] The device sends emotional data to the server.

[1554] The device sends the collected emotional data to the server using an HTTP request. Specifically, it sends the user's facial expression data and voice tone data. The server receives this data and proceeds to the next step.

[1555] Step 6:

[1556] The server analyzes the emotion data and trains an emotion recognition model.

[1557] The server analyzes the received emotion data and trains an emotion recognition model using machine learning algorithms (such as TensorFlow or PyTorch). For example, it can determine whether the user is smiling based on facial expression data. The trained model is stored on the server.

[1558] Step 7:

[1559] The server beautifies the illustrations according to the user's emotions.

[1560] The server uses a trained emotion recognition model to beautify illustrations according to the user's emotions. For example, if the user is smiling, warmer tones are added to the illustration. The beautified illustration is then saved on the server.

[1561] Step 8:

[1562] The server sends the beautified illustration to the terminal.

[1563] The server sends the beautified illustration to the terminal using an HTTP response. Specifically, the beautified illustration data is sent. The terminal receives this and proceeds to the next step.

[1564] Step 9:

[1565] The device displays an aesthetically pleasing illustration to the user.

[1566] The device displays the received, beautified illustration to the user. For example, the illustration is displayed in a smartphone app. The user can then see the beautified illustration.

[1567] (Application Example 3)

[1568] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 as a "terminal".

[1569] Conventional image beautification systems had the problem that it was difficult for users to beautify illustrations according to their own emotions and preferences. Furthermore, the lack of an intuitive interface for adjusting beautification parameters often made the process cumbersome. In addition, the absence of technology to recognize and beautify illustrations based on user emotions meant that optimal beautification tailored to the user's feelings could not be achieved.

[1570] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1571] In this invention, the server includes means for automatically beautifying illustrations drawn by the user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for allowing the user to adjust beautification parameters, means including an emotion recognition engine for recognizing the user's emotions, and means for beautifying the illustration based on the recognized emotions. This allows the user to beautify illustrations according to their emotions and preferences, and to easily adjust the beautification parameters through an intuitive interface. Furthermore, by using an emotion recognition engine, optimal beautification according to the user's emotions can be achieved.

[1572] "Advanced image processing technology" refers to sophisticated algorithms and techniques for beautifying images by adjusting their visual characteristics, such as hue, brightness, and contrast.

[1573] "User interface" is a general term for the screens and input devices that users use to operate a system, and includes UI elements such as sliders and dials that can be operated intuitively.

[1574] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast, and can be freely adjusted by the user.

[1575] An "emotion recognition engine" is a system that collects data such as the user's facial expressions, voice tone, and text input, and uses machine learning algorithms to recognize the user's emotions.

[1576] "A method for beautifying illustrations based on recognized emotions" is a technique that automatically adjusts the illustration's beautification parameters according to the user's emotions recognized by an emotion recognition engine, thereby achieving optimal beautification.

[1577] A system for carrying out this invention includes advanced image processing technology for automatically beautifying illustrations drawn by a user, a user interface that can be operated intuitively by the user, means for the user to adjust beautification parameters, an emotion recognition engine for recognizing the user's emotions, and means for beautifying the illustration based on the recognized emotions.

[1578] System Configuration

[1579] Hardware: Smartphone

[1580] Software: Python, OpenCV, Keras, emotion recognition library

[1581] Program Processing Description

[1582] 1. Image Loading: Load images taken by the user from the smartphone's camera or gallery. Image data is obtained using OpenCV.

[1583] 2. Emotion Recognition: Using an emotion recognition model trained on Keras, we recognize emotions from the user's facial expressions, tone of voice, text input, etc. We collect and analyze user emotion data using an emotion recognition library.

[1584] 3. Adjusting Beautification Parameters: Provide intuitive UI elements such as sliders and dials for adjusting the intensity and brightness of the color tone specified by the user. Users can freely adjust the beautification parameters by manipulating these UI elements.

[1585] 4. Emotion-Based Beautification: The image beautification parameters are automatically adjusted based on the recognized emotion. For example, if the user is perceived as "happy," the image will be beautified by setting the color intensity to 1.2 and the brightness to 30.

[1586] 5. Saving the results: Save the beautified image to the smartphone's gallery. Use OpenCV to save the processed image data.

[1587] Specific example

[1588] Input image: Landscape photo taken by the user

[1589] Emotion: When the user is perceived as "happy"

[1590] Beautification parameters: Saturation 1.2, Brightness 30

[1591] Example of a prompt

[1592] If a landscape photo taken by the user is used as the input image and the emotion engine recognizes it as "happy," please beautify it by setting the color intensity to 1.2 and the brightness to 30.

[1593] In this way, users can beautify illustrations according to their emotions and preferences, and easily adjust the beautification parameters through an intuitive interface. Furthermore, by using an emotion recognition engine, optimal beautification tailored to the user's emotions can be achieved.

[1594] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[1595] Step 1:

[1596] The device reads images taken by the user from the smartphone's camera or gallery. Specifically, it uses OpenCV to acquire image data. The input is an image file selected by the user, and the output is an array of image data.

[1597] Step 2:

[1598] The server uses an emotion recognition model trained with Keras to recognize emotions from the user's facial expressions, tone of voice, text input, etc. Specifically, it uses an emotion recognition library to collect and analyze the user's emotion data. The input is an array of image data, and the output is the label of the recognized emotion.

[1599] Step 3:

[1600] The device provides intuitive UI elements such as sliders and dials for adjusting the intensity and brightness of the color tones specified by the user. By manipulating these UI elements, the user can freely adjust the beautification parameters. The input is the user's actions, and the output is the adjusted beautification parameters.

[1601] Step 4:

[1602] The server automatically adjusts image beautification parameters based on the recognized emotion. Specifically, it sets the color intensity and brightness according to the emotion recognized by the emotion recognition engine. The input is the recognized emotion label and the user-adjusted beautification parameters, and the output is the optimized beautification parameters.

[1603] Step 5:

[1604] The device beautifies images based on beautification parameters. Specifically, it uses OpenCV to adjust the intensity and brightness of the colors and process the images. The input is an array of optimized beautification parameters and image data, and the output is the beautified image data.

[1605] Step 6:

[1606] The device saves the beautified image to the smartphone's gallery. Specifically, it uses OpenCV to save the processed image data. The input is the beautified image data, and the output is the saved image file.

[1607] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1608] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1609] Other examples of generative AI include Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are some examples.

[1610] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1611] [Third Embodiment]

[1612] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1613] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1614] The data processing device 12 includes a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a “computer” related to the technology of this disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32.

[1615] The processor 28, RAM 30, and storage 32 are connected to the bus 34. The database 24 and communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to the network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1616] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1617] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1618] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1619] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1620] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1621] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1622] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1623] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1624] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[1625] "Example of form 1"

[1626] One embodiment of the present invention provides a system that automatically beautifies illustrations drawn by a user. This system has the function of automatically beautifying illustrations drawn by a user using advanced image processing technology. Specifically, it receives an illustration drawn by a user as input, automatically adjusts the visual characteristics of the illustration such as color tone, brightness, and contrast, and outputs a beautified illustration.

[1627] "Example of form 2"

[1628] Furthermore, as one embodiment of the present invention, an intuitively operable user interface is provided. This user interface includes intuitively operable UI elements such as sliders and dials. Specifically, the user can adjust the beautification parameters by operating the sliders and dials. This allows the user to adjust the degree of beautification of the illustration to their liking.

[1629] "Example of form 3"

[1630] Furthermore, in one embodiment of the present invention, a means is provided that allows the user to adjust the beautification parameters. Specifically, the user can freely adjust the degree of beautification of the illustration by directly inputting the beautification parameters. For example, the user can beautify the illustration to their liking by adjusting parameters such as "color intensity" and "brightness adjustment".

[1631] The following describes the processing flow for each example form.

[1632] "Example of form 1"

[1633] Step 1: Input the illustration drawn by the user into the system. This input can be done via file upload, drag and drop, etc.

[1634] Step 2: The system analyzes the input illustration and automatically adjusts its visual characteristics such as hue, brightness, and contrast. This adjustment is performed using a pre-trained machine learning model.

[1635] Step 3: The system outputs the adjusted illustration. The output can be displayed directly on the screen or downloaded as a file.

[1636] "Example of form 2"

[1637] Step 1: The user operates the system's user interface. Specifically, they adjust the beautification parameters by manipulating sliders and dials.

[1638] Step 2: The system adjusts the beautification parameters according to the user's input and beautifies the illustration.

[1639] Step 3: The system outputs the beautified illustration. The output can be displayed directly on the screen or downloaded as a file.

[1640] "Example of form 3"

[1641] Step 1: The user directly inputs the beautification parameters. Specifically, parameters such as "color intensity" and "brightness adjustment" are specified using numerical values ​​or sliders.

[1642] Step 2: The system beautifies the illustration based on the beautification parameters entered by the user.

[1643] Step 3: The system outputs the beautified illustration. The output can be displayed directly on the screen or downloaded as a file.

[1644] (Example 1)

[1645] Next, we will describe Embodiment 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1646] Traditional illustration beautification systems required users to manually adjust color tone, brightness, and contrast, resulting in a cumbersome operation. Furthermore, they lacked an intuitive user interface, making the beautification process difficult for users. Additionally, the lack of real-time progress displays and completion notifications resulted in a poor user experience.

[1647] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes means for the user to upload an illustration, means for the server to receive the illustration, means for the server to adjust the color tone, means for the server to adjust the brightness, means for the server to adjust the contrast, means for the server to generate a beautified illustration, and means for the server to provide the beautified illustration to the user. This makes it possible for the user to easily beautify an illustration through an intuitive interface. In addition, the user experience is improved by displaying the progress in real time and notifying the user when the processing is complete.

[1648] "Advanced image processing technology" refers to technology that automatically adjusts the visual characteristics of an image, such as its hue, brightness, and contrast.

[1649] A "user interface" is an interface that includes screens and input devices for a user to operate a system.

[1650] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[1651] "Means for users to upload illustrations" refers to a function that allows users to send illustration files from their devices to the system.

[1652] "The means by which the server receives illustrations" refers to the function that allows the server to receive and save illustration files sent by users.

[1653] "Means for the server to adjust the color tone" refers to a function that allows the server to automatically change the hue and saturation of an illustration.

[1654] "A means for the server to adjust brightness" refers to a function that allows the server to automatically change the brightness of the illustration.

[1655] "A means for the server to adjust contrast" refers to a function that allows the server to automatically change the difference in brightness between light and dark areas in an illustration.

[1656] "Means for the server to generate beautified illustrations" refers to a function that allows the server to create new illustrations after adjusting the color tone, brightness, and contrast.

[1657] "Means by which the server provides beautified illustrations to users" refers to a function that provides users with beautified illustrations generated by the server in a downloadable format.

[1658] This invention relates to a system that automatically beautifies illustrations drawn by a user. The system provides an intuitive user interface and allows the user to adjust beautification parameters for adjusting visual characteristics such as hue, brightness, and contrast.

[1659] Hardware and software to be used

[1660] The server is a computer system for running Python programs, using OpenCV and Pillow as image processing libraries. Users access the system using internet-connected devices (such as PCs, smartphones, and tablets).

[1661] Program Processing Description

[1662] The server receives the illustration uploaded by the user. Next, the server adjusts the color tone of the image using OpenCV. Specifically, it converts the image to HSV (Hue, Saturation, Value) space, adjusts the hue appropriately, and then converts it back to RGB space.

[1663] The server then uses Pillow to adjust the image brightness. Specifically, it adjusts the brightness using Pillow's ImageEnhance.Brightness class. Next, it adjusts the contrast using Pillow's ImageEnhance.Contrast class.

[1664] After these processes, an enhanced illustration is generated and provided to the user. The user can obtain the enhanced illustration by clicking the download link on their device.

[1665] Specific example

[1666] As a concrete example, let's consider the case of beautifying an illustration "example_illustration.png" drawn by a user.

[1667] 1. User actions:

[1668] The user uploads "example_illustration.png" from their device to the system.

[1669] 2. Server processing:

[1670] The server receives the uploaded image and adjusts its color tone, brightness, and contrast.

[1671] Save the adjusted image as "beautified_illustration.png".

[1672] 3. Output to the user:

[1673] The server provides the user with a beautified illustration, "beautified_illustration.png".

[1674] An example of a prompt message is as follows:

[1675] Please upload your user-created illustration, "example_illustration.png". The system will automatically adjust the color tone, brightness, and contrast to provide an enhanced illustration. You will receive a notification when processing is complete.

[1676] In this way, the system works to automatically beautify illustrations drawn by users.

[1677] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1678] Step 1:

[1679] The user uploads an illustration.

[1680] Users upload their own illustration files (e.g., "example_illustration.png") from their terminal via the system's web interface. The input is the user's illustration file, and the output is the illustration file sent to the server.

[1681] Step 2:

[1682] The server receives the illustration.

[1683] The server receives illustration files uploaded by users and saves them to a specified directory. The input is the illustration file sent by the user, and the output is the illustration file saved on the server. Specifically, the server checks the file format and size and verifies that it is in an appropriate format.

[1684] Step 3:

[1685] The server adjusts the color tone.

[1686] The server loads the illustration using the OpenCV cv2.imread function. Next, it converts the image to HSV (Hue, Saturation, Value) space using the cv2.cvtColor function. It then adjusts the hue appropriately and converts it back to RGB space. The input is the illustration file stored on the server, and the output is the color-adjusted illustration. Specifically, the hue is shifted by 10 degrees.

[1687] Step 4:

[1688] The server adjusts the brightness.

[1689] The server uses Pillow's ImageEnhance.Brightness class to adjust the brightness of the image. Specifically, it increases the brightness by 1.2 times. The input is an illustration with adjusted color tone, and the output is an illustration with adjusted brightness.

[1690] Step 5:

[1691] The server adjusts the contrast.

[1692] The server uses Pillow's ImageEnhance.Contrast class to adjust the contrast of the image. Specifically, it increases the contrast by 1.3 times. The input is an illustration with adjusted brightness, and the output is an illustration with adjusted contrast.

[1693] Step 6:

[1694] The server generates an aesthetically pleasing illustration.

[1695] The server saves the adjusted image as "beautified_illustration.png". The input is an illustration with adjusted contrast, and the output is a beautified illustration file.

[1696] Step 7:

[1697] The server provides users with an enhanced version of the illustration.

[1698] The server provides the user with a beautified illustration file, "beautified_illustration.png". The user can obtain the beautified illustration by clicking the download link from their device. The input is the beautified illustration file, and the output is the download link provided to the user. Specifically, the server displays the progress in real time and sends a notification to the user when the process is complete.

[1699] (Application Example 1)

[1700] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1701] Traditional illustration beautification systems offer the functionality to automatically beautify user-submitted illustrations, but they lack a convenient way to share these beautified illustrations with content distribution services. Furthermore, the customization options for the beautification process are limited, making it difficult for users to adjust the process to their own preferences.

[1702] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1703] In this invention, the server includes means for automatically beautifying user-drawn illustrations using advanced image processing technology, means for providing an intuitive user interface, means for allowing the user to adjust beautification parameters, means for directly sharing the beautified illustrations with content distribution services, and means for providing customization options for the beautification process. This makes it possible for users to easily share beautified illustrations and further customize the beautification process to their liking.

[1704] "Advanced image processing technology" refers to technology that automatically adjusts the visual characteristics of an image, such as its hue, brightness, and contrast.

[1705] "User-generated illustrations" refer to paintings or drawings created by users, either by hand or using digital tools.

[1706] "Methods for automatically beautifying" refer to algorithms and software that improve the visual characteristics of illustrations without user intervention.

[1707] A "user interface that users can operate intuitively" is an interface designed to allow users to operate it easily and intuitively.

[1708] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[1709] "User-adjustable means" refers to a feature that allows users to manually change beautification parameters.

[1710] A "content distribution service" is a service that distributes digital content such as images and videos over the internet.

[1711] "Shareable means" refers to features that allow users to share the beautified illustrations with other users or platforms.

[1712] "Customization options" are settings and features that allow users to adjust the beautification process to their own preferences.

[1713] The following system can be constructed as an embodiment of this invention.

[1714] System Overview

[1715] This system automatically beautifies user-generated illustrations and shares those beautified illustrations with content distribution services. The system includes a smartphone, a server, and software utilizing advanced image processing technology.

[1716] Hardware and software to be used

[1717] Hardware: Smartphone (iOS or Android), Server

[1718] Software: Python, OpenCV (image processing library)

[1719] Data processing and data calculation

[1720] 1. Loading images

[1721] Users take photos of or upload illustrations they have drawn on their smartphones. This utilizes the smartphone's camera function and file upload function.

[1722] 2. Adjusting the color tone

[1723] The server converts the image to the HSV color space and increases its saturation. This makes the illustration's colors more vibrant.

[1724] 3. Adjusting Brightness

[1725] The server performs a scaling transformation to increase the brightness of the image. This makes the illustration brighter.

[1726] 4. Adjust the contrast

[1727] The server converts the image to the LAB color space and performs histogram equalization of the L channel to improve contrast. This enhances the visual characteristics of the illustration.

[1728] 5. Saving and sharing images

[1729] The beautified images are stored on the server, and users can share them directly to content distribution services via smartphone applications.

[1730] Specific example

[1731] Users take a photo of an illustration they drew on their smartphone and upload it to the app, whereupon the server automatically beautifies the illustration. The beautified illustration can then be adjusted in terms of color tone, brightness, and contrast to suit the user's preferences. Finally, the beautified illustration can be shared on social media and other content distribution services.

[1732] Example of a prompt

[1733] "Please develop an app that beautifies user-submitted illustrations. The app should automatically adjust the color tone, brightness, and contrast of the illustrations, and output the beautified versions."

[1734] In this way, a system can be implemented that allows users to easily create and share beautified illustrations.

[1735] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1736] Step 1:

[1737] The user takes a picture of an illustration they drew on their smartphone or uploads it. The input is the illustration image drawn by the user, and the output is the image file saved on the smartphone. Specifically, the user takes a picture of the illustration using the smartphone's camera function or uploads an existing image file to the application.

[1738] Step 2:

[1739] The device sends captured or uploaded illustration images to the server. The input is an image file stored on the smartphone, and the output is the image data transferred to the server. Specifically, the smartphone application calls an API to send the image file to the server.

[1740] Step 3:

[1741] The server reads the received illustration image. The input is the image data transferred to the server, and the output is the image object loaded into memory. Specifically, the server reads the image file using Python and OpenCV.

[1742] Step 4:

[1743] The server adjusts the color tone of the image. The input is an image object loaded into memory, and the output is an image object with adjusted color tone. Specifically, the server converts the image to the HSV color space and increases its saturation.

[1744] Step 5:

[1745] The server adjusts the brightness of the image. The input is a color-adjusted image object, and the output is a brightness-adjusted image object. Specifically, the server performs a scaling transformation to increase the brightness of the image.

[1746] Step 6:

[1747] The server adjusts the contrast of the image. The input is an image object with adjusted brightness, and the output is an image object with adjusted contrast. Specifically, the server converts the image to the LAB color space and performs histogram equalization of the L channel.

[1748] Step 7:

[1749] The server saves the beautified image. The input is an image object with adjusted contrast, and the output is a beautified image file saved on the server. Specifically, the server saves the beautified image as a file.

[1750] Step 8:

[1751] The device retrieves a beautified image from the server. The input is a beautified image file stored on the server, and the output is a beautified image file downloaded to the smartphone. Specifically, the smartphone application calls an API to download the beautified image from the server.

[1752] Step 9:

[1753] Users share beautified illustrations to content distribution services. The input is a beautified image file downloaded to a smartphone, and the output is a beautified image uploaded to a content distribution service. Specifically, users share beautified illustrations to social media and other content distribution services using a smartphone application.

[1754] (Example 2)

[1755] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1756] Conventional image processing systems required specialized knowledge and complex operations to beautify user-generated illustrations, lacking an intuitive user interface. Furthermore, adjusting beautification parameters was difficult, making it challenging for users to customize their illustrations to their preferences. Additionally, the beautification process using generative AI models was not performed in real time, resulting in delayed feedback from user input.

[1757] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1758] In this invention, the server includes means for converting beautification parameters operated by the user into prompt sentences, means for inputting the prompt sentences into a generation AI model to beautify the illustration, and means for displaying the beautified illustration to the user. This allows the user to adjust the beautification parameters and view the beautified illustration in real time through an intuitively operable user interface.

[1759] "Advanced image processing technology" is a general term for advanced algorithms and techniques used to automatically beautify illustrations drawn by users.

[1760] "User interface" is a general term for the screens and input devices that users use to operate a system, and includes intuitively operable UI elements.

[1761] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, such as its color tone, brightness, and contrast.

[1762] A "prompt message" is a text-based instruction used to input the beautification parameters set by the user into the generating AI model.

[1763] A "generative AI model" is an artificial intelligence model used to beautify illustrations based on input prompt text, and includes, for example, "Stable Diffusion" and "DALL-E".

[1764] A "slider" is a UI element that allows users to adjust values ​​by dragging.

[1765] A "dial" is a UI element that allows users to adjust values ​​by rotating it.

[1766] A "server" is a computer system that receives prompt messages, inputs them into a generation AI model, generates an aesthetically pleasing illustration, and sends it back to the terminal.

[1767] A "terminal" is a device operated by a user, which provides a user interface, retrieves beautification parameters, generates prompt messages, and sends them to the server.

[1768] This invention is a system that automatically beautifies illustrations drawn by users, and provides a user interface that allows users to operate it intuitively. The system converts the beautification parameters operated by the user into prompt sentences, inputs them into a generating AI model, and beautifies the illustration. A specific embodiment of this system is described below.

[1769] Hardware and software to be used

[1770] hardware

[1771] Terminal: A device operated by the user, providing a user interface, obtaining beautification parameters, and generating and sending prompt messages to the server. Specifically, this includes personal computers, tablets, and smartphones.

[1772] Server: A computer system that receives prompt messages, inputs them into a generation AI model, generates an aesthetically pleasing illustration, and sends it back to the terminal.

[1773] software

[1774] User Interface: Includes UI elements that allow users to operate intuitively. Specifically, this includes sliders, dials, and similar elements.

[1775] Generative AI Model: This is an artificial intelligence model used to beautify illustrations based on prompt text. Specifically, models such as "Stable Diffusion" and "DALL-E" are used.

[1776] Data processing and data calculation

[1777] User actions

[1778] The user interacts with the user interface on the device. Specifically, they adjust beautification parameters using sliders and dials. For example, if the user moves the slider to the right, the beautification parameter increases.

[1779] Terminal processing

[1780] The terminal acquires real-time position information of sliders and dials operated by the user. This position information is interpreted as a beautification parameter. For example, if the slider is set to its maximum value, the beautification parameter will be "100". The terminal generates a prompt message based on the acquired beautification parameter. For example, if the beautification parameter is "100", it will generate a prompt message that says "Beautification parameter: 100".

[1781] Server Processing

[1782] The server receives the prompt message sent from the terminal. It inputs the received prompt message into the generating AI model. The generating AI model beautifies the illustration based on the input prompt message. For example, if the prompt message is "Beautification parameter: 100", it processes the illustration with the degree of beautification set to the maximum. The server receives the beautified illustration from the generating AI model and sends it back to the terminal.

[1783] Terminal display

[1784] The device receives the beautified illustration sent back from the server and displays it to the user. This allows the user to see the illustration based on the beautification parameters they have set.

[1785] Specific example

[1786] Example of a prompt

[1787] "Beautification parameter: 100"

[1788] By inputting this prompt into the AI ​​model, you can set the level of illustration beautification to the maximum.

[1789] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1790] Step 1:

[1791] The user interacts with UI elements.

[1792] The user operates the user interface on the device, adjusting beautification parameters using sliders and dials. For example, moving the slider to the right increases the beautification parameter. The input is the user's actions, and the output is the adjusted beautification parameter.

[1793] Step 2:

[1794] The device retrieves the beautification parameters.

[1795] The terminal acquires real-time position information of sliders and dials operated by the user. This position information is interpreted as a beautification parameter. For example, if the slider is set to its maximum value, the beautification parameter will be "100". The input is the user's operation position information, and the output is the beautification parameter.

[1796] Step 3:

[1797] The terminal generates the prompt message.

[1798] The terminal generates a prompt message based on the acquired beautification parameter. For example, if the beautification parameter is "100", it will generate the prompt message "Beautification parameter: 100". The input is the beautification parameter, and the output is the prompt message.

[1799] Step 4:

[1800] The terminal sends a prompt message to the server.

[1801] The terminal sends the generated prompt message to the server. This transmission is performed using a communication protocol such as an HTTP request. The input is the prompt message, and the output is a notification to the server that the transmission has been completed.

[1802] Step 5:

[1803] The server receives the prompt message.

[1804] The server receives a prompt message sent from the terminal. The received prompt message is used in the next processing step. The input is the prompt message, and the output is an acknowledgment.

[1805] Step 6:

[1806] The server inputs prompt messages into the generated AI model.

[1807] The server inputs the received prompt message into the generating AI model. For example, it can use generating AI models such as "Stable Diffusion" or "DALL-E". The input is the prompt message, and the output is a notification to the generating AI model that input is complete.

[1808] Step 7:

[1809] The generative AI model beautifies the illustration.

[1810] The generative AI model beautifies illustrations based on the input prompt text. For example, if the prompt text "Beautification parameter: 100" is input, the model will set the illustration's beautification level to the maximum and process it. The input is the prompt text, and the output is the beautified illustration.

[1811] Step 8:

[1812] The server sends the beautified illustration back to the terminal.

[1813] The server receives the beautified illustration generated by the AI ​​model and sends it back to the terminal. This return is also done using a communication protocol such as an HTTP response. The input is the beautified illustration, and the output is a notification to the terminal that the transmission is complete.

[1814] Step 9:

[1815] The device displays an aesthetically pleasing illustration.

[1816] The terminal receives the beautified illustration sent back from the server and displays it to the user. This allows the user to see the illustration based on the beautification parameters they have set. The input is the beautified illustration, and the output is the display to the user.

[1817] (Application Example 2)

[1818] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1819] Conventional image beautification systems lacked an intuitive user interface for beautifying user-generated illustrations and photographs, making real-time preview and saving / sharing of beautified images difficult. Furthermore, users struggled to fine-tune visual characteristics when adjusting beautification parameters.

[1820] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1821] In this invention, the server includes means for automatically beautifying user-drawn illustrations using advanced image processing technology, means for providing an intuitive user interface, means for allowing the user to adjust beautification parameters, means for previewing user-uploaded images in real time, and means for saving or sharing the beautified images. This allows the user to intuitively operate the server, view the beautified image in real time, adjust the visual characteristics to their liking, and save or share the final image.

[1822] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing, transforming, and correcting images.

[1823] "User-created illustrations" refers to drawings and diagrams that users have manually created.

[1824] "Automatic beautification methods" refer to technologies and devices that improve the quality and appearance of images without user intervention.

[1825] A "user interface that users can operate intuitively" refers to an interface designed to allow users to operate it easily and naturally.

[1826] "Beautification parameters" refer to settings used to adjust the visual characteristics of an image, such as color tone, brightness, and contrast.

[1827] "User-adjustable means" refers to features that allow users to change settings and parameters to suit their preferences.

[1828] "Real-time preview" refers to a function that displays the results immediately when a user performs an action.

[1829] "Means for saving or sharing beautified images" refers to functions for saving processed images to a device or sharing them with other users.

[1830] The system for carrying out this invention automatically beautifies illustrations and photographs drawn by the user and provides an intuitive user interface. Specific embodiments of this system are described below.

[1831] System Configuration

[1832] hardware

[1833] Smartphone: A device used by users to upload and manipulate images.

[1834] Server: Provides computing resources for image processing.

[1835] software

[1836] OpenCV: An image processing library. It performs image analysis, transformation, and modification.

[1837] Tkinter: A GUI library used to build user interfaces.

[1838] Data processing and data calculation

[1839] Image loading

[1840] The user uploads an image using their smartphone. The server reads this image using OpenCV.

[1841] Adjusting the beautification parameters

[1842] Users adjust beautification parameters (smoothness, brightness, contrast) by operating sliders and dials displayed on their smartphone screen. This allows users to see the changes in the image in real time.

[1843] Real-time preview

[1844] The server processes the image in real time based on the beautification parameters adjusted by the user and sends a preview to the smartphone. This allows the user to see the results immediately.

[1845] Saving and sharing images

[1846] Users can save images they are satisfied with to their smartphones. They can also share them with other users via social media or email.

[1847] Specific example

[1848] Users open the app on their smartphones and upload photos. They use sliders to adjust smoothness, brightness, and contrast, and preview the beautified photos in real time. Once satisfied, they save the photos or share them on social media.

[1849] Example of a prompt

[1850] "Create an app that lets users upload photos from their smartphones, adjust smoothness, brightness, and contrast using sliders, preview the beautified photos in real time, and then save or share them."

[1851] In this way, the invention allows users to intuitively operate the system, view beautified images in real time, adjust visual characteristics to their liking, and save or share the final image.

[1852] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1853] Step 1:

[1854] Upload image

[1855] The user uses their smartphone to launch the application, select an image, and upload it.

[1856] Input: The image file selected by the user.

[1857] Output: Image data sent to the server.

[1858] Specific operation: The user selects an image from their smartphone's gallery and presses the "Upload" button. The image data is sent to the server.

[1859] Step 2:

[1860] Image loading

[1861] The server reads the received image data using OpenCV.

[1862] Input: Image data sent to the server.

[1863] Output: Image object loaded with OpenCV.

[1864] Specific operation: The server receives the image data and loads the image into memory using the OpenCV cv2.imread function.

[1865] Step 3:

[1866] Initial settings for beautification parameters

[1867] The server sets the initial beautification parameters (smoothness, brightness, contrast).

[1868] Input: None (default setting).

[1869] Output: Initialized beautification parameters.

[1870] Specific operation: The server sets initial values ​​for smoothness, brightness, and contrast, and retains these values.

[1871] Step 4:

[1872] Display of User Interface

[1873] The terminal displays a user interface and provides sliders and dials.

[1874] Input: Initial beautification parameters.

[1875] Output: Display of the user interface.

[1876] Specific operation: The terminal uses Tkinter to display a user interface including sliders and dials.

[1877] Step 5:

[1878] Adjusting the beautification parameters

[1879] The user adjusts the beautification parameters by operating sliders and dials.

[1880] Input: The value of a slider or dial as determined by the user.

[1881] Output: Adjusted beautification parameters.

[1882] Specific operation: The user operates a slider or dial and sends a new parameter value to the server.

[1883] Step 6:

[1884] Image beautification processing

[1885] The server processes the image based on the adjusted beautification parameters.

[1886] Input: Adjusted beautification parameters, loaded image object.

[1887] Output: A beautified image object.

[1888] Specific operation: The server uses OpenCV functions to adjust smoothness, brightness, and contrast to beautify the image.

[1889] Step 7:

[1890] Displaying real-time preview

[1891] The device displays a real-time preview of the beautified image.

[1892] Input: A beautified image object.

[1893] Output: The beautified image displayed on the device.

[1894] Specific operation: The server sends the beautified image to the terminal, and the terminal displays it on the screen.

[1895] Step 8:

[1896] Saving and sharing images

[1897] Users save or share the beautified images.

[1898] Input: A beautified image object.

[1899] Output: Saved image file or shared image link.

[1900] Specific action: The user presses the "Save" or "Share" button, and the device saves the image to local storage or shares it via social media or email.

[1901] (Example 3)

[1902] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1903] Conventional illustration beautification systems often made it difficult for users to intuitively adjust beautification parameters, and the generated illustrations frequently failed to meet user expectations. Furthermore, the lack of a process to properly analyze user-input parameters and generate prompts suitable for the AI ​​model resulted in inconsistent levels of illustration beautification.

[1904] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[1905] In this invention, the server includes means for automatically beautifying illustrations drawn by the user, means for providing a user interface that the user can operate intuitively, means for the user to adjust beautification parameters, means for sending the beautification parameters entered by the user to the server, means for the server to analyze the received beautification parameters and generate prompt sentences suitable for the generating AI model, means for the generating AI model to beautify the illustration based on the prompt sentences, and means for the server to send the beautified illustration to the user's terminal. This makes it possible for the user to intuitively adjust the beautification parameters and obtain a stable beautified illustration using the generating AI model.

[1906] "Advanced image processing technology" refers to the technology of using computers to analyze, correct, and transform images, and is particularly focused on achieving highly accurate and complex processing.

[1907] A "user interface" is a means for a user to interact with a system, and it includes both visual and manipulative elements.

[1908] "Beautification parameters" are settings used to adjust the visual characteristics of an illustration, and include factors such as hue, brightness, and contrast.

[1909] A "generative AI model" is an algorithm or system that uses artificial intelligence to generate or transform data, and is particularly specialized for image generation.

[1910] A "prompt message" is text-based input data used to give instructions to a generative AI model.

[1911] A "server" is a computer system that provides services to clients over a network.

[1912] A "device" is a device that a user directly operates, and includes computers, smartphones, tablets, and other similar devices.

[1913] "Analysis" is the process of examining data in detail and understanding its structure and meaning.

[1914] "Transmission" is the act of moving data from one location to another.

[1915] An "illustration" is an image intended for visual expression, created using hand-drawn or digital tools.

[1916] Modes for carrying out the invention

[1917] This invention is a system that automatically beautifies illustrations drawn by a user, provides an intuitive user interface, and includes means for the user to adjust the beautification parameters. Specific embodiments of this system are described below.

[1918] Hardware and software to be used

[1919] This system uses the following hardware and software:

[1920] Server: A computer system that provides services to clients over a network.

[1921] Device: A device that a user directly interacts with (e.g., computer, smartphone, tablet).

[1922] Generative AI models: Algorithms and systems that use artificial intelligence to generate or transform data (e.g., Stable Diffusion, DALL-E).

[1923] System Processing Overview

[1924] Users input beautification parameters using their devices. For example, users set parameters such as adjusting color intensity and brightness using a dedicated application on their device or a web interface. Specifically, users use sliders or text boxes to set "color intensity" to 50 and "brightness adjustment" to 70.

[1925] The terminal sends the beautification parameters entered by the user to the server. This transmission is done using communication protocols such as HTTP requests or WebSockets. The server analyzes the received beautification parameters and generates prompts suitable for the AI ​​model. For example, the server might generate a prompt such as, "Generate an illustration with a hue intensity of 50 and a brightness adjustment of 70."

[1926] The server inputs prompt text into the generation AI model, requesting it to beautify an illustration. In doing so, the server calls the generation AI model's API. The generation AI model beautifies the illustration based on the prompt text and returns the result to the server. The returned data is an image file of the beautified illustration.

[1927] The server sends the beautified illustration received from the generating AI model to the terminal. This transmission also uses communication protocols such as HTTP responses or WebSockets. The user then views the beautified illustration on the terminal. The terminal's application or web interface has the functionality to display the received illustration.

[1928] Specific example

[1929] If the user uses their device to set "Color Intensity" to 50 and "Brightness Adjustment" to 70, the server will input the following prompt message into the AI ​​model:

[1930] Example of a prompt:

[1931] "Please generate an illustration with a hue intensity of 50 and a brightness adjustment of 70."

[1932] The generative AI model receives this prompt and beautifies the illustration based on the specified parameters. As a result, a beautified illustration tailored to the user's preferences is generated.

[1933] In this way, users can intuitively adjust the beautification parameters and obtain stable, beautified illustrations using the generation AI model. The flow of the specific processing in Example 3 will be explained with reference to Figure 15.

[1934] Step 1:

[1935] The user enters beautification parameters using a terminal.

[1936] Input: Users set parameters such as color intensity and brightness using a dedicated application or web interface on their device.

[1937] Specific action: The user uses a slider or text box to set the "Tone Intensity" to 50 and the "Brightness Adjustment" to 70.

[1938] Output: Beautification parameters set on the terminal.

[1939] Step 2:

[1940] The terminal sends the entered beautification parameters to the server.

[1941] Input: Beautification parameters set on the device.

[1942] Specific operation: The terminal creates an HTTP POST request and sends the beautification parameters in JSON format to the server endpoint.

[1943] Output: Beautification parameters sent to the server.

[1944] Step 3:

[1945] The server analyzes the beautification parameters it receives and generates prompt sentences suitable for the AI ​​model.

[1946] Input: Beautification parameters sent to the server.

[1947] Specific operation: The server parses the JSON data it receives and performs string manipulation to generate a prompt message. For example, it generates a prompt message such as "Generate an illustration with a hue intensity of 50 and a brightness adjustment of 70."

[1948] Output: The generated prompt message.

[1949] Step 4:

[1950] The server inputs prompt messages into the generated AI model, requesting it to beautify the illustrations.

[1951] Input: The generated prompt message.

[1952] Specific operation: The server sends an HTTP POST request to the API endpoint of the generated AI model, sending data including a prompt message.

[1953] Output: The prompt message sent to the generating AI model.

[1954] Step 5:

[1955] The AI ​​model generates an aesthetically pleasing illustration and sends it back to the server.

[1956] Input: The prompt text sent to the generating AI model.

[1957] Specific operation: The generation AI model internally processes the illustration for aesthetics and then returns the generated illustration to the server as binary data.

[1958] Output: Binary data of the beautified illustration sent back to the server.

[1959] Step 6:

[1960] The server sends the beautified illustration to the terminal.

[1961] Input: Binary data of the beautified illustration sent back to the server.

[1962] Specific operation: The server sends the received binary data to the terminal as an HTTP response.

[1963] Output: A beautified illustration sent to the terminal.

[1964] Step 7:

[1965] The user views the beautified illustration on their device.

[1966] Input: A beautified illustration sent to the terminal.

[1967] Specific operation: The application displays the illustration received by the device, and the user confirms the illustration.

[1968] Output: User-confirmed, beautified illustration.

[1969] (Application Example 3)

[1970] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1971] Conventional image beautification systems often lack an intuitive user interface when automatically beautifying user-generated illustrations. Furthermore, they often lack features such as real-time preview of the adjusted illustration or the ability to save adjustments and share them with other content distribution services. This presents a challenge for users, as it makes it difficult to beautify illustrations to their liking and easily share them.

[1972] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1973] In this invention, the server includes means for automatically beautifying illustrations drawn by the user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for the user to adjust beautification parameters, means for previewing the adjusted illustration in real time, and means for saving the adjustments and sharing them with other content distribution services. This makes it possible for users to beautify illustrations to their liking and easily share them.

[1974] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing and transforming images.

[1975] "User-created illustrations" refers to drawings and diagrams that users have manually created.

[1976] "Automatic beautification methods" refer to technologies that improve the quality and appearance of images without user intervention.

[1977] A "user interface that users can operate intuitively" refers to an interface that users can easily understand and operate.

[1978] "Beautification parameters" refer to settings used to adjust the visual characteristics of an image, such as color tone, brightness, and contrast.

[1979] "A means of previewing the adjusted illustration in real time" refers to a technology that instantly displays the results when a user changes parameters.

[1980] "Means of saving adjustments and sharing them with other content distribution services" refers to technology that records image adjustments made by users and shares them with other online platforms.

[1981] The system for implementing this invention automatically beautifies illustrations drawn by the user and provides a user interface that the user can operate intuitively. The system is implemented as an application that runs on devices such as smartphones and tablets.

[1982] Hardware and software to be used

[1983] Hardware: Smartphones, tablets

[1984] Software: Python, OpenCV, Tkinter

[1985] System Configuration

[1986] 1. Image Loading: Users upload photos and illustrations taken with their devices to the application.

[1987] 2. Adjusting beautification parameters: Users can adjust beautification parameters such as "color intensity" and "brightness adjustment" using sliders and dials within the application.

[1988] 3. Real-time preview: Image beautification is performed in real time based on the adjusted parameters, and the results are displayed instantly.

[1989] 4. Saving and sharing adjustments: Users can save their satisfactory adjustment results and share them on social media and other content distribution services.

[1990] Processing flow

[1991] 1. Image Loading: Select an image from your device's camera or gallery and upload it to the application. Load the image using OpenCV and start processing.

[1992] 2. Adjusting Beautification Parameters: Using Tkinter, provide sliders and dials that users can intuitively operate. Users will adjust the beautification parameters by manipulating these UI elements.

[1993] 3. Real-time preview: Every time the user changes a parameter, the brightness and contrast of the image are adjusted using OpenCV's convertScaleAbs function, and the results are displayed in real time.

[1994] 4. Saving and sharing adjustments: Provides a function to save the adjusted image to the device and share it on social media or other content distribution services.

[1995] Specific example

[1996] For example, a user can upload a landscape photo taken with their smartphone to the application, enhance the colors, slightly reduce the brightness, and then share it on Instagram.

[1997] Example of a prompt

[1998] "Upload a landscape photo, enhance the colors, slightly reduce the brightness, and share it on Instagram."

[1999] In this way, users can beautify illustrations to their liking and easily share them.

[2000] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[2001] Step 1:

[2002] Users upload photos and illustrations taken with their devices to the application.

[2003] Input: An image file selected from the device's camera or gallery.

[2004] Output: Image data loaded into the application.

[2005] Specific operation: The user launches the application, presses the image upload button, and selects an image from the device's camera roll or gallery. The selected image is loaded into the application.

[2006] Step 2:

[2007] The image data read by the device is processed using OpenCV.

[2008] Input: Loaded image data.

[2009] Output: An image object that can be processed by OpenCV.

[2010] Specific operation: The application uses the OpenCV library to convert the loaded image data into an internal format, making it ready for processing.

[2011] Step 3:

[2012] The user operates sliders and dials to adjust the beautification parameters.

[2013] Input: The value of a slider or dial as determined by the user.

[2014] Output: Adjusted beautification parameters.

[2015] Specific operation: The user operates sliders and dials within the application to set parameters such as "color intensity" and "brightness adjustment." The set values ​​are reflected in the application in real time.

[2016] Step 4:

[2017] The device previews the image in real time based on the adjusted beautification parameters.

[2018] Input: Adjusted beautification parameters and original image data.

[2019] Output: The adjusted image is previewed in real time.

[2020] Specific operation: The application uses the OpenCV convertScaleAbs function to adjust the brightness and contrast of an image based on the beautification parameters set by the user, and displays the results in real time.

[2021] Step 5:

[2022] Users save the adjusted image and share it with other content distribution services.

[2023] Input: Adjusted image data.

[2024] Output: Saved image files and shared images.

[2025] Specific actions: The user presses the save button to save the adjusted image to their device. They then press the share button to upload the image to Instagram or other social media platforms.

[2026] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[2027] "Example of form 1"

[2028] One embodiment of the present invention is an illustration beautification system incorporating an emotion engine. This system includes a function to automatically beautify illustrations drawn by the user, a function to provide an intuitive user interface, a function to allow the user to adjust beautification parameters, and an emotion engine that recognizes the user's emotions. The emotion engine recognizes the user's emotions from the user's facial expressions, tone of voice, text input, etc., and adjusts the beautification parameters according to those emotions. Specifically, it is ...

Claims

[Claim 1] A means of obtaining image data representing an illustration drawn by a user, Means for receiving input of beautification parameters, including color tone, brightness, or contrast, which are applied to the image data, via a user interface including a slider or dial that can be operated by the user, A means for recognizing the user's emotions during the process of drawing the illustration, using an emotion engine, Means for updating the beautification parameters based on the user's emotions, means for generating a prompt statement that instructs to adjust the visual characteristics of the image data based on the updated beautification parameters, A means for inputting the aforementioned prompt sentence into a generative AI and causing the generative AI to adjust the visual characteristics of the image data, A system that includes this.

Images