System

Abstract

To provide a system that automatically makes an illustration drawn by a user beautiful using advanced image processing techniques.SOLUTION: A system comprises: means for acquiring image data representing an illustration drawn by a user; means for automatically adjusting visual characteristics of the image data; means for recognizing the user's feeling using a feeling engine; and means for adjusting the visual characteristics of the image data by using a prompt sentence that instructs adjustment of the visual characteristics of the image data and generative AI on the basis of the user's feeling.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Current image editing tools require advanced skills, making them difficult for average users to use freely. In addition, there is no easy way to enhance illustrations that you have drawn yourself. [Means for solving the problem]

[0005] This invention provides a system that uses advanced image processing technology to automatically beautify illustrations drawn by users. It also provides an intuitive user interface and a means for users to adjust beautification parameters, creating an environment in which users can freely beautify their illustrations. [Brief explanation of the drawings]

[0006] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] One embodiment of the present invention provides a system for automatically beautifying user-drawn illustrations. This system uses advanced image processing technology to automatically beautify user-drawn illustrations. Specifically, the system receives a user-drawn illustration as input, automatically adjusts the visual characteristics of the illustration, such as color tone, brightness, and contrast, and outputs a beautified illustration.

[0029] "Example 2"

[0030] In addition, as one embodiment of the present invention, an intuitively operable user interface is provided. This user interface includes intuitively operable UI elements such as sliders and dials. Specifically, the user can adjust beautification parameters by operating the sliders and dials. This allows the user to adjust the degree of beautification of the illustration to suit their preferences.

[0031] "Example 3"

[0032] Furthermore, as one embodiment of the present invention, a means is provided for allowing a user to adjust beautification parameters. Specifically, the user can freely adjust the degree of beautification of an illustration by directly inputting beautification parameters. For example, the user can beautify an illustration to suit their preferences by adjusting parameters such as "color intensity" and "brightness adjustment."

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: The user inputs the illustration they have drawn into the system. This can be done by file upload, drag and drop, etc.

[0036] Step 2: The system analyzes the input illustration and automatically adjusts visual characteristics such as color tone, brightness, and contrast. This adjustment is done using a pre-trained machine learning model.

[0037] Step 3: The system outputs the adjusted illustration, which can be displayed directly on the screen or downloaded as a file.

[0038] "Example 2"

[0039] Step 1: The user operates the system's user interface, specifically, by manipulating sliders and dials to adjust the beautification parameters.

[0040] Step 2: The system adjusts the beautification parameters according to the user's operations and beautifies the illustration.

[0041] Step 3: The system outputs the beautified illustration, which can be displayed directly on the screen or downloaded as a file.

[0042] "Example 3"

[0043] Step 1: The user directly inputs the beautification parameters, such as "color intensity" and "brightness adjustment," using numerical values ​​or sliders.

[0044] Step 2: The system beautifies the illustration based on the beautification parameters entered by the user.

[0045] Step 3: The system outputs the beautified illustration, which can be displayed directly on the screen or downloaded as a file.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0048] Conventional illustration beautification systems require users to manually adjust color, brightness, and contrast, making operation cumbersome. Furthermore, they lack an intuitive user interface, making it difficult for users to easily complete the beautification process. Furthermore, the lack of real-time progress indicators and notification of process completion negatively impacted the user experience.

[0049] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for a user to upload an illustration, a means for the server to receive the illustration, a means for the server to adjust color tone, a means for the server to adjust brightness, a means for the server to adjust contrast, a means for the server to generate a beautified illustration, and a means for the server to provide the beautified illustration to the user. This allows the user to easily beautify an illustration through an intuitive interface. In addition, the user experience is improved by real-time progress display and processing completion notification.

[0050] "Advanced image processing technology" refers to technology for automatically adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[0051] A "user interface" is an interface that includes a screen and input devices that allow a user to operate a system.

[0052] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[0053] "Means for users to upload illustrations" refers to a function that allows users to send illustration files to the system from their own devices.

[0054] "Means for the server to receive illustrations" refers to the function by which the server receives and stores illustration files sent by users.

[0055] "Means for the server to adjust color tone" is a function that allows the server to automatically change the hue and saturation of an illustration.

[0056] "Means for the server to adjust brightness" is a function that allows the server to automatically change the brightness of the illustration.

[0057] "Means for the server to adjust contrast" is a function that allows the server to automatically change the difference in brightness of the illustration.

[0058] The "means for the server to generate a beautified illustration" is a function for the server to create a new illustration after adjusting the color tone, brightness, and contrast.

[0059] "Means for the server to provide beautified illustrations to users" refers to a function for providing beautified illustrations generated by the server to users in a downloadable format.

[0060] This invention relates to a system for automatically beautifying illustrations drawn by users. The system provides an intuitive user interface and allows users to adjust beautification parameters to adjust visual characteristics such as color tone, brightness, and contrast.

[0061] Hardware and software used

[0062] The server is a computer system that runs Python programs and uses OpenCV and Pillow as image processing libraries. Users access the system using devices connected to the Internet (PCs, smartphones, tablets, etc.).

[0063] Program processing explanation

[0064] The server receives the illustration uploaded by the user and then adjusts the image's color using OpenCV. Specifically, the server converts the image to HSV (hue, saturation, brightness) space, adjusts the hue appropriately, and then converts it back to RGB space.

[0065] The server then uses Pillow to adjust the image brightness, specifically using Pillow's ImageEnhance.Brightness class to adjust the brightness, and then using Pillow's ImageEnhance.Contrast class to adjust the contrast.

[0066] Through these processes, a beautified illustration is generated and provided to the user, who can then obtain it by clicking the download link on their device.

[0067] Specific examples

[0068] As a concrete example, consider the case of beautifying an illustration drawn by a user, "example_illustration.png."

[0069] 1. User Action:

[0070] The user uploads "example_illustration.png" to the system from their device.

[0071] 2. Server processing:

[0072] The server receives the uploaded image and adjusts the color, brightness, and contrast.

[0073] Save the adjusted image as "beautified_illustration.png".

[0074] 3. Output to the user:

[0075] The server provides the beautified illustration "beautified_illustration.png" to the user.

[0076] An example of a prompt is as follows:

[0077] Upload your own illustration "example_illustration.png". The system will automatically adjust the color, brightness, and contrast to provide you with a beautified illustration. You will receive a notification when the process is complete.

[0078] This is how the system works, automatically beautifying illustrations drawn by users.

[0079] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0080] Step 1:

[0081] Users upload illustrations.

[0082] A user uploads an illustration file (e.g., "example_illustration.png") that they have drawn from their terminal through the system's web interface. The input is the user's illustration file, and the output is the illustration file sent to the server.

[0083] Step 2:

[0084] The server receives the illustration.

[0085] The server receives the illustration file uploaded by the user and saves it in the specified directory. The input is the illustration file sent by the user, and the output is the illustration file saved on the server. Specifically, the server checks the file format and size and verifies that it is in the appropriate format.

[0086] Step 3:

[0087] The server adjusts the color tone.

[0088] The server loads the illustration using OpenCV's cv2.imread function. Next, it converts the image to HSV (hue, saturation, brightness) space using the cv2.cvtColor function. The hue is adjusted appropriately and then converted back to RGB space. The input is an illustration file stored on the server, and the output is an illustration with adjusted color tones. Specifically, the hue is shifted by 10 degrees.

[0089] Step 4:

[0090] The server adjusts the brightness.

[0091] The server uses Pillow's ImageEnhance.Brightness class to adjust the brightness of the image. Specifically, it increases the brightness by 1.2 times. The input is an illustration with adjusted color tone, and the output is an illustration with adjusted brightness.

[0092] Step 5:

[0093] The server adjusts the contrast.

[0094] The server uses Pillow's ImageEnhance.Contrast class to adjust the contrast of the image. Specifically, it increases the contrast by 1.3 times. The input is an illustration with adjusted brightness, and the output is an illustration with adjusted contrast.

[0095] Step 6:

[0096] The server generates the beautified illustration.

[0097] The server saves the adjusted image as "beautified_illustration.png". The input is a contrast-adjusted illustration, and the output is a beautified illustration file.

[0098] Step 7:

[0099] The server provides the beautified illustrations to the user.

[0100] The server provides the beautified illustration "beautified_illustration.png" to the user. The user can obtain the beautified illustration by clicking the download link from their device. The input is the beautified illustration file, and the output is the download link provided to the user. Specifically, the server displays the progress in real time and notifies the user after the process is completed.

[0101] (Application example 1)

[0102] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] While existing illustration beautification systems automatically beautify user-drawn illustrations, they lack a way to easily share the beautified illustrations with content distribution services. Additionally, the customization options for the beautification process are limited, making it difficult for users to customize the beautification process to their own preferences.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0105] In this invention, the server includes means for automatically beautifying illustrations drawn by users using advanced image processing technology, means for providing a user interface that can be intuitively operated by users, means for allowing users to adjust beautification parameters, means for allowing users to directly share beautified illustrations to a content distribution service, and means for providing customization options for the beautification process, thereby enabling users to easily share beautified illustrations and further customize the beautification process to their own preferences.

[0106] "Advanced image processing technology" refers to technology for automatically adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[0107] "User-drawn illustrations" are paintings or drawings created by users, either by hand or using digital tools.

[0108] "Automatic beautification" refers to algorithms or software that enhance the visual characteristics of illustrations without user intervention.

[0109] A "user interface that users can operate intuitively" is an interface that is designed so that users can operate it easily and intuitively.

[0110] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[0111] "User adjustable means" refers to a feature that allows the user to manually change the beautification parameters.

[0112] A "content distribution service" is a service that distributes digital content such as images and videos over the Internet.

[0113] "Shareable means" refers to the ability to share aestheticized illustrations with other users and platforms.

[0114] "Customization Options" are settings and features that allow users to tailor the beautification process to their own preferences.

[0115] As an embodiment of the present invention, the following system can be constructed.

[0116] System Overview

[0117] The system automatically beautifies illustrations drawn by users and shares the beautified illustrations with a content distribution service. The system includes a smartphone, a server, and software using advanced image processing technology.

[0118] Hardware and software used

[0119] Hardware: Smartphone (iOS or ANDROID (registered trademark)), server

[0120] Software: Python, OpenCV (image processing library)

[0121] Data processing and calculation

[0122] 1. Loading images

[0123] Users can take a photo of an illustration they have drawn on their smartphone or upload it using the smartphone's camera or file upload function.

[0124] 2. Adjusting the color tone

[0125] The server converts the image to the HSV color space and increases the saturation, which makes the illustration appear more vibrant.

[0126] 3. Adjusting the brightness

[0127] The server performs a scale transformation to increase the brightness of the image, which makes the illustration brighter.

[0128] 4. Adjusting the contrast

[0129] The server converts the image to the LAB color space and performs histogram equalization on the L channel to enhance the contrast, which improves the visual characteristics of the illustration.

[0130] 5. Save and share images

[0131] The beautified images are stored on a server, and users can share them directly to content distribution services via smartphone applications.

[0132] Specific examples

[0133] Users can take a photo of an illustration they've drawn with their smartphone and upload it to the app, where the server automatically beautifies it. The beautified illustration can then be adjusted in color, brightness, and contrast to suit the user's preferences. Finally, the beautified illustration can be shared on social media and other content distribution services.

[0134] Prompt Sentence Examples

[0135] "Develop an app that beautifies illustrations drawn by users. The app will automatically adjust the color tone, brightness, and contrast of the illustration and output the beautified illustration."

[0136] In this way, a system can be realized that allows users to easily create and share aesthetic illustrations.

[0137] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0138] Step 1:

[0139] The user takes a photo of an illustration drawn on their smartphone or uploads it. The input is the illustration image drawn by the user, and the output is an image file saved on the smartphone. Specifically, the user takes a photo of the illustration using the smartphone's camera function, or uploads an existing image file to the application.

[0140] Step 2:

[0141] The device sends the photographed or uploaded illustration image to the server. The input is the image file saved on the smartphone, and the output is the image data transferred to the server. Specifically, the smartphone application calls an API to send the image file to the server.

[0142] Step 3:

[0143] The server reads the received illustration image. The input is the image data transferred to the server, and the output is an image object loaded into memory. Specifically, the server reads the image file using Python and OpenCV.

[0144] Step 4:

[0145] The server adjusts the color tone of an image. The input is an image object loaded into memory, and the output is an image object with the adjusted color tone. Specifically, the server converts the image to the HSV color space and increases the saturation.

[0146] Step 5:

[0147] The server adjusts the brightness of the image. The input is a color-adjusted image object, and the output is a brightness-adjusted image object. Specifically, the server performs a scale transformation to increase the brightness of the image.

[0148] Step 6:

[0149] The server adjusts the contrast of an image. The input is a brightness-adjusted image object, and the output is a contrast-adjusted image object. Specifically, the server converts the image to the LAB color space and performs histogram equalization on the L channel.

[0150] Step 7:

[0151] The server saves the beautified image. The input is a contrast-adjusted image object, and the output is a beautified image file saved on the server. Specifically, the server saves the beautified image as a file.

[0152] Step 8:

[0153] The device obtains the beautified image from the server. The input is the beautified image file stored on the server, and the output is the beautified image file downloaded to the smartphone. Specifically, the smartphone application calls an API to download the beautified image from the server.

[0154] Step 9:

[0155] The user shares the beautified illustration on a content distribution service. The input is a beautified image file downloaded to a smartphone, and the output is a beautified image uploaded to the content distribution service. Specifically, the user uses a smartphone application to share the beautified illustration on social media or other content distribution services.

[0156] Example 2

[0157] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0158] With conventional image processing systems, beautifying user-drawn illustrations required specialized knowledge and complex operations, and lacked an intuitive user interface. It was also difficult to adjust the beautification parameters, making it difficult for users to beautify illustrations to their own preferences. Furthermore, beautification processing using generative AI models was not performed in real time, resulting in delayed feedback to user operations.

[0159] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0160] In this invention, the server includes a means for converting beautification parameters operated by the user into a prompt sentence, a means for inputting the prompt sentence into a generative AI model to beautify the illustration, and a means for displaying the beautified illustration to the user, thereby enabling the user to adjust the beautification parameters and check the beautified illustration in real time through an intuitive user interface.

[0161] "Advanced image processing technology" is a general term for advanced algorithms and methods used to automatically beautify user-drawn illustrations.

[0162] "User interface" is a general term for the screens and input devices that users use to operate a system, and includes UI elements that can be operated intuitively.

[0163] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[0164] A "prompt" is a textual instruction used to input user-set beautification parameters into a generative AI model.

[0165] A "generative AI model" is an artificial intelligence model that beautifies illustrations based on input prompts, and examples include "Stable Diffusion" and "DALL-E."

[0166] A "slider" is a UI element that allows the user to adjust a value by dragging it.

[0167] A "dial" is a UI element that allows the user to adjust a value by rotating it.

[0168] The "server" is a computer system that receives prompt text, inputs it into a generative AI model, generates a beautified illustration, and sends it back to the device.

[0169] A "terminal" is a device operated by a user, which has the functions of providing a user interface, acquiring beautification parameters, generating prompt sentences, and sending them to a server.

[0170] This invention is a system for automatically beautifying illustrations drawn by users, providing a user interface that allows users to operate intuitively. The system converts beautification parameters operated by the user into prompt sentences, which are then input into a generative AI model to beautify the illustration. A specific embodiment of this system is described below.

[0171] Hardware and software used

[0172] Hardware

[0173] Terminal: A device operated by the user that provides a user interface, acquires beautification parameters, generates prompts, and sends them to the server. Examples include PCs, tablets, and smartphones.

[0174] Server: A computer system that receives prompt text, inputs it into a generative AI model, generates a beautified illustration, and sends it back to the device.

[0175] software

[0176] User Interface: Includes UI elements that users can operate intuitively, such as sliders and dials.

[0177] Generative AI model: An artificial intelligence model that beautifies illustrations based on prompts. Specifically, "Stable Diffusion" and "DALL-E" are used.

[0178] Data processing and calculation

[0179] User operations

[0180] The user operates the user interface on the device, specifically, by using sliders and dials to adjust the beautification parameters. For example, when the user moves the slider to the right, the beautification parameter increases.

[0181] Terminal handling

[0182] The device obtains the position information of the slider or dial operated by the user in real time. This position information is interpreted as a beautification parameter. For example, if the slider is set to the maximum value, the beautification parameter is "100". The device generates a prompt sentence based on the obtained beautification parameter. For example, if the beautification parameter is "100", the device generates the prompt sentence "Beautification parameter: 100".

[0183] Server Processing

[0184] The server receives the prompt sent from the device. It inputs the received prompt into the generative AI model. The generative AI model beautifies the illustration based on the input prompt. For example, if a prompt with "beautification parameter: 100" is input, the beautification level of the illustration is set to the maximum and processing is performed. The server receives the beautified illustration from the generative AI model and sends it back to the device.

[0185] Terminal display

[0186] The terminal receives the beautified illustration sent back from the server and displays it to the user, who can then check the illustration based on the beautification parameters he or she has set.

[0187] Specific examples

[0188] Prompt Sentence Examples

[0189] "Beautification parameter: 100"

[0190] By inputting this prompt into the generative AI model, the degree of beautification of the illustration can be set to the maximum.

[0191] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0192] Step 1:

[0193] The user interacts with a UI element.

[0194] The user operates the user interface on the device and adjusts the beautification parameters using sliders and dials. For example, when the user moves the slider to the right, the beautification parameter increases. The input is the user's operation, and the output is the adjusted beautification parameter.

[0195] Step 2:

[0196] The terminal acquires the beautification parameters.

[0197] The device obtains the position information of the slider or dial operated by the user in real time. This position information is interpreted as a beautification parameter. For example, if the slider is set to the maximum value, the beautification parameter is "100". The input is the user's operation position information, and the output is the beautification parameter.

[0198] Step 3:

[0199] The terminal generates a prompt.

[0200] The terminal generates a prompt sentence based on the acquired beautification parameter. For example, if the beautification parameter is "100", the terminal generates the prompt sentence "Beautification parameter: 100". The input is the beautification parameter, and the output is the prompt sentence.

[0201] Step 4:

[0202] The terminal sends the prompt to the server.

[0203] The terminal sends the generated prompt text to the server. This transmission is performed using a communication protocol such as an HTTP request. The input is the prompt text, and the output is a transmission completion notification to the server.

[0204] Step 5:

[0205] The server receives the prompt.

[0206] The server receives the prompt sent from the terminal. The received prompt is used in the next processing step. The input is the prompt, and the output is a confirmation of receipt.

[0207] Step 6:

[0208] The server inputs a prompt sentence into the generative AI model.

[0209] The server inputs the received prompt sentence into a generative AI model. For example, generative AI models such as "Stable Diffusion" or "DALL-E" can be used. The input is the prompt sentence, and the output is a notification to the generative AI model that the input has been completed.

[0210] Step 7:

[0211] Generative AI models beautify illustrations.

[0212] The generative AI model beautifies the illustration based on the input prompt. For example, if a prompt with "Beautification parameter: 100" is input, the beautification level of the illustration is set to the maximum and processing is performed. The input is the prompt, and the output is a beautified illustration.

[0213] Step 8:

[0214] The server returns the beautified illustration to the device.

[0215] The server receives the beautified illustration from the generative AI model and returns it to the device. This return is also done using a communication protocol such as an HTTP response. The input is the beautified illustration, and the output is a notification of completion of transmission to the device.

[0216] Step 9:

[0217] The device displays a beautified illustration.

[0218] The terminal receives the beautified illustration sent back from the server and displays it to the user. The user can then check the illustration based on the beautification parameters they set. The input is the beautified illustration, and the output is the display to the user.

[0219] (Application example 2)

[0220] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0221] Conventional image beautification systems lack an intuitive user interface when beautifying user-drawn illustrations or photos, making it difficult to preview in real time or save or share beautified images. Another issue is that it is difficult for users to fine-tune visual characteristics when adjusting beautification parameters.

[0222] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0223] In this invention, the server includes means for automatically beautifying illustrations drawn by a user using advanced image processing technology, means for providing a user interface that can be operated intuitively by the user, means for allowing the user to adjust beautification parameters, means for previewing images uploaded by the user in real time, and means for saving or sharing the beautified images. This allows the user to intuitively operate the server, check the beautified images in real time, adjust visual characteristics to suit their preferences, and save or share the final images.

[0224] "Advanced image processing techniques" refers to advanced algorithms and methods for analyzing, transforming, and modifying images.

[0225] "User-drawn illustrations" refers to pictures or drawings manually created by a user.

[0226] "Automatic beautification" refers to techniques or devices that enhance the quality or appearance of images without user intervention.

[0227] "A user interface that users can operate intuitively" refers to an interface that is designed so that users can operate it easily and naturally.

[0228] "Beautification parameters" refer to settings for adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[0229] "User-adjustable means" refers to features that allow users to change settings or parameters to suit their preferences.

[0230] "Means for real-time preview" refers to a function that displays the results immediately when the user performs an operation.

[0231] "Means to save or share beautified images" refers to the functionality to save the beautified images to your device or share them with other users.

[0232] The system for implementing this invention automatically beautifies illustrations and photographs drawn by the user and provides an intuitive user interface. Specific embodiments of this system are described below.

[0233] System Configuration

[0234] Hardware

[0235] Smartphone: A device that allows users to upload and manipulate images.

[0236] Server: Provides the computing resources for image processing.

[0237] software

[0238] OpenCV: Image processing library for analyzing, transforming, and modifying images.

[0239] Tkinter: A GUI library used to build user interfaces.

[0240] Data processing and calculation

[0241] Loading images

[0242] A user uploads an image using their smartphone, and the server reads the image using OpenCV.

[0243] Adjusting beautification parameters

[0244] Users can adjust the beautification parameters (smoothness, brightness, contrast) by using the sliders and dials displayed on the smartphone screen, allowing them to see the changes in the image in real time.

[0245] Real-time preview

[0246] The server processes the image in real time based on the beautification parameters adjusted by the user and sends a preview to the smartphone, allowing the user to see the results instantly.

[0247] Save and share images

[0248] Once users are satisfied with the image, they can save it on their smartphone and share it with other users via social media or email.

[0249] Specific examples

[0250] Users open the app on their smartphone, upload a photo, use sliders to adjust smoothness, brightness, and contrast, preview the beautified photo in real time, and once satisfied, save the photo or share it on social media.

[0251] Prompt Sentence Examples

[0252] "Create an app where you upload a photo on your smartphone, use sliders to adjust smoothness, brightness, and contrast, preview the beautified photo in real time, and save or share it."

[0253] In this way, the invention allows users to intuitively view beautified images in real time, adjust visual characteristics to their liking, and save or share the final image.

[0254] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0255] Step 1:

[0256] Uploading an image

[0257] The user uses a smartphone to launch the application, select an image, and upload it.

[0258] Input: An image file selected by the user.

[0259] Output: Image data sent to the server.

[0260] Specific operation: The user selects an image from the smartphone gallery and presses the "Upload" button. The image data is sent to the server.

[0261] Step 2:

[0262] Loading images

[0263] The server reads the received image data using OpenCV.

[0264] Input: Image data sent to the server.

[0265] Output: Image object loaded by OpenCV.

[0266] Specific operation: The server receives the image data and reads the image into memory using OpenCV's cv2.imread function.

[0267] Step 3:

[0268] Initial settings for beautification parameters

[0269] The server sets the initial beautification parameters (smoothness, brightness, contrast).

[0270] Input: None (default value).

[0271] Output: Initialized beautification parameters.

[0272] What happens: The server sets initial values ​​for smoothness, brightness, and contrast, and then persists these values.

[0273] Step 4:

[0274] User Interface Display

[0275] The terminal displays a user interface and provides sliders and dials.

[0276] Input: Initialized beautification parameters.

[0277] Output: Display of the user interface.

[0278] What it does: The terminal uses Tkinter to display a user interface, including sliders and dials.

[0279] Step 5:

[0280] Adjusting beautification parameters

[0281] The user adjusts the beautification parameters by manipulating sliders and dials.

[0282] Input: The value of a slider or dial as manipulated by the user.

[0283] Output: Adjusted beautification parameters.

[0284] What happens: The user operates a slider or dial, sending a new parameter value to the server.

[0285] Step 6:

[0286] Image beautification

[0287] The server processes the image based on the adjusted beautification parameters.

[0288] Input: adjusted beautification parameters, loaded image object.

[0289] Output: A beautified image object.

[0290] What it does: The server uses OpenCV functions to adjust smoothness, brightness, and contrast to beautify the image.

[0291] Step 7:

[0292] View real-time preview

[0293] The device will display a preview of the beautified image in real time.

[0294] Input: A beautified image object.

[0295] Output: The beautified image displayed on the device.

[0296] Specific operation: The server sends the beautified image to the device, which displays it on the screen.

[0297] Step 8:

[0298] Save and share images

[0299] The user saves or shares the beautified image.

[0300] Input: A beautified image object.

[0301] Output: Saved image file or shared image link.

[0302] Specific behavior: The user presses the "Save" or "Share" button, and the device saves the image to local storage or shares it via social media or email.

[0303] Example 3

[0304] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0305] In conventional illustration beautification systems, it was difficult for users to intuitively adjust beautification parameters, and the generated illustrations often did not meet the user's expectations. Furthermore, there was a lack of a process to properly analyze the parameters entered by the user and generate prompts appropriate for the generative AI model, which resulted in inconsistent beautification of the illustrations.

[0306] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0307] In this invention, the server includes means for automatically beautifying an illustration drawn by a user, means for providing a user interface that the user can intuitively operate, means for the user to adjust beautification parameters, means for transmitting the beautification parameters input by the user to the server, means for the server to analyze the received beautification parameters and generate a prompt sentence suitable for the generative AI model, means for the generative AI model to beautify the illustration based on the prompt sentence, and means for the server to transmit the beautified illustration to the user's terminal. This allows the user to intuitively adjust the beautification parameters and obtain a consistently beautified illustration using the generative AI model.

[0308] "Advanced image processing technology" refers to technology that uses computers to analyze, modify, and convert images, and is particularly capable of achieving highly accurate and complex processing.

[0309] A "user interface" is the means by which a user interacts with a system, including both visual and operational elements.

[0310] "Beautification parameters" are setting values ​​for adjusting the visual characteristics of an illustration, and include color tone, brightness, contrast, and the like.

[0311] A "generative AI model" is an algorithm or system that uses artificial intelligence to generate or transform data, specifically for image generation.

[0312] A "prompt sentence" is text-based input data used to give instructions to a generative AI model.

[0313] A "server" is a computer system that provides services to clients over a network.

[0314] A "terminal" is a device that a user directly operates, including a computer, smartphone, tablet, etc.

[0315] "Analysis" is the process of examining data in detail to understand its structure and meaning.

[0316] "Transmitting" is the act of moving data from one place to another.

[0317] An "illustration" is an image intended for visual expression, created by hand or using digital tools.

[0318] MODE FOR CARRYING OUT THE INVENTION

[0319] This invention is a system for automatically beautifying illustrations drawn by a user, providing a user interface that can be operated intuitively by the user and including means for allowing the user to adjust beautification parameters. Specific embodiments of this system are described below.

[0320] Hardware and software used

[0321] The system uses the following hardware and software:

[0322] Server: A computer system that provides services to clients over a network.

[0323] Terminal: A device that is directly operated by a user (e.g., computer, smartphone, tablet).

[0324] Generative AI Model: An algorithm or system that uses artificial intelligence to generate or transform data (e.g., Stable Diffusion, DALL-E).

[0325] System processing overview

[0326] Users input beautification parameters using their devices. For example, they can use a dedicated device application or a web interface to set parameters such as color intensity and brightness adjustment. Specifically, they can use sliders and text boxes to set "color intensity" to 50 and "brightness adjustment" to 70.

[0327] The device sends the beautification parameters entered by the user to the server. This transmission is performed using a communication protocol such as an HTTP request or WebSocket. The server analyzes the received beautification parameters and generates a prompt appropriate for the generative AI model. For example, the server generates a prompt such as, "Please generate an illustration with color intensity: 50, brightness adjustment: 70."

[0328] The server inputs a prompt into the generative AI model and requests it to beautify the illustration. At this time, the server calls the generative AI model's API. The generative AI model beautifies the illustration based on the prompt and returns the results to the server. The returned data is an image file of the beautified illustration.

[0329] The server sends the beautified illustration received from the generative AI model to the device. This transmission is also done using a communication protocol such as HTTP response or WebSocket. The user checks the beautified illustration on the device. The device application or web interface has the function to display the received illustration.

[0330] Specific examples

[0331] If a user uses their device to set "Color Intensity" to 50 and "Brightness Adjustment" to 70, the server will input the following prompt to the generative AI model:

[0332] Example prompt sentence:

[0333] "Create an illustration with color intensity: 50, brightness adjustment: 70."

[0334] The generative AI model receives this prompt and beautifies the illustration based on the specified parameters, resulting in a beautified illustration tailored to the user's preferences.

[0335] In this way, the user can intuitively adjust the beautification parameters and obtain a stable beautified illustration using the generative AI model. The flow of the specific processing in Example 3 will be described with reference to FIG.

[0336] Step 1:

[0337] The user inputs the beautification parameters using a terminal.

[0338] Input: The user sets parameters such as color intensity and brightness using a dedicated application on the device or a web interface.

[0339] What it does: The user uses the sliders and text boxes to set "Color Intensity" to 50 and "Brightness Adjustment" to 70.

[0340] Output: Beautification parameters set to the terminal.

[0341] Step 2:

[0342] The terminal transmits the input beautification parameters to the server.

[0343] Input: Beautification parameters set on the device.

[0344] Specific operation: The terminal makes an HTTP POST request and sends the beautification parameters in JSON format to the server endpoint.

[0345] Output: Beautification parameters sent to the server.

[0346] Step 3:

[0347] The server analyzes the received beautification parameters and generates a prompt sentence suitable for the generative AI model.

[0348] Input: The beautification parameters sent to the server.

[0349] Specific operation: The server analyzes the received JSON data and performs string manipulation to generate a prompt. For example, it generates a prompt such as "Please generate an illustration with color intensity: 50, brightness adjustment: 70."

[0350] Output: The generated prompt statement.

[0351] Step 4:

[0352] The server inputs a prompt into the generative AI model, requesting it to beautify the illustration.

[0353] Input: The generated prompt statement.

[0354] Specific operation: The server sends an HTTP POST request to the API endpoint of the generated AI model and sends data including the prompt text.

[0355] Output: The prompt sent to the generative AI model.

[0356] Step 5:

[0357] The generative AI model generates a beautified illustration and sends it back to the server.

[0358] Input: The prompt sent to the generative AI model.

[0359] Specific operation: The generative AI model internally beautifies the illustration and sends the generated illustration back to the server as binary data.

[0360] Output: Binary data of the beautified illustration sent back to the server.

[0361] Step 6:

[0362] The server sends the beautified illustration to the device.

[0363] Input: Binary data of the aestheticized illustration sent back to the server.

[0364] Specific operation: The binary data received by the server is sent to the terminal as an HTTP response.

[0365] Output: A beautified illustration sent to the device.

[0366] Step 7:

[0367] The user checks the beautified illustration on their device.

[0368] Input: A glorified illustration sent to the device.

[0369] Specific operation: The illustration received by the device is displayed within the application, and the user confirms the illustration.

[0370] Output: A beautified illustration reviewed by the user.

[0371] (Application example 3)

[0372] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0373] Conventional image beautification systems often lack an intuitive interface for users when automatically beautifying illustrations drawn by users. They also lack the ability to preview the adjusted illustration in real time or the ability to save and share the adjustments on other content distribution services. This makes it difficult for users to beautify illustrations to their liking and easily share them.

[0374] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0375] In this invention, the server includes means for automatically beautifying illustrations drawn by users using advanced image processing technology, means for providing a user interface that users can operate intuitively, means for allowing users to adjust beautification parameters, means for previewing the adjusted illustration in real time, and means for saving the adjustments and sharing them with other content distribution services. This allows users to beautify illustrations to their liking and easily share them.

[0376] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing and transforming images.

[0377] "User-drawn illustrations" refers to pictures or drawings manually created by a user.

[0378] "Automatic beautification" refers to techniques that improve the quality or appearance of images without user intervention.

[0379] An "intuitive user interface" refers to an interface that users can easily understand and operate.

[0380] "Beautification parameters" refer to settings for adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[0381] "A means of previewing the adjusted illustration in real time" refers to technology that instantly displays the results when the user changes parameters.

[0382] "Means for saving adjustments and sharing them with other content distribution services" refers to technology that allows users to record image adjustments they make and share them with other online platforms.

[0383] The system for implementing this invention automatically beautifies illustrations drawn by users and provides a user interface that allows users to operate intuitively. The system is implemented as an application that runs on devices such as smartphones and tablets.

[0384] Hardware and software used

[0385] Hardware: Smartphones, tablets

[0386] Software: Python, OpenCV, Tkinter

[0387] System configuration

[0388] 1. Image loading: Users can take photos or illustrations on their devices and upload them to the application.

[0389] 2. Adjusting beautification parameters: Users can use sliders and dials within the application to adjust beautification parameters such as "color intensity" and "brightness adjustment."

[0390] 3. Real-time preview: Based on the adjusted parameters, the image beautification process will be carried out in real time and the results will be displayed instantly.

[0391] 4. Saving and sharing adjustments: Users can save the adjustment results they are satisfied with and share them on social media and other content distribution services.

[0392] Processing flow

[0393] 1. Loading an image: Select an image from the device's camera or gallery and upload it to the application. OpenCV is used to load the image and begin processing.

[0394] 2. Adjusting beautification parameters: We use Tkinter to provide intuitive sliders and dials for users to operate. Users can adjust beautification parameters by manipulating these UI elements.

[0395] 3. Real-time preview: Every time the user changes a parameter, the brightness and contrast of the image are adjusted using the OpenCV convertScaleAbs function, and the results are displayed in real time.

[0396] 4. Saving and sharing adjustments: Provides the ability to save the adjusted image on the device and share it on social media and other content distribution services.

[0397] Specific examples

[0398] For example, a user can upload a landscape photo taken with their smartphone to the application, enhance the color tone, slightly reduce the brightness, and share it on Instagram.

[0399] Prompt Sentence Examples

[0400] "Upload a landscape photo, intensify the colors, lower the brightness a bit, and share it to Instagram."

[0401] In this way, users can customize their illustrations to their liking and share them easily.

[0402] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0403] Step 1:

[0404] Users can upload photos and illustrations taken on their devices to the application.

[0405] Input: An image file selected from the device's camera or gallery.

[0406] Output: Image data loaded into the application.

[0407] Specific behavior: A user launches the application, clicks the image upload button, and selects an image from the device's camera roll or gallery. The selected image is then loaded into the application.

[0408] Step 2:

[0409] The device starts processing the image data read using OpenCV.

[0410] Input: The loaded image data.

[0411] Output: An image object that can be processed with OpenCV.

[0412] Specific operation: The application uses the OpenCV library to convert the loaded image data into an internal format so that it can be processed.

[0413] Step 3:

[0414] The user operates sliders and dials to adjust the beautification parameters.

[0415] Input: The value of a slider or dial as manipulated by the user.

[0416] Output: Adjusted beautification parameters.

[0417] Specific operation: The user operates sliders and dials within the application to set parameters such as "color intensity" and "brightness adjustment." The set values ​​are reflected in the application in real time.

[0418] Step 4:

[0419] The device will preview the image in real time based on the adjusted beautification parameters.

[0420] Input: adjusted beautification parameters and original image data.

[0421] Output: Real-time preview of the adjusted image.

[0422] Specific operation: The application uses OpenCV's convertScaleAbs function to adjust the brightness and contrast of the image based on the beautification parameters set by the user, and displays the results in real time.

[0423] Step 5:

[0424] Users can save the adjusted images and share them with other content distribution services.

[0425] Input: Adjusted image data.

[0426] Output: Saved image files and shared images.

[0427] Specific operation: The user presses the save button to save the adjusted image to the device, and then presses the share button to upload the image to Instagram or other social media.

[0428] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0429] "Example 1"

[0430] One embodiment of the present invention is an illustration beautification system incorporating an emotion engine. This system includes a function to automatically beautify illustrations drawn by a user, a function to provide a user interface that allows the user to operate intuitively, a function that allows the user to adjust beautification parameters, and an emotion engine that recognizes the user's emotions. The emotion engine recognizes the user's emotions from the user's facial expressions, tone of voice, text input, etc., and adjusts the beautification parameters according to the emotions. Specifically, when the user is feeling happy, the color tone of the illustration is adjusted to be brighter, and when the user is feeling sad, the color tone is adjusted to be darker.

[0431] It is possible to beautify illustrations according to the user's emotions, such as by adding a "beautifying" effect to the illustration.

[0432] "Example 2"

[0433] The emotion engine also has the ability to recognize changes in a user's emotions in real time and dynamically adjust the beautification parameters accordingly. For example, if a user starts drawing feeling happy but then changes to sadness midway through, the emotion engine will capture that change in real time and instantly adjust the beautification parameters. This makes it possible to beautify illustrations in response to changes in the user's emotions.

[0434] "Example 3"

[0435] Furthermore, the emotion engine also has a learning function to recognize the user's emotions. Specifically, it collects data such as the user's facial expressions, tone of voice, and text input, and uses a machine learning algorithm to train an emotion recognition model. This learning enables the emotion engine to recognize the user's emotions more accurately, enabling it to beautify illustrations that better suit the user's emotions.

[0436] The processing flow of each embodiment will be described below.

[0437] "Example 1"

[0438] Step 1: The user inputs their illustration into the system.

[0439] Step 2: The system uses an emotion engine to recognize the user's emotions based on their facial expressions, tone of voice, text input, etc.

[0440] Step 3: The emotion engine adjusts the beautification parameters according to the recognized emotion.

[0441] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[0442] "Example 2"

[0443] Step 1: The user inputs their illustration into the system.

[0444] Step 2: The system uses an emotion engine to recognize the user's emotions in real time based on their facial expressions, tone of voice, text input, etc.

[0445] Step 3: The emotion engine dynamically adjusts the beautification parameters according to the changes in the recognized emotions.

[0446] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[0447] "Example 3"

[0448] Step 1: Collect data such as the user's facial expressions, tone of voice, and text input.

[0449] Step 2: Using the collected data, the emotion engine uses machine learning algorithms to train an emotion recognition model.

[0450] Step 3: Using the trained emotion recognition model, recognize the user's emotion and adjust the beautification parameters according to that emotion.

[0451] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[0452] Example 1

[0453] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0454] Conventional illustration beautification systems were unable to take the user's emotions into account when automatically beautifying illustrations drawn by the user. As a result, beautification was not tailored to the user's emotions, making it difficult to increase user satisfaction. Furthermore, if the user interface was not intuitive, the operation was complicated and difficult to use.

[0455] The identification process by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes: means for acquiring image data representing an illustration drawn by a user; means for automatically adjusting the visual characteristics of the image data; means for recognizing the user's emotion using an emotion engine; a prompt statement instructing the adjustment of the visual characteristics of the image data based on the user's emotion; and means for adjusting the visual characteristics of the image data using a generative AI. This enables the illustration to be beautify in accordance with the user's emotion, thereby increasing user satisfaction. The server also includes means for adjusting the visual characteristics of the image data via a user interface that can be operated by the user. The automatic adjustment means adjusts the color tone, brightness, and contrast of the image data.

[0456] "Advanced image processing technology" refers to technology for adjusting visual characteristics such as color tone, brightness, and contrast of an image, and is technology that uses libraries such as OpenCV and TENSORFLOW (registered trademark).

[0457] "User-drawn illustrations" are image data created by users using hand-drawn or digital tools.

[0458] "Automatic beautification means" refers to algorithms or programs that automatically adjust the visual characteristics of illustrations drawn by users to improve their appearance.

[0459] An "intuitive user interface" is an interface designed to be easy for users to understand and operate, and includes UI elements such as sliders and dials.

[0460] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[0461] "Means for recognizing user emotions" refers to algorithms or programs that analyze the user's facial expressions, tone of voice, text input, etc. to determine the user's emotions.

[0462] The "means for adjusting beautification parameters" refers to an algorithm or program for changing the beautification parameters in response to the user's emotions and adjusting the visual characteristics of the illustration.

[0463] "Means for returning a beautified illustration to the user" refers to communication and display means for providing the adjusted illustration to the user.

[0464] The present invention relates to a system for automatically beautifying illustrations drawn by a user. A specific embodiment of this system will be described below.

[0465] System Overview

[0466] This system takes an illustration drawn by a user as input, automatically adjusts the visual characteristics of the illustration, such as color tone, brightness, and contrast, and outputs a beautified illustration. It also has the ability to recognize the user's emotions and adjust the beautification parameters according to those emotions.

[0467] Hardware and software used

[0468] Server: A server with high-performance computing power is used. The server receives illustration data from users and performs image processing and emotion recognition.

[0469] Device: A device such as a smartphone or PC used by a user. The device uploads illustrations and displays the beautified illustrations.

[0470] software:

[0471] OpenCV: An image processing library used to adjust the color, brightness, and contrast of illustrations.

[0472] TensorFlow: A machine learning library used to build an emotion recognition engine.

[0473] Program processing

[0474] The server receives the illustration uploaded by the user, and then automatically adjusts the visual characteristics of the illustration, such as color tone, brightness, and contrast, using OpenCV. Specifically, the server performs the following processes:

[0475] Color Tone Adjustment: Use OpenCV's cv2.cvtColor function to convert the color space of the illustration and adjust the color tone.

[0476] Brightness adjustment: Use the cv2.convertScaleAbs function to adjust the brightness of the illustration.

[0477] Contrast adjustment: Use the cv2.equalizeHist function to adjust the contrast of the illustration.

[0478] Furthermore, in systems incorporating an emotion engine, the server analyzes the user's facial expressions, tone of voice, and text input to recognize the user's emotions. The emotion engine uses a neural network model built using TensorFlow. If the user is happy, the server adjusts the color tone of the illustration to be brighter, and if the user is sad, the server adjusts the color tone to be darker.

[0479] Specific examples

[0480] As a concrete example, we show how a user can upload an illustration they have drawn and the system will beautify it.

[0481] Users upload illustrations: Users open the dedicated app on their smartphones, select the illustration they have drawn, and tap the "Upload" button.

[0482] The server receives the illustration: The server receives the uploaded illustration file and saves it in the / uploads directory.

[0483] The server adjusts the visual characteristics of the illustration: The server reads the saved illustration file and adjusts the color tone, brightness, and contrast using OpenCV.

[0484] The server uses an emotion engine to recognize the user's emotions: The server analyzes the user's facial expression image and determines that the user is happy.

[0485] The server uses generative AI to beautify the illustration according to the emotion: The server uses generative AI and OpenCV to brighten the color tone of the illustration and generate a beautified illustration.

[0486] The server returns the beautified illustration to the user: The server sends the beautified illustration as an HTTP response and displays it on the user's smartphone.

[0487] Prompt Sentence Examples

[0488] The user is happy. Use OpenCV to adjust the color tone, brightness, and contrast of the user's illustration based on the user's emotions.

[0489] In this way, a system is realized that automatically beautifies illustrations drawn by users and adjusts them according to the user's emotions.

[0490] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0491] Step 1:

[0492] The user uploads an illustration. Using a device (smartphone or PC), the user uploads the illustration they have drawn to the system. Specifically, they select the illustration file through a web browser or a dedicated app and click the upload button. The input is the illustration file selected by the user, and the output is the illustration data sent to the server.

[0493] Step 2:

[0494] The server receives the illustration. The server receives the illustration file uploaded by the user. The server temporarily stores the illustration data sent via the HTTP request. The input is the illustration data sent by the user, and the output is the illustration file saved on the server.

[0495] Step 3:

[0496] The server adjusts the visual characteristics of the illustration. The server uses the OpenCV library to automatically adjust the visual characteristics of the illustration, such as color tone, brightness, and contrast. The specific process is as follows:

[0497] Color Tone Adjustment: Use the cv2.cvtColor function to convert the color space of the illustration and adjust the color tone. The input is an illustration file saved on the server, and the output is the color-adjusted illustration data.

[0498] Brightness adjustment: Use the cv2.convertScaleAbs function to adjust the brightness of the illustration. The input is the color-adjusted illustration data, and the output is the brightness-adjusted illustration data.

[0499] Contrast adjustment: Use the cv2.equalizeHist function to adjust the contrast of the illustration. The input is the illustration data with brightness adjustment, and the output is the illustration data with contrast adjustment.

[0500] Step 4:

[0501] The server uses an emotion engine to recognize the user's emotions. The server uses an emotion engine to recognize the user's emotions. The emotion engine uses a neural network model built using TensorFlow. It analyzes the user's facial expression images, voice data, and text input to determine the user's emotions. The input is the user's facial expression images, voice data, and text input, and the output is the user's emotional data.

[0502] Step 5:

[0503] The server uses generative AI and OpenCV to beautify the illustration according to the user's emotions. The server beautifies the illustration according to the user's recognized emotions. For example, if the user is happy, the server adjusts the color tone of the illustration to be brighter. Specifically, it changes the color adjustment parameters and processes the illustration again using OpenCV. The input is the user's emotional data and illustration data with contrast adjusted, and the output is illustration data beautified according to the emotion.

[0504] Step 6:

[0505] The server returns the beautified illustration to the user. Specifically, the server sends the illustration data as an HTTP response and displays it on the user's device. The input is illustration data beautified according to the emotion, and the output is a beautified illustration displayed on the user's device. The server then provides a user interface that can be operated by the user for adjusting the visual characteristics of the image data. This user interface includes intuitive UI elements such as sliders and dials, allowing the user to make final adjustments to the visual characteristics.

[0506] (Application example 1)

[0507] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0508] Conventional illustration beautification systems provide a function to automatically beautify illustrations drawn by users, but because they cannot adjust beautification parameters according to the user's emotions, it is difficult to obtain optimal beautification results that match the user's emotions. Furthermore, there is a demand for a user interface that can be intuitively operated on mobile devices such as smartphones. Therefore, the challenge is to provide a system that can obtain beautification results that users are more satisfied with.

[0509] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0510] In this invention, the server includes means for automatically beautifying illustrations drawn by a user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for allowing the user to adjust beautification parameters, means for recognizing the user's emotions and adjusting the beautification parameters in accordance with the emotions, and means for operating as an application installed on a smartphone, thereby making it possible to obtain optimal beautification results in accordance with the user's emotions.

[0511] "Advanced image processing technology" is a technology that automatically adjusts visual characteristics such as color tone, brightness, and contrast of an image.

[0512] "User-drawn illustrations" are images created by users using hand-drawn or digital tools.

[0513] "Automatic beautification" refers to technology that has the ability to improve the visual characteristics of illustrations without user intervention.

[0514] A "user interface that users can operate intuitively" is an interface that users can easily understand and operate.

[0515] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[0516] "User-adjustable means" refers to a technology that allows users to change the beautification parameters according to their preferences.

[0517] "Means for recognizing user emotions" refers to technology that detects emotions from the user's facial expressions, tone of voice, etc.

[0518] The "means for adjusting beautification parameters according to emotions" is a technology that has the function of automatically changing beautification parameters based on the recognized emotions of the user.

[0519] An "application installed on a smartphone" is a software program that runs on a smartphone.

[0520] In one embodiment of the present invention, the EmotionArt application is installed on a smartphone. The EmotionArt application automatically beautifies illustrations drawn by the user and adjusts the visual characteristics of the illustration according to the user's emotions.

[0521] The server includes the following means:

[0522] 1. A means of automatically beautifying illustrations drawn by users using advanced image processing technology.

[0523] 2. A means of providing a user interface that is intuitive to the user.

[0524] 3. User adjustable means of beautification parameters.

[0525] 4. A means of recognizing user emotions and adjusting beautification parameters accordingly.

[0526] 5. A means that operates as an application installed on a smartphone.

[0527] This makes it possible to obtain optimal beautification results according to the user's feelings.

[0528] Hardware and software used

[0529] Hardware:

[0530] Smartphone (camera, microphone, display)

[0531] software:

[0532] OpenCV (image processing library)

[0533] EmotionEngine (emotion recognition engine)

[0534] ImageBeautifier (Image Beautifier)

[0535] Processing flow

[0536] 1. Loading an image: The user takes a photo of the illustration they have drawn using their smartphone camera and saves it as an image file.

[0537] 2. Emotion Recognition: Using the EmotionEngine, emotions are recognized from the user's facial expressions and tone of voice.

[0538] 3. Adjusting beautification parameters: Based on the recognized emotion, ImageBeautifier adjusts the visual characteristics of the illustration, such as color tone, brightness, and contrast.

[0539] 4. Saving the beautified image: The beautified illustration is saved as an image file and provided to the user.

[0540] Specific examples

[0541] Users draw illustrations on their smartphones and upload them to the app. The app captures the user's facial expressions with its camera and recognizes their emotions. If the recognized emotion is "joy," the app brightens the illustration's color tone. The beautified illustration is then displayed to the user and saved.

[0542] Prompt Sentence Examples

[0543] We are developing an application that beautifies illustrations drawn by users. Please generate a program for this application that has the function of adjusting the color tone, brightness, and contrast of the illustration according to the user's emotions. The user's emotions will be recognized from facial expressions and tone of voice. The libraries used are OpenCV, EmotionEngine, and ImageBeautifier.

[0544] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0545] Step 1:

[0546] The user draws an illustration on their smartphone and then takes a photo of it with their camera.

[0547] Input: User-drawn illustration

[0548] Output: Photographed illustration image file

[0549] Specific operation: The user launches the smartphone camera app and takes a picture of the illustration they have drawn. The captured image is then saved in the smartphone's storage.

[0550] Step 2:

[0551] The device reads the photographed illustration image file.

[0552] Input: Photographed illustration image file

[0553] Output: Image data

[0554] Specific operation: The EmotionArt application on the smartphone reads the image file and stores it in memory as image data.

[0555] Step 3:

[0556] The device uses EmotionEngine to recognize the user's emotions.

[0557] Input: User facial expressions and tone of voice

[0558] Output: Recognized emotion data

[0559] How it works: Using the smartphone's camera and microphone, it captures the user's facial expressions and tone of voice, and the Emotion Engine analyzes this data to recognize the user's emotions.

[0560] Step 4:

[0561] The device adjusts beautification parameters based on the recognized emotion.

[0562] Input: Recognized emotion data

[0563] Output: Adjusted beautification parameters

[0564] Specific operation: The EmotionArt application automatically sets ImageBeautifier's beautification parameters (color tone, brightness, contrast, etc.) based on the recognized emotion data.

[0565] Step 5:

[0566] The device uses ImageBeautifier to beautify the illustration.

[0567] Input: Image data, adjusted beautification parameters

[0568] Output: Beautified illustration image data

[0569] Specific operation: ImageBeautifier uses the image data and adjusted beautification parameters to adjust the color tone, brightness, contrast, etc. of the illustration to generate a beautified illustration.

[0570] Step 6:

[0571] The device saves the beautified illustration and displays it to the user.

[0572] Input: Beautified illustration image data

[0573] Output: Saved beautified illustration image file, beautified illustration displayed to the user

[0574] Specific operation: The beautified illustration image data is saved to the smartphone's storage and displayed on the application screen. The user can check the beautified illustration and make further adjustments if necessary.

[0575] Example 2

[0576] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0577] Conventional illustration beautification systems require users to manually adjust beautification parameters, making operation cumbersome. Furthermore, they lacked the functionality to dynamically adjust the beautification level of an illustration according to changes in the user's emotions, making it difficult to beautify illustrations in accordance with the user's emotions. This led to a problem of reduced user satisfaction.

[0578] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0579] In this invention, the server includes means for automatically beautifying illustrations drawn by a user using advanced image processing technology, means for providing a user interface that can be operated intuitively by the user, means for allowing the user to adjust beautification parameters, and means for recognizing changes in the user's emotions in real time and dynamically adjusting the beautification parameters in response to those changes, thereby enabling the user to beautify illustrations in accordance with their emotions while operating intuitively.

[0580] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing, modifying, and transforming images.

[0581] "User interface" refers to the screens and operating means that allow a user to interact with a system.

[0582] "Beautification parameters" refer to the settings used to adjust the visual characteristics of an illustration, such as color tone, brightness, and contrast.

[0583] A "slider" is a UI element that allows the user to adjust a value by dragging it.

[0584] A "dial" refers to a UI element that allows the user to adjust a value by rotating it.

[0585] "Emotional changes" refers to the user's emotional state changing over time.

[0586] "Real-time recognition" refers to instantly detecting and processing changes in a user's emotions.

[0587] "Dynamic adjustment" refers to automatically changing the setting value depending on the situation.

[0588] MODE FOR CARRYING OUT THE INVENTION

[0589] This invention is a system that automatically beautifies illustrations drawn by users and dynamically adjusts beautification parameters according to changes in the user's emotions. This system consists of three main components: a server, a terminal, and a user.

[0590] server

[0591] The server performs the following functions:

[0592] 1. Providing a user interface

[0593] The server generates a user interface (UI) and sends it to the device. This UI includes intuitive elements such as sliders and dials.

[0594] Software used: UI libraries such as React, Vue.js.

[0595] 2. Receiving user actions

[0596] The server receives user operation data sent from the device, including the position of the slider and the angle of the dial.

[0597] Software used: WebSocket, HTTP requests.

[0598] 3. Calculation of beautification parameters

[0599] The server calculates beautification parameters based on the received operation data, which determine the degree of beautification of the illustration.

[0600] Software used: Programming languages ​​such as Python, Node.js, etc.

[0601] 4. Emotion Data Analysis

[0602] The server receives and analyzes the user's emotional data sent from the device, and uses an emotion engine to recognize changes in the user's emotions in real time.

[0603] Software used: Sentiment analysis engines such as Microsoft® Azure® Emotion API.

[0604] 5. Dynamic adjustment of beautification parameters

[0605] The server dynamically adjusts the beautification parameters based on the analyzed emotional data, making it possible to beautify illustrations according to the user's emotions.

[0606] Terminal

[0607] The terminal performs the following functions:

[0608] 1. Displaying the user interface

[0609] The device displays the user interface received from the server, which includes UI elements such as sliders and dials.

[0610] Hardware used: PC, tablet, smartphone.

[0611] Software required: Browser, HTML, JavaScript (registered trademark).

[0612] 2. Sending user actions

[0613] When a user operates a slider or dial, the device transmits the operation data to the server.

[0614] Software used: JavaScript event listeners, WebSockets, HTTP requests.

[0615] 3. Collecting Emotional Data

[0616] The device uses sensors such as a camera and microphone to collect user emotional data.

[0617] Software used: WebRTC, MediaStream API.

[0618] User

[0619] The user performs the following operations:

[0620] 1. User Interface Operation

[0621] Users adjust the beautification parameters of the illustration by operating sliders and dials on their device.

[0622] What it does: The user drags a slider or rotates a dial using a mouse or touchscreen.

[0623] 2. Emotional changes

[0624] The user's emotions change as they draw the illustration, and these changes in emotions are collected by the device and sent to the server.

[0625] Specific behavior: Emotion data changes when the user smiles or frowns.

[0626] Specific examples

[0627] Hardware and software used

[0628] Hardware: PCs, tablets, smartphones

[0629] Software: User interface (UI) libraries (e.g., React, Vue.js), sentiment analysis engines (e.g., Microsoft Azure Emotion API)

[0630] Prompt Sentence Examples

[0631] "Please describe a system where the user can adjust the beautification parameters of an illustration by manipulating sliders. Also include the ability to dynamically adjust the beautification parameters in response to changes in the user's emotions."

[0632] By inputting this prompt into a generative AI model, a detailed description of the system can be obtained.

[0633] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0634] Step 1:

[0635] Providing a user interface

[0636] The server generates a user interface (UI) and sends it to the device. It uses a UI library (e.g., React or Vue.js) as input and generates HTML and JavaScript code as output. Specifically, the server uses the UI library to build a UI that includes intuitively operable elements such as sliders and dials, and sends this to the device.

[0637] Step 2:

[0638] User Interface Display

[0639] The device displays the user interface received from the server. It receives HTML and JavaScript code sent from the server as input and renders the UI in the browser as output. Specifically, the device uses the browser to parse the received code and display the UI, including sliders and dials.

[0640] Step 3:

[0641] Sending user actions

[0642] When a user operates a slider or dial, the device sends the operation data to the server. It receives user operation events as input and sends the operation data to the server as output. Specifically, the device uses a JavaScript event listener to capture user operations and sends them to the server via WebSocket or HTTP request.

[0643] Step 4:

[0644] Receiving user actions

[0645] The server receives user operation data sent from the device. It receives the operation data from the device as input and uses it as output to calculate beautification parameters. Specifically, the server receives user operation data in real time via WebSocket or HTTP request.

[0646] Step 5:

[0647] Calculation of beautification parameters

[0648] The server calculates beautification parameters based on the received operation data. It receives user operation data as input and generates beautification parameters as output. Specifically, the server uses a programming language such as Python or Node.js to execute the beautification algorithm and calculate the parameters.

[0649] Step 6:

[0650] Collecting Emotional Data

[0651] The device uses sensors such as a camera and microphone to collect user emotional data. It receives sensor data from the camera and microphone as input and sends emotional data to the server as output. Specifically, the device uses WebRTC and MediaStream API to collect data from the camera and microphone and send it to the server.

[0652] Step 7:

[0653] Emotional Data Analysis

[0654] The server receives and analyzes the user's emotional data sent from the device. It receives the emotional data from the device as input and generates the analysis results as output. Specifically, the server uses an emotion analysis engine such as the Microsoft Azure Emotion API to analyze the user's emotional data.

[0655] Step 8:

[0656] Dynamic adjustment of beautification parameters

[0657] The server dynamically adjusts the beautification parameters based on the analyzed emotion data. It receives the emotion analysis results as input and generates updated beautification parameters as output. Specifically, the server recalculates and updates the parameters of the beautification algorithm based on the emotion analysis results.

[0658] Step 9:

[0659] Illustration update

[0660] The terminal updates the illustration in real time based on the updated beautification parameters received from the server. The terminal receives the updated beautification parameters from the server as input and displays the updated illustration as output. Specifically, the terminal uses the browser to redraw the illustration based on the received beautification parameters.

[0661] (Application example 2)

[0662] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0663] Conventional illustration beautification systems require users to manually adjust beautification parameters, making it difficult to dynamically beautify illustrations in response to changes in the user's emotions. Furthermore, they lack an intuitive user interface, making it difficult to improve the user experience.

[0664] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0665] In this invention, the server includes means for automatically beautifying illustrations drawn by a user using advanced image processing technology, means for providing a user interface that can be operated intuitively by the user, means for allowing the user to adjust beautification parameters, and means for recognizing the user's emotions in real time and dynamically adjusting the beautification parameters in response to the user's emotions. This enables dynamic beautification of illustrations in response to changes in the user's emotions, improving the user experience through intuitive operation.

[0666] "Advanced image processing techniques" refers to advanced algorithms and methods for analyzing, transforming, and modifying images.

[0667] "User-drawn illustrations" refers to visual works created by users, either manually or using digital tools.

[0668] "Automatic beautification" refers to techniques or algorithms that enhance the visual characteristics of illustrations without user intervention.

[0669] "A user interface that users can operate intuitively" refers to an interface that is designed so that users can operate it easily and naturally.

[0670] "Beautification parameters" refer to the settings used to adjust the visual characteristics of an illustration, such as color tone, brightness, and contrast.

[0671] "User-adjustable means" refers to features and interfaces that allow users to change settings to suit their preferences.

[0672] "Means of recognizing user emotions in real time" refers to technology or systems that analyze a user's facial expressions and behavior to instantly determine their emotions at any given time.

[0673] "Means for dynamically adjusting beautification parameters according to emotions" refers to technologies or algorithms that automatically change beautification parameters based on recognized user emotions.

[0674] The following system configuration will be described as an embodiment of the present invention.

[0675] System Configuration

[0676] This system recognizes emotions in real time when a user draws an illustration using a smartphone, and dynamically adjusts the beautification parameters of the illustration according to those emotions. The system consists of the following main components:

[0677] 1. Smartphone

[0678] Camera: Captures the user's face and provides data for emotion recognition.

[0679] Touchscreen: Provides an interface for users to draw illustrations.

[0680] Processor: Performs image processing, emotion recognition, and UI operations.

[0681] 2. Software Components

[0682] OpenCV: Image processing library for beautifying illustrations.

[0683] EmotionRecognizer: An emotion recognition library that recognizes user emotions in real time.

[0684] UI Elements: A library that provides intuitive UI elements such as sliders and dials.

[0685] Program processing

[0686] The server implements the system using the following means:

[0687] 1. A method to automatically beautify user-drawn illustrations using advanced image processing technology

[0688] Using OpenCV, we apply beautification processing to illustrations drawn by users, specifically by applying filters such as Gaussian blur to improve the visual characteristics of the illustrations.

[0689] 2. A means of providing a user interface that users can operate intuitively

[0690] The UI Elements library provides intuitive UI elements such as sliders and dials, allowing users to easily adjust beautification parameters.

[0691] 3. User-adjustable beautification parameters

[0692] Allow users to adjust beautification parameters according to their preferences using sliders and dials, such as hue, brightness, contrast, etc.

[0693] 4. A means to recognize user emotions in real time and dynamically adjust beautification parameters according to those emotions.

[0694] EmotionRecognizer captures the user's face and recognizes their emotions in real time. Based on the recognized emotion, the beautification parameters are dynamically adjusted. For example, if the user feels "joy," the beautification parameters are set high, and if they feel "sad," the parameters are set low.

[0695] Specific examples

[0696] For example, if a user is drawing an illustration on their smartphone, the camera captures the user's face and the emotion recognition library detects "joy," the beautification parameters will be set higher, making the illustration more vivid and bright. Conversely, if "sadness" is detected, the beautification parameters will be set lower, making the illustration a little darker and more subdued.

[0697] Prompt Sentence Examples

[0698] When a user draws an illustration on their smartphone, the camera captures the user's face. If the emotion recognition library detects "joy," the beautification parameters are set higher, making the illustration more vivid and bright. Conversely, if it detects "sadness," the beautification parameters are set lower, making the illustration a little darker and more subdued.

[0699] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0700] Step 1:

[0701] The device captures the user's face using a camera. The input is the user's face image, and the output is the captured face image data. This face image data is used as preprocessing for emotion recognition.

[0702] Step 2:

[0703] The device uses an emotion recognition library (EmotionRecognizer) to recognize the user's emotions in real time from the captured facial image data. The input is the facial image data, and the output is the recognized emotion (e.g., joy, sadness, surprise, etc.). This emotion data is used to adjust the subsequent beautification parameters.

[0704] Step 3:

[0705] The device allows users to draw illustrations using a touch screen. The input is the user's touch operation, and the output is the drawn illustration data. This illustration data is then subjected to beautification processing.

[0706] Step 4:

[0707] The device uses user interface (UI Elements) to display sliders and dials to allow the user to adjust beautification parameters. The input is the user's operation of the slider or dial, and the output is the adjusted beautification parameters. These beautification parameters are used to beautify the illustration.

[0708] Step 5:

[0709] The device dynamically adjusts the beautification parameters based on the recognized emotion data. The input is the emotion data and the beautification parameters adjusted by the user, and the output is the final adjusted beautification parameters. For example, if the user feels "joy," the beautification parameters are set high, and if they feel "sad," they are set low.

[0710] Step 6:

[0711] The device uses OpenCV to beautify the illustration based on the final adjusted beautification parameters. The input is the illustration data and the adjusted beautification parameters, and the output is the beautified illustration data. Specifically, filters such as Gaussian blur are applied to improve the visual characteristics of the illustration.

[0712] Step 7:

[0713] The terminal displays the beautified illustration to the user. The input is beautified illustration data, and the output is a beautified illustration that the user can visually confirm. This allows the user to check the beautified illustration in real time.

[0714] Example 3

[0715] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0716] Conventional illustration beautification systems allow users to adjust beautification parameters, but they are unable to beautify illustrations in accordance with the user's emotions. This makes it difficult to beautify illustrations in accordance with the user's emotions, and it is difficult to increase user satisfaction. Furthermore, while systems with emotion recognition capabilities exist, the technology to apply this to beautifying illustrations has not yet been established.

[0717] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0718] In this invention, the server includes means for collecting user emotion data, means for analyzing the collected emotion data and training an emotion recognition model, and means for beautifying illustrations in accordance with the user's emotions, thereby enabling beautification of illustrations in accordance with the user's emotions.

[0719] "Advanced image processing technology" refers to technology for adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[0720] A "user interface" is an interface that allows a user to operate a system, and includes UI elements that can be operated intuitively, such as sliders and dials.

[0721] "Beautification parameters" are parameters for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[0722] "User emotional data" refers to data about emotions collected from a user's facial expressions, tone of voice, text input, etc.

[0723] An "emotion recognition model" is a model trained using machine learning algorithms to recognize a user's emotions.

[0724] "Illustration beautification" is a process that makes illustrations drawn by users more attractive by adjusting their visual characteristics such as color tone, brightness, and contrast.

[0725] MODE FOR CARRYING OUT THE INVENTION

[0726] This invention is a system that automatically beautifies illustrations drawn by a user using advanced image processing technology, and further adjusts the beautification in response to the user's emotions. Specific embodiments of this system will be described below.

[0727] Hardware and software used

[0728] Hardware: User's device (PC, smartphone, etc.), camera, microphone

[0729] Software: Web browser, dedicated application, image processing library (OpenCV, etc.), machine learning library (TensorFlow, PyTorch, etc.)

[0730] Program processing

[0731] Adjusting beautification parameters

[0732] The user inputs beautification parameters such as "color intensity" and "brightness adjustment" through the device interface. For example, using a smartphone app, the user moves the slider to set color intensity to 50. The device then sends these parameters to the server, which then beautifies the illustration based on the received parameters.

[0733] Emotion data collection and analysis

[0734] The user's device uses a camera and microphone to collect the user's facial expressions and tone of voice. For example, the camera takes a picture of the user's face and captures facial expression data. The device then sends the collected emotional data to a server, which uses a machine learning algorithm to train an emotion recognition model. The trained model recognizes the user's emotions and beautifies the illustration according to those emotions.

[0735] Beautification and display of illustrations

[0736] The server uses the learned emotion recognition model to beautify the illustration according to the user's emotions. For example, if the user is smiling, it adds warm colors to the illustration. Finally, the beautified illustration is sent to the device and displayed to the user.

[0737] Specific examples

[0738] The user opens the app on their smartphone and sets the color intensity to 50 and the brightness adjustment to 20. The device sends these parameters to the server, which then beautifies the illustration based on the received parameters. Next, if the user smiles through the camera, the device sends the facial expression data to the server. The server trains an emotion recognition model and adds warm color tones to the illustration according to the user's emotions. Finally, the beautified illustration is sent to the device and displayed to the user.

[0739] Prompt Sentence Examples

[0740] "Please describe a system in which a user sets parameters for beautifying an illustration, and an emotion engine recognizes the user's emotions and beautifies the illustration. Please include the names of the specific hardware and software."

[0741] In this way, the present invention provides a function for beautifying an illustration based on the user's input and emotions. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[0742] Step 1:

[0743] The user inputs the beautification parameters.

[0744] Users input beautification parameters such as "color intensity" and "brightness adjustment" through the device interface. For example, using a smartphone app, they can move the slider to set color intensity to 50. The input parameters are saved on the device.

[0745] Step 2:

[0746] The terminal transmits the beautification parameters to the server.

[0747] The device sends the beautification parameters entered by the user to the server using an HTTP request, such as a color intensity of 50 and brightness adjustment of 20. The server receives this and proceeds to the next step.

[0748] Step 3:

[0749] The server receives the beautification parameters and beautifies the illustration.

[0750] The server uses an image processing library (such as OpenCV) to beautify the illustration based on the received beautification parameters. For example, it sets the color intensity to 50 and adjusts the brightness to 20. The beautified illustration is saved on the server.

[0751] Step 4:

[0752] Collect user sentiment data.

[0753] The user's device uses a camera and microphone to collect the user's facial expressions and tone of voice. For example, the camera takes a picture of the user's face and captures facial expression data. The collected emotional data is stored on the device.

[0754] Step 5:

[0755] The device transmits the emotion data to the server.

[0756] The device sends the collected emotional data to the server using an HTTP request. Specifically, the user's facial expression data and tone of voice data are sent. The server receives this data and proceeds to the next step.

[0757] Step 6:

[0758] The server analyzes the emotion data and trains an emotion recognition model.

[0759] The server analyzes the received emotion data and uses machine learning algorithms (such as TensorFlow or PyTorch) to train an emotion recognition model. For example, it can determine whether the user is smiling based on facial expression data. The trained model is stored on the server.

[0760] Step 7:

[0761] The server beautifies the illustration according to the user's emotions.

[0762] The server uses the trained emotion recognition model to beautify the illustration according to the user's emotion. For example, if the user is smiling, it adds warm colors to the illustration. The beautified illustration is then stored on the server.

[0763] Step 8:

[0764] The server sends the beautified illustration to the device.

[0765] The server sends the beautified illustration to the terminal using an HTTP response. Specifically, the beautified illustration data is sent. The terminal receives this and proceeds to the next process.

[0766] Step 9:

[0767] The device displays a beautified illustration to the user.

[0768] The device displays the beautified illustration to the user. For example, the illustration is displayed in a smartphone app. The user can then check the beautified illustration.

[0769] (Application example 3)

[0770] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0771] Conventional image beautification systems have had the problem of making it difficult for users to beautify illustrations according to their own emotions and preferences. Furthermore, the lack of an intuitive interface for users to adjust beautification parameters often made operation cumbersome. Furthermore, there was no technology available to recognize the user's emotions and beautify illustrations based on those emotions, making it impossible to achieve optimal beautification according to the user's emotions.

[0772] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0773] In this invention, the server includes means for automatically beautifying illustrations drawn by a user using advanced image processing technology, means for providing a user interface that the user can intuitively operate, means for the user to adjust beautification parameters, means including an emotion recognition engine for recognizing the user's emotions, and means for beautifying the illustration based on the recognized emotions. This allows the user to beautify the illustration according to their own emotions and preferences, and the beautification parameters can be easily adjusted through the intuitively operable interface. Furthermore, the use of the emotion recognition engine enables optimal beautification according to the user's emotions.

[0774] "Advanced image processing technology" refers to advanced algorithms and techniques for adjusting visual characteristics such as color tone, brightness, and contrast of an image to enhance it.

[0775] "User interface" is a general term for the screens and input devices that users use to operate a system, and includes UI elements such as sliders and dials that can be operated intuitively.

[0776] "Beautification parameters" are setting values ​​for adjusting the visual characteristics of an illustration, such as color tone, brightness, and contrast, and can be freely adjusted by the user.

[0777] An "emotion recognition engine" is a system that collects data such as a user's facial expressions, tone of voice, and text input, and uses machine learning algorithms to recognize the user's emotions.

[0778] The "means for beautifying illustrations based on recognized emotions" is a method for automatically adjusting the beautification parameters of an illustration in accordance with the user's emotions recognized by an emotion recognition engine, thereby achieving optimal beautification.

[0779] A system for implementing this invention includes advanced image processing technology for automatically beautifying illustrations drawn by a user, a user interface that can be operated intuitively by the user, means for allowing the user to adjust beautification parameters, an emotion recognition engine for recognizing the user's emotions, and means for beautifying the illustration based on the recognized emotions.

[0780] System configuration

[0781] Hardware: Smartphone

[0782] Software: Python, OpenCV, Keras, emotion recognition library

[0783] Program processing explanation

[0784] 1. Image Loading: Load an image taken by the user from the smartphone camera or gallery. Use OpenCV to obtain the image data.

[0785] 2. Emotion Recognition: Using an emotion recognition model trained with Keras, we recognize emotions from the user's facial expressions, tone of voice, text input, etc. We use an emotion recognition library to collect and analyze user emotion data.

[0786] 3. Adjusting beautification parameters: We provide intuitive UI elements such as sliders and dials to adjust the intensity and brightness of the color tone specified by the user. By manipulating these UI elements, users can freely adjust the beautification parameters.

[0787] 4. Emotion-based beautification: Automatically adjust image beautification parameters based on the recognized emotion. For example, if the user is recognized as "happy," the image will be beautified by setting the color intensity to 1.2 and the brightness to 30.

[0788] 5. Save the result: Save the beautified image to your smartphone's gallery. Use OpenCV to save the processed image data.

[0789] Specific examples

[0790] Input image: A landscape photo taken by the user

[0791] Emotion: When a user is perceived as "happy"

[0792] Beautification parameters: Color intensity 1.2, Brightness 30

[0793] Prompt Sentence Examples

[0794] Use a landscape photo taken by the user as the input image, and if the emotion engine recognizes it as "happy," beautify it by setting the color intensity to 1.2 and the brightness to 30.

[0795] In this way, users can beautify illustrations according to their own emotions and preferences, and the beautification parameters can be easily adjusted through an intuitive interface. In addition, by using an emotion recognition engine, optimal beautification can be achieved according to the user's emotions.

[0796] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0797] Step 1:

[0798] The device reads images taken by the user from the smartphone camera or gallery. Specifically, it uses OpenCV to acquire image data. The input is an image file selected by the user, and the output is an array of image data.

[0799] Step 2:

[0800] The server uses an emotion recognition model trained with Keras to recognize emotions from the user's facial expressions, tone of voice, text input, etc. Specifically, it uses an emotion recognition library to collect and analyze user emotion data. The input is an array of image data, and the output is a label of the recognized emotion.

[0801] Step 3:

[0802] The device provides intuitive UI elements such as sliders and dials for adjusting the intensity and brightness of the user-specified color tone. By manipulating these UI elements, the user can freely adjust the beautification parameters. The input is the user's operation, and the output is the adjusted beautification parameters.

[0803] Step 4:

[0804] The server automatically adjusts image beautification parameters based on the recognized emotion. Specifically, it sets color intensity and brightness according to the emotion recognized by the emotion recognition engine. The input is the recognized emotion label and the beautification parameters adjusted by the user, and the output is the optimized beautification parameters.

[0805] Step 5:

[0806] The device beautifies the image based on the beautification parameters. Specifically, it uses OpenCV to adjust the color intensity and brightness and process the image. The input is the optimized beautification parameters and an array of image data, and the output is the beautified image data.

[0807] Step 6:

[0808] The device saves the beautified image to the smartphone's gallery. Specifically, OpenCV is used to save the processed image data. The input is the beautified image data, and the output is the saved image file.

[0809] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0810] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0811] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0812] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0813] [Second embodiment]

[0814] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0815] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0816] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0817] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0818] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0819] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0820] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0821] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0822] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0823] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0824] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0825] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0826] "Example 1"

[0827] One embodiment of the present invention provides a system for automatically beautifying user-drawn illustrations. This system uses advanced image processing technology to automatically beautify user-drawn illustrations. Specifically, the system receives a user-drawn illustration as input, automatically adjusts the visual characteristics of the illustration, such as color tone, brightness, and contrast, and outputs a beautified illustration.

[0828] "Example 2"

[0829] In addition, as one embodiment of the present invention, an intuitively operable user interface is provided. This user interface includes intuitively operable UI elements such as sliders and dials. Specifically, the user can adjust beautification parameters by operating the sliders and dials. This allows the user to adjust the degree of beautification of the illustration to suit their preferences.

[0830] "Example 3"

[0831] Furthermore, as one embodiment of the present invention, a means is provided for allowing a user to adjust beautification parameters. Specifically, the user can freely adjust the degree of beautification of an illustration by directly inputting beautification parameters. For example, the user can beautify an illustration to suit their preferences by adjusting parameters such as "color intensity" and "brightness adjustment."

[0832] The processing flow of each embodiment will be described below.

[0833] "Example 1"

[0834] Step 1: The user inputs the illustration they have drawn into the system. This can be done by file upload, drag and drop, etc.

[0835] Step 2: The system analyzes the input illustration and automatically adjusts visual characteristics such as color tone, brightness, and contrast. This adjustment is done using a pre-trained machine learning model.

[0836] Step 3: The system outputs the adjusted illustration, which can be displayed directly on the screen or downloaded as a file.

[0837] "Example 2"

[0838] Step 1: The user operates the system's user interface, specifically, by manipulating sliders and dials to adjust the beautification parameters.

[0839] Step 2: The system adjusts the beautification parameters according to the user's operations and beautifies the illustration.

[0840] Step 3: The system outputs the beautified illustration, which can be displayed directly on the screen or downloaded as a file.

[0841] "Example 3"

[0842] Step 1: The user directly inputs the beautification parameters, such as "color intensity" and "brightness adjustment," using numerical values ​​or sliders.

[0843] Step 2: The system beautifies the illustration based on the beautification parameters entered by the user.

[0844] Step 3: The system outputs the beautified illustration, which can be displayed directly on the screen or downloaded as a file.

[0845] Example 1

[0846] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0847] Conventional illustration beautification systems require users to manually adjust color, brightness, and contrast, making operation cumbersome. Furthermore, they lack an intuitive user interface, making it difficult for users to easily complete the beautification process. Furthermore, the lack of real-time progress indicators and notification of process completion negatively impacted the user experience.

[0848] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for a user to upload an illustration, a means for the server to receive the illustration, a means for the server to adjust color tone, a means for the server to adjust brightness, a means for the server to adjust contrast, a means for the server to generate a beautified illustration, and a means for the server to provide the beautified illustration to the user. This allows the user to easily beautify an illustration through an intuitive interface. In addition, the user experience is improved by real-time progress display and processing completion notification.

[0849] "Advanced image processing technology" refers to technology for automatically adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[0850] A "user interface" is an interface that includes a screen and input devices that allow a user to operate a system.

[0851] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[0852] "Means for users to upload illustrations" refers to a function that allows users to send illustration files to the system from their own devices.

[0853] "Means for the server to receive illustrations" refers to the function by which the server receives and stores illustration files sent by users.

[0854] "Means for the server to adjust color tone" is a function that allows the server to automatically change the hue and saturation of an illustration.

[0855] "Means for the server to adjust brightness" is a function that allows the server to automatically change the brightness of the illustration.

[0856] "Means for the server to adjust contrast" is a function that allows the server to automatically change the difference in brightness of the illustration.

[0857] The "means for the server to generate a beautified illustration" is a function for the server to create a new illustration after adjusting the color tone, brightness, and contrast.

[0858] "Means for the server to provide beautified illustrations to users" refers to a function for providing beautified illustrations generated by the server to users in a downloadable format.

[0859] This invention relates to a system for automatically beautifying illustrations drawn by users. The system provides an intuitive user interface and allows users to adjust beautification parameters to adjust visual characteristics such as color tone, brightness, and contrast.

[0860] Hardware and software used

[0861] The server is a computer system that runs Python programs and uses OpenCV and Pillow as image processing libraries. Users access the system using devices connected to the Internet (PCs, smartphones, tablets, etc.).

[0862] Program processing explanation

[0863] The server receives the illustration uploaded by the user and then adjusts the image's color using OpenCV. Specifically, the server converts the image to HSV (hue, saturation, brightness) space, adjusts the hue appropriately, and then converts it back to RGB space.

[0864] The server then uses Pillow to adjust the image brightness, specifically using Pillow's ImageEnhance.Brightness class to adjust the brightness, and then using Pillow's ImageEnhance.Contrast class to adjust the contrast.

[0865] Through these processes, a beautified illustration is generated and provided to the user, who can then obtain it by clicking the download link on their device.

[0866] Specific examples

[0867] As a concrete example, consider the case of beautifying an illustration drawn by a user, "example_illustration.png."

[0868] 1. User Action:

[0869] The user uploads "example_illustration.png" to the system from their device.

[0870] 2. Server processing:

[0871] The server receives the uploaded image and adjusts the color, brightness, and contrast.

[0872] Save the adjusted image as "beautified_illustration.png".

[0873] 3. Output to the user:

[0874] The server provides the beautified illustration "beautified_illustration.png" to the user.

[0875] An example of a prompt is as follows:

[0876] Upload your own illustration "example_illustration.png". The system will automatically adjust the color, brightness, and contrast to provide you with a beautified illustration. You will receive a notification when the process is complete.

[0877] This is how the system works, automatically beautifying illustrations drawn by users.

[0878] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0879] Step 1:

[0880] Users upload illustrations.

[0881] A user uploads an illustration file (e.g., "example_illustration.png") that they have drawn from their terminal through the system's web interface. The input is the user's illustration file, and the output is the illustration file sent to the server.

[0882] Step 2:

[0883] The server receives the illustration.

[0884] The server receives the illustration file uploaded by the user and saves it in the specified directory. The input is the illustration file sent by the user, and the output is the illustration file saved on the server. Specifically, the server checks the file format and size and verifies that it is in the appropriate format.

[0885] Step 3:

[0886] The server adjusts the color tone.

[0887] The server loads the illustration using OpenCV's cv2.imread function. Next, it converts the image to HSV (hue, saturation, brightness) space using the cv2.cvtColor function. The hue is adjusted appropriately and then converted back to RGB space. The input is an illustration file stored on the server, and the output is an illustration with adjusted color tones. Specifically, the hue is shifted by 10 degrees.

[0888] Step 4:

[0889] The server adjusts the brightness.

[0890] The server uses Pillow's ImageEnhance.Brightness class to adjust the brightness of the image. Specifically, it increases the brightness by 1.2 times. The input is an illustration with adjusted color tone, and the output is an illustration with adjusted brightness.

[0891] Step 5:

[0892] The server adjusts the contrast.

[0893] The server uses Pillow's ImageEnhance.Contrast class to adjust the contrast of the image. Specifically, it increases the contrast by 1.3 times. The input is an illustration with adjusted brightness, and the output is an illustration with adjusted contrast.

[0894] Step 6:

[0895] The server generates the beautified illustration.

[0896] The server saves the adjusted image as "beautified_illustration.png". The input is a contrast-adjusted illustration, and the output is a beautified illustration file.

[0897] Step 7:

[0898] The server provides the beautified illustrations to the user.

[0899] The server provides the beautified illustration "beautified_illustration.png" to the user. The user can obtain the beautified illustration by clicking the download link from their device. The input is the beautified illustration file, and the output is the download link provided to the user. Specifically, the server displays the progress in real time and notifies the user after the process is completed.

[0900] (Application example 1)

[0901] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0902] While existing illustration beautification systems automatically beautify user-drawn illustrations, they lack a way to easily share the beautified illustrations with content distribution services. Additionally, the customization options for the beautification process are limited, making it difficult for users to customize the beautification process to their own preferences.

[0903] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0904] In this invention, the server includes means for automatically beautifying illustrations drawn by users using advanced image processing technology, means for providing a user interface that can be intuitively operated by users, means for allowing users to adjust beautification parameters, means for allowing users to directly share beautified illustrations to a content distribution service, and means for providing customization options for the beautification process, thereby enabling users to easily share beautified illustrations and further customize the beautification process to their own preferences.

[0905] "Advanced image processing technology" refers to technology for automatically adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[0906] "User-drawn illustrations" are paintings or drawings created by users, either by hand or using digital tools.

[0907] "Automatic beautification" refers to algorithms or software that enhance the visual characteristics of illustrations without user intervention.

[0908] A "user interface that users can operate intuitively" is an interface that is designed so that users can operate it easily and intuitively.

[0909] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[0910] "User adjustable means" refers to a feature that allows the user to manually change the beautification parameters.

[0911] A "content distribution service" is a service that distributes digital content such as images and videos over the Internet.

[0912] "Shareable means" refers to the ability to share aestheticized illustrations with other users and platforms.

[0913] "Customization Options" are settings and features that allow users to tailor the beautification process to their own preferences.

[0914] As an embodiment of the present invention, the following system can be constructed.

[0915] System Overview

[0916] The system automatically beautifies illustrations drawn by users and shares the beautified illustrations with a content distribution service. The system includes a smartphone, a server, and software using advanced image processing technology.

[0917] Hardware and software used

[0918] Hardware: Smartphone (iOS or Android), server

[0919] Software: Python, OpenCV (image processing library)

[0920] Data processing and calculation

[0921] 1. Loading images

[0922] Users can take a photo of an illustration they have drawn on their smartphone or upload it using the smartphone's camera or file upload function.

[0923] 2. Adjusting the color tone

[0924] The server converts the image to the HSV color space and increases the saturation, which makes the illustration appear more vibrant.

[0925] 3. Adjusting the brightness

[0926] The server performs a scale transformation to increase the brightness of the image, which makes the illustration brighter.

[0927] 4. Adjusting the contrast

[0928] The server converts the image to the LAB color space and performs histogram equalization on the L channel to enhance the contrast, which improves the visual characteristics of the illustration.

[0929] 5. Save and share images

[0930] The beautified images are stored on a server, and users can share them directly to content distribution services via smartphone applications.

[0931] Specific examples

[0932] Users can take a photo of an illustration they've drawn with their smartphone and upload it to the app, where the server automatically beautifies it. The beautified illustration can then be adjusted in color, brightness, and contrast to suit the user's preferences. Finally, the beautified illustration can be shared on social media and other content distribution services.

[0933] Prompt Sentence Examples

[0934] "Develop an app that beautifies illustrations drawn by users. The app will automatically adjust the color tone, brightness, and contrast of the illustration and output the beautified illustration."

[0935] In this way, a system can be realized that allows users to easily create and share aesthetic illustrations.

[0936] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0937] Step 1:

[0938] The user takes a photo of an illustration drawn on their smartphone or uploads it. The input is the illustration image drawn by the user, and the output is an image file saved on the smartphone. Specifically, the user takes a photo of the illustration using the smartphone's camera function, or uploads an existing image file to the application.

[0939] Step 2:

[0940] The device sends the photographed or uploaded illustration image to the server. The input is the image file saved on the smartphone, and the output is the image data transferred to the server. Specifically, the smartphone application calls an API to send the image file to the server.

[0941] Step 3:

[0942] The server reads the received illustration image. The input is the image data transferred to the server, and the output is an image object loaded into memory. Specifically, the server reads the image file using Python and OpenCV.

[0943] Step 4:

[0944] The server adjusts the color tone of an image. The input is an image object loaded into memory, and the output is an image object with the adjusted color tone. Specifically, the server converts the image to the HSV color space and increases the saturation.

[0945] Step 5:

[0946] The server adjusts the brightness of the image. The input is a color-adjusted image object, and the output is a brightness-adjusted image object. Specifically, the server performs a scale transformation to increase the brightness of the image.

[0947] Step 6:

[0948] The server adjusts the contrast of an image. The input is a brightness-adjusted image object, and the output is a contrast-adjusted image object. Specifically, the server converts the image to the LAB color space and performs histogram equalization on the L channel.

[0949] Step 7:

[0950] The server saves the beautified image. The input is a contrast-adjusted image object, and the output is a beautified image file saved on the server. Specifically, the server saves the beautified image as a file.

[0951] Step 8:

[0952] The device obtains the beautified image from the server. The input is the beautified image file stored on the server, and the output is the beautified image file downloaded to the smartphone. Specifically, the smartphone application calls an API to download the beautified image from the server.

[0953] Step 9:

[0954] The user shares the beautified illustration on a content distribution service. The input is a beautified image file downloaded to a smartphone, and the output is a beautified image uploaded to the content distribution service. Specifically, the user uses a smartphone application to share the beautified illustration on social media or other content distribution services.

[0955] Example 2

[0956] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0957] With conventional image processing systems, beautifying user-drawn illustrations required specialized knowledge and complex operations, and lacked an intuitive user interface. It was also difficult to adjust the beautification parameters, making it difficult for users to beautify illustrations to their own preferences. Furthermore, beautification processing using generative AI models was not performed in real time, resulting in delayed feedback to user operations.

[0958] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0959] In this invention, the server includes a means for converting beautification parameters operated by the user into a prompt sentence, a means for inputting the prompt sentence into a generative AI model to beautify the illustration, and a means for displaying the beautified illustration to the user, thereby enabling the user to adjust the beautification parameters and check the beautified illustration in real time through an intuitive user interface.

[0960] "Advanced image processing technology" is a general term for advanced algorithms and methods used to automatically beautify user-drawn illustrations.

[0961] "User interface" is a general term for the screens and input devices that users use to operate a system, and includes UI elements that can be operated intuitively.

[0962] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[0963] A "prompt" is a textual instruction used to input user-set beautification parameters into a generative AI model.

[0964] A "generative AI model" is an artificial intelligence model that beautifies illustrations based on input prompts, and examples include "Stable Diffusion" and "DALL-E."

[0965] A "slider" is a UI element that allows the user to adjust a value by dragging it.

[0966] A "dial" is a UI element that allows the user to adjust a value by rotating it.

[0967] The "server" is a computer system that receives prompt text, inputs it into a generative AI model, generates a beautified illustration, and sends it back to the device.

[0968] A "terminal" is a device operated by a user, which has the functions of providing a user interface, acquiring beautification parameters, generating prompt sentences, and sending them to a server.

[0969] This invention is a system for automatically beautifying illustrations drawn by users, providing a user interface that allows users to operate intuitively. The system converts beautification parameters operated by the user into prompt sentences, which are then input into a generative AI model to beautify the illustration. A specific embodiment of this system is described below.

[0970] Hardware and software used

[0971] Hardware

[0972] Terminal: A device operated by the user that provides a user interface, acquires beautification parameters, generates prompts, and sends them to the server. Examples include PCs, tablets, and smartphones.

[0973] Server: A computer system that receives prompt text, inputs it into a generative AI model, generates a beautified illustration, and sends it back to the device.

[0974] software

[0975] User Interface: Includes UI elements that users can operate intuitively, such as sliders and dials.

[0976] Generative AI model: An artificial intelligence model that beautifies illustrations based on prompts. Specifically, "Stable Diffusion" and "DALL-E" are used.

[0977] Data processing and calculation

[0978] User operations

[0979] The user operates the user interface on the device, specifically, by using sliders and dials to adjust the beautification parameters. For example, when the user moves the slider to the right, the beautification parameter increases.

[0980] Terminal handling

[0981] The device obtains the position information of the slider or dial operated by the user in real time. This position information is interpreted as a beautification parameter. For example, if the slider is set to the maximum value, the beautification parameter is "100". The device generates a prompt sentence based on the obtained beautification parameter. For example, if the beautification parameter is "100", the device generates the prompt sentence "Beautification parameter: 100".

[0982] Server Processing

[0983] The server receives the prompt sent from the device. It inputs the received prompt into the generative AI model. The generative AI model beautifies the illustration based on the input prompt. For example, if a prompt with "beautification parameter: 100" is input, the beautification level of the illustration is set to the maximum and processing is performed. The server receives the beautified illustration from the generative AI model and sends it back to the device.

[0984] Terminal display

[0985] The terminal receives the beautified illustration sent back from the server and displays it to the user, who can then check the illustration based on the beautification parameters he or she has set.

[0986] Specific examples

[0987] Prompt Sentence Examples

[0988] "Beautification parameter: 100"

[0989] By inputting this prompt into the generative AI model, the degree of beautification of the illustration can be set to the maximum.

[0990] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0991] Step 1:

[0992] The user interacts with a UI element.

[0993] The user operates the user interface on the device and adjusts the beautification parameters using sliders and dials. For example, when the user moves the slider to the right, the beautification parameter increases. The input is the user's operation, and the output is the adjusted beautification parameter.

[0994] Step 2:

[0995] The terminal acquires the beautification parameters.

[0996] The device obtains the position information of the slider or dial operated by the user in real time. This position information is interpreted as a beautification parameter. For example, if the slider is set to the maximum value, the beautification parameter is "100". The input is the user's operation position information, and the output is the beautification parameter.

[0997] Step 3:

[0998] The terminal generates a prompt.

[0999] The terminal generates a prompt sentence based on the acquired beautification parameter. For example, if the beautification parameter is "100", the terminal generates the prompt sentence "Beautification parameter: 100". The input is the beautification parameter, and the output is the prompt sentence.

[1000] Step 4:

[1001] The terminal sends the prompt to the server.

[1002] The terminal sends the generated prompt text to the server. This transmission is performed using a communication protocol such as an HTTP request. The input is the prompt text, and the output is a transmission completion notification to the server.

[1003] Step 5:

[1004] The server receives the prompt.

[1005] The server receives the prompt sent from the terminal. The received prompt is used in the next processing step. The input is the prompt, and the output is a confirmation of receipt.

[1006] Step 6:

[1007] The server inputs a prompt sentence into the generative AI model.

[1008] The server inputs the received prompt sentence into a generative AI model. For example, generative AI models such as "Stable Diffusion" or "DALL-E" can be used. The input is the prompt sentence, and the output is a notification to the generative AI model that the input has been completed.

[1009] Step 7:

[1010] Generative AI models beautify illustrations.

[1011] The generative AI model beautifies the illustration based on the input prompt. For example, if a prompt with "Beautification parameter: 100" is input, the beautification level of the illustration is set to the maximum and processing is performed. The input is the prompt, and the output is a beautified illustration.

[1012] Step 8:

[1013] The server returns the beautified illustration to the device.

[1014] The server receives the beautified illustration from the generative AI model and returns it to the device. This return is also done using a communication protocol such as an HTTP response. The input is the beautified illustration, and the output is a notification of completion of transmission to the device.

[1015] Step 9:

[1016] The device displays a beautified illustration.

[1017] The terminal receives the beautified illustration sent back from the server and displays it to the user. The user can then check the illustration based on the beautification parameters they set. The input is the beautified illustration, and the output is the display to the user.

[1018] (Application example 2)

[1019] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1020] Conventional image beautification systems lack an intuitive user interface when beautifying user-drawn illustrations or photos, making it difficult to preview in real time or save or share beautified images. Another issue is that it is difficult for users to fine-tune visual characteristics when adjusting beautification parameters.

[1021] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1022] In this invention, the server includes means for automatically beautifying illustrations drawn by a user using advanced image processing technology, means for providing a user interface that can be operated intuitively by the user, means for allowing the user to adjust beautification parameters, means for previewing images uploaded by the user in real time, and means for saving or sharing the beautified images. This allows the user to intuitively operate the server, check the beautified images in real time, adjust visual characteristics to suit their preferences, and save or share the final images.

[1023] "Advanced image processing techniques" refers to advanced algorithms and methods for analyzing, transforming, and modifying images.

[1024] "User-drawn illustrations" refers to pictures or drawings manually created by a user.

[1025] "Automatic beautification" refers to techniques or devices that enhance the quality or appearance of images without user intervention.

[1026] "A user interface that users can operate intuitively" refers to an interface that is designed so that users can operate it easily and naturally.

[1027] "Beautification parameters" refer to settings for adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[1028] "User-adjustable means" refers to features that allow users to change settings or parameters to suit their preferences.

[1029] "Means for real-time preview" refers to a function that displays the results immediately when the user performs an operation.

[1030] "Means to save or share beautified images" refers to the functionality to save the beautified images to your device or share them with other users.

[1031] The system for implementing this invention automatically beautifies illustrations and photographs drawn by the user and provides an intuitive user interface. Specific embodiments of this system are described below.

[1032] System Configuration

[1033] Hardware

[1034] Smartphone: A device that allows users to upload and manipulate images.

[1035] Server: Provides the computing resources for image processing.

[1036] software

[1037] OpenCV: Image processing library for analyzing, transforming, and modifying images.

[1038] Tkinter: A GUI library used to build user interfaces.

[1039] Data processing and calculation

[1040] Loading images

[1041] A user uploads an image using their smartphone, and the server reads the image using OpenCV.

[1042] Adjusting beautification parameters

[1043] Users can adjust the beautification parameters (smoothness, brightness, contrast) by using the sliders and dials displayed on the smartphone screen, allowing them to see the changes in the image in real time.

[1044] Real-time preview

[1045] The server processes the image in real time based on the beautification parameters adjusted by the user and sends a preview to the smartphone, allowing the user to see the results instantly.

[1046] Save and share images

[1047] Once users are satisfied with the image, they can save it on their smartphone and share it with other users via social media or email.

[1048] Specific examples

[1049] Users open the app on their smartphone, upload a photo, use sliders to adjust smoothness, brightness, and contrast, preview the beautified photo in real time, and once satisfied, save the photo or share it on social media.

[1050] Prompt Sentence Examples

[1051] "Create an app where you upload a photo on your smartphone, use sliders to adjust smoothness, brightness, and contrast, preview the beautified photo in real time, and save or share it."

[1052] In this way, the invention allows users to intuitively view beautified images in real time, adjust visual characteristics to their liking, and save or share the final image.

[1053] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1054] Step 1:

[1055] Uploading an image

[1056] The user uses a smartphone to launch the application, select an image, and upload it.

[1057] Input: An image file selected by the user.

[1058] Output: Image data sent to the server.

[1059] Specific operation: The user selects an image from the smartphone gallery and presses the "Upload" button. The image data is sent to the server.

[1060] Step 2:

[1061] Loading images

[1062] The server reads the received image data using OpenCV.

[1063] Input: Image data sent to the server.

[1064] Output: Image object loaded by OpenCV.

[1065] Specific operation: The server receives the image data and reads the image into memory using OpenCV's cv2.imread function.

[1066] Step 3:

[1067] Initial settings for beautification parameters

[1068] The server sets the initial beautification parameters (smoothness, brightness, contrast).

[1069] Input: None (default value).

[1070] Output: Initialized beautification parameters.

[1071] What happens: The server sets initial values ​​for smoothness, brightness, and contrast, and then persists these values.

[1072] Step 4:

[1073] User Interface Display

[1074] The terminal displays a user interface and provides sliders and dials.

[1075] Input: Initialized beautification parameters.

[1076] Output: Display of the user interface.

[1077] What it does: The terminal uses Tkinter to display a user interface, including sliders and dials.

[1078] Step 5:

[1079] Adjusting beautification parameters

[1080] The user adjusts the beautification parameters by manipulating sliders and dials.

[1081] Input: The value of a slider or dial as manipulated by the user.

[1082] Output: Adjusted beautification parameters.

[1083] What happens: The user operates a slider or dial, sending a new parameter value to the server.

[1084] Step 6:

[1085] Image beautification

[1086] The server processes the image based on the adjusted beautification parameters.

[1087] Input: adjusted beautification parameters, loaded image object.

[1088] Output: A beautified image object.

[1089] What it does: The server uses OpenCV functions to adjust smoothness, brightness, and contrast to beautify the image.

[1090] Step 7:

[1091] View real-time preview

[1092] The device will display a preview of the beautified image in real time.

[1093] Input: A beautified image object.

[1094] Output: The beautified image displayed on the device.

[1095] Specific operation: The server sends the beautified image to the device, which displays it on the screen.

[1096] Step 8:

[1097] Save and share images

[1098] The user saves or shares the beautified image.

[1099] Input: A beautified image object.

[1100] Output: Saved image file or shared image link.

[1101] Specific behavior: The user presses the "Save" or "Share" button, and the device saves the image to local storage or shares it via social media or email.

[1102] Example 3

[1103] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1104] In conventional illustration beautification systems, it was difficult for users to intuitively adjust beautification parameters, and the generated illustrations often did not meet the user's expectations. Furthermore, there was a lack of a process to properly analyze the parameters entered by the user and generate prompts appropriate for the generative AI model, which resulted in inconsistent beautification of the illustrations.

[1105] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1106] In this invention, the server includes means for automatically beautifying an illustration drawn by a user, means for providing a user interface that the user can intuitively operate, means for the user to adjust beautification parameters, means for transmitting the beautification parameters input by the user to the server, means for the server to analyze the received beautification parameters and generate a prompt sentence suitable for the generative AI model, means for the generative AI model to beautify the illustration based on the prompt sentence, and means for the server to transmit the beautified illustration to the user's terminal. This allows the user to intuitively adjust the beautification parameters and obtain a consistently beautified illustration using the generative AI model.

[1107] "Advanced image processing technology" refers to technology that uses computers to analyze, modify, and convert images, and is particularly capable of achieving highly accurate and complex processing.

[1108] A "user interface" is the means by which a user interacts with a system, including both visual and operational elements.

[1109] "Beautification parameters" are setting values ​​for adjusting the visual characteristics of an illustration, and include color tone, brightness, contrast, and the like.

[1110] A "generative AI model" is an algorithm or system that uses artificial intelligence to generate or transform data, specifically for image generation.

[1111] A "prompt sentence" is text-based input data used to give instructions to a generative AI model.

[1112] A "server" is a computer system that provides services to clients over a network.

[1113] A "terminal" is a device that a user directly operates, including a computer, smartphone, tablet, etc.

[1114] "Analysis" is the process of examining data in detail to understand its structure and meaning.

[1115] "Transmitting" is the act of moving data from one place to another.

[1116] An "illustration" is an image intended for visual expression, created by hand or using digital tools.

[1117] MODE FOR CARRYING OUT THE INVENTION

[1118] This invention is a system for automatically beautifying illustrations drawn by a user, providing a user interface that can be operated intuitively by the user and including means for allowing the user to adjust beautification parameters. Specific embodiments of this system are described below.

[1119] Hardware and software used

[1120] The system uses the following hardware and software:

[1121] Server: A computer system that provides services to clients over a network.

[1122] Terminal: A device that is directly operated by a user (e.g., computer, smartphone, tablet).

[1123] Generative AI Model: An algorithm or system that uses artificial intelligence to generate or transform data (e.g., Stable Diffusion, DALL-E).

[1124] System processing overview

[1125] Users input beautification parameters using their devices. For example, they can use a dedicated device application or a web interface to set parameters such as color intensity and brightness adjustment. Specifically, they can use sliders and text boxes to set "color intensity" to 50 and "brightness adjustment" to 70.

[1126] The device sends the beautification parameters entered by the user to the server. This transmission is performed using a communication protocol such as an HTTP request or WebSocket. The server analyzes the received beautification parameters and generates a prompt appropriate for the generative AI model. For example, the server generates a prompt such as, "Please generate an illustration with color intensity: 50, brightness adjustment: 70."

[1127] The server inputs a prompt into the generative AI model and requests it to beautify the illustration. At this time, the server calls the generative AI model's API. The generative AI model beautifies the illustration based on the prompt and returns the results to the server. The returned data is an image file of the beautified illustration.

[1128] The server sends the beautified illustration received from the generative AI model to the device. This transmission is also done using a communication protocol such as HTTP response or WebSocket. The user checks the beautified illustration on the device. The device application or web interface has the function to display the received illustration.

[1129] Specific examples

[1130] If a user uses their device to set "Color Intensity" to 50 and "Brightness Adjustment" to 70, the server will input the following prompt to the generative AI model:

[1131] Example prompt sentence:

[1132] "Create an illustration with color intensity: 50, brightness adjustment: 70."

[1133] The generative AI model receives this prompt and beautifies the illustration based on the specified parameters, resulting in a beautified illustration tailored to the user's preferences.

[1134] In this way, the user can intuitively adjust the beautification parameters and obtain a stable beautified illustration using the generative AI model. The flow of the specific processing in Example 3 will be described with reference to FIG.

[1135] Step 1:

[1136] The user inputs the beautification parameters using a terminal.

[1137] Input: The user sets parameters such as color intensity and brightness using a dedicated application on the device or a web interface.

[1138] What it does: The user uses the sliders and text boxes to set "Color Intensity" to 50 and "Brightness Adjustment" to 70.

[1139] Output: Beautification parameters set to the terminal.

[1140] Step 2:

[1141] The terminal transmits the input beautification parameters to the server.

[1142] Input: Beautification parameters set on the device.

[1143] Specific operation: The terminal makes an HTTP POST request and sends the beautification parameters in JSON format to the server endpoint.

[1144] Output: Beautification parameters sent to the server.

[1145] Step 3:

[1146] The server analyzes the received beautification parameters and generates a prompt sentence suitable for the generative AI model.

[1147] Input: The beautification parameters sent to the server.

[1148] Specific operation: The server analyzes the received JSON data and performs string manipulation to generate a prompt. For example, it generates a prompt such as "Please generate an illustration with color intensity: 50, brightness adjustment: 70."

[1149] Output: The generated prompt statement.

[1150] Step 4:

[1151] The server inputs a prompt into the generative AI model, requesting it to beautify the illustration.

[1152] Input: The generated prompt statement.

[1153] Specific operation: The server sends an HTTP POST request to the API endpoint of the generated AI model and sends data including the prompt text.

[1154] Output: The prompt sent to the generative AI model.

[1155] Step 5:

[1156] The generative AI model generates a beautified illustration and sends it back to the server.

[1157] Input: The prompt sent to the generative AI model.

[1158] Specific operation: The generative AI model internally beautifies the illustration and sends the generated illustration back to the server as binary data.

[1159] Output: Binary data of the beautified illustration sent back to the server.

[1160] Step 6:

[1161] The server sends the beautified illustration to the device.

[1162] Input: Binary data of the aestheticized illustration sent back to the server.

[1163] Specific operation: The binary data received by the server is sent to the terminal as an HTTP response.

[1164] Output: A beautified illustration sent to the device.

[1165] Step 7:

[1166] The user checks the beautified illustration on their device.

[1167] Input: A glorified illustration sent to the device.

[1168] Specific operation: The illustration received by the device is displayed within the application, and the user confirms the illustration.

[1169] Output: A beautified illustration reviewed by the user.

[1170] (Application example 3)

[1171] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1172] Conventional image beautification systems often lack an intuitive interface for users when automatically beautifying illustrations drawn by users. They also lack the ability to preview the adjusted illustration in real time or the ability to save and share the adjustments on other content distribution services. This makes it difficult for users to beautify illustrations to their liking and easily share them.

[1173] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1174] In this invention, the server includes means for automatically beautifying illustrations drawn by users using advanced image processing technology, means for providing a user interface that users can operate intuitively, means for allowing users to adjust beautification parameters, means for previewing the adjusted illustration in real time, and means for saving the adjustments and sharing them with other content distribution services. This allows users to beautify illustrations to their liking and easily share them.

[1175] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing and transforming images.

[1176] "User-drawn illustrations" refers to pictures or drawings manually created by a user.

[1177] "Automatic beautification" refers to techniques that improve the quality or appearance of images without user intervention.

[1178] An "intuitive user interface" refers to an interface that users can easily understand and operate.

[1179] "Beautification parameters" refer to settings for adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[1180] "A means of previewing the adjusted illustration in real time" refers to technology that instantly displays the results when the user changes parameters.

[1181] "Means for saving adjustments and sharing them with other content distribution services" refers to technology that allows users to record image adjustments they make and share them with other online platforms.

[1182] The system for implementing this invention automatically beautifies illustrations drawn by users and provides a user interface that allows users to operate intuitively. The system is implemented as an application that runs on devices such as smartphones and tablets.

[1183] Hardware and software used

[1184] Hardware: Smartphones, tablets

[1185] Software: Python, OpenCV, Tkinter

[1186] System configuration

[1187] 1. Image loading: Users can take photos or illustrations on their devices and upload them to the application.

[1188] 2. Adjusting beautification parameters: Users can use sliders and dials within the application to adjust beautification parameters such as "color intensity" and "brightness adjustment."

[1189] 3. Real-time preview: Based on the adjusted parameters, the image beautification process will be carried out in real time and the results will be displayed instantly.

[1190] 4. Saving and sharing adjustments: Users can save the adjustment results they are satisfied with and share them on social media and other content distribution services.

[1191] Processing flow

[1192] 1. Loading an image: Select an image from the device's camera or gallery and upload it to the application. OpenCV is used to load the image and begin processing.

[1193] 2. Adjusting beautification parameters: We use Tkinter to provide intuitive sliders and dials for users to operate. Users can adjust beautification parameters by manipulating these UI elements.

[1194] 3. Real-time preview: Every time the user changes a parameter, the brightness and contrast of the image are adjusted using the OpenCV convertScaleAbs function, and the results are displayed in real time.

[1195] 4. Saving and sharing adjustments: Provides the ability to save the adjusted image on the device and share it on social media and other content distribution services.

[1196] Specific examples

[1197] For example, a user can upload a landscape photo taken with their smartphone to the application, enhance the color tone, slightly reduce the brightness, and share it on Instagram.

[1198] Prompt Sentence Examples

[1199] "Upload a landscape photo, intensify the colors, lower the brightness a bit, and share it to Instagram."

[1200] In this way, users can customize their illustrations to their liking and share them easily.

[1201] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1202] Step 1:

[1203] Users can upload photos and illustrations taken on their devices to the application.

[1204] Input: An image file selected from the device's camera or gallery.

[1205] Output: Image data loaded into the application.

[1206] Specific behavior: A user launches the application, clicks the image upload button, and selects an image from the device's camera roll or gallery. The selected image is then loaded into the application.

[1207] Step 2:

[1208] The device starts processing the image data read using OpenCV.

[1209] Input: The loaded image data.

[1210] Output: An image object that can be processed with OpenCV.

[1211] Specific operation: The application uses the OpenCV library to convert the loaded image data into an internal format so that it can be processed.

[1212] Step 3:

[1213] The user operates sliders and dials to adjust the beautification parameters.

[1214] Input: The value of a slider or dial as manipulated by the user.

[1215] Output: Adjusted beautification parameters.

[1216] Specific operation: The user operates sliders and dials within the application to set parameters such as "color intensity" and "brightness adjustment." The set values ​​are reflected in the application in real time.

[1217] Step 4:

[1218] The device will preview the image in real time based on the adjusted beautification parameters.

[1219] Input: adjusted beautification parameters and original image data.

[1220] Output: Real-time preview of the adjusted image.

[1221] Specific operation: The application uses OpenCV's convertScaleAbs function to adjust the brightness and contrast of the image based on the beautification parameters set by the user, and displays the results in real time.

[1222] Step 5:

[1223] Users can save the adjusted images and share them with other content distribution services.

[1224] Input: Adjusted image data.

[1225] Output: Saved image files and shared images.

[1226] Specific operation: The user presses the save button to save the adjusted image to the device, and then presses the share button to upload the image to Instagram or other social media.

[1227] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1228] "Example 1"

[1229] One embodiment of the present invention is an illustration beautification system incorporating an emotion engine. This system includes a function for automatically beautifying illustrations drawn by a user, a function for providing a user interface that allows the user to intuitively operate the system, a function for allowing the user to adjust beautification parameters, and an emotion engine that recognizes the user's emotions. The emotion engine recognizes the user's emotions from the user's facial expressions, tone of voice, text input, etc., and adjusts the beautification parameters according to the user's emotions. Specifically, the system can beautify the illustration according to the user's emotions, such as by adjusting the color tone of the illustration to be brighter when the user is feeling happy and adjusting the color tone to be darker when the user is feeling sad.

[1230] "Example 2"

[1231] The emotion engine also has the ability to recognize changes in a user's emotions in real time and dynamically adjust the beautification parameters accordingly. For example, if a user starts drawing feeling happy but then changes to sadness midway through, the emotion engine will capture that change in real time and instantly adjust the beautification parameters. This makes it possible to beautify illustrations in response to changes in the user's emotions.

[1232] "Example 3"

[1233] Furthermore, the emotion engine also has a learning function to recognize the user's emotions. Specifically, it collects data such as the user's facial expressions, tone of voice, and text input, and uses a machine learning algorithm to train an emotion recognition model. This learning enables the emotion engine to recognize the user's emotions more accurately, enabling it to beautify illustrations that better suit the user's emotions.

[1234] The processing flow of each embodiment will be described below.

[1235] "Example 1"

[1236] Step 1: The user inputs their illustration into the system.

[1237] Step 2: The system uses an emotion engine to recognize the user's emotions based on their facial expressions, tone of voice, text input, etc.

[1238] Step 3: The emotion engine adjusts the beautification parameters according to the recognized emotion.

[1239] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[1240] "Example 2"

[1241] Step 1: The user inputs their illustration into the system.

[1242] Step 2: The system uses an emotion engine to recognize the user's emotions in real time based on their facial expressions, tone of voice, text input, etc.

[1243] Step 3: The emotion engine dynamically adjusts the beautification parameters according to the changes in the recognized emotions.

[1244] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[1245] "Example 3"

[1246] Step 1: Collect data such as the user's facial expressions, tone of voice, and text input.

[1247] Step 2: Using the collected data, the emotion engine uses machine learning algorithms to train an emotion recognition model.

[1248] Step 3: Using the trained emotion recognition model, recognize the user's emotion and adjust the beautification parameters according to that emotion.

[1249] Step 4: Based on the adjusted beautification parameters, the system automatically beautifies the illustration.

[1250] Example 1

[1251] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1252] Conventional illustration beautification systems were unable to take the user's emotions into account when automatically beautifying illustrations drawn by the user. As a result, beautification was not tailored to the user's emotions, making it difficult to increase user satisfaction. Furthermore, if the user interface was not intuitive, the operation was complicated and difficult to use.

[1253] The identification process by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes: means for acquiring image data representing an illustration drawn by a user; means for automatically adjusting the visual characteristics of the image data; means for recognizing the user's emotion using an emotion engine; a prompt statement instructing the adjustment of the visual characteristics of the image data based on the user's emotion; and means for adjusting the visual characteristics of the image data using a generative AI. This enables the illustration to be beautify in accordance with the user's emotion, thereby increasing user satisfaction. The server also includes means for adjusting the visual characteristics of the image data via a user interface that can be operated by the user. The automatic adjustment means adjusts the color tone, brightness, and contrast of the image data.

[1254] "Advanced image processing technology" refers to technology for adjusting visual characteristics such as color tone, brightness, and contrast of an image, and is technology that uses libraries such as OpenCV and TENSORFLOW (registered trademark).

[1255] "User-drawn illustrations" are image data created by users using hand-drawn or digital tools.

[1256] "Automatic beautification means" refers to algorithms or programs that automatically adjust the visual characteristics of illustrations drawn by users to improve their appearance.

[1257] An "intuitive user interface" is an interface designed to be easy for users to understand and operate, and includes UI elements such as sliders and dials.

[1258] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[1259] "Means for recognizing user emotions" refers to algorithms or programs that analyze the user's facial expressions, tone of voice, text input, etc. to determine the user's emotions.

[1260] The "means for adjusting beautification parameters" refers to an algorithm or program for changing the beautification parameters in response to the user's emotions and adjusting the visual characteristics of the illustration.

[1261] "Means for returning a beautified illustration to the user" refers to communication and display means for providing the adjusted illustration to the user.

[1262] The present invention relates to a system for automatically beautifying illustrations drawn by a user. A specific embodiment of this system will be described below.

[1263] System Overview

[1264] This system takes an illustration drawn by a user as input, automatically adjusts the visual characteristics of the illustration, such as color tone, brightness, and contrast, and outputs a beautified illustration. It also has the ability to recognize the user's emotions and adjust the beautification parameters according to those emotions.

[1265] Hardware and software used

[1266] Server: A server with high-performance computing power is used. The server receives illustration data from users and performs image processing and emotion recognition.

[1267] Device: A device such as a smartphone or PC used by a user. The device uploads illustrations and displays the beautified illustrations.

[1268] software:

[1269] OpenCV: An image processing library used to adjust the color, brightness, and contrast of illustrations.

[1270] TensorFlow: A machine learning library used to build an emotion recognition engine.

[1271] Program processing

[1272] The server receives the illustration uploaded by the user, and then automatically adjusts the visual characteristics of the illustration, such as color tone, brightness, and contrast, using OpenCV. Specifically, the server performs the following processes:

[1273] Color Tone Adjustment: Use OpenCV's cv2.cvtColor function to convert the color space of the illustration and adjust the color tone.

[1274] Brightness adjustment: Use the cv2.convertScaleAbs function to adjust the brightness of the illustration.

[1275] Contrast adjustment: Use the cv2.equalizeHist function to adjust the contrast of the illustration.

[1276] Furthermore, in systems incorporating an emotion engine, the server analyzes the user's facial expressions, tone of voice, and text input to recognize the user's emotions. The emotion engine uses a neural network model built using TensorFlow. If the user is happy, the server adjusts the color tone of the illustration to be brighter, and if the user is sad, the server adjusts the color tone to be darker.

[1277] Specific examples

[1278] As a concrete example, we show how a user can upload an illustration they have drawn and the system will beautify it.

[1279] Users upload illustrations: Users open the dedicated app on their smartphones, select the illustration they have drawn, and tap the "Upload" button.

[1280] The server receives the illustration: The server receives the uploaded illustration file and saves it in the / uploads directory.

[1281] The server adjusts the visual characteristics of the illustration: The server reads the saved illustration file and adjusts the color tone, brightness, and contrast using OpenCV.

[1282] The server uses an emotion engine to recognize the user's emotions: The server analyzes the user's facial expression image and determines that the user is happy.

[1283] The server uses generative AI to beautify the illustration according to the emotion: The server uses generative AI and OpenCV to brighten the color tone of the illustration and generate a beautified illustration.

[1284] The server returns the beautified illustration to the user: The server sends the beautified illustration as an HTTP response and displays it on the user's smartphone.

[1285] Prompt Sentence Examples

[1286] The user is happy. Use OpenCV to adjust the color tone, brightness, and contrast of the user's illustration based on the user's emotions.

[1287] In this way, a system is realized that automatically beautifies illustrations drawn by users and adjusts them according to the user's emotions.

[1288] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1289] Step 1:

[1290] The user uploads an illustration. Using a device (smartphone or PC), the user uploads the illustration they have drawn to the system. Specifically, they select the illustration file through a web browser or a dedicated app and click the upload button. The input is the illustration file selected by the user, and the output is the illustration data sent to the server.

[1291] Step 2:

[1292] The server receives the illustration. The server receives the illustration file uploaded by the user. The server temporarily stores the illustration data sent via the HTTP request. The input is the illustration data sent by the user, and the output is the illustration file saved on the server.

[1293] Step 3:

[1294] The server adjusts the visual characteristics of the illustration. The server uses the OpenCV library to automatically adjust the visual characteristics of the illustration, such as color tone, brightness, and contrast. The specific process is as follows:

[1295] Color Tone Adjustment: Use the cv2.cvtColor function to convert the color space of the illustration and adjust the color tone. The input is an illustration file saved on the server, and the output is the color-adjusted illustration data.

[1296] Brightness adjustment: Use the cv2.convertScaleAbs function to adjust the brightness of the illustration. The input is the color-adjusted illustration data, and the output is the brightness-adjusted illustration data.

[1297] Contrast adjustment: Use the cv2.equalizeHist function to adjust the contrast of the illustration. The input is the illustration data with brightness adjustment, and the output is the illustration data with contrast adjustment.

[1298] Step 4:

[1299] The server uses an emotion engine to recognize the user's emotions. The server uses an emotion engine to recognize the user's emotions. The emotion engine uses a neural network model built using TensorFlow. It analyzes the user's facial expression images, voice data, and text input to determine the user's emotions. The input is the user's facial expression images, voice data, and text input, and the output is the user's emotional data.

[1300] Step 5:

[1301] The server uses generative AI and OpenCV to beautify the illustration according to the user's emotions. The server beautifies the illustration according to the user's recognized emotions. For example, if the user is happy, the server adjusts the color tone of the illustration to be brighter. Specifically, it changes the color adjustment parameters and processes the illustration again using OpenCV. The input is the user's emotional data and illustration data with contrast adjusted, and the output is illustration data beautified according to the emotion.

[1302] Step 6:

[1303] The server returns the beautified illustration to the user. Specifically, the server sends the illustration data as an HTTP response and displays it on the user's device. The input is illustration data beautified according to the emotion, and the output is a beautified illustration displayed on the user's device. The server then provides a user interface that can be operated by the user for adjusting the visual characteristics of the image data. This user interface includes intuitive UI elements such as sliders and dials, allowing the user to make final adjustments to the visual characteristics.

[1304] (Application example 1)

[1305] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1306] Conventional illustration beautification systems provide a function to automatically beautify illustrations drawn by users, but because they cannot adjust beautification parameters according to the user's emotions, it is difficult to obtain optimal beautification results that match the user's emotions. Furthermore, there is a demand for a user interface that can be intuitively operated on mobile devices such as smartphones. Therefore, the challenge is to provide a system that can obtain beautification results that users are more satisfied with.

[1307] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1308] In this invention, the server includes means for automatically beautifying illustrations drawn by a user using advanced image processing technology, means for providing a user interface that the user can operate intuitively, means for allowing the user to adjust beautification parameters, means for recognizing the user's emotions and adjusting the beautification parameters in accordance with the emotions, and means for operating as an application installed on a smartphone, thereby making it possible to obtain optimal beautification results in accordance with the user's emotions.

[1309] "Advanced image processing technology" is a technology that automatically adjusts visual characteristics such as color tone, brightness, and contrast of an image.

[1310] "User-drawn illustrations" are images created by users using hand-drawn or digital tools.

[1311] "Automatic beautification" refers to technology that has the ability to improve the visual characteristics of illustrations without user intervention.

[1312] A "user interface that users can operate intuitively" is an interface that users can easily understand and operate.

[1313] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[1314] "User-adjustable means" refers to a technology that allows users to change the beautification parameters according to their preferences.

[1315] "Means for recognizing user emotions" refers to technology that detects emotions from the user's facial expressions, tone of voice, etc.

[1316] The "means for adjusting beautification parameters according to emotions" is a technology that has the function of automatically changing beautification parameters based on the recognized emotions of the user.

[1317] An "application installed on a smartphone" is a software program that runs on a smartphone.

[1318] In one embodiment of the present invention, the EmotionArt application is installed on a smartphone. The EmotionArt application automatically beautifies illustrations drawn by the user and adjusts the visual characteristics of the illustration according to the user's emotions.

[1319] The server includes the following means:

[1320] 1. A means of automatically beautifying illustrations drawn by users using advanced image processing technology.

[1321] 2. A means of providing a user interface that is intuitive to the user.

[1322] 3. User adjustable means of beautification parameters.

[1323] 4. A means of recognizing user emotions and adjusting beautification parameters accordingly.

[1324] 5. A means that operates as an application installed on a smartphone.

[1325] This makes it possible to obtain optimal beautification results according to the user's feelings.

[1326] Hardware and software used

[1327] Hardware:

[1328] Smartphone (camera, microphone, display)

[1329] software:

[1330] OpenCV (image processing library)

[1331] EmotionEngine (emotion recognition engine)

[1332] ImageBeautifier (Image Beautifier)

[1333] Processing flow

[1334] 1. Loading an image: The user takes a photo of the illustration they have drawn using their smartphone camera and saves it as an image file.

[1335] 2. Emotion Recognition: Using the EmotionEngine, emotions are recognized from the user's facial expressions and tone of voice.

[1336] 3. Adjusting beautification parameters: Based on the recognized emotion, ImageBeautifier adjusts the visual characteristics of the illustration, such as color tone, brightness, and contrast.

[1337] 4. Saving the beautified image: The beautified illustration is saved as an image file and provided to the user.

[1338] Specific examples

[1339] Users draw illustrations on their smartphones and upload them to the app. The app captures the user's facial expressions with its camera and recognizes their emotions. If the recognized emotion is "joy," the app brightens the illustration's color tone. The beautified illustration is then displayed to the user and saved.

[1340] Prompt Sentence Examples

[1341] We are developing an application that beautifies illustrations drawn by users. Please generate a program for this application that has the function of adjusting the color tone, brightness, and contrast of the illustration according to the user's emotions. The user's emotions will be recognized from facial expressions and tone of voice. The libraries used are OpenCV, EmotionEngine, and ImageBeautifier.

[1342] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1343] Step 1:

[1344] The user draws an illustration on their smartphone and then takes a photo of it with their camera.

[1345] Input: User-drawn illustration

[1346] Output: Photographed illustration image file

[1347] Specific operation: The user launches the smartphone camera app and takes a picture of the illustration they have drawn. The captured image is then saved in the smartphone's storage.

[1348] Step 2:

[1349] The device reads the photographed illustration image file.

[1350] Input: Photographed illustration image file

[1351] Output: Image data

[1352] Specific operation: The EmotionArt application on the smartphone reads the image file and stores it in memory as image data.

[1353] Step 3:

[1354] The device uses EmotionEngine to recognize the user's emotions.

[1355] Input: User facial expressions and tone of voice

[1356] Output: Recognized emotion data

[1357] How it works: Using the smartphone's camera and microphone, it captures the user's facial expressions and tone of voice, and the Emotion Engine analyzes this data to recognize the user's emotions.

[1358] Step 4:

[1359] The device adjusts beautification parameters based on the recognized emotion.

[1360] Input: Recognized emotion data

[1361] Output: Adjusted beautification parameters

[1362] Specific operation: The EmotionArt application automatically sets ImageBeautifier's beautification parameters (color tone, brightness, contrast, etc.) based on the recognized emotion data.

[1363] Step 5:

[1364] The device uses ImageBeautifier to beautify the illustration.

[1365] Input: Image data, adjusted beautification parameters

[1366] Output: Beautified illustration image data

[1367] Specific operation: ImageBeautifier uses the image data and adjusted beautification parameters to adjust the color tone, brightness, contrast, etc. of the illustration to generate a beautified illustration.

[1368] Step 6:

[1369] The device saves the beautified illustration and displays it to the user.

[1370] Input: Beautified illustration image data

[1371] Output: Saved beautified illustration image file, beautified illustration displayed to the user

[1372] Specific operation: The beautified illustration image data is saved to the smartphone's storage and displayed on the application screen. The user can check the beautified illustration and make further adjustments if necessary.

[1373] Example 2

[1374] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1375] Conventional illustration beautification systems require users to manually adjust beautification parameters, making operation cumbersome. Furthermore, they lacked the functionality to dynamically adjust the beautification level of an illustration according to changes in the user's emotions, making it difficult to beautify illustrations in accordance with the user's emotions. This led to a problem of reduced user satisfaction.

[1376] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1377] In this invention, the server includes means for automatically beautifying illustrations drawn by a user using advanced image processing technology, means for providing a user interface that can be operated intuitively by the user, means for allowing the user to adjust beautification parameters, and means for recognizing changes in the user's emotions in real time and dynamically adjusting the beautification parameters in response to those changes, thereby enabling the user to beautify illustrations in accordance with their emotions while operating intuitively.

[1378] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing, modifying, and transforming images.

[1379] "User interface" refers to the screens and operating means that allow a user to interact with a system.

[1380] "Beautification parameters" refer to the settings used to adjust the visual characteristics of an illustration, such as color tone, brightness, and contrast.

[1381] A "slider" is a UI element that allows the user to adjust a value by dragging it.

[1382] A "dial" refers to a UI element that allows the user to adjust a value by rotating it.

[1383] "Emotional changes" refers to the user's emotional state changing over time.

[1384] "Real-time recognition" refers to instantly detecting and processing changes in a user's emotions.

[1385] "Dynamic adjustment" refers to automatically changing the setting value depending on the situation.

[1386] MODE FOR CARRYING OUT THE INVENTION

[1387] This invention is a system that automatically beautifies illustrations drawn by users and dynamically adjusts beautification parameters according to changes in the user's emotions. This system consists of three main components: a server, a terminal, and a user.

[1388] server

[1389] The server performs the following functions:

[1390] 1. Providing a user interface

[1391] The server generates a user interface (UI) and sends it to the device. This UI includes intuitive elements such as sliders and dials.

[1392] Software used: UI libraries such as React, Vue.js.

[1393] 2. Receiving user actions

[1394] The server receives user operation data sent from the device, including the position of the slider and the angle of the dial.

[1395] Software used: WebSocket, HTTP requests.

[1396] 3. Calculation of beautification parameters

[1397] The server calculates beautification parameters based on the received operation data, which determine the degree of beautification of the illustration.

[1398] Software used: Programming languages ​​such as Python, Node.js, etc.

[1399] 4. Emotion Data Analysis

[1400] The server receives and analyzes the user's emotional data sent from the device, and uses an emotion engine to recognize changes in the user's emotions in real time.

[1401] Software used: Sentiment analysis engines such as Microsoft Azure Emotion API.

[1402] 5. Dynamic adjustment of beautification parameters

[1403] The server dynamically adjusts the beautification parameters based on the analyzed emotional data, making it possible to beautify illustrations according to the user's emotions.

[1404] Terminal

[1405] The terminal performs the following functions:

[1406] 1. Displaying the user interface

[1407] The device displays the user interface received from the server, which includes UI elements such as sliders and dials.

[1408] Hardware used: PC, tablet, smartphone.

[1409] Software required: Browser, HTML, JavaScript.

[1410] 2. Sending user actions

[1411] When a user operates a slider or dial, the device transmits the operation data to the server.

[1412] Software used: JavaScript event listeners, WebSockets, HTTP requests.

[1413] 3. Collecting Emotional Data

[1414] The device uses sensors such as a camera and microphone to collect user emotional data.

[1415] Software used: WebRTC, MediaStream API.

[1416] User

[1417] The user performs the following operations:

[1418] 1. User Interface Operation

[1419] Users adjust the beautification parameters of the illustration by operating sliders and dials on their device.

[1420] What it does: The user drags a slider or rotates a dial using a mouse or touchscreen.

[1421] 2. Emotional changes

[1422] The user's emotions change as they draw the illustration, and these changes in emotions are collected by the device and sent to the server.

[1423] Specific behavior: Emotion data changes when the user smiles or frowns.

[1424] Specific examples

[1425] Hardware and software used

[1426] Hardware: PCs, tablets, smartphones

[1427] Software: User interface (UI) libraries (e.g., React, Vue.js), sentiment analysis engines (e.g., Microsoft Azure Emotion API)

[1428] Prompt Sentence Examples

[1429] "Please describe a system where the user can adjust the beautification parameters of an illustration by manipulating sliders. Also include the ability to dynamically adjust the beautification parameters in response to changes in the user's emotions."

[1430] By inputting this prompt into a generative AI model, a detailed description of the system can be obtained.

[1431] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1432] Step 1:

[1433] Providing a user interface

[1434] The server generates a user interface (UI) and sends it to the device. It uses a UI library (e.g., React or Vue.js) as input and generates HTML and JavaScript code as output. Specifically, the server uses the UI library to build a UI that includes intuitively operable elements such as sliders and dials, and sends this to the device.

[1435] Step 2:

[1436] User Interface Display

[1437] The device displays the user interface received from the server. It receives HTML and JavaScript code sent from the server as input and renders the UI in the browser as output. Specifically, the device uses the browser to parse the received code and display the UI, including sliders and dials.

[1438] Step 3:

[1439] Sending user actions

[1440] When a user operates a slider or dial, the device sends the operation data to the server. It receives user operation events as input and sends the operation data to the server as output. Specifically, the device uses a JavaScript event listener to capture user operations and sends them to the server via WebSocket or HTTP request.

[1441] Step 4:

[1442] Receiving user actions

[1443] The server receives user operation data sent from the device. It receives the operation data from the device as input and uses it as output to calculate beautification parameters. Specifically, the server receives user operation data in real time via WebSocket or HTTP request.

[1444] Step 5:

[1445] Calculation of beautification parameters

[1446] The server calculates beautification parameters based on the received operation data. It receives user operation data as input and generates beautification parameters as output. Specifically, the server uses a programming language such as Python or Node.js to execute the beautification algorithm and calculate the parameters.

[1447] Step 6:

[1448] Collecting Emotional Data

[1449] The device uses sensors such as a camera and microphone to collect user emotional data. It receives sensor data from the camera and microphone as input and sends emotional data to the server as output. Specifically, the device uses WebRTC and MediaStream API to collect data from the camera and microphone and send it to the server.

[1450] Step 7:

[1451] Emotional Data Analysis

[1452] The server receives and analyzes the user's emotional data sent from the device. It receives the emotional data from the device as input and generates the analysis results as output. Specifically, the server uses an emotion analysis engine such as the Microsoft Azure Emotion API to analyze the user's emotional data.

[1453] Step 8:

[1454] Dynamic adjustment of beautification parameters

[1455] The server dynamically adjusts the beautification parameters based on the analyzed emotion data. It receives the emotion analysis results as input and generates updated beautification parameters as output. Specifically, the server recalculates and updates the parameters of the beautification algorithm based on the emotion analysis results.

[1456] Step 9:

[1457] Illustration update

[1458] The terminal updates the illustration in real time based on the updated beautification parameters received from the server. The terminal receives the updated beautification parameters from the server as input and displays the updated illustration as output. Specifically, the terminal uses the browser to redraw the illustration based on the received beautification parameters.

[1459] (Application example 2)

[1460] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1461] Conventional illustration beautification systems require users to manually adjust beautification parameters, making it difficult to dynamically beautify illustrations in response to changes in the user's emotions. Furthermore, they lack an intuitive user interface, making it difficult to improve the user experience.

[1462] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1463] In this invention, the server includes means for automatically beautifying illustrations drawn by a user using advanced image processing technology, means for providing a user interface that can be operated intuitively by the user, means for allowing the user to adjust beautification parameters, and means for recognizing the user's emotions in real time and dynamically adjusting the beautification parameters in response to the user's emotions. This enables dynamic beautification of illustrations in response to changes in the user's emotions, improving the user experience through intuitive operation.

[1464] "Advanced image processing techniques" refers to advanced algorithms and methods for analyzing, transforming, and modifying images.

[1465] "User-drawn illustrations" refers to visual works created by users, either manually or using digital tools.

[1466] "Automatic beautification" refers to techniques or algorithms that enhance the visual characteristics of illustrations without user intervention.

[1467] "A user interface that users can operate intuitively" refers to an interface that is designed so that users can operate it easily and naturally.

[1468] "Beautification parameters" refer to the settings used to adjust the visual characteristics of an illustration, such as color tone, brightness, and contrast.

[1469] "User-adjustable means" refers to features and interfaces that allow users to change settings to suit their preferences.

[1470] "Means of recognizing user emotions in real time" refers to technology or systems that analyze a user's facial expressions and behavior to instantly determine their emotions at any given time.

[1471] "Means for dynamically adjusting beautification parameters according to emotions" refers to technologies or algorithms that automatically change beautification parameters based on recognized user emotions.

[1472] The following system configuration will be described as an embodiment of the present invention.

[1473] System Configuration

[1474] This system recognizes emotions in real time when a user draws an illustration using a smartphone, and dynamically adjusts the beautification parameters of the illustration according to those emotions. The system consists of the following main components:

[1475] 1. Smartphone

[1476] Camera: Captures the user's face and provides data for emotion recognition.

[1477] Touchscreen: Provides an interface for users to draw illustrations.

[1478] Processor: Performs image processing, emotion recognition, and UI operations.

[1479] 2. Software Components

[1480] OpenCV: Image processing library for beautifying illustrations.

[1481] EmotionRecognizer: An emotion recognition library that recognizes user emotions in real time.

[1482] UI Elements: A library that provides intuitive UI elements such as sliders and dials.

[1483] Program processing

[1484] The server implements the system using the following means:

[1485] 1. A method to automatically beautify user-drawn illustrations using advanced image processing technology

[1486] Using OpenCV, we apply beautification processing to illustrations drawn by users, specifically by applying filters such as Gaussian blur to improve the visual characteristics of the illustrations.

[1487] 2. A means of providing a user interface that users can operate intuitively

[1488] The UI Elements library provides intuitive UI elements such as sliders and dials, allowing users to easily adjust beautification parameters.

[1489] 3. User-adjustable beautification parameters

[1490] Allow users to adjust beautification parameters according to their preferences using sliders and dials, such as hue, brightness, contrast, etc.

[1491] 4. A means to recognize user emotions in real time and dynamically adjust beautification parameters according to those emotions.

[1492] EmotionRecognizer captures the user's face and recognizes their emotions in real time. Based on the recognized emotion, the beautification parameters are dynamically adjusted. For example, if the user feels "joy," the beautification parameters are set high, and if they feel "sad," the parameters are set low.

[1493] Specific examples

[1494] For example, if a user is drawing an illustration on their smartphone, the camera captures the user's face and the emotion recognition library detects "joy," the beautification parameters will be set higher, making the illustration more vivid and bright. Conversely, if "sadness" is detected, the beautification parameters will be set lower, making the illustration a little darker and more subdued.

[1495] Prompt Sentence Examples

[1496] When a user draws an illustration on their smartphone, the camera captures the user's face. If the emotion recognition library detects "joy," the beautification parameters are set higher, making the illustration more vivid and bright. Conversely, if it detects "sadness," the beautification parameters are set lower, making the illustration a little darker and more subdued.

[1497] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1498] Step 1:

[1499] The device captures the user's face using a camera. The input is the user's face image, and the output is the captured face image data. This face image data is used as preprocessing for emotion recognition.

[1500] Step 2:

[1501] The device uses an emotion recognition library (EmotionRecognizer) to recognize the user's emotions in real time from the captured facial image data. The input is the facial image data, and the output is the recognized emotion (e.g., joy, sadness, surprise, etc.). This emotion data is used to adjust the subsequent beautification parameters.

[1502] Step 3:

[1503] The device allows users to draw illustrations using a touch screen. The input is the user's touch operation, and the output is the drawn illustration data. This illustration data is then subjected to beautification processing.

[1504] Step 4:

[1505] The device uses user interface (UI Elements) to display sliders and dials to allow the user to adjust beautification parameters. The input is the user's operation of the slider or dial, and the output is the adjusted beautification parameters. These beautification parameters are used to beautify the illustration.

[1506] Step 5:

[1507] The device dynamically adjusts the beautification parameters based on the recognized emotion data. The input is the emotion data and the beautification parameters adjusted by the user, and the output is the final adjusted beautification parameters. For example, if the user feels "joy," the beautification parameters are set high, and if they feel "sad," they are set low.

[1508] Step 6:

[1509] The device uses OpenCV to beautify the illustration based on the final adjusted beautification parameters. The input is the illustration data and the adjusted beautification parameters, and the output is the beautified illustration data. Specifically, filters such as Gaussian blur are applied to improve the visual characteristics of the illustration.

[1510] Step 7:

[1511] The terminal displays the beautified illustration to the user. The input is beautified illustration data, and the output is a beautified illustration that the user can visually confirm. This allows the user to check the beautified illustration in real time.

[1512] Example 3

[1513] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1514] Conventional illustration beautification systems allow users to adjust beautification parameters, but they are unable to beautify illustrations in accordance with the user's emotions. This makes it difficult to beautify illustrations in accordance with the user's emotions, and it is difficult to increase user satisfaction. Furthermore, while systems with emotion recognition capabilities exist, the technology to apply this to beautifying illustrations has not yet been established.

[1515] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1516] In this invention, the server includes means for collecting user emotion data, means for analyzing the collected emotion data and training an emotion recognition model, and means for beautifying illustrations in accordance with the user's emotions, thereby enabling beautification of illustrations in accordance with the user's emotions.

[1517] "Advanced image processing technology" refers to technology for adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[1518] A "user interface" is an interface that allows a user to operate a system, and includes UI elements that can be operated intuitively, such as sliders and dials.

[1519] "Beautification parameters" are parameters for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[1520] "User emotional data" refers to data about emotions collected from a user's facial expressions, tone of voice, text input, etc.

[1521] An "emotion recognition model" is a model trained using machine learning algorithms to recognize a user's emotions.

[1522] "Illustration beautification" is a process that makes illustrations drawn by users more attractive by adjusting their visual characteristics such as color tone, brightness, and contrast.

[1523] MODE FOR CARRYING OUT THE INVENTION

[1524] This invention is a system that automatically beautifies illustrations drawn by a user using advanced image processing technology, and further adjusts the beautification in response to the user's emotions. Specific embodiments of this system will be described below.

[1525] Hardware and software used

[1526] Hardware: User's device (PC, smartphone, etc.), camera, microphone

[1527] Software: Web browser, dedicated application, image processing library (OpenCV, etc.), machine learning library (TensorFlow, PyTorch, etc.)

[1528] Program processing

[1529] Adjusting beautification parameters

[1530] The user inputs beautification parameters such as "color intensity" and "brightness adjustment" through the device interface. For example, using a smartphone app, the user moves the slider to set color intensity to 50. The device then sends these parameters to the server, which then beautifies the illustration based on the received parameters.

[1531] Emotion data collection and analysis

[1532] The user's device uses a camera and microphone to collect the user's facial expressions and tone of voice. For example, the camera takes a picture of the user's face and captures facial expression data. The device then sends the collected emotional data to a server, which uses a machine learning algorithm to train an emotion recognition model. The trained model recognizes the user's emotions and beautifies the illustration according to those emotions.

[1533] Beautification and display of illustrations

[1534] The server uses the learned emotion recognition model to beautify the illustration according to the user's emotions. For example, if the user is smiling, it adds warm colors to the illustration. Finally, the beautified illustration is sent to the device and displayed to the user.

[1535] Specific examples

[1536] The user opens the app on their smartphone and sets the color intensity to 50 and the brightness adjustment to 20. The device sends these parameters to the server, which then beautifies the illustration based on the received parameters. Next, if the user smiles through the camera, the device sends the facial expression data to the server. The server trains an emotion recognition model and adds warm color tones to the illustration according to the user's emotions. Finally, the beautified illustration is sent to the device and displayed to the user.

[1537] Prompt Sentence Examples

[1538] "Please describe a system in which a user sets parameters for beautifying an illustration, and an emotion engine recognizes the user's emotions and beautifies the illustration. Please include the names of the specific hardware and software."

[1539] In this way, the present invention provides a function for beautifying an illustration based on the user's input and emotions. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[1540] Step 1:

[1541] The user inputs the beautification parameters.

[1542] Users input beautification parameters such as "color intensity" and "brightness adjustment" through the device interface. For example, using a smartphone app, they can move the slider to set color intensity to 50. The input parameters are saved on the device.

[1543] Step 2:

[1544] The terminal transmits the beautification parameters to the server.

[1545] The device sends the beautification parameters entered by the user to the server using an HTTP request, such as a color intensity of 50 and brightness adjustment of 20. The server receives this and proceeds to the next step.

[1546] Step 3:

[1547] The server receives the beautification parameters and beautifies the illustration.

[1548] The server uses an image processing library (such as OpenCV) to beautify the illustration based on the received beautification parameters. For example, it sets the color intensity to 50 and adjusts the brightness to 20. The beautified illustration is saved on the server.

[1549] Step 4:

[1550] Collect user sentiment data.

[1551] The user's device uses a camera and microphone to collect the user's facial expressions and tone of voice. For example, the camera takes a picture of the user's face and captures facial expression data. The collected emotional data is stored on the device.

[1552] Step 5:

[1553] The device transmits the emotion data to the server.

[1554] The device sends the collected emotional data to the server using an HTTP request. Specifically, the user's facial expression data and tone of voice data are sent. The server receives this data and proceeds to the next step.

[1555] Step 6:

[1556] The server analyzes the emotion data and trains an emotion recognition model.

[1557] The server analyzes the received emotion data and uses machine learning algorithms (such as TensorFlow or PyTorch) to train an emotion recognition model. For example, it can determine whether the user is smiling based on facial expression data. The trained model is stored on the server.

[1558] Step 7:

[1559] The server beautifies the illustration according to the user's emotions.

[1560] The server uses the trained emotion recognition model to beautify the illustration according to the user's emotion. For example, if the user is smiling, it adds warm colors to the illustration. The beautified illustration is then stored on the server.

[1561] Step 8:

[1562] The server sends the beautified illustration to the device.

[1563] The server sends the beautified illustration to the terminal using an HTTP response. Specifically, the beautified illustration data is sent. The terminal receives this and proceeds to the next process.

[1564] Step 9:

[1565] The device displays a beautified illustration to the user.

[1566] The device displays the beautified illustration to the user. For example, the illustration is displayed in a smartphone app. The user can then check the beautified illustration.

[1567] (Application example 3)

[1568] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1569] Conventional image beautification systems have had the problem of making it difficult for users to beautify illustrations according to their own emotions and preferences. Furthermore, the lack of an intuitive interface for users to adjust beautification parameters often made operation cumbersome. Furthermore, there was no technology available to recognize the user's emotions and beautify illustrations based on those emotions, making it impossible to achieve optimal beautification according to the user's emotions.

[1570] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1571] In this invention, the server includes means for automatically beautifying illustrations drawn by a user using advanced image processing technology, means for providing a user interface that the user can intuitively operate, means for the user to adjust beautification parameters, means including an emotion recognition engine for recognizing the user's emotions, and means for beautifying the illustration based on the recognized emotions. This allows the user to beautify the illustration according to their own emotions and preferences, and the beautification parameters can be easily adjusted through the intuitively operable interface. Furthermore, the use of the emotion recognition engine enables optimal beautification according to the user's emotions.

[1572] "Advanced image processing technology" refers to advanced algorithms and techniques for adjusting visual characteristics such as color tone, brightness, and contrast of an image to enhance it.

[1573] "User interface" is a general term for the screens and input devices that users use to operate a system, and includes UI elements such as sliders and dials that can be operated intuitively.

[1574] "Beautification parameters" are setting values ​​for adjusting the visual characteristics of an illustration, such as color tone, brightness, and contrast, and can be freely adjusted by the user.

[1575] An "emotion recognition engine" is a system that collects data such as a user's facial expressions, tone of voice, and text input, and uses machine learning algorithms to recognize the user's emotions.

[1576] The "means for beautifying illustrations based on recognized emotions" is a method for automatically adjusting the beautification parameters of an illustration in accordance with the user's emotions recognized by an emotion recognition engine, thereby achieving optimal beautification.

[1577] A system for implementing this invention includes advanced image processing technology for automatically beautifying illustrations drawn by a user, a user interface that can be operated intuitively by the user, means for allowing the user to adjust beautification parameters, an emotion recognition engine for recognizing the user's emotions, and means for beautifying the illustration based on the recognized emotions.

[1578] System configuration

[1579] Hardware: Smartphone

[1580] Software: Python, OpenCV, Keras, emotion recognition library

[1581] Program processing explanation

[1582] 1. Image Loading: Load an image taken by the user from the smartphone camera or gallery. Use OpenCV to obtain the image data.

[1583] 2. Emotion Recognition: Using an emotion recognition model trained with Keras, we recognize emotions from the user's facial expressions, tone of voice, text input, etc. We use an emotion recognition library to collect and analyze user emotion data.

[1584] 3. Adjusting beautification parameters: We provide intuitive UI elements such as sliders and dials to adjust the intensity and brightness of the color tone specified by the user. By manipulating these UI elements, users can freely adjust the beautification parameters.

[1585] 4. Emotion-based beautification: Automatically adjust image beautification parameters based on the recognized emotion. For example, if the user is recognized as "happy," the image will be beautified by setting the color intensity to 1.2 and the brightness to 30.

[1586] 5. Save the result: Save the beautified image to your smartphone's gallery. Use OpenCV to save the processed image data.

[1587] Specific examples

[1588] Input image: A landscape photo taken by the user

[1589] Emotion: When a user is perceived as "happy"

[1590] Beautification parameters: Color intensity 1.2, Brightness 30

[1591] Prompt Sentence Examples

[1592] Use a landscape photo taken by the user as the input image, and if the emotion engine recognizes it as "happy," beautify it by setting the color intensity to 1.2 and the brightness to 30.

[1593] In this way, users can beautify illustrations according to their own emotions and preferences, and the beautification parameters can be easily adjusted through an intuitive interface. In addition, by using an emotion recognition engine, optimal beautification can be achieved according to the user's emotions.

[1594] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1595] Step 1:

[1596] The device reads images taken by the user from the smartphone camera or gallery. Specifically, it uses OpenCV to acquire image data. The input is an image file selected by the user, and the output is an array of image data.

[1597] Step 2:

[1598] The server uses an emotion recognition model trained with Keras to recognize emotions from the user's facial expressions, tone of voice, text input, etc. Specifically, it uses an emotion recognition library to collect and analyze user emotion data. The input is an array of image data, and the output is a label of the recognized emotion.

[1599] Step 3:

[1600] The device provides intuitive UI elements such as sliders and dials for adjusting the intensity and brightness of the user-specified color tone. By manipulating these UI elements, the user can freely adjust the beautification parameters. The input is the user's operation, and the output is the adjusted beautification parameters.

[1601] Step 4:

[1602] The server automatically adjusts image beautification parameters based on the recognized emotion. Specifically, it sets color intensity and brightness according to the emotion recognized by the emotion recognition engine. The input is the recognized emotion label and the beautification parameters adjusted by the user, and the output is the optimized beautification parameters.

[1603] Step 5:

[1604] The device beautifies the image based on the beautification parameters. Specifically, it uses OpenCV to adjust the color intensity and brightness and process the image. The input is the optimized beautification parameters and an array of image data, and the output is the beautified image data.

[1605] Step 6:

[1606] The device saves the beautified image to the smartphone's gallery. Specifically, OpenCV is used to save the processed image data. The input is the beautified image data, and the output is the saved image file.

[1607] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1608] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1609] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1610] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1611] [Third embodiment]

[1612] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1613] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1614] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32.

[1615] The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1616] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1617] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1618] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1619] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1620] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1621] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1622] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1623] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1624] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1625] "Example 1"

[1626] One embodiment of the present invention provides a system for automatically beautifying user-drawn illustrations. This system uses advanced image processing technology to automatically beautify user-drawn illustrations. Specifically, the system receives a user-drawn illustration as input, automatically adjusts the visual characteristics of the illustration, such as color tone, brightness, and contrast, and outputs a beautified illustration.

[1627] "Example 2"

[1628] In addition, as one embodiment of the present invention, an intuitively operable user interface is provided. This user interface includes intuitively operable UI elements such as sliders and dials. Specifically, the user can adjust beautification parameters by operating the sliders and dials. This allows the user to adjust the degree of beautification of the illustration to suit their preferences.

[1629] "Example 3"

[1630] Furthermore, as one embodiment of the present invention, a means is provided for allowing a user to adjust beautification parameters. Specifically, the user can freely adjust the degree of beautification of an illustration by directly inputting beautification parameters. For example, the user can beautify an illustration to suit their preferences by adjusting parameters such as "color intensity" and "brightness adjustment."

[1631] The processing flow of each embodiment will be described below.

[1632] "Example 1"

[1633] Step 1: The user inputs the illustration they have drawn into the system. This can be done by file upload, drag and drop, etc.

[1634] Step 2: The system analyzes the input illustration and automatically adjusts visual characteristics such as color tone, brightness, and contrast. This adjustment is done using a pre-trained machine learning model.

[1635] Step 3: The system outputs the adjusted illustration, which can be displayed directly on the screen or downloaded as a file.

[1636] "Example 2"

[1637] Step 1: The user operates the system's user interface, specifically, by manipulating sliders and dials to adjust the beautification parameters.

[1638] Step 2: The system adjusts the beautification parameters according to the user's operations and beautifies the illustration.

[1639] Step 3: The system outputs the beautified illustration, which can be displayed directly on the screen or downloaded as a file.

[1640] "Example 3"

[1641] Step 1: The user directly inputs the beautification parameters, such as "color intensity" and "brightness adjustment," using numerical values ​​or sliders.

[1642] Step 2: The system beautifies the illustration based on the beautification parameters entered by the user.

[1643] Step 3: The system outputs the beautified illustration, which can be displayed directly on the screen or downloaded as a file.

[1644] Example 1

[1645] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1646] Conventional illustration beautification systems require users to manually adjust color, brightness, and contrast, making operation cumbersome. Furthermore, they lack an intuitive user interface, making it difficult for users to easily complete the beautification process. Furthermore, the lack of real-time progress indicators and notification of process completion negatively impacted the user experience.

[1647] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for a user to upload an illustration, a means for the server to receive the illustration, a means for the server to adjust color tone, a means for the server to adjust brightness, a means for the server to adjust contrast, a means for the server to generate a beautified illustration, and a means for the server to provide the beautified illustration to the user. This allows the user to easily beautify an illustration through an intuitive interface. In addition, the user experience is improved by real-time progress display and processing completion notification.

[1648] "Advanced image processing technology" refers to technology for automatically adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[1649] A "user interface" is an interface that includes a screen and input devices that allow a user to operate a system.

[1650] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[1651] "Means for users to upload illustrations" refers to a function that allows users to send illustration files to the system from their own devices.

[1652] "Means for the server to receive illustrations" refers to the function by which the server receives and stores illustration files sent by users.

[1653] "Means for the server to adjust color tone" is a function that allows the server to automatically change the hue and saturation of an illustration.

[1654] "Means for the server to adjust brightness" is a function that allows the server to automatically change the brightness of the illustration.

[1655] "Means for the server to adjust contrast" is a function that allows the server to automatically change the difference in brightness of the illustration.

[1656] The "means for the server to generate a beautified illustration" is a function for the server to create a new illustration after adjusting the color tone, brightness, and contrast.

[1657] "Means for the server to provide beautified illustrations to users" refers to a function for providing beautified illustrations generated by the server to users in a downloadable format.

[1658] This invention relates to a system for automatically beautifying illustrations drawn by users. The system provides an intuitive user interface and allows users to adjust beautification parameters to adjust visual characteristics such as color tone, brightness, and contrast.

[1659] Hardware and software used

[1660] The server is a computer system that runs Python programs and uses OpenCV and Pillow as image processing libraries. Users access the system using devices connected to the Internet (PCs, smartphones, tablets, etc.).

[1661] Program processing explanation

[1662] The server receives the illustration uploaded by the user and then adjusts the image's color using OpenCV. Specifically, the server converts the image to HSV (hue, saturation, brightness) space, adjusts the hue appropriately, and then converts it back to RGB space.

[1663] The server then uses Pillow to adjust the image brightness, specifically using Pillow's ImageEnhance.Brightness class to adjust the brightness, and then using Pillow's ImageEnhance.Contrast class to adjust the contrast.

[1664] Through these processes, a beautified illustration is generated and provided to the user, who can then obtain it by clicking the download link on their device.

[1665] Specific examples

[1666] As a concrete example, consider the case of beautifying an illustration drawn by a user, "example_illustration.png."

[1667] 1. User Action:

[1668] The user uploads "example_illustration.png" to the system from their device.

[1669] 2. Server processing:

[1670] The server receives the uploaded image and adjusts the color, brightness, and contrast.

[1671] Save the adjusted image as "beautified_illustration.png".

[1672] 3. Output to the user:

[1673] The server provides the beautified illustration "beautified_illustration.png" to the user.

[1674] An example of a prompt is as follows:

[1675] Upload your own illustration "example_illustration.png". The system will automatically adjust the color, brightness, and contrast to provide you with a beautified illustration. You will receive a notification when the process is complete.

[1676] This is how the system works, automatically beautifying illustrations drawn by users.

[1677] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1678] Step 1:

[1679] Users upload illustrations.

[1680] A user uploads an illustration file (e.g., "example_illustration.png") that they have drawn from their terminal through the system's web interface. The input is the user's illustration file, and the output is the illustration file sent to the server.

[1681] Step 2:

[1682] The server receives the illustration.

[1683] The server receives the illustration file uploaded by the user and saves it in the specified directory. The input is the illustration file sent by the user, and the output is the illustration file saved on the server. Specifically, the server checks the file format and size and verifies that it is in the appropriate format.

[1684] Step 3:

[1685] The server adjusts the color tone.

[1686] The server loads the illustration using OpenCV's cv2.imread function. Next, it converts the image to HSV (hue, saturation, brightness) space using the cv2.cvtColor function. The hue is adjusted appropriately and then converted back to RGB space. The input is an illustration file stored on the server, and the output is an illustration with adjusted color tones. Specifically, the hue is shifted by 10 degrees.

[1687] Step 4:

[1688] The server adjusts the brightness.

[1689] The server uses Pillow's ImageEnhance.Brightness class to adjust the brightness of the image. Specifically, it increases the brightness by 1.2 times. The input is an illustration with adjusted color tone, and the output is an illustration with adjusted brightness.

[1690] Step 5:

[1691] The server adjusts the contrast.

[1692] The server uses Pillow's ImageEnhance.Contrast class to adjust the contrast of the image. Specifically, it increases the contrast by 1.3 times. The input is an illustration with adjusted brightness, and the output is an illustration with adjusted contrast.

[1693] Step 6:

[1694] The server generates the beautified illustration.

[1695] The server saves the adjusted image as "beautified_illustration.png". The input is a contrast-adjusted illustration, and the output is a beautified illustration file.

[1696] Step 7:

[1697] The server provides the beautified illustrations to the user.

[1698] The server provides the beautified illustration "beautified_illustration.png" to the user. The user can obtain the beautified illustration by clicking the download link from their device. The input is the beautified illustration file, and the output is the download link provided to the user. Specifically, the server displays the progress in real time and notifies the user after the process is completed.

[1699] (Application example 1)

[1700] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1701] While existing illustration beautification systems automatically beautify user-drawn illustrations, they lack a way to easily share the beautified illustrations with content distribution services. Additionally, the customization options for the beautification process are limited, making it difficult for users to customize the beautification process to their own preferences.

[1702] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1703] In this invention, the server includes means for automatically beautifying illustrations drawn by users using advanced image processing technology, means for providing a user interface that can be intuitively operated by users, means for allowing users to adjust beautification parameters, means for allowing users to directly share beautified illustrations to a content distribution service, and means for providing customization options for the beautification process, thereby enabling users to easily share beautified illustrations and further customize the beautification process to their own preferences.

[1704] "Advanced image processing technology" refers to technology for automatically adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[1705] "User-drawn illustrations" are paintings or drawings created by users, either by hand or using digital tools.

[1706] "Automatic beautification" refers to algorithms or software that enhance the visual characteristics of illustrations without user intervention.

[1707] A "user interface that users can operate intuitively" is an interface that is designed so that users can operate it easily and intuitively.

[1708] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[1709] "User adjustable means" refers to a feature that allows the user to manually change the beautification parameters.

[1710] A "content distribution service" is a service that distributes digital content such as images and videos over the Internet.

[1711] "Shareable means" refers to the ability to share aestheticized illustrations with other users and platforms.

[1712] "Customization Options" are settings and features that allow users to tailor the beautification process to their own preferences.

[1713] As an embodiment of the present invention, the following system can be constructed.

[1714] System Overview

[1715] The system automatically beautifies illustrations drawn by users and shares the beautified illustrations with a content distribution service. The system includes a smartphone, a server, and software using advanced image processing technology.

[1716] Hardware and software used

[1717] Hardware: Smartphone (iOS or Android), server

[1718] Software: Python, OpenCV (image processing library)

[1719] Data processing and calculation

[1720] 1. Loading images

[1721] Users can take a photo of an illustration they have drawn on their smartphone or upload it using the smartphone's camera or file upload function.

[1722] 2. Adjusting the color tone

[1723] The server converts the image to the HSV color space and increases the saturation, which makes the illustration appear more vibrant.

[1724] 3. Adjusting the brightness

[1725] The server performs a scale transformation to increase the brightness of the image, which makes the illustration brighter.

[1726] 4. Adjusting the contrast

[1727] The server converts the image to the LAB color space and performs histogram equalization on the L channel to enhance the contrast, which improves the visual characteristics of the illustration.

[1728] 5. Save and share images

[1729] The beautified images are stored on a server, and users can share them directly to content distribution services via smartphone applications.

[1730] Specific examples

[1731] Users can take a photo of an illustration they've drawn with their smartphone and upload it to the app, where the server automatically beautifies it. The beautified illustration can then be adjusted in color, brightness, and contrast to suit the user's preferences. Finally, the beautified illustration can be shared on social media and other content distribution services.

[1732] Prompt Sentence Examples

[1733] "Develop an app that beautifies illustrations drawn by users. The app will automatically adjust the color tone, brightness, and contrast of the illustration and output the beautified illustration."

[1734] In this way, a system can be realized that allows users to easily create and share aesthetic illustrations.

[1735] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1736] Step 1:

[1737] The user takes a photo of an illustration drawn on their smartphone or uploads it. The input is the illustration image drawn by the user, and the output is an image file saved on the smartphone. Specifically, the user takes a photo of the illustration using the smartphone's camera function, or uploads an existing image file to the application.

[1738] Step 2:

[1739] The device sends the photographed or uploaded illustration image to the server. The input is the image file saved on the smartphone, and the output is the image data transferred to the server. Specifically, the smartphone application calls an API to send the image file to the server.

[1740] Step 3:

[1741] The server reads the received illustration image. The input is the image data transferred to the server, and the output is an image object loaded into memory. Specifically, the server reads the image file using Python and OpenCV.

[1742] Step 4:

[1743] The server adjusts the color tone of an image. The input is an image object loaded into memory, and the output is an image object with the adjusted color tone. Specifically, the server converts the image to the HSV color space and increases the saturation.

[1744] Step 5:

[1745] The server adjusts the brightness of the image. The input is a color-adjusted image object, and the output is a brightness-adjusted image object. Specifically, the server performs a scale transformation to increase the brightness of the image.

[1746] Step 6:

[1747] The server adjusts the contrast of an image. The input is a brightness-adjusted image object, and the output is a contrast-adjusted image object. Specifically, the server converts the image to the LAB color space and performs histogram equalization on the L channel.

[1748] Step 7:

[1749] The server saves the beautified image. The input is a contrast-adjusted image object, and the output is a beautified image file saved on the server. Specifically, the server saves the beautified image as a file.

[1750] Step 8:

[1751] The device obtains the beautified image from the server. The input is the beautified image file stored on the server, and the output is the beautified image file downloaded to the smartphone. Specifically, the smartphone application calls an API to download the beautified image from the server.

[1752] Step 9:

[1753] The user shares the beautified illustration on a content distribution service. The input is a beautified image file downloaded to a smartphone, and the output is a beautified image uploaded to the content distribution service. Specifically, the user uses a smartphone application to share the beautified illustration on social media or other content distribution services.

[1754] Example 2

[1755] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1756] With conventional image processing systems, beautifying user-drawn illustrations required specialized knowledge and complex operations, and lacked an intuitive user interface. It was also difficult to adjust the beautification parameters, making it difficult for users to beautify illustrations to their own preferences. Furthermore, beautification processing using generative AI models was not performed in real time, resulting in delayed feedback to user operations.

[1757] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1758] In this invention, the server includes a means for converting beautification parameters operated by the user into a prompt sentence, a means for inputting the prompt sentence into a generative AI model to beautify the illustration, and a means for displaying the beautified illustration to the user, thereby enabling the user to adjust the beautification parameters and check the beautified illustration in real time through an intuitive user interface.

[1759] "Advanced image processing technology" is a general term for advanced algorithms and methods used to automatically beautify user-drawn illustrations.

[1760] "User interface" is a general term for the screens and input devices that users use to operate a system, and includes UI elements that can be operated intuitively.

[1761] "Beautification parameters" are setting values ​​for adjusting visual characteristics such as color tone, brightness, and contrast of an illustration.

[1762] A "prompt" is a textual instruction used to input user-set beautification parameters into a generative AI model.

[1763] A "generative AI model" is an artificial intelligence model that beautifies illustrations based on input prompts, and examples include "Stable Diffusion" and "DALL-E."

[1764] A "slider" is a UI element that allows the user to adjust a value by dragging it.

[1765] A "dial" is a UI element that allows the user to adjust a value by rotating it.

[1766] The "server" is a computer system that receives prompt text, inputs it into a generative AI model, generates a beautified illustration, and sends it back to the device.

[1767] A "terminal" is a device operated by a user, which has the functions of providing a user interface, acquiring beautification parameters, generating prompt sentences, and sending them to a server.

[1768] This invention is a system for automatically beautifying illustrations drawn by users, providing a user interface that allows users to operate intuitively. The system converts beautification parameters operated by the user into prompt sentences, which are then input into a generative AI model to beautify the illustration. A specific embodiment of this system is described below.

[1769] Hardware and software used

[1770] Hardware

[1771] Terminal: A device operated by the user that provides a user interface, acquires beautification parameters, generates prompts, and sends them to the server. Examples include PCs, tablets, and smartphones.

[1772] Server: A computer system that receives prompt text, inputs it into a generative AI model, generates a beautified illustration, and sends it back to the device.

[1773] software

[1774] User Interface: Includes UI elements that users can operate intuitively, such as sliders and dials.

[1775] Generative AI model: An artificial intelligence model that beautifies illustrations based on prompts. Specifically, "Stable Diffusion" and "DALL-E" are used.

[1776] Data processing and calculation

[1777] User operations

[1778] The user operates the user interface on the device, specifically, by using sliders and dials to adjust the beautification parameters. For example, when the user moves the slider to the right, the beautification parameter increases.

[1779] Terminal handling

[1780] The device obtains the position information of the slider or dial operated by the user in real time. This position information is interpreted as a beautification parameter. For example, if the slider is set to the maximum value, the beautification parameter is "100". The device generates a prompt sentence based on the obtained beautification parameter. For example, if the beautification parameter is "100", the device generates the prompt sentence "Beautification parameter: 100".

[1781] Server Processing

[1782] The server receives the prompt sent from the device. It inputs the received prompt into the generative AI model. The generative AI model beautifies the illustration based on the input prompt. For example, if a prompt with "beautification parameter: 100" is input, the beautification level of the illustration is set to the maximum and processing is performed. The server receives the beautified illustration from the generative AI model and sends it back to the device.

[1783] Terminal display

[1784] The terminal receives the beautified illustration sent back from the server and displays it to the user, who can then check the illustration based on the beautification parameters he or she has set.

[1785] Specific examples

[1786] Prompt Sentence Examples

[1787] "Beautification parameter: 100"

[1788] By inputting this prompt into the generative AI model, the degree of beautification of the illustration can be set to the maximum.

[1789] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1790] Step 1:

[1791] The user interacts with a UI element.

[1792] The user operates the user interface on the device and adjusts the beautification parameters using sliders and dials. For example, when the user moves the slider to the right, the beautification parameter increases. The input is the user's operation, and the output is the adjusted beautification parameter.

[1793] Step 2:

[1794] The terminal acquires the beautification parameters.

[1795] The device obtains the position information of the slider or dial operated by the user in real time. This position information is interpreted as a beautification parameter. For example, if the slider is set to the maximum value, the beautification parameter is "100". The input is the user's operation position information, and the output is the beautification parameter.

[1796] Step 3:

[1797] The terminal generates a prompt.

[1798] The terminal generates a prompt sentence based on the acquired beautification parameter. For example, if the beautification parameter is "100", the terminal generates the prompt sentence "Beautification parameter: 100". The input is the beautification parameter, and the output is the prompt sentence.

[1799] Step 4:

[1800] The terminal sends the prompt to the server.

[1801] The terminal sends the generated prompt text to the server. This transmission is performed using a communication protocol such as an HTTP request. The input is the prompt text, and the output is a transmission completion notification to the server.

[1802] Step 5:

[1803] The server receives the prompt.

[1804] The server receives the prompt sent from the terminal. The received prompt is used in the next processing step. The input is the prompt, and the output is a confirmation of receipt.

[1805] Step 6:

[1806] The server inputs a prompt sentence into the generative AI model.

[1807] The server inputs the received prompt sentence into a generative AI model. For example, generative AI models such as "Stable Diffusion" or "DALL-E" can be used. The input is the prompt sentence, and the output is a notification to the generative AI model that the input has been completed.

[1808] Step 7:

[1809] Generative AI models beautify illustrations.

[1810] The generative AI model beautifies the illustration based on the input prompt. For example, if a prompt with "Beautification parameter: 100" is input, the beautification level of the illustration is set to the maximum and processing is performed. The input is the prompt, and the output is a beautified illustration.

[1811] Step 8:

[1812] The server returns the beautified illustration to the device.

[1813] The server receives the beautified illustration from the generative AI model and returns it to the device. This return is also done using a communication protocol such as an HTTP response. The input is the beautified illustration, and the output is a notification of completion of transmission to the device.

[1814] Step 9:

[1815] The device displays a beautified illustration.

[1816] The terminal receives the beautified illustration sent back from the server and displays it to the user. The user can then check the illustration based on the beautification parameters they set. The input is the beautified illustration, and the output is the display to the user.

[1817] (Application example 2)

[1818] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1819] Conventional image beautification systems lack an intuitive user interface when beautifying user-drawn illustrations or photos, making it difficult to preview in real time or save or share beautified images. Another issue is that it is difficult for users to fine-tune visual characteristics when adjusting beautification parameters.

[1820] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1821] In this invention, the server includes means for automatically beautifying illustrations drawn by a user using advanced image processing technology, means for providing a user interface that can be operated intuitively by the user, means for allowing the user to adjust beautification parameters, means for previewing images uploaded by the user in real time, and means for saving or sharing the beautified images. This allows the user to intuitively operate the server, check the beautified images in real time, adjust visual characteristics to suit their preferences, and save or share the final images.

[1822] "Advanced image processing techniques" refers to advanced algorithms and methods for analyzing, transforming, and modifying images.

[1823] "User-drawn illustrations" refers to pictures or drawings manually created by a user.

[1824] "Automatic beautification" refers to techniques or devices that enhance the quality or appearance of images without user intervention.

[1825] "A user interface that users can operate intuitively" refers to an interface that is designed so that users can operate it easily and naturally.

[1826] "Beautification parameters" refer to settings for adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[1827] "User-adjustable means" refers to features that allow users to change settings or parameters to suit their preferences.

[1828] "Means for real-time preview" refers to a function that displays the results immediately when the user performs an operation.

[1829] "Means to save or share beautified images" refers to the functionality to save the beautified images to your device or share them with other users.

[1830] The system for implementing this invention automatically beautifies illustrations and photographs drawn by the user and provides an intuitive user interface. Specific embodiments of this system are described below.

[1831] System Configuration

[1832] Hardware

[1833] Smartphone: A device that allows users to upload and manipulate images.

[1834] Server: Provides the computing resources for image processing.

[1835] software

[1836] OpenCV: Image processing library for analyzing, transforming, and modifying images.

[1837] Tkinter: A GUI library used to build user interfaces.

[1838] Data processing and calculation

[1839] Loading images

[1840] A user uploads an image using their smartphone, and the server reads the image using OpenCV.

[1841] Adjusting beautification parameters

[1842] Users can adjust the beautification parameters (smoothness, brightness, contrast) by using the sliders and dials displayed on the smartphone screen, allowing them to see the changes in the image in real time.

[1843] Real-time preview

[1844] The server processes the image in real time based on the beautification parameters adjusted by the user and sends a preview to the smartphone, allowing the user to see the results instantly.

[1845] Save and share images

[1846] Once users are satisfied with the image, they can save it on their smartphone and share it with other users via social media or email.

[1847] Specific examples

[1848] Users open the app on their smartphone, upload a photo, use sliders to adjust smoothness, brightness, and contrast, preview the beautified photo in real time, and once satisfied, save the photo or share it on social media.

[1849] Prompt Sentence Examples

[1850] "Create an app where you upload a photo on your smartphone, use sliders to adjust smoothness, brightness, and contrast, preview the beautified photo in real time, and save or share it."

[1851] In this way, the invention allows users to intuitively view beautified images in real time, adjust visual characteristics to their liking, and save or share the final image.

[1852] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1853] Step 1:

[1854] Uploading an image

[1855] The user uses a smartphone to launch the application, select an image, and upload it.

[1856] Input: An image file selected by the user.

[1857] Output: Image data sent to the server.

[1858] Specific operation: The user selects an image from the smartphone gallery and presses the "Upload" button. The image data is sent to the server.

[1859] Step 2:

[1860] Loading images

[1861] The server reads the received image data using OpenCV.

[1862] Input: Image data sent to the server.

[1863] Output: Image object loaded by OpenCV.

[1864] Specific operation: The server receives the image data and reads the image into memory using OpenCV's cv2.imread function.

[1865] Step 3:

[1866] Initial settings for beautification parameters

[1867] The server sets the initial beautification parameters (smoothness, brightness, contrast).

[1868] Input: None (default value).

[1869] Output: Initialized beautification parameters.

[1870] What happens: The server sets initial values ​​for smoothness, brightness, and contrast, and then persists these values.

[1871] Step 4:

[1872] User Interface Display

[1873] The terminal displays a user interface and provides sliders and dials.

[1874] Input: Initialized beautification parameters.

[1875] Output: Display of the user interface.

[1876] What it does: The terminal uses Tkinter to display a user interface, including sliders and dials.

[1877] Step 5:

[1878] Adjusting beautification parameters

[1879] The user adjusts the beautification parameters by manipulating sliders and dials.

[1880] Input: The value of a slider or dial as manipulated by the user.

[1881] Output: Adjusted beautification parameters.

[1882] What happens: The user operates a slider or dial, sending a new parameter value to the server.

[1883] Step 6:

[1884] Image beautification

[1885] The server processes the image based on the adjusted beautification parameters.

[1886] Input: adjusted beautification parameters, loaded image object.

[1887] Output: A beautified image object.

[1888] What it does: The server uses OpenCV functions to adjust smoothness, brightness, and contrast to beautify the image.

[1889] Step 7:

[1890] View real-time preview

[1891] The device will display a preview of the beautified image in real time.

[1892] Input: A beautified image object.

[1893] Output: The beautified image displayed on the device.

[1894] Specific operation: The server sends the beautified image to the device, which displays it on the screen.

[1895] Step 8:

[1896] Save and share images

[1897] The user saves or shares the beautified image.

[1898] Input: A beautified image object.

[1899] Output: Saved image file or shared image link.

[1900] Specific behavior: The user presses the "Save" or "Share" button, and the device saves the image to local storage or shares it via social media or email.

[1901] Example 3

[1902] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1903] In conventional illustration beautification systems, it was difficult for users to intuitively adjust beautification parameters, and the generated illustrations often did not meet the user's expectations. Furthermore, there was a lack of a process to properly analyze the parameters entered by the user and generate prompts appropriate for the generative AI model, which resulted in inconsistent beautification of the illustrations.

[1904] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1905] In this invention, the server includes means for automatically beautifying an illustration drawn by a user, means for providing a user interface that the user can intuitively operate, means for the user to adjust beautification parameters, means for transmitting the beautification parameters input by the user to the server, means for the server to analyze the received beautification parameters and generate a prompt sentence suitable for the generative AI model, means for the generative AI model to beautify the illustration based on the prompt sentence, and means for the server to transmit the beautified illustration to the user's terminal. This allows the user to intuitively adjust the beautification parameters and obtain a consistently beautified illustration using the generative AI model.

[1906] "Advanced image processing technology" refers to technology that uses computers to analyze, modify, and convert images, and is particularly capable of achieving highly accurate and complex processing.

[1907] A "user interface" is the means by which a user interacts with a system, including both visual and operational elements.

[1908] "Beautification parameters" are setting values ​​for adjusting the visual characteristics of an illustration, and include color tone, brightness, contrast, and the like.

[1909] A "generative AI model" is an algorithm or system that uses artificial intelligence to generate or transform data, specifically for image generation.

[1910] A "prompt sentence" is text-based input data used to give instructions to a generative AI model.

[1911] A "server" is a computer system that provides services to clients over a network.

[1912] A "terminal" is a device that a user directly operates, including a computer, smartphone, tablet, etc.

[1913] "Analysis" is the process of examining data in detail to understand its structure and meaning.

[1914] "Transmitting" is the act of moving data from one place to another.

[1915] An "illustration" is an image intended for visual expression, created by hand or using digital tools.

[1916] MODE FOR CARRYING OUT THE INVENTION

[1917] This invention is a system for automatically beautifying illustrations drawn by a user, providing a user interface that can be operated intuitively by the user and including means for allowing the user to adjust beautification parameters. Specific embodiments of this system are described below.

[1918] Hardware and software used

[1919] The system uses the following hardware and software:

[1920] Server: A computer system that provides services to clients over a network.

[1921] Terminal: A device that is directly operated by a user (e.g., computer, smartphone, tablet).

[1922] Generative AI Model: An algorithm or system that uses artificial intelligence to generate or transform data (e.g., Stable Diffusion, DALL-E).

[1923] System processing overview

[1924] Users input beautification parameters using their devices. For example, they can use a dedicated device application or a web interface to set parameters such as color intensity and brightness adjustment. Specifically, they can use sliders and text boxes to set "color intensity" to 50 and "brightness adjustment" to 70.

[1925] The device sends the beautification parameters entered by the user to the server. This transmission is performed using a communication protocol such as an HTTP request or WebSocket. The server analyzes the received beautification parameters and generates a prompt appropriate for the generative AI model. For example, the server generates a prompt such as, "Please generate an illustration with color intensity: 50, brightness adjustment: 70."

[1926] The server inputs a prompt into the generative AI model and requests it to beautify the illustration. At this time, the server calls the generative AI model's API. The generative AI model beautifies the illustration based on the prompt and returns the results to the server. The returned data is an image file of the beautified illustration.

[1927] The server sends the beautified illustration received from the generative AI model to the device. This transmission is also done using a communication protocol such as HTTP response or WebSocket. The user checks the beautified illustration on the device. The device application or web interface has the function to display the received illustration.

[1928] Specific examples

[1929] If a user uses their device to set "Color Intensity" to 50 and "Brightness Adjustment" to 70, the server will input the following prompt to the generative AI model:

[1930] Example prompt sentence:

[1931] "Create an illustration with color intensity: 50, brightness adjustment: 70."

[1932] The generative AI model receives this prompt and beautifies the illustration based on the specified parameters, resulting in a beautified illustration tailored to the user's preferences.

[1933] In this way, the user can intuitively adjust the beautification parameters and obtain a stable beautified illustration using the generative AI model. The flow of the specific processing in Example 3 will be described with reference to FIG.

[1934] Step 1:

[1935] The user inputs the beautification parameters using a terminal.

[1936] Input: The user sets parameters such as color intensity and brightness using a dedicated application on the device or a web interface.

[1937] What it does: The user uses the sliders and text boxes to set "Color Intensity" to 50 and "Brightness Adjustment" to 70.

[1938] Output: Beautification parameters set to the terminal.

[1939] Step 2:

[1940] The terminal transmits the input beautification parameters to the server.

[1941] Input: Beautification parameters set on the device.

[1942] Specific operation: The terminal makes an HTTP POST request and sends the beautification parameters in JSON format to the server endpoint.

[1943] Output: Beautification parameters sent to the server.

[1944] Step 3:

[1945] The server analyzes the received beautification parameters and generates a prompt sentence suitable for the generative AI model.

[1946] Input: The beautification parameters sent to the server.

[1947] Specific operation: The server analyzes the received JSON data and performs string manipulation to generate a prompt. For example, it generates a prompt such as "Please generate an illustration with color intensity: 50, brightness adjustment: 70."

[1948] Output: The generated prompt statement.

[1949] Step 4:

[1950] The server inputs a prompt into the generative AI model, requesting it to beautify the illustration.

[1951] Input: The generated prompt statement.

[1952] Specific operation: The server sends an HTTP POST request to the API endpoint of the generated AI model and sends data including the prompt text.

[1953] Output: The prompt sent to the generative AI model.

[1954] Step 5:

[1955] The generative AI model generates a beautified illustration and sends it back to the server.

[1956] Input: The prompt sent to the generative AI model.

[1957] Specific operation: The generative AI model internally beautifies the illustration and sends the generated illustration back to the server as binary data.

[1958] Output: Binary data of the beautified illustration sent back to the server.

[1959] Step 6:

[1960] The server sends the beautified illustration to the device.

[1961] Input: Binary data of the aestheticized illustration sent back to the server.

[1962] Specific operation: The binary data received by the server is sent to the terminal as an HTTP response.

[1963] Output: A beautified illustration sent to the device.

[1964] Step 7:

[1965] The user checks the beautified illustration on their device.

[1966] Input: A glorified illustration sent to the device.

[1967] Specific operation: The illustration received by the device is displayed within the application, and the user confirms the illustration.

[1968] Output: A beautified illustration reviewed by the user.

[1969] (Application example 3)

[1970] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1971] Conventional image beautification systems often lack an intuitive interface for users when automatically beautifying illustrations drawn by users. They also lack the ability to preview the adjusted illustration in real time or the ability to save and share the adjustments on other content distribution services. This makes it difficult for users to beautify illustrations to their liking and easily share them.

[1972] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1973] In this invention, the server includes means for automatically beautifying illustrations drawn by users using advanced image processing technology, means for providing a user interface that users can operate intuitively, means for allowing users to adjust beautification parameters, means for previewing the adjusted illustration in real time, and means for saving the adjustments and sharing them with other content distribution services. This allows users to beautify illustrations to their liking and easily share them.

[1974] "Advanced image processing technology" refers to advanced algorithms and methods for analyzing and transforming images.

[1975] "User-drawn illustrations" refers to pictures or drawings manually created by a user.

[1976] "Automatic beautification" refers to techniques that improve the quality or appearance of images without user intervention.

[1977] An "intuitive user interface" refers to an interface that users can easily understand and operate.

[1978] "Beautification parameters" refer to settings for adjusting visual characteristics such as color tone, brightness, and contrast of an image.

[1979] "A means of previewing the adjusted illustration in real time" refers to technology that instantly displays the results when the user changes parameters.

[1980] "Means for saving adjustments and sharing them with other content distribution services" refers to technology that allows users to record image adjustments they make and share them with other online platforms.

[1981] The system for implementing this invention automatically beautifies illustrations drawn by users and provides a user interface that allows users to operate intuitively. The system is implemented as an application that runs on devices such as smartphones and tablets.

[1982] Hardware and software used

[1983] Hardware: Smartphones, tablets

[1984] Software: Python, OpenCV, Tkinter

[1985] System configuration

[1986] 1. Image loading: Users can take photos or illustrations on their devices and upload them to the application.

[1987] 2. Adjusting beautification parameters: Users can use sliders and dials within the application to adjust beautification parameters such as "color intensity" and "brightness adjustment."

[1988] 3. Real-time preview: Based on the adjusted parameters, the image beautification process will be carried out in real time and the results will be displayed instantly.

[1989] 4. Saving and sharing adjustments: Users can save the adjustment results they are satisfied with and share them on social media and other content distribution services.

[1990] Processing flow

[1991] 1. Loading an image: Select an image from the device's camera or gallery and upload it to the application. OpenCV is used to load the image and begin processing.

[1992] 2. Adjusting beautification parameters: We use Tkinter to provide intuitive sliders and dials for users to operate. Users can adjust beautification parameters by manipulating these UI elements.

[1993] 3. Real-time preview: Every time the user changes a parameter, the brightness and contrast of the image are adjusted using the OpenCV convertScaleAbs function, and the results are displayed in real time.

[1994] 4. Saving and sharing adjustments: Provides the ability to save the adjusted image on the device and share it on social media and other content distribution services.

[1995] Specific examples

[1996] For example, a user can upload a landscape photo taken with their smartphone to the application, enhance the color tone, slightly reduce the brightness, and share it on Instagram.

[1997] Prompt Sentence Examples

[1998] "Upload a landscape photo, intensify the colors, lower the brightness a bit, and share it to Instagram."

[1999] In this way, users can customize their illustrations to their liking and share them easily.

[2000] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[2001] Step 1:

[2002] Users can upload photos and illustrations taken on their devices to the application.

[2003] Input: An image file selected from the device's camera or gallery.

[2004] Output: Image data loaded into the application.

[2005] Specific behavior: A user launches the application, clicks the image upload button, and selects an image from the device's camera roll or gallery. The selected image is then loaded into the application.

[2006] Step 2:

[2007] The device starts processing the image data read using OpenCV.

[2008] Input: The loaded image data.

[2009] Output: An image object that can be processed with OpenCV.

[2010] Specific operation: The application uses the OpenCV library to convert the loaded image data into an internal format so that it can be processed.

[2011] Step 3:

[2012] The user operates sliders and dials to adjust the beautification parameters.

[2013] Input: The value of a slider or dial as manipulated by the user.

[2014] Output: Adjusted beautification parameters.

[2015] Specific operation: The user operates sliders and dials within the application to set parameters such as "color intensity" and "brightness adjustment." The set values ​​are reflected in the application in real time.

[2016] Step 4:

[2017] The device will preview the image in real time based on the adjusted beautification parameters.

[2018] Input: adjusted beautification parameters and original image data.

[2019] Output: Real-time preview of the adjusted image.

[2020] Specific operation: The application uses OpenCV's convertScaleAbs function to adjust the brightness and contrast of the image based on the beautification parameters set by the user, and displays the results in real time.

[2021] Step 5:

[2022] Users can save the adjusted images and share them with other content distribution services.

[2023] Input: Adjusted image data.

[2024] Output: Saved image files and shared images.

[2025] Specific operation: The user presses the save button to save the adjusted image to the device, and then presses the share button to upload the image to Instagram or other social media.

[2026] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2027] "Example 1"

[2028] One embodiment of the present invention is an illustration beautification system incorporating an emotion engine. This system includes a function for automatically beautifying illustrations drawn by a user, a function for providing a user interface that allows the user to intuitively operate the system, a function for allowing the user to adjust beautification parameters, and an emotion engine that recognizes the user's emotions. The emotion engine recognizes the user's emotions from the user's facial expressions, tone of voice, text input, etc., and adjusts the beautification parameters according to the user's emotions. Specifically, the system can beautify the illustration according to the user's emotions, such as by adjusting the color tone of the illustration to be brighter when the user is feeling happy and adjusting the color tone to be darker when the user is feeling sad.

[2029] "Example 2"

[2030] The emotion engine also has the ability to recognize changes in a user's emotions in real time and dynamically adjust the beautification parameters accordingly. For example, if a user starts drawing feeling happy but then changes to sadness midway through, the emotion engine will capture that change in real time and instantly adjust the beautification parameters. This makes it possible to beautify illustrations in response to changes in the user's emotions.

[2031] "Example 3"

[2032] Furthermore, the emotion engine also has a learning function to recognize the user's emotions. Specifically, it collects data such as the user's facial expressions, tone of voice, and text input, and uses a machine learning algorithm to train an emotion recognition model. This learning enables the emotion ...

Claims

1. A means for acquiring image data representing an illustration drawn by a user; means for automatically adjusting visual characteristics of the image data; means for recognizing emotions of the user using an emotion engine; a means for adjusting visual characteristics of the image data using a prompt sentence that instructs the user to adjust visual characteristics of the image data based on the user's emotion and a generative AI; A system including:

2. 10. The system of claim 1, further comprising means for adjusting visual characteristics of said image data via a user-operable user interface.

3. 2. The system of claim 1, wherein the automatically adjusting means adjusts hue, brightness, and contrast of the image data.

Images