System

Abstract

A system is provided.SOLUTION: A system comprising: means for uploading an incomplete illustration file to be rendered through a user interface; means for optionally selecting a style or display setting; means for transmitting the uploaded illustration file and the selected option to a server; means for processing the illustration file using a AI image processing engine by the server; and means for transmitting a processing completion notice and a processing result file from the server to a client.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern manga production requires a great deal of time and effort, with line drawing and coloring tasks placing a particularly heavy burden on creators. Furthermore, the long production period can limit the frequency and quality of work. These challenges have led to a decline in productivity throughout the manga industry, creating an environment that makes it difficult for new talent to develop. The purpose of this invention is to dramatically improve the efficiency of manga production, significantly shorten production time, and reduce the burden on creators. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for uploading an unfinished illustration file to be drawn through a user interface, a means for selecting an optional style and display settings, a means for transmitting the uploaded illustration file and the selected options to a server, a means for the server to process the illustration file using an AI image processing engine, a means for transmitting a processing progress report from the server to a terminal, a means for transmitting a processing completion notice and a processing result file from the server to the terminal, and a means for displaying a processing completion notice to the user and allowing the terminal to download the result file. This allows users to automate part of the manga production process with simple operations, significantly reducing production time. Furthermore, the system also includes a means for the server to select and operate different AI image processing engines based on the style selected by the user, and a means for the terminal to display the progress report transmitted from the server to the user in real time, enabling flexible and real-time production support.

[0006] A "user interface" is an interface that provides a screen and input means for users to operate a system or software.

[0007] An "illustration file" is image data that has been drawn by a user and saved in digital format.

[0008] "Style" is an option that refers to drawing and color settings based on a specific art style or design principles.

[0009] "Display settings" are optional settings that allow the user to specify desired drawing details and effects.

[0010] A "server" is a computer system that processes requests from client devices or terminals and provides the required data or services.

[0011] An "AI image processing engine" is a software module that uses artificial intelligence technology to analyze, convert, and modify image data.

[0012] "Progress" is information indicating the current progress and completion status of image processing performed by the server.

[0013] The "processing completion notification" is a message sent by the server to the terminal indicating the completion of image processing.

[0014] The "result file" is the final illustration data after being processed by the AI ​​image processing engine.

[0015] "Downloading" is the operation of transferring data or files stored on a server to a user's terminal via the Internet. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention provides a means to efficiently automate part of the manga production process through a system that operates via a server and terminals. Specifically, it is a system that automatically processes unfinished illustration files drawn by users using an AI image processing engine to complete them in a short amount of time.

[0038] User Interface Parts

[0039] Users operate the system through a dedicated user interface (UI). Users access the system from their device's browser and upload unfinished illustration files. The UI displays an "Upload Line Art" button, and clicking this displays a file selection dialog. Here, users can select an illustration file from their local disk and upload it.

[0040] Additionally, the UI provides "Style Selection" and "Display Settings" options, allowing users to select the style and effect they desire.

[0041] Sending part of the request

[0042] The selected illustration file and settings are sent from the device to the server as an HTTP POST request, which includes the file data and the user-specified style and display settings.

[0043] Server-side processing

[0044] The server receives the illustration file and setting information sent from the device and temporarily stores them. The server then starts the AI ​​image processing engine and provides the received data to the processing engine. The AI ​​image processing engine uses a deep learning model to analyze the line drawing of the illustration and color it in the specified style.

[0045] Progress management and notifications

[0046] The server manages the processing progress of the AI ​​image processing engine and periodically checks its progress. This progress information is sent from the server to the device in real time. The device displays the received progress information on the UI, allowing the user to understand the current progress.

[0047] Returning and downloading results

[0048] When the processing is complete, the server generates a processing result file and sends a processing completion notification to the device. This completion notification includes a download link for the result file. Based on the received completion notification, the device displays a pop-up or dialog box on the UI and generates a "Download" button.

[0049] The user confirms the completion notification and clicks the download button to save the resulting file to their local disk, allowing them to obtain the colored finished illustration in a short time.

[0050] Specific examples

[0051] For example, consider the case where a user uploads a monochrome line drawing illustration and chooses to colorize it in a vibrant style.

[0052] 1. The user uses the device to click the "Upload Line Art" button on the UI to upload an unfinished monochrome line art file.

[0053] 2. The user selects the "Colorful" style from the "Style Selection" drop-down menu.

[0054] 3. The device sends the file and style information to the server.

[0055] 4. The server processes the file using an AI image processing engine and periodically sends progress updates to the device.

[0056] 5. When the progress reaches 100%, the server will send a notification of completion and a download link for the result file to the device.

[0057] 6. The device displays a completion notice to the user and provides a "Download" button.

[0058] 7. The user clicks the download button and receives the completed color illustration file.

[0059] This invention significantly reduces the time and effort required to create manga, providing an environment in which creators can concentrate on creating their work. It also enhances creators' creativity by supporting a variety of styles and settings.

[0060] The processing flow will be explained below.

[0061] Step 1:

[0062] The user accesses the terminal's user interface. The user interface can be accessed from a browser or a dedicated application, and a login screen is displayed. The user enters login information and logs into the system.

[0063] Step 2:

[0064] The user uploads a line drawing file. The user clicks the "Upload Line Drawing" button on the UI and selects a line drawing file from the local disk in the file selection dialog that appears. The device prepares to send the selected file to the server.

[0065] Step 3:

[0066] The user selects a style or display setting. The UI displays a drop-down menu of style choices and other options. The user selects the style or display setting they want.

[0067] Step 4:

[0068] The device sends the uploaded line art file and selected options to the server as an HTTP POST request, which includes the file data and style information.

[0069] Step 5:

[0070] The server receives the request, temporarily stores the received data, and saves the received line drawing file in an appropriate folder, while also recording the style information specified by the user.

[0071] Step 6:

[0072] The server starts the AI ​​image processing engine and provides the received data to the processing engine. The server selects an AI model corresponding to the specified style and processes the line drawing file.

[0073] Step 7:

[0074] The server manages the progress of the processing and periodically notifies the device. As the AI ​​image processing progresses, the progress status is reported to the server, which then periodically sends it to the device.

[0075] Step 8:

[0076] The terminal displays the received progress information on the UI so that the user can check the current processing status in real time.

[0077] Step 9:

[0078] After the process is complete, the server generates a result file and sends a notification of the completion of the process to the device, along with a download link for the result file.

[0079] Step 10:

[0080] The device displays a pop-up or dialog box on the UI based on the received completion notification, and generates a "Download" button to allow the user to download the result file.

[0081] Step 11:

[0082] The user downloads the resulting file. After confirming the completion notification, they click the download button and save it to their local disk. The completed color illustration file is provided.

[0083] This series of processes allows users to quickly and efficiently automate parts of manga production, significantly reducing production time and effort.

[0084] Example 1

[0085] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0086] Traditional manga production has the problem of requiring a great deal of time and effort to color images and reflect styles. Manual coloring, in particular, is a factor that reduces productivity, and many creators are seeking more efficient methods. Furthermore, there is often a lack of smooth communication methods for checking progress and obtaining results. To solve these issues, a system is needed that automates image processing, allowing users to check progress in real time and obtain results quickly.

[0087] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0088] In this invention, the server includes a means for uploading an unfinished image file to be drawn through a user interface, a means for selecting an optional style and display setting, a means for transmitting the uploaded image file and the selected option to the server, a means for processing the image file using an artificial intelligence image processing engine by the server, a means for transmitting a processing progress report from the server to the terminal, a means for transmitting a processing completion notice and a processing result file from the server to the terminal, and a means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file, thereby enabling the user to quickly automatically colorize and style images, check the progress in real time, and reliably obtain the results.

[0089] A "user interface" is a dedicated interface for users to perform operations, and includes, for example, buttons and menus displayed on the screen.

[0090] "Image files" refers to digital files such as line drawings and illustrations, and includes formats such as JPEG, PNG, and BMP.

[0091] "Style" refers to the design and effect settings applied in image processing, and includes different methods of expression such as "vivid colors" and "monochrome."

[0092] "Display settings" refers to various settings related to the display of images and user interfaces, including, for example, brightness and contrast adjustments.

[0093] A "server" refers to a computer system that accepts requests from client terminals via a network and processes and stores data.

[0094] "Artificial intelligence image processing engine" refers to software that analyzes and processes images using deep learning models and other artificial intelligence technologies.

[0095] "Processing progress" refers to information indicating the progress of the processing currently being performed by the image processing engine.

[0096] A "processing completion notification" is a message from the system notifying you that image processing is complete, and typically includes a download link.

[0097] The "download link" indicates the URL for the user to obtain the result file.

[0098] "Terminal" refers to an electronic device such as a computer or smartphone that is operated by a user.

[0099] The present invention is a system that automatically processes unfinished image files drawn by users using an artificial intelligence image processing engine to complete them in a short time. The system is operated through a user interface and operates via a server and terminals.

[0100] First, the user operates the system through a dedicated user interface (UI). The user accesses the system from the device's browser and uploads an unfinished image file. The UI displays an "Upload Image" button, and clicking this displays a file selection dialog. The user selects an image file from the local disk and uploads it. Furthermore, the UI provides options for "Style Selection" and "Display Settings," allowing the user to select the desired style and effect.

[0101] The image file and settings information selected by the user are sent from the device to the server as an HTTP POST request. This request includes the file data and the style and display settings specified by the user. The server receives the image file and settings information sent from the device and temporarily stores them in a temporary directory on the server. The server then starts an artificial intelligence image processing engine and provides the received data to the processing engine. The artificial intelligence image processing engine uses deep learning models such as TensorFlow and PyTorch to analyze the image and apply styles.

[0102] As the processing progresses, the server monitors the progress of the artificial intelligence image processing engine and sends progress information to the device in real time at regular intervals. The device displays the received progress information on the UI, allowing the user to understand the current progress. After the processing is complete, the server generates a processing result file and sends a processing completion notification to the device. This completion notification includes a download link for the result file. Based on the received completion notification, the device displays a pop-up or dialog box on the UI and generates a "Download" button. The user can click this download button to save the result file to their local disk.

[0103] For example, suppose a user uploads a monochrome line drawing illustration and chooses to colorize it in a vibrant style. Here is an example prompt:

[0104] Example prompt sentence:

[0105] The user uses the device to click the "Upload Image" button on the UI to upload an unfinished monochrome line drawing file. The user selects the "Vivid" style from the "Style Selection" drop-down menu. The device sends the file and style information to the server. The server processes the file using an artificial intelligence image processing engine and periodically sends progress updates to the device. Once processing is complete, the server sends a processing completion notification and a download link for the resulting file to the device. The device displays a completion notification to the user and provides a "Download" button. The user clicks the download button to receive the completed color illustration file.

[0106] This invention will significantly reduce the time and effort required to create manga, allowing creators to focus on creating their works. It will also enhance creators' creativity by supporting a variety of styles and settings.

[0107] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0108] Step 1:

[0109] The user uses a terminal to access a dedicated user interface. The user clicks the "Upload Image" button and selects and uploads an unfinished image file from the local disk. Based on the input, the UI receives the selected file "ImageFile.png". This file is uploaded and ready for the next step.

[0110] Specific behavior: A user opens a browser, accesses the system's web page, clicks the "Upload Image" button, selects "Image File.png" in the file selection dialog, and uploads it.

[0111] Step 2:

[0112] The user selects the desired style and effect from the "Style Selection" and "Display Settings" options. Based on the input, the UI receives the selected style "Vivid" and display settings. The setting information is ready to be sent to the server.

[0113] What it does: The user opens the "Style Selection" drop-down menu on the UI, selects the "Vivid" style, and then adjusts other effects in the "Display Settings" options.

[0114] Step 3:

[0115] The device sends the image file and setting information selected by the user to the server as an HTTP POST request. Based on the input, the file data "Image file.png" and style information "Vivid" arrive at the server. The server receives this data and temporarily stores it.

[0116] Specific operation: The device generates a POST request and sends it to the server's image processing API endpoint, including the file "image file.png" and the "vivid" style information. The server receives it and saves it in the " / tmp / uploads / " directory.

[0117] Step 4:

[0118] The server launches an artificial intelligence image processing engine and provides the saved image file and configuration information to the processing engine. Based on the input, a deep learning model (e.g., TensorFlow) is used to analyze and style the image. As an output, a processed image file is generated.

[0119] Specific operation: The server runs the Python script "process_image.py" to process "image_file.png" using the TensorFlow model. The process takes a few seconds to a few minutes.

[0120] Step 5:

[0121] The server monitors the processing progress of the AI ​​image processing engine. The progress status is checked at regular intervals and the progress information is sent to the terminal in real time. Based on the input, processing progress information is generated and sent to the terminal.

[0122] Specific operation: The server checks the progress of the processing every 5 seconds and sends a message containing progress information (e.g., processing progress 50%) to the terminal via WebSocket communication.

[0123] Step 6:

[0124] Once the processing is complete, the server generates a processing result file. The server then sends a processing completion notification to the device, which includes a download link for the processing result file. Based on the input, the processing result "processed_image.png" is generated and a download link is created.

[0125] Specific operation: The server generates a file called "processed_image.png" and sends a message to the terminal containing a processing completion notification and a download link (e.g., "http: / / example.com / download / processed_image.png").

[0126] Step 7:

[0127] The device displays a pop-up or dialog box on the UI based on the received processing completion notification. It generates a "Download" button to allow the user to download the result file. Based on the input, the UI displays a download link so that the user can obtain the result file.

[0128] Specific operation: The device receives the completion notification and displays a "Download" button on the UI. The user clicks this button and saves the resulting file "processed_image.png" to the local disk.

[0129] By following these steps, users can obtain a finished image that is automatically processed in a short time.

[0130] (Application example 1)

[0131] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0132] There is a need for a system that can efficiently complete unfinished design files drawn by users and instantly display the completed designs in a virtual store. Conventional methods require specialized skills and a significant amount of time for users to complete the design. In addition, displaying the completed design must be done manually, which is a time-consuming process.

[0133] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0134] In this invention, the server includes means for uploading an unfinished design file drawn through a user interface, means for selecting a design style and display settings as options, means for transmitting the uploaded design file and the selected options to the server, means for the server to process the design file using a machine learning model, means for transmitting a processing progress status from the server to the terminal, means for transmitting a processing completion notice and a processing result file from the server to the terminal, means for the terminal to display a processing completion notice to the user and allow the user to download the result file, and means for automatically uploading and displaying the completed design file to a virtual store, thereby enabling the user to efficiently complete a design and immediately display the design in the virtual store.

[0135] A "user interface" is a software component that provides a screen and input means for a user to access and operate a system.

[0136] A "design file" is a digital file representing an unfinished illustration or design drawn by a user.

[0137] "Options" are additional setting items such as design styles and display settings that can be selected by the user.

[0138] "Server" means a computer system that receives and processes design files and options uploaded by users.

[0139] "Machine Learning Model" means a computational model utilizing machine learning algorithms used to process design files.

[0140] "Processing progress" is information that the server uses to notify the user of the processing status of the design file in real time.

[0141] The "processing completion notification" is notification information that the server uses to notify the user that processing of the design file has been completed.

[0142] "Result File" means the final design file processed by the machine learning model.

[0143] A "virtual store" is an online platform such as a shop or exhibition hall that is virtually set up on the Internet.

[0144] A system for implementing the present invention comprises a user interface, a server, a machine learning model, a terminal, and a virtual store.

[0145] Users upload unfinished design files through their device's browser. The user interface displays a "File Upload" button, which when clicked displays a file selection dialog, allowing users to select and upload a design file from their local disk. In addition, the user interface provides options for "Design Style Selection" and "Display Settings," allowing users to select the style and effect they desire.

[0146] The selected design file and settings information are sent from the device to the server as an HTTP POST request. This request includes the file data and the style and display settings specified by the user. The server receives this data and temporarily stores it.

[0147] The server then launches an environment for the machine learning model (for example, one using TensorFlow or PyTorch) and provides the received data to the model, which uses deep learning techniques to analyze the design file and colorize it in the specified style.

[0148] The server manages the progress of this processing and periodically transmits progress information to the terminal in real time. The terminal displays the received progress information on a user interface, allowing the user to grasp the current progress.

[0149] When the processing is complete, the server generates a processing result file and sends a processing completion notification to the terminal. This completion notification includes a download link for the result file. Based on the received completion notification, the terminal displays a pop-up or dialog box on the user interface and generates a "Download" button. The user can click this to save the completed design file to their local disk.

[0150] In addition, the completed design file is automatically uploaded to the virtual store by the server and instantly displayed online, allowing users to instantly share and exhibit their created designs.

[0151] A concrete example scenario might be a scenario where a user uses smart glasses to take a photo of a sketch character they've drawn in the real world, then uploads it to an app and specifies a "fantasy style" coloring. An example prompt for this would be:

[0152] "Color the line art below in a vibrant, fantasy style."

[0153] Contains the user's line drawing illustration data.

[0154] This system allows users to easily complete designs and instantly display them in a virtual store.

[0155] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0156] Step 1:

[0157] The user opens the user interface via a browser on the device. As input, the user uses the browser on the device. The user clicks the "File Upload" button and selects and uploads an unfinished design file from the local disk in the file selection dialog. As output, the selected design file is obtained.

[0158] Step 2:

[0159] On the user interface, the user selects the desired style from the "Select Design Style" drop-down menu. As input, there is the style information selected by the user. As output, the design file together with the style information is sent to the server.

[0160] Step 3:

[0161] The device sends the uploaded design file and selected style information to the server as an HTTP POST request. The file data and style setting information from the device are used as input. The request arrives at the server as output.

[0162] Step 4:

[0163] The server receives and temporarily stores design files and style information. The input is the file and information sent from the device. The output is the file and information saved in the server's storage.

[0164] Step 5:

[0165] The server launches a machine learning model (e.g., a model using TensorFlow or PyTorch) and provides it with the incoming data. As input, it uses the saved design file and style information. The machine learning model processes the data and colorizes it with the specified style. As output, it generates the processed finished design file.

[0166] Step 6:

[0167] The server manages the processing progress and periodically sends progress information to the device. The input is the processing status of the machine learning model. The output is the progress information sent from the server to the device via data communication.

[0168] Step 7:

[0169] The progress information received by the terminal is displayed on the user interface in real time. As input, there is progress information sent from the server. As output, there is real-time progress displayed on the user interface.

[0170] Step 8:

[0171] When the server completes the process, it generates a result file and sends a notification of completion to the device. The input is the completed design file processed by the machine learning model. The output is a notification of completion and a download link provided by the server to the device.

[0172] Step 9:

[0173] Based on the completion notification received by the terminal, a pop-up or dialog box is displayed on the user interface and a "Download" button is generated. The input is a completion notification from the server. The output is a pop-up or dialog box displayed on the user interface.

[0174] Step 10:

[0175] The user clicks the "Download" button to save the completed design file to their local disk. As input, there is a "Download" button in a popup or dialog box. As output, there is a completed design file saved to their local disk.

[0176] Step 11:

[0177] The server automatically uploads the completed design file to the virtual store and displays it. The input is the processed completed design file. The output is the design file displayed in the virtual store.

[0178] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0179] This invention is a system that reduces the burden on users and improves efficiency in manga production, and in particular, it employs a configuration that combines an AI image processing engine and an emotion engine. Specifically, it is a system that improves the user experience by recognizing the user's emotions and providing dynamic responses accordingly.

[0180] User Interface Parts

[0181] Users operate the system through a dedicated user interface (UI), which can be accessed from a browser or a dedicated application. Users can access the system from a login screen and perform the necessary operations.

[0182] Example: A user opens a browser, accesses the system's URL, enters login information, and logs in.

[0183] Sending part of a user request

[0184] 1. The user uploads a line art file. The UI displays an "Upload Line Art" button, and clicking it displays a file selection dialog. The user selects an illustration file from their local disk and uploads it.

[0185] 2. The user selects the style and display settings. Select the desired option from the "Style Selection" and "Display Settings" options displayed on the UI.

[0186] 3. The device prepares to send the line drawing file and selected options to the server.

[0187] Example: A user uploads an unfinished line art file and selects a colorful style.

[0188] The working parts of the emotion engine

[0189] The emotion engine has the ability to analyze the user's facial expressions and voice. This engine works in conjunction with the user interface to recognize the user's emotions in real time.

[0190] Example: The user allows the camera and microphone, and the emotion engine recognizes joy or displeasure from the user's facial expressions and voice.

[0191] The emotion engine dynamically adjusts the UI based on the emotional data it recognizes. If the user is dissatisfied, the UI will respond by suggesting assistance.

[0192] Server-side processing

[0193] 1. The server receives the request sent from the device and temporarily stores it.

[0194] 2. The server starts the AI ​​image processing engine to process the received data, and selects and operates different AI models depending on the specified style.

[0195] Example: The server selects an AI model that corresponds to a colorful style and processes a line art file.

[0196] 3. The server periodically notifies the terminal of the progress of the processing, and the progress information is displayed to the user in real time.

[0197] Returning and downloading results

[0198] 1. After the processing is complete, the server generates a result file and sends a notification to the device along with a download link for the result file.

[0199] 2. The device notifies the user on the UI based on the received completion notification and generates a download button, which the user can click to download the result file.

[0200] Example: The user sees the completion notification and clicks the download button to receive the completed color illustration file.

[0201] The system's unique feature is its ability to improve the user experience by incorporating an emotion engine that dynamically responds to the user's emotional state. The emotion engine recognizes the user's emotional state and uses that information to change the screen display and operation guides, supporting efficient and smooth work progress. Furthermore, by linking with an AI image processing engine, the entire manga production process can be automated, significantly shortening production time.

[0202] This system reduces the amount of tedious manual work required for users, providing an environment where users can focus more on creative work. It also enables efficient workflow, allowing users to quickly produce high-quality works.

[0203] The processing flow will be explained below.

[0204] Step 1:

[0205] The user opens the user interface (UI) of the device, launches a browser or a dedicated application, and accesses the system login screen. The user enters the username and password to log in.

[0206] Step 2:

[0207] The user uploads an unfinished illustration file by clicking the "Upload Line Art" button on the UI, selecting a line art file saved on the local disk from the file selection dialog, and uploading it.

[0208] Step 3:

[0209] The user selects the style and display settings. Select the desired style or effect from the "Style Selection" drop-down menu or other options in the UI. The device records the selection.

[0210] Step 4:

[0211] The device sends the uploaded illustration file and the selected style settings to the server as an HTTP POST request, which includes the file data and style information.

[0212] Step 5:

[0213] The server receives the request and temporarily saves the file data. The server saves the received illustration file in a specified folder and records the style setting information.

[0214] Step 6:

[0215] The server activates the emotion engine and analyzes the user's facial expressions and voice data sent from the user interface. The server recognizes the user's emotional state in real time through the emotion engine.

[0216] Step 7:

[0217] The server dynamically adjusts the UI display based on the emotional data it recognizes. For example, if the user is frustrated, a support message or assistance options will be displayed.

[0218] Step 8:

[0219] The server processes the illustration file using an AI image processing engine, selects the corresponding AI model based on the specified style information, and applies coloring and effects to the line art file.

[0220] Step 9:

[0221] The server manages the progress of the process in real time and notifies the device. Each time the progress of the process is updated, the progress is sent to the device.

[0222] Step 10:

[0223] The terminal displays the received progress information on the user interface, allowing the user to check the progress of the processing in real time.

[0224] Step 11:

[0225] When the server completes the process, it generates a result file and sends a notification to the device, which includes a download link for the result file.

[0226] Step 12:

[0227] The terminal displays a completion notification to the user and provides a download button, after which the user can confirm the completion notification and click the download button to download the resulting file.

[0228] Step 13:

[0229] The user clicks the download button to save the completed color illustration file to their local disk. The user can then check the completed work and further process or use it as needed.

[0230] This system allows users to quickly and efficiently automate parts of manga production, significantly reducing production time and effort. Furthermore, by combining it with an emotion engine, it provides flexible support according to the user's state, improving the user experience.

[0231] Example 2

[0232] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0233] The manga production process is extremely time-consuming and labor-intensive, requiring a great deal of manual work. As a result, the burden on the workers increases, leading to problems with reduced production efficiency. Another issue is that the system cannot adequately respond to changes in the user's emotions that occur during the process, resulting in a poor user experience. Furthermore, efficient progress management is required to quickly produce high-quality works.

[0234] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0235] In this invention, the server includes means for uploading unfinished image files to be drawn through a user interface, means for selecting appearances and settings as options, means for transmitting the uploaded image files and the selected options to the server, means for processing the image files using an artificial intelligence image processing engine by the server, means for recognizing emotions from the user's facial expressions and voice using an emotion engine, means for dynamically adjusting the user interface based on the user's emotional state, means for transmitting a processing progress status from the server to the terminal, means for transmitting a processing completion notice and a processing result file from the server to the terminal, and means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file. This allows the system to dynamically respond to the user's emotional state, enabling efficient progress management and rapid high-quality manga production.

[0236] A "user interface" is a screen or application that allows a user to interact with a system and perform various operations.

[0237] An "unfinished image file" is an image that is being uploaded by a user and is in a state before processing or editing.

[0238] "Options" are appearance and setting choices that the user can specify, including image processing styles, display settings, and the like.

[0239] A "server" is a computer system that receives requests from users and processes images and stores and manages data.

[0240] An "artificial intelligence image processing engine" is software equipped with AI technology used to analyze and process image data.

[0241] An "emotion engine" is a technology that recognizes emotions from a user's facial expressions and voice, and adjusts the system's behavior based on that emotional state.

[0242] "Progress" is information indicating the progress of image processing, and displays in real time how far the processing has progressed.

[0243] The "processing completion notification" is a message for notifying the user that image processing has been completed.

[0244] "Processing result file" refers to the final image file that has been processed and edited by the AI ​​image processing engine.

[0245] A "terminal" is a device that a user uses to access the system and perform various operations and check the processing results.

[0246] "Downloading" refers to the act of saving a processing result file sent from a server to a terminal.

[0247] "Dynamic adjustment" means changing the system's display and operation methods in real time according to the user's emotions, progress, etc.

[0248] This system reduces the burden on users and improves efficiency in manga production, and in particular employs a configuration that combines an image processing engine and an emotion engine that utilize artificial intelligence. The entire system operates through a combination of a user interface, terminals, and a server.

[0249] First, the user operates the system through a dedicated user interface (UI). This UI can be accessed from a standard web browser or a dedicated application. The user enters their authentication information on the login screen and logs into the system. After logging in, the user is redirected to the dashboard, where they can click the "Upload Line Art" button to select and upload an unfinished image file from their local disk.

[0250] Next, the user selects style and display settings as options on the UI. Specifically, for example, they select "Watercolor" from the "Style Selection" menu, and then adjust "Brightness" and "Contrast" in the display settings options. This selected data is then prepared by the device to be sent to the server. For transmission, the HTTP protocol or other communication technology is used.

[0251] After receiving the request from the device, the server temporarily stores the data in storage. It then launches an artificial intelligence image processing engine using an AI framework such as TensorFlow or PyTorch to process the received data. The appropriate AI model is selected based on the specified style, and processing is performed. For example, if the user selects a "watercolor" style, the AI ​​model corresponding to the style is applied, converting the line art file into a beautiful watercolor-style color illustration.

[0252] The emotion engine analyzes the user's facial expressions and voice while they are operating the device to recognize their current emotions. This function collects data in real time through the camera and microphone and uses machine learning algorithms to determine their emotional state. For example, if the user allows the camera and microphone to be used, the emotion engine recognizes emotions such as "happiness" or "dissatisfaction" from the user's facial expressions. Based on the recognized emotional state, the system dynamically adjusts the UI. Specifically, if the user is dissatisfied, the UI will automatically suggest assistance.

[0253] The server periodically notifies the device of the progress of image processing, which is displayed in real time on the UI. This allows the user to check the progress of the processing. The progress information is displayed as a message such as "50% complete." Finally, after processing is complete, the server generates a result file and sends a processing completion notification to the device. This notification also includes a download link for the result file. The user can receive the notification and download the result file by clicking the "Download" button on the UI.

[0254] In this way, this system significantly reduces the user's manual work and supports high-quality manga production by utilizing generative AI models. Furthermore, the introduction of an emotion engine enables dynamic responses according to the user's emotional state, providing a better user experience.

[0255] Examples and prompts:

[0256] As a concrete example, we will describe how a user accesses the system, uploads a line drawing file, selects "watercolor" as the style, and authorizes the camera and microphone. We will also explain the process in which the emotion engine recognizes the user's frustration and the UI suggests assistance. Finally, we will explain the entire process in which the user downloads the watercolor color illustration file generated by the server after processing is complete.

[0257] Example prompt sentence:

[0258] "Please explain in detail the process by which a user opens a browser, logs into the system, uploads a line drawing file, and selects style and display settings. Also, please explain in detail the entire process by which the emotion engine recognizes the user's frustration, the server processes the image using an AI image processing engine, and finally the user downloads the resulting file."

[0259] This system reduces the amount of tedious manual work required for users, providing an environment where users can focus more on creative work. It also enables efficient workflow, allowing users to quickly produce high-quality works.

[0260] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0261] Step 1:

[0262] A user accesses the system from a browser or a dedicated application and enters their username and password on the login screen. The entered authentication information is sent to the server, and if authentication is successful, the dashboard screen is displayed. When login is successful, the output is the dashboard screen.

[0263] Specific operation: The user opens a browser, accesses the system's URL, enters login information, and clicks the "Login" button.

[0264] ---

[0265] Step 2:

[0266] The user clicks the "Upload Line Art" button on the dashboard, selects an unfinished image file from the local disk, and uploads it. At this time, the file path and file name are also sent to the server, which receives and temporarily saves the file. The input is the unfinished image file selected by the user, and the output is the unfinished image file saved on the server.

[0267] Specific operation: The user clicks the "Upload line art" button on the dashboard, selects the appropriate line art file from the file selection dialog, and uploads it.

[0268] ---

[0269] Step 3:

[0270] Select the style and display settings for the image file uploaded by the user. Select the desired style from the "Style Selection" menu and make detailed settings with the "Display Settings" option. The input is the style and display settings selected by the user, and the output is the style and display setting information.

[0271] Specific behavior: The user selects "Watercolor" from the "Style Selection" menu and adjusts "Brightness" and "Contrast" in the display settings options.

[0272] ---

[0273] Step 4:

[0274] The device sends the uploaded image file and the style and display settings selected by the user to the server. This sending process uses HTTP requests and API calls to securely transfer data. The input is the image file and style settings, and the output is the data sent to the server.

[0275] Specific operation: The device sends the line drawing file and the "watercolor" style setting to the server via an HTTP request.

[0276] ---

[0277] Step 5:

[0278] The emotion engine recognizes the user's facial expressions and voice and analyzes emotions in real time. It uses machine learning algorithms to determine the user's emotional state based on data collected from the camera and microphone. The input is data from the camera and microphone, and the output is the recognized emotional state.

[0279] Specific operation: The user allows the camera and microphone, and the emotion engine analyzes emotions such as "happiness" or "satisfaction" from the user's facial expression.

[0280] ---

[0281] Step 6:

[0282] The server launches an artificial intelligence image processing engine and processes the received image file in the specified style. It uses TensorFlow and PyTorch to analyze and process the image data. The input is the image file and style settings uploaded by the user, and the output is the processed image file.

[0283] Specific operation: The server applies the "watercolor style" model to process the unfinished image file into a color illustration.

[0284] ---

[0285] Step 7:

[0286] The server notifies the terminal of the progress of the processing. The terminal reflects the received progress status in the user interface in real time and displays it to the user. The input is the progress information calculated by the server, and the output is the progress status displayed on the terminal.

[0287] Specific operation: When the server has completed 50% of the processing, it notifies the terminal of the progress and displays the message "50% complete" on the UI.

[0288] ---

[0289] Step 8:

[0290] After the processing is complete, the server generates a result file and sends a processing completion notification to the device. This notification includes a download link for the result file. The input is the processed image file, and the output is the completion notification and the download link.

[0291] What it does: The server generates the final watercolor-style color illustration file and sends a notification saying "Processing complete. You can download it here" along with a link.

[0292] ---

[0293] Step 9:

[0294] The user receives a notification of the completion of the process and clicks the "Download" button on the UI to download the generated result file. The input is the completion notification from the server, and the output is the result file saved locally by the user.

[0295] Specific operation: The user confirms the notification and clicks the "Download" button on the UI to save the generated watercolor-style color illustration file.

[0296] (Application example 2)

[0297] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0298] In manga production, there is a need to reduce the burden on users and improve efficiency. In particular, there is a lack of systems that provide dynamic support according to the user's emotional state and enable smooth progress in the work, making it difficult to provide an environment where users can concentrate on creative work. Another issue is that existing systems only provide simple functions without taking the user's emotional state into consideration.

[0299] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for uploading an unfinished illustration file to be drawn through a user interface; means for selecting an optional style and display settings; means for analyzing the user's emotional state using an emotion recognition engine; means for transmitting the uploaded illustration file, the selected options, and the analyzed emotional state to the server; means for processing the illustration file using an AI image processing engine by the server; means for transmitting a processing progress status from the server to the terminal; means for transmitting a processing completion notice and a processing result file from the server to the terminal; means for dynamically adjusting the user interface according to the emotional state; and means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file. This provides interactive support according to the user's emotional state, improves the efficiency of creative work, and enables the user to engage in creative activities with a sense of satisfaction.

[0300] (definition statement)

[0301] A "user interface" is an interface that provides a screen and operating means for a user to access and operate a system.

[0302] An "illustration file" refers to image data that a user is currently creating or has completed, and is specifically a file that contains manga line drawings and sketches.

[0303] "Options" are choices that a user can set within the system, including style and display settings.

[0304] An "emotion recognition engine" is an engine that analyzes a user's facial expressions and voice to recognize their emotional state in real time.

[0305] The "AI image processing engine" is an engine that uses artificial intelligence to process uploaded illustration files based on a specific style.

[0306] "Progress" is information that indicates the progress of work while the system is processing it.

[0307] The "processing completion notification" is a notification that notifies the user that the server has completed processing of the illustration file.

[0308] "Result file" refers to the illustration file after it has been processed by the AI ​​image processing engine.

[0309] "Dynamic adjustment" means that the display content and operation guide of the user interface are changed according to the emotional state of the user.

[0310] A "terminal" is a device used by a user to access the system, including a smartphone, a personal computer, etc.

[0311] This invention is a system for reducing the user's burden and improving efficiency in manga production. In particular, it has a function to provide dynamic support according to the user's emotional state. This system consists of a user interface (UI), an emotion recognition engine, an AI image processing engine, a server, and a terminal.

[0312] Hardware and software used

[0313] Hardware: Smartphone, PC (with camera and microphone)

[0314] software:

[0315] AI image processing engine: TensorFlow

[0316] Emotion recognition engine: OpenCV (facial expression recognition), Google Cloud Speech-to-Text API (voice recognition)

[0317] Web server: Node.js

[0318] System configuration

[0319] 1. User Interface (UI):

[0320] Users can upload unfinished illustration files and select style and display settings through a dedicated UI, which can be accessed from a browser or a dedicated application.

[0321] 2. Emotion Recognition Engine:

[0322] The emotion recognition engine analyzes the user's facial expressions and voice in real time to recognize their emotional state. For example, when a user accesses the emotion recognition engine using a camera and microphone, the system can detect emotions such as joy or displeasure.

[0323] 3. AI image processing engine:

[0324] The AI ​​image processing engine running on the server processes the uploaded illustration file according to the selected style, allowing the illustration to be automatically colored in the style specified by the user.

[0325] 4. Server:

[0326] The server temporarily stores the illustration file and settings information received from the user, then starts the AI ​​image processing engine to process it. It also has a function to send progress and processing completion notifications to the terminal.

[0327] 5. Terminal:

[0328] The terminal is the device that the user uses to access the system, such as a smartphone or PC. The terminal displays the progress and processing completion notifications sent from the server to the user in real time, and allows the user to download the result file.

[0329] Specific examples

[0330] Consider a scenario where a user uploads a line art file and selects the "Realistic Style." The user enters the following prompt:

[0331] Example prompt sentence:

[0332] "User ID: example_user_id, Request type: Image processing, Style: Realistic, Image file path: / path / to / uploaded / line_art.png"

[0333] This prompt is an instruction for the system's AI image processing engine to select an appropriate model and color the illustration. Additionally, if the user allows the camera and microphone, the emotion recognition engine will recognize joy or displeasure from facial expressions and voice in real time and dynamically adjust the UI, significantly improving user efficiency.

[0334] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0335] (Processing steps of patent embodiment)

[0336] Step 1:

[0337] Users access the system through a dedicated user interface (UI) and upload unfinished illustration files.

[0338] Input: Unfinished illustration file

[0339] Operation: Click the "Upload Line Art" button on the user interface and select an illustration file from your local disk from the file selection dialog.

[0340] Output: Temporarily save uploaded illustration files

[0341] Step 2:

[0342] The user selects style and display settings from options in the UI.

[0343] Input: Style selection, display settings

[0344] Action: Select the desired option from the "Style Selection" or "Display Settings" drop-down menu.

[0345] Output: Information about selected styles and display settings

[0346] Step 3:

[0347] An emotion recognition engine analyzes the user's emotional state.

[0348] Input: User's facial and voice data

[0349] How it works: It uses a camera and microphone to collect and analyze the user's facial expressions and voice in real time.

[0350] Output: Analyzed emotional state data

[0351] Step 4:

[0352] The terminal prepares to send the uploaded illustration file, the selected options, and the analyzed emotional state to the server.

[0353] Input: illustration file, style settings, emotional state

[0354] Operation: This data is compiled into a prompt statement format and prepared to be sent to the server.

[0355] Output: Data in prompt format

[0356] Step 5:

[0357] The server temporarily stores the received illustration file, options, and emotional state data, and then activates an AI image processing engine to process the illustration file.

[0358] Input: Data in prompt format (illustration file, style settings, emotional state)

[0359] How it works: Analyzes the received data, selects the AI ​​image processing engine that corresponds to the specified style, and colors the line drawing.

[0360] Output: Colored illustration file

[0361] Step 6:

[0362] The server sends the progress of the illustration processing to the terminal.

[0363] Input: Processing progress information

[0364] How it works: The AI ​​image processing engine reports progress in real time and sends that information to your device.

[0365] Output: Progress notification

[0366] Step 7:

[0367] The server sends a processing completion notice and a result file to the terminal.

[0368] Input: Processing completion information, result file

[0369] Operation: Once processing is complete, a notification of completion will be sent to the device along with a download link for the generated color illustration file.

[0370] Output: Processing completion notification and download link

[0371] Step 8:

[0372] The terminal displays a notification to the user that the process is complete and allows the result file to be downloaded.

[0373] Input: Processing completion notification, download link

[0374] Behavior: Displays a notification in the UI and generates a "Download" button that the user clicks to download the resulting file.

[0375] Output: Downloaded color illustration file

[0376] In this way, specific actions are performed at each step, allowing users to create manga efficiently. Interactive support is also provided according to the user's emotional state, creating a more creative work environment.

[0377] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0378] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0379] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0380] [Second embodiment]

[0381] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0382] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0383] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0384] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0385] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0386] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0387] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0388] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0389] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0390] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0391] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0392] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0393] This invention provides a means to efficiently automate part of the manga production process through a system that operates via a server and terminals. Specifically, it is a system that automatically processes unfinished illustration files drawn by users using an AI image processing engine to complete them in a short amount of time.

[0394] User Interface Parts

[0395] Users operate the system through a dedicated user interface (UI). Users access the system from their device's browser and upload unfinished illustration files. The UI displays an "Upload Line Art" button, and clicking this displays a file selection dialog. Here, users can select an illustration file from their local disk and upload it.

[0396] Additionally, the UI provides "Style Selection" and "Display Settings" options, allowing users to select the style and effect they desire.

[0397] Sending part of the request

[0398] The selected illustration file and settings are sent from the device to the server as an HTTP POST request, which includes the file data and the user-specified style and display settings.

[0399] Server-side processing

[0400] The server receives the illustration file and setting information sent from the device and temporarily stores them. The server then starts the AI ​​image processing engine and provides the received data to the processing engine. The AI ​​image processing engine uses a deep learning model to analyze the line drawing of the illustration and color it in the specified style.

[0401] Progress management and notifications

[0402] The server manages the processing progress of the AI ​​image processing engine and periodically checks its progress. This progress information is sent from the server to the device in real time. The device displays the received progress information on the UI, allowing the user to understand the current progress.

[0403] Returning and downloading results

[0404] When the processing is complete, the server generates a processing result file and sends a processing completion notification to the device. This completion notification includes a download link for the result file. Based on the received completion notification, the device displays a pop-up or dialog box on the UI and generates a "Download" button.

[0405] The user confirms the completion notification and clicks the download button to save the resulting file to their local disk, allowing them to obtain the colored finished illustration in a short time.

[0406] Specific examples

[0407] For example, consider the case where a user uploads a monochrome line drawing illustration and chooses to colorize it in a vibrant style.

[0408] 1. The user uses the device to click the "Upload Line Art" button on the UI to upload an unfinished monochrome line art file.

[0409] 2. The user selects the "Colorful" style from the "Style Selection" drop-down menu.

[0410] 3. The device sends the file and style information to the server.

[0411] 4. The server processes the file using an AI image processing engine and periodically sends progress updates to the device.

[0412] 5. When the progress reaches 100%, the server will send a notification of completion and a download link for the result file to the device.

[0413] 6. The device displays a completion notice to the user and provides a "Download" button.

[0414] 7. The user clicks the download button and receives the completed color illustration file.

[0415] This invention significantly reduces the time and effort required to create manga, providing an environment in which creators can concentrate on creating their work. It also enhances creators' creativity by supporting a variety of styles and settings.

[0416] The processing flow will be explained below.

[0417] Step 1:

[0418] The user accesses the terminal's user interface. The user interface can be accessed from a browser or a dedicated application, and a login screen is displayed. The user enters login information and logs into the system.

[0419] Step 2:

[0420] The user uploads a line drawing file. The user clicks the "Upload Line Drawing" button on the UI and selects a line drawing file from the local disk in the file selection dialog that appears. The device prepares to send the selected file to the server.

[0421] Step 3:

[0422] The user selects a style or display setting. The UI displays a drop-down menu of style choices and other options. The user selects the style or display setting they want.

[0423] Step 4:

[0424] The device sends the uploaded line art file and selected options to the server as an HTTP POST request, which includes the file data and style information.

[0425] Step 5:

[0426] The server receives the request, temporarily stores the received data, and saves the received line drawing file in an appropriate folder, while also recording the style information specified by the user.

[0427] Step 6:

[0428] The server starts the AI ​​image processing engine and provides the received data to the processing engine. The server selects an AI model corresponding to the specified style and processes the line drawing file.

[0429] Step 7:

[0430] The server manages the progress of the processing and periodically notifies the device. As the AI ​​image processing progresses, the progress status is reported to the server, which then periodically sends it to the device.

[0431] Step 8:

[0432] The terminal displays the received progress information on the UI so that the user can check the current processing status in real time.

[0433] Step 9:

[0434] After the process is complete, the server generates a result file and sends a notification of the completion of the process to the device, along with a download link for the result file.

[0435] Step 10:

[0436] The device displays a pop-up or dialog box on the UI based on the received completion notification, and generates a "Download" button to allow the user to download the result file.

[0437] Step 11:

[0438] The user downloads the resulting file. After confirming the completion notification, they click the download button and save it to their local disk. The completed color illustration file is provided.

[0439] This series of processes allows users to quickly and efficiently automate parts of manga production, significantly reducing production time and effort.

[0440] Example 1

[0441] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0442] Traditional manga production has the problem of requiring a great deal of time and effort to color images and reflect styles. Manual coloring, in particular, is a factor that reduces productivity, and many creators are seeking more efficient methods. Furthermore, there is often a lack of smooth communication methods for checking progress and obtaining results. To solve these issues, a system is needed that automates image processing, allowing users to check progress in real time and obtain results quickly.

[0443] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0444] In this invention, the server includes a means for uploading an unfinished image file to be drawn through a user interface, a means for selecting an optional style and display setting, a means for transmitting the uploaded image file and the selected option to the server, a means for processing the image file using an artificial intelligence image processing engine by the server, a means for transmitting a processing progress report from the server to the terminal, a means for transmitting a processing completion notice and a processing result file from the server to the terminal, and a means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file, thereby enabling the user to quickly automatically colorize and style images, check the progress in real time, and reliably obtain the results.

[0445] A "user interface" is a dedicated interface for users to perform operations, and includes, for example, buttons and menus displayed on the screen.

[0446] "Image files" refers to digital files such as line drawings and illustrations, and includes formats such as JPEG, PNG, and BMP.

[0447] "Style" refers to the design and effect settings applied in image processing, and includes different methods of expression such as "vivid colors" and "monochrome."

[0448] "Display settings" refers to various settings related to the display of images and user interfaces, including, for example, brightness and contrast adjustments.

[0449] A "server" refers to a computer system that accepts requests from client terminals via a network and processes and stores data.

[0450] "Artificial intelligence image processing engine" refers to software that analyzes and processes images using deep learning models and other artificial intelligence technologies.

[0451] "Processing progress" refers to information indicating the progress of the processing currently being performed by the image processing engine.

[0452] A "processing completion notification" is a message from the system notifying you that image processing is complete, and typically includes a download link.

[0453] The "download link" indicates the URL for the user to obtain the result file.

[0454] "Terminal" refers to an electronic device such as a computer or smartphone that is operated by a user.

[0455] The present invention is a system that automatically processes unfinished image files drawn by users using an artificial intelligence image processing engine to complete them in a short time. The system is operated through a user interface and operates via a server and terminals.

[0456] First, the user operates the system through a dedicated user interface (UI). The user accesses the system from the device's browser and uploads an unfinished image file. The UI displays an "Upload Image" button, and clicking this displays a file selection dialog. The user selects an image file from the local disk and uploads it. Furthermore, the UI provides options for "Style Selection" and "Display Settings," allowing the user to select the desired style and effect.

[0457] The image file and settings information selected by the user are sent from the device to the server as an HTTP POST request. This request includes the file data and the style and display settings specified by the user. The server receives the image file and settings information sent from the device and temporarily stores them in a temporary directory on the server. The server then starts an artificial intelligence image processing engine and provides the received data to the processing engine. The artificial intelligence image processing engine uses deep learning models such as TensorFlow and PyTorch to analyze the image and apply styles.

[0458] As the processing progresses, the server monitors the progress of the artificial intelligence image processing engine and sends progress information to the device in real time at regular intervals. The device displays the received progress information on the UI, allowing the user to understand the current progress. After the processing is complete, the server generates a processing result file and sends a processing completion notification to the device. This completion notification includes a download link for the result file. Based on the received completion notification, the device displays a pop-up or dialog box on the UI and generates a "Download" button. The user can click this download button to save the result file to their local disk.

[0459] For example, suppose a user uploads a monochrome line drawing illustration and chooses to colorize it in a vibrant style. Here is an example prompt:

[0460] Example prompt sentence:

[0461] The user uses the device to click the "Upload Image" button on the UI to upload an unfinished monochrome line drawing file. The user selects the "Vivid" style from the "Style Selection" drop-down menu. The device sends the file and style information to the server. The server processes the file using an artificial intelligence image processing engine and periodically sends progress updates to the device. Once processing is complete, the server sends a processing completion notification and a download link for the resulting file to the device. The device displays a completion notification to the user and provides a "Download" button. The user clicks the download button to receive the completed color illustration file.

[0462] This invention will significantly reduce the time and effort required to create manga, allowing creators to focus on creating their works. It will also enhance creators' creativity by supporting a variety of styles and settings.

[0463] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0464] Step 1:

[0465] The user uses a terminal to access a dedicated user interface. The user clicks the "Upload Image" button and selects and uploads an unfinished image file from the local disk. Based on the input, the UI receives the selected file "ImageFile.png". This file is uploaded and ready for the next step.

[0466] Specific behavior: A user opens a browser, accesses the system's web page, clicks the "Upload Image" button, selects "Image File.png" in the file selection dialog, and uploads it.

[0467] Step 2:

[0468] The user selects the desired style and effect from the "Style Selection" and "Display Settings" options. Based on the input, the UI receives the selected style "Vivid" and display settings. The setting information is ready to be sent to the server.

[0469] What it does: The user opens the "Style Selection" drop-down menu on the UI, selects the "Vivid" style, and then adjusts other effects in the "Display Settings" options.

[0470] Step 3:

[0471] The device sends the image file and setting information selected by the user to the server as an HTTP POST request. Based on the input, the file data "Image file.png" and style information "Vivid" arrive at the server. The server receives this data and temporarily stores it.

[0472] Specific operation: The device generates a POST request and sends it to the server's image processing API endpoint, including the file "image file.png" and the "vivid" style information. The server receives it and saves it in the " / tmp / uploads / " directory.

[0473] Step 4:

[0474] The server launches an artificial intelligence image processing engine and provides the saved image file and configuration information to the processing engine. Based on the input, a deep learning model (e.g., TensorFlow) is used to analyze and style the image. As an output, a processed image file is generated.

[0475] Specific operation: The server runs the Python script "process_image.py" to process "image_file.png" using the TensorFlow model. The process takes a few seconds to a few minutes.

[0476] Step 5:

[0477] The server monitors the processing progress of the AI ​​image processing engine. The progress status is checked at regular intervals and the progress information is sent to the terminal in real time. Based on the input, processing progress information is generated and sent to the terminal.

[0478] Specific operation: The server checks the progress of the processing every 5 seconds and sends a message containing progress information (e.g., processing progress 50%) to the terminal via WebSocket communication.

[0479] Step 6:

[0480] Once the processing is complete, the server generates a processing result file. The server then sends a processing completion notification to the device, which includes a download link for the processing result file. Based on the input, the processing result "processed_image.png" is generated and a download link is created.

[0481] Specific operation: The server generates a file called "processed_image.png" and sends a message to the terminal containing a processing completion notification and a download link (e.g., "http: / / example.com / download / processed_image.png").

[0482] Step 7:

[0483] The device displays a pop-up or dialog box on the UI based on the received processing completion notification. It generates a "Download" button to allow the user to download the result file. Based on the input, the UI displays a download link so that the user can obtain the result file.

[0484] Specific operation: The device receives the completion notification and displays a "Download" button on the UI. The user clicks this button and saves the resulting file "processed_image.png" to the local disk.

[0485] By following these steps, users can obtain a finished image that is automatically processed in a short time.

[0486] (Application example 1)

[0487] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0488] There is a need for a system that can efficiently complete unfinished design files drawn by users and instantly display the completed designs in a virtual store. Conventional methods require specialized skills and a significant amount of time for users to complete the design. In addition, displaying the completed design must be done manually, which is a time-consuming process.

[0489] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0490] In this invention, the server includes means for uploading an unfinished design file drawn through a user interface, means for selecting a design style and display settings as options, means for transmitting the uploaded design file and the selected options to the server, means for the server to process the design file using a machine learning model, means for transmitting a processing progress status from the server to the terminal, means for transmitting a processing completion notice and a processing result file from the server to the terminal, means for the terminal to display a processing completion notice to the user and allow the user to download the result file, and means for automatically uploading and displaying the completed design file to a virtual store, thereby enabling the user to efficiently complete a design and immediately display the design in the virtual store.

[0491] A "user interface" is a software component that provides a screen and input means for a user to access and operate a system.

[0492] A "design file" is a digital file representing an unfinished illustration or design drawn by a user.

[0493] "Options" are additional setting items such as design styles and display settings that can be selected by the user.

[0494] "Server" means a computer system that receives and processes design files and options uploaded by users.

[0495] "Machine Learning Model" means a computational model utilizing machine learning algorithms used to process design files.

[0496] "Processing progress" is information that the server uses to notify the user of the processing status of the design file in real time.

[0497] The "processing completion notification" is notification information that the server uses to notify the user that processing of the design file has been completed.

[0498] "Result File" means the final design file processed by the machine learning model.

[0499] A "virtual store" is an online platform such as a shop or exhibition hall that is virtually set up on the Internet.

[0500] A system for implementing the present invention comprises a user interface, a server, a machine learning model, a terminal, and a virtual store.

[0501] Users upload unfinished design files through their device's browser. The user interface displays a "File Upload" button, which when clicked displays a file selection dialog, allowing users to select and upload a design file from their local disk. In addition, the user interface provides options for "Design Style Selection" and "Display Settings," allowing users to select the style and effect they desire.

[0502] The selected design file and settings information are sent from the device to the server as an HTTP POST request. This request includes the file data and the style and display settings specified by the user. The server receives this data and temporarily stores it.

[0503] The server then launches an environment for the machine learning model (for example, one using TensorFlow or PyTorch) and provides the received data to the model, which uses deep learning techniques to analyze the design file and colorize it in the specified style.

[0504] The server manages the progress of this processing and periodically transmits progress information to the terminal in real time. The terminal displays the received progress information on a user interface, allowing the user to grasp the current progress.

[0505] When the processing is complete, the server generates a processing result file and sends a processing completion notification to the terminal. This completion notification includes a download link for the result file. Based on the received completion notification, the terminal displays a pop-up or dialog box on the user interface and generates a "Download" button. The user can click this to save the completed design file to their local disk.

[0506] In addition, the completed design file is automatically uploaded to the virtual store by the server and instantly displayed online, allowing users to instantly share and exhibit their created designs.

[0507] A concrete example scenario might be a scenario where a user uses smart glasses to take a photo of a sketch character they've drawn in the real world, then uploads it to an app and specifies a "fantasy style" coloring. An example prompt for this would be:

[0508] "Color the line art below in a vibrant, fantasy style."

[0509] Contains the user's line drawing illustration data.

[0510] This system allows users to easily complete designs and instantly display them in a virtual store.

[0511] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0512] Step 1:

[0513] The user opens the user interface via a browser on the device. As input, the user uses the browser on the device. The user clicks the "File Upload" button and selects and uploads an unfinished design file from the local disk in the file selection dialog. As output, the selected design file is obtained.

[0514] Step 2:

[0515] On the user interface, the user selects the desired style from the "Select Design Style" drop-down menu. As input, there is the style information selected by the user. As output, the design file together with the style information is sent to the server.

[0516] Step 3:

[0517] The device sends the uploaded design file and selected style information to the server as an HTTP POST request. The file data and style setting information from the device are used as input. The request arrives at the server as output.

[0518] Step 4:

[0519] The server receives and temporarily stores design files and style information. The input is the file and information sent from the device. The output is the file and information saved in the server's storage.

[0520] Step 5:

[0521] The server launches a machine learning model (e.g., a model using TensorFlow or PyTorch) and provides it with the incoming data. As input, it uses the saved design file and style information. The machine learning model processes the data and colorizes it with the specified style. As output, it generates the processed finished design file.

[0522] Step 6:

[0523] The server manages the processing progress and periodically sends progress information to the device. The input is the processing status of the machine learning model. The output is the progress information sent from the server to the device via data communication.

[0524] Step 7:

[0525] The progress information received by the terminal is displayed on the user interface in real time. As input, there is progress information sent from the server. As output, there is real-time progress displayed on the user interface.

[0526] Step 8:

[0527] When the server completes the process, it generates a result file and sends a notification of completion to the device. The input is the completed design file processed by the machine learning model. The output is a notification of completion and a download link provided by the server to the device.

[0528] Step 9:

[0529] Based on the completion notification received by the terminal, a pop-up or dialog box is displayed on the user interface and a "Download" button is generated. The input is a completion notification from the server. The output is a pop-up or dialog box displayed on the user interface.

[0530] Step 10:

[0531] The user clicks the "Download" button to save the completed design file to their local disk. As input, there is a "Download" button in a popup or dialog box. As output, there is a completed design file saved to their local disk.

[0532] Step 11:

[0533] The server automatically uploads the completed design file to the virtual store and displays it. The input is the processed completed design file. The output is the design file displayed in the virtual store.

[0534] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0535] This invention is a system that reduces the burden on users and improves efficiency in manga production, and in particular, it employs a configuration that combines an AI image processing engine and an emotion engine. Specifically, it is a system that improves the user experience by recognizing the user's emotions and providing dynamic responses accordingly.

[0536] User Interface Parts

[0537] Users operate the system through a dedicated user interface (UI), which can be accessed from a browser or a dedicated application. Users can access the system from a login screen and perform the necessary operations.

[0538] Example: A user opens a browser, accesses the system's URL, enters login information, and logs in.

[0539] Sending part of a user request

[0540] 1. The user uploads a line art file. The UI displays an "Upload Line Art" button, and clicking it displays a file selection dialog. The user selects an illustration file from their local disk and uploads it.

[0541] 2. The user selects the style and display settings. Select the desired option from the "Style Selection" and "Display Settings" options displayed on the UI.

[0542] 3. The device prepares to send the line drawing file and selected options to the server.

[0543] Example: A user uploads an unfinished line art file and selects a colorful style.

[0544] The working parts of the emotion engine

[0545] The emotion engine has the ability to analyze the user's facial expressions and voice. This engine works in conjunction with the user interface to recognize the user's emotions in real time.

[0546] Example: The user allows the camera and microphone, and the emotion engine recognizes joy or displeasure from the user's facial expressions and voice.

[0547] The emotion engine dynamically adjusts the UI based on the emotional data it recognizes. If the user is dissatisfied, the UI will respond by suggesting assistance.

[0548] Server-side processing

[0549] 1. The server receives the request sent from the device and temporarily stores it.

[0550] 2. The server starts the AI ​​image processing engine to process the received data, and selects and operates different AI models depending on the specified style.

[0551] Example: The server selects an AI model that corresponds to a colorful style and processes a line art file.

[0552] 3. The server periodically notifies the terminal of the progress of the processing, and the progress information is displayed to the user in real time.

[0553] Returning and downloading results

[0554] 1. After the processing is complete, the server generates a result file and sends a notification to the device along with a download link for the result file.

[0555] 2. The device notifies the user on the UI based on the received completion notification and generates a download button, which the user can click to download the result file.

[0556] Example: The user sees the completion notification and clicks the download button to receive the completed color illustration file.

[0557] The system's unique feature is its ability to improve the user experience by incorporating an emotion engine that dynamically responds to the user's emotional state. The emotion engine recognizes the user's emotional state and uses that information to change the screen display and operation guides, supporting efficient and smooth work progress. Furthermore, by linking with an AI image processing engine, the entire manga production process can be automated, significantly shortening production time.

[0558] This system reduces the amount of tedious manual work required for users, providing an environment where users can focus more on creative work. It also enables efficient workflow, allowing users to quickly produce high-quality works.

[0559] The processing flow will be explained below.

[0560] Step 1:

[0561] The user opens the user interface (UI) of the device, launches a browser or a dedicated application, and accesses the system login screen. The user enters the username and password to log in.

[0562] Step 2:

[0563] The user uploads an unfinished illustration file by clicking the "Upload Line Art" button on the UI, selecting a line art file saved on the local disk from the file selection dialog, and uploading it.

[0564] Step 3:

[0565] The user selects the style and display settings. Select the desired style or effect from the "Style Selection" drop-down menu or other options in the UI. The device records the selection.

[0566] Step 4:

[0567] The device sends the uploaded illustration file and the selected style settings to the server as an HTTP POST request, which includes the file data and style information.

[0568] Step 5:

[0569] The server receives the request and temporarily saves the file data. The server saves the received illustration file in a specified folder and records the style setting information.

[0570] Step 6:

[0571] The server activates the emotion engine and analyzes the user's facial expressions and voice data sent from the user interface. The server recognizes the user's emotional state in real time through the emotion engine.

[0572] Step 7:

[0573] The server dynamically adjusts the UI display based on the emotional data it recognizes. For example, if the user is frustrated, a support message or assistance options will be displayed.

[0574] Step 8:

[0575] The server processes the illustration file using an AI image processing engine, selects the corresponding AI model based on the specified style information, and applies coloring and effects to the line art file.

[0576] Step 9:

[0577] The server manages the progress of the process in real time and notifies the device. Each time the progress of the process is updated, the progress is sent to the device.

[0578] Step 10:

[0579] The terminal displays the received progress information on the user interface, allowing the user to check the progress of the processing in real time.

[0580] Step 11:

[0581] When the server completes the process, it generates a result file and sends a notification to the device, which includes a download link for the result file.

[0582] Step 12:

[0583] The terminal displays a completion notification to the user and provides a download button, after which the user can confirm the completion notification and click the download button to download the resulting file.

[0584] Step 13:

[0585] The user clicks the download button to save the completed color illustration file to their local disk. The user can then check the completed work and further process or use it as needed.

[0586] This system allows users to quickly and efficiently automate parts of manga production, significantly reducing production time and effort. Furthermore, by combining it with an emotion engine, it provides flexible support according to the user's state, improving the user experience.

[0587] Example 2

[0588] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0589] The manga production process is extremely time-consuming and labor-intensive, requiring a great deal of manual work. As a result, the burden on the workers increases, leading to problems with reduced production efficiency. Another issue is that the system cannot adequately respond to changes in the user's emotions that occur during the process, resulting in a poor user experience. Furthermore, efficient progress management is required to quickly produce high-quality works.

[0590] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0591] In this invention, the server includes means for uploading unfinished image files to be drawn through a user interface, means for selecting appearances and settings as options, means for transmitting the uploaded image files and the selected options to the server, means for processing the image files using an artificial intelligence image processing engine by the server, means for recognizing emotions from the user's facial expressions and voice using an emotion engine, means for dynamically adjusting the user interface based on the user's emotional state, means for transmitting a processing progress status from the server to the terminal, means for transmitting a processing completion notice and a processing result file from the server to the terminal, and means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file. This allows the system to dynamically respond to the user's emotional state, enabling efficient progress management and rapid high-quality manga production.

[0592] A "user interface" is a screen or application that allows a user to interact with a system and perform various operations.

[0593] An "unfinished image file" is an image that is being uploaded by a user and is in a state before processing or editing.

[0594] "Options" are appearance and setting choices that the user can specify, including image processing styles, display settings, and the like.

[0595] A "server" is a computer system that receives requests from users and processes images and stores and manages data.

[0596] An "artificial intelligence image processing engine" is software equipped with AI technology used to analyze and process image data.

[0597] An "emotion engine" is a technology that recognizes emotions from a user's facial expressions and voice, and adjusts the system's behavior based on that emotional state.

[0598] "Progress" is information indicating the progress of image processing, and displays in real time how far the processing has progressed.

[0599] The "processing completion notification" is a message for notifying the user that image processing has been completed.

[0600] "Processing result file" refers to the final image file that has been processed and edited by the AI ​​image processing engine.

[0601] A "terminal" is a device that a user uses to access the system and perform various operations and check the processing results.

[0602] "Downloading" refers to the act of saving a processing result file sent from a server to a terminal.

[0603] "Dynamic adjustment" means changing the system's display and operation methods in real time according to the user's emotions, progress, etc.

[0604] This system reduces the burden on users and improves efficiency in manga production, and in particular employs a configuration that combines an image processing engine and an emotion engine that utilize artificial intelligence. The entire system operates through a combination of a user interface, terminals, and a server.

[0605] First, the user operates the system through a dedicated user interface (UI). This UI can be accessed from a standard web browser or a dedicated application. The user enters their authentication information on the login screen and logs into the system. After logging in, the user is redirected to the dashboard, where they can click the "Upload Line Art" button to select and upload an unfinished image file from their local disk.

[0606] Next, the user selects style and display settings as options on the UI. Specifically, for example, they select "Watercolor" from the "Style Selection" menu, and then adjust "Brightness" and "Contrast" in the display settings options. This selected data is then prepared by the device to be sent to the server. For transmission, the HTTP protocol or other communication technology is used.

[0607] After receiving the request from the device, the server temporarily stores the data in storage. It then launches an artificial intelligence image processing engine using an AI framework such as TensorFlow or PyTorch to process the received data. The appropriate AI model is selected based on the specified style, and processing is performed. For example, if the user selects a "watercolor" style, the AI ​​model corresponding to the style is applied, converting the line art file into a beautiful watercolor-style color illustration.

[0608] The emotion engine analyzes the user's facial expressions and voice while they are operating the device to recognize their current emotions. This function collects data in real time through the camera and microphone and uses machine learning algorithms to determine their emotional state. For example, if the user allows the camera and microphone to be used, the emotion engine recognizes emotions such as "happiness" or "dissatisfaction" from the user's facial expressions. Based on the recognized emotional state, the system dynamically adjusts the UI. Specifically, if the user is dissatisfied, the UI will automatically suggest assistance.

[0609] The server periodically notifies the device of the progress of image processing, which is displayed in real time on the UI. This allows the user to check the progress of the processing. The progress information is displayed as a message such as "50% complete." Finally, after processing is complete, the server generates a result file and sends a processing completion notification to the device. This notification also includes a download link for the result file. The user can receive the notification and download the result file by clicking the "Download" button on the UI.

[0610] In this way, this system significantly reduces the user's manual work and supports high-quality manga production by utilizing generative AI models. Furthermore, the introduction of an emotion engine enables dynamic responses according to the user's emotional state, providing a better user experience.

[0611] Examples and prompts:

[0612] As a concrete example, we will describe how a user accesses the system, uploads a line drawing file, selects "watercolor" as the style, and authorizes the camera and microphone. We will also explain the process in which the emotion engine recognizes the user's frustration and the UI suggests assistance. Finally, we will explain the entire process in which the user downloads the watercolor color illustration file generated by the server after processing is complete.

[0613] Example prompt sentence:

[0614] "Please explain in detail the process by which a user opens a browser, logs into the system, uploads a line drawing file, and selects style and display settings. Also, please explain in detail the entire process by which the emotion engine recognizes the user's frustration, the server processes the image using an AI image processing engine, and finally the user downloads the resulting file."

[0615] This system reduces the amount of tedious manual work required for users, providing an environment where users can focus more on creative work. It also enables efficient workflow, allowing users to quickly produce high-quality works.

[0616] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0617] Step 1:

[0618] A user accesses the system from a browser or a dedicated application and enters their username and password on the login screen. The entered authentication information is sent to the server, and if authentication is successful, the dashboard screen is displayed. When login is successful, the output is the dashboard screen.

[0619] Specific operation: The user opens a browser, accesses the system's URL, enters login information, and clicks the "Login" button.

[0620] ---

[0621] Step 2:

[0622] The user clicks the "Upload Line Art" button on the dashboard, selects an unfinished image file from the local disk, and uploads it. At this time, the file path and file name are also sent to the server, which receives and temporarily saves the file. The input is the unfinished image file selected by the user, and the output is the unfinished image file saved on the server.

[0623] Specific operation: The user clicks the "Upload line art" button on the dashboard, selects the appropriate line art file from the file selection dialog, and uploads it.

[0624] ---

[0625] Step 3:

[0626] Select the style and display settings for the image file uploaded by the user. Select the desired style from the "Style Selection" menu and make detailed settings with the "Display Settings" option. The input is the style and display settings selected by the user, and the output is the style and display setting information.

[0627] Specific behavior: The user selects "Watercolor" from the "Style Selection" menu and adjusts "Brightness" and "Contrast" in the display settings options.

[0628] ---

[0629] Step 4:

[0630] The device sends the uploaded image file and the style and display settings selected by the user to the server. This sending process uses HTTP requests and API calls to securely transfer data. The input is the image file and style settings, and the output is the data sent to the server.

[0631] Specific operation: The device sends the line drawing file and the "watercolor" style setting to the server via an HTTP request.

[0632] ---

[0633] Step 5:

[0634] The emotion engine recognizes the user's facial expressions and voice and analyzes emotions in real time. It uses machine learning algorithms to determine the user's emotional state based on data collected from the camera and microphone. The input is data from the camera and microphone, and the output is the recognized emotional state.

[0635] Specific operation: The user allows the camera and microphone, and the emotion engine analyzes emotions such as "happiness" or "satisfaction" from the user's facial expression.

[0636] ---

[0637] Step 6:

[0638] The server launches an artificial intelligence image processing engine and processes the received image file in the specified style. It uses TensorFlow and PyTorch to analyze and process the image data. The input is the image file and style settings uploaded by the user, and the output is the processed image file.

[0639] Specific operation: The server applies the "watercolor style" model to process the unfinished image file into a color illustration.

[0640] ---

[0641] Step 7:

[0642] The server notifies the terminal of the progress of the processing. The terminal reflects the received progress status in the user interface in real time and displays it to the user. The input is the progress information calculated by the server, and the output is the progress status displayed on the terminal.

[0643] Specific operation: When the server has completed 50% of the processing, it notifies the terminal of the progress and displays the message "50% complete" on the UI.

[0644] ---

[0645] Step 8:

[0646] After the processing is complete, the server generates a result file and sends a processing completion notification to the device. This notification includes a download link for the result file. The input is the processed image file, and the output is the completion notification and the download link.

[0647] What it does: The server generates the final watercolor-style color illustration file and sends a notification saying "Processing complete. You can download it here" along with a link.

[0648] ---

[0649] Step 9:

[0650] The user receives a notification of the completion of the process and clicks the "Download" button on the UI to download the generated result file. The input is the completion notification from the server, and the output is the result file saved locally by the user.

[0651] Specific operation: The user confirms the notification and clicks the "Download" button on the UI to save the generated watercolor-style color illustration file.

[0652] (Application example 2)

[0653] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0654] In manga production, there is a need to reduce the burden on users and improve efficiency. In particular, there is a lack of systems that provide dynamic support according to the user's emotional state and enable smooth progress in the work, making it difficult to provide an environment where users can concentrate on creative work. Another issue is that existing systems only provide simple functions without taking the user's emotional state into consideration.

[0655] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for uploading an unfinished illustration file to be drawn through a user interface; means for selecting an optional style and display settings; means for analyzing the user's emotional state using an emotion recognition engine; means for transmitting the uploaded illustration file, the selected options, and the analyzed emotional state to the server; means for processing the illustration file using an AI image processing engine by the server; means for transmitting a processing progress status from the server to the terminal; means for transmitting a processing completion notice and a processing result file from the server to the terminal; means for dynamically adjusting the user interface according to the emotional state; and means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file. This provides interactive support according to the user's emotional state, improves the efficiency of creative work, and enables the user to engage in creative activities with a sense of satisfaction.

[0656] (definition statement)

[0657] A "user interface" is an interface that provides a screen and operating means for a user to access and operate a system.

[0658] An "illustration file" refers to image data that a user is currently creating or has completed, and is specifically a file that contains manga line drawings and sketches.

[0659] "Options" are choices that a user can set within the system, including style and display settings.

[0660] An "emotion recognition engine" is an engine that analyzes a user's facial expressions and voice to recognize their emotional state in real time.

[0661] The "AI image processing engine" is an engine that uses artificial intelligence to process uploaded illustration files based on a specific style.

[0662] "Progress" is information that indicates the progress of work while the system is processing it.

[0663] The "processing completion notification" is a notification that notifies the user that the server has completed processing of the illustration file.

[0664] "Result file" refers to the illustration file after it has been processed by the AI ​​image processing engine.

[0665] "Dynamic adjustment" means that the display content and operation guide of the user interface are changed according to the emotional state of the user.

[0666] A "terminal" is a device used by a user to access the system, including a smartphone, a personal computer, etc.

[0667] This invention is a system for reducing the user's burden and improving efficiency in manga production. In particular, it has a function to provide dynamic support according to the user's emotional state. This system consists of a user interface (UI), an emotion recognition engine, an AI image processing engine, a server, and a terminal.

[0668] Hardware and software used

[0669] Hardware: Smartphone, PC (with camera and microphone)

[0670] software:

[0671] AI image processing engine: TensorFlow

[0672] Emotion recognition engine: OpenCV (facial expression recognition), Google Cloud Speech-to-Text API (voice recognition)

[0673] Web server: Node.js

[0674] System configuration

[0675] 1. User Interface (UI):

[0676] Users can upload unfinished illustration files and select style and display settings through a dedicated UI, which can be accessed from a browser or a dedicated application.

[0677] 2. Emotion Recognition Engine:

[0678] The emotion recognition engine analyzes the user's facial expressions and voice in real time to recognize their emotional state. For example, when a user accesses the emotion recognition engine using a camera and microphone, the system can detect emotions such as joy or displeasure.

[0679] 3. AI image processing engine:

[0680] The AI ​​image processing engine running on the server processes the uploaded illustration file according to the selected style, allowing the illustration to be automatically colored in the style specified by the user.

[0681] 4. Server:

[0682] The server temporarily stores the illustration file and settings information received from the user, then starts the AI ​​image processing engine to process it. It also has a function to send progress and processing completion notifications to the terminal.

[0683] 5. Terminal:

[0684] The terminal is the device that the user uses to access the system, such as a smartphone or PC. The terminal displays the progress and processing completion notifications sent from the server to the user in real time, and allows the user to download the result file.

[0685] Specific examples

[0686] Consider a scenario where a user uploads a line art file and selects the "Realistic Style." The user enters the following prompt:

[0687] Example prompt sentence:

[0688] "User ID: example_user_id, Request type: Image processing, Style: Realistic, Image file path: / path / to / uploaded / line_art.png"

[0689] This prompt is an instruction for the system's AI image processing engine to select an appropriate model and color the illustration. Additionally, if the user allows the camera and microphone, the emotion recognition engine will recognize joy or displeasure from facial expressions and voice in real time and dynamically adjust the UI, significantly improving user efficiency.

[0690] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0691] (Processing steps of patent embodiment)

[0692] Step 1:

[0693] Users access the system through a dedicated user interface (UI) and upload unfinished illustration files.

[0694] Input: Unfinished illustration file

[0695] Operation: Click the "Upload Line Art" button on the user interface and select an illustration file from your local disk from the file selection dialog.

[0696] Output: Temporarily save uploaded illustration files

[0697] Step 2:

[0698] The user selects style and display settings from options in the UI.

[0699] Input: Style selection, display settings

[0700] Action: Select the desired option from the "Style Selection" or "Display Settings" drop-down menu.

[0701] Output: Information about selected styles and display settings

[0702] Step 3:

[0703] An emotion recognition engine analyzes the user's emotional state.

[0704] Input: User's facial and voice data

[0705] How it works: It uses a camera and microphone to collect and analyze the user's facial expressions and voice in real time.

[0706] Output: Analyzed emotional state data

[0707] Step 4:

[0708] The terminal prepares to send the uploaded illustration file, the selected options, and the analyzed emotional state to the server.

[0709] Input: illustration file, style settings, emotional state

[0710] Operation: This data is compiled into a prompt statement format and prepared to be sent to the server.

[0711] Output: Data in prompt format

[0712] Step 5:

[0713] The server temporarily stores the received illustration file, options, and emotional state data, and then activates an AI image processing engine to process the illustration file.

[0714] Input: Data in prompt format (illustration file, style settings, emotional state)

[0715] How it works: Analyzes the received data, selects the AI ​​image processing engine that corresponds to the specified style, and colors the line drawing.

[0716] Output: Colored illustration file

[0717] Step 6:

[0718] The server sends the progress of the illustration processing to the terminal.

[0719] Input: Processing progress information

[0720] How it works: The AI ​​image processing engine reports progress in real time and sends that information to your device.

[0721] Output: Progress notification

[0722] Step 7:

[0723] The server sends a processing completion notice and a result file to the terminal.

[0724] Input: Processing completion information, result file

[0725] Operation: Once processing is complete, a notification of completion will be sent to the device along with a download link for the generated color illustration file.

[0726] Output: Processing completion notification and download link

[0727] Step 8:

[0728] The terminal displays a notification to the user that the process is complete and allows the result file to be downloaded.

[0729] Input: Processing completion notification, download link

[0730] Behavior: Displays a notification in the UI and generates a "Download" button that the user clicks to download the resulting file.

[0731] Output: Downloaded color illustration file

[0732] In this way, specific actions are performed at each step, allowing users to create manga efficiently. Interactive support is also provided according to the user's emotional state, creating a more creative work environment.

[0733] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0734] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0735] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0736] [Third embodiment]

[0737] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0738] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0739] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0740] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0741] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0742] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0743] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0744] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0745] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0746] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0747] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0748] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0749] This invention provides a means to efficiently automate part of the manga production process through a system that operates via a server and terminals. Specifically, it is a system that automatically processes unfinished illustration files drawn by users using an AI image processing engine to complete them in a short amount of time.

[0750] User Interface Parts

[0751] Users operate the system through a dedicated user interface (UI). Users access the system from their device's browser and upload unfinished illustration files. The UI displays an "Upload Line Art" button, and clicking this displays a file selection dialog. Here, users can select an illustration file from their local disk and upload it.

[0752] Additionally, the UI provides "Style Selection" and "Display Settings" options, allowing users to select the style and effect they desire.

[0753] Sending part of the request

[0754] The selected illustration file and settings are sent from the device to the server as an HTTP POST request, which includes the file data and the user-specified style and display settings.

[0755] Server-side processing

[0756] The server receives the illustration file and setting information sent from the device and temporarily stores them. The server then starts the AI ​​image processing engine and provides the received data to the processing engine. The AI ​​image processing engine uses a deep learning model to analyze the line drawing of the illustration and color it in the specified style.

[0757] Progress management and notifications

[0758] The server manages the processing progress of the AI ​​image processing engine and periodically checks its progress. This progress information is sent from the server to the device in real time. The device displays the received progress information on the UI, allowing the user to understand the current progress.

[0759] Returning and downloading results

[0760] When the processing is complete, the server generates a processing result file and sends a processing completion notification to the device. This completion notification includes a download link for the result file. Based on the received completion notification, the device displays a pop-up or dialog box on the UI and generates a "Download" button.

[0761] The user confirms the completion notification and clicks the download button to save the resulting file to their local disk, allowing them to obtain the colored finished illustration in a short time.

[0762] Specific examples

[0763] For example, consider the case where a user uploads a monochrome line drawing illustration and chooses to colorize it in a vibrant style.

[0764] 1. The user uses the device to click the "Upload Line Art" button on the UI to upload an unfinished monochrome line art file.

[0765] 2. The user selects the "Colorful" style from the "Style Selection" drop-down menu.

[0766] 3. The device sends the file and style information to the server.

[0767] 4. The server processes the file using an AI image processing engine and periodically sends progress updates to the device.

[0768] 5. When the progress reaches 100%, the server will send a notification of completion and a download link for the result file to the device.

[0769] 6. The device displays a completion notice to the user and provides a "Download" button.

[0770] 7. The user clicks the download button and receives the completed color illustration file.

[0771] This invention significantly reduces the time and effort required to create manga, providing an environment in which creators can concentrate on creating their work. It also enhances creators' creativity by supporting a variety of styles and settings.

[0772] The processing flow will be explained below.

[0773] Step 1:

[0774] The user accesses the terminal's user interface. The user interface can be accessed from a browser or a dedicated application, and a login screen is displayed. The user enters login information and logs into the system.

[0775] Step 2:

[0776] The user uploads a line drawing file. The user clicks the "Upload Line Drawing" button on the UI and selects a line drawing file from the local disk in the file selection dialog that appears. The device prepares to send the selected file to the server.

[0777] Step 3:

[0778] The user selects a style or display setting. The UI displays a drop-down menu of style choices and other options. The user selects the style or display setting they want.

[0779] Step 4:

[0780] The device sends the uploaded line art file and selected options to the server as an HTTP POST request, which includes the file data and style information.

[0781] Step 5:

[0782] The server receives the request, temporarily stores the received data, and saves the received line drawing file in an appropriate folder, while also recording the style information specified by the user.

[0783] Step 6:

[0784] The server starts the AI ​​image processing engine and provides the received data to the processing engine. The server selects an AI model corresponding to the specified style and processes the line drawing file.

[0785] Step 7:

[0786] The server manages the progress of the processing and periodically notifies the device. As the AI ​​image processing progresses, the progress status is reported to the server, which then periodically sends it to the device.

[0787] Step 8:

[0788] The terminal displays the received progress information on the UI so that the user can check the current processing status in real time.

[0789] Step 9:

[0790] After the process is complete, the server generates a result file and sends a notification of the completion of the process to the device, along with a download link for the result file.

[0791] Step 10:

[0792] The device displays a pop-up or dialog box on the UI based on the received completion notification, and generates a "Download" button to allow the user to download the result file.

[0793] Step 11:

[0794] The user downloads the resulting file. After confirming the completion notification, they click the download button and save it to their local disk. The completed color illustration file is provided.

[0795] This series of processes allows users to quickly and efficiently automate parts of manga production, significantly reducing production time and effort.

[0796] Example 1

[0797] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0798] Traditional manga production has the problem of requiring a great deal of time and effort to color images and reflect styles. Manual coloring, in particular, is a factor that reduces productivity, and many creators are seeking more efficient methods. Furthermore, there is often a lack of smooth communication methods for checking progress and obtaining results. To solve these issues, a system is needed that automates image processing, allowing users to check progress in real time and obtain results quickly.

[0799] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0800] In this invention, the server includes a means for uploading an unfinished image file to be drawn through a user interface, a means for selecting an optional style and display setting, a means for transmitting the uploaded image file and the selected option to the server, a means for processing the image file using an artificial intelligence image processing engine by the server, a means for transmitting a processing progress report from the server to the terminal, a means for transmitting a processing completion notice and a processing result file from the server to the terminal, and a means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file, thereby enabling the user to quickly automatically colorize and style images, check the progress in real time, and reliably obtain the results.

[0801] A "user interface" is a dedicated interface for users to perform operations, and includes, for example, buttons and menus displayed on the screen.

[0802] "Image files" refers to digital files such as line drawings and illustrations, and includes formats such as JPEG, PNG, and BMP.

[0803] "Style" refers to the design and effect settings applied in image processing, and includes different methods of expression such as "vivid colors" and "monochrome."

[0804] "Display settings" refers to various settings related to the display of images and user interfaces, including, for example, brightness and contrast adjustments.

[0805] A "server" refers to a computer system that accepts requests from client terminals via a network and processes and stores data.

[0806] "Artificial intelligence image processing engine" refers to software that analyzes and processes images using deep learning models and other artificial intelligence technologies.

[0807] "Processing progress" refers to information indicating the progress of the processing currently being performed by the image processing engine.

[0808] A "processing completion notification" is a message from the system notifying you that image processing is complete, and typically includes a download link.

[0809] The "download link" indicates the URL for the user to obtain the result file.

[0810] "Terminal" refers to an electronic device such as a computer or smartphone that is operated by a user.

[0811] The present invention is a system that automatically processes unfinished image files drawn by users using an artificial intelligence image processing engine to complete them in a short time. The system is operated through a user interface and operates via a server and terminals.

[0812] First, the user operates the system through a dedicated user interface (UI). The user accesses the system from the device's browser and uploads an unfinished image file. The UI displays an "Upload Image" button, and clicking this displays a file selection dialog. The user selects an image file from the local disk and uploads it. Furthermore, the UI provides options for "Style Selection" and "Display Settings," allowing the user to select the desired style and effect.

[0813] The image file and settings information selected by the user are sent from the device to the server as an HTTP POST request. This request includes the file data and the style and display settings specified by the user. The server receives the image file and settings information sent from the device and temporarily stores them in a temporary directory on the server. The server then starts an artificial intelligence image processing engine and provides the received data to the processing engine. The artificial intelligence image processing engine uses deep learning models such as TensorFlow and PyTorch to analyze the image and apply styles.

[0814] As the processing progresses, the server monitors the progress of the artificial intelligence image processing engine and sends progress information to the device in real time at regular intervals. The device displays the received progress information on the UI, allowing the user to understand the current progress. After the processing is complete, the server generates a processing result file and sends a processing completion notification to the device. This completion notification includes a download link for the result file. Based on the received completion notification, the device displays a pop-up or dialog box on the UI and generates a "Download" button. The user can click this download button to save the result file to their local disk.

[0815] For example, suppose a user uploads a monochrome line drawing illustration and chooses to colorize it in a vibrant style. Here is an example prompt:

[0816] Example prompt sentence:

[0817] The user uses the device to click the "Upload Image" button on the UI to upload an unfinished monochrome line drawing file. The user selects the "Vivid" style from the "Style Selection" drop-down menu. The device sends the file and style information to the server. The server processes the file using an artificial intelligence image processing engine and periodically sends progress updates to the device. Once processing is complete, the server sends a processing completion notification and a download link for the resulting file to the device. The device displays a completion notification to the user and provides a "Download" button. The user clicks the download button to receive the completed color illustration file.

[0818] This invention will significantly reduce the time and effort required to create manga, allowing creators to focus on creating their works. It will also enhance creators' creativity by supporting a variety of styles and settings.

[0819] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0820] Step 1:

[0821] The user uses a terminal to access a dedicated user interface. The user clicks the "Upload Image" button and selects and uploads an unfinished image file from the local disk. Based on the input, the UI receives the selected file "ImageFile.png". This file is uploaded and ready for the next step.

[0822] Specific behavior: A user opens a browser, accesses the system's web page, clicks the "Upload Image" button, selects "Image File.png" in the file selection dialog, and uploads it.

[0823] Step 2:

[0824] The user selects the desired style and effect from the "Style Selection" and "Display Settings" options. Based on the input, the UI receives the selected style "Vivid" and display settings. The setting information is ready to be sent to the server.

[0825] What it does: The user opens the "Style Selection" drop-down menu on the UI, selects the "Vivid" style, and then adjusts other effects in the "Display Settings" options.

[0826] Step 3:

[0827] The device sends the image file and setting information selected by the user to the server as an HTTP POST request. Based on the input, the file data "Image file.png" and style information "Vivid" arrive at the server. The server receives this data and temporarily stores it.

[0828] Specific operation: The device generates a POST request and sends it to the server's image processing API endpoint, including the file "image file.png" and the "vivid" style information. The server receives it and saves it in the " / tmp / uploads / " directory.

[0829] Step 4:

[0830] The server launches an artificial intelligence image processing engine and provides the saved image file and configuration information to the processing engine. Based on the input, a deep learning model (e.g., TensorFlow) is used to analyze and style the image. As an output, a processed image file is generated.

[0831] Specific operation: The server runs the Python script "process_image.py" to process "image_file.png" using the TensorFlow model. The process takes a few seconds to a few minutes.

[0832] Step 5:

[0833] The server monitors the processing progress of the AI ​​image processing engine. The progress status is checked at regular intervals and the progress information is sent to the terminal in real time. Based on the input, processing progress information is generated and sent to the terminal.

[0834] Specific operation: The server checks the progress of the processing every 5 seconds and sends a message containing progress information (e.g., processing progress 50%) to the terminal via WebSocket communication.

[0835] Step 6:

[0836] Once the processing is complete, the server generates a processing result file. The server then sends a processing completion notification to the device, which includes a download link for the processing result file. Based on the input, the processing result "processed_image.png" is generated and a download link is created.

[0837] Specific operation: The server generates a file called "processed_image.png" and sends a message to the terminal containing a processing completion notification and a download link (e.g., "http: / / example.com / download / processed_image.png").

[0838] Step 7:

[0839] The device displays a pop-up or dialog box on the UI based on the received processing completion notification. It generates a "Download" button to allow the user to download the result file. Based on the input, the UI displays a download link so that the user can obtain the result file.

[0840] Specific operation: The device receives the completion notification and displays a "Download" button on the UI. The user clicks this button and saves the resulting file "processed_image.png" to the local disk.

[0841] By following these steps, users can obtain a finished image that is automatically processed in a short time.

[0842] (Application example 1)

[0843] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0844] There is a need for a system that can efficiently complete unfinished design files drawn by users and instantly display the completed designs in a virtual store. Conventional methods require specialized skills and a significant amount of time for users to complete the design. In addition, displaying the completed design must be done manually, which is a time-consuming process.

[0845] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0846] In this invention, the server includes means for uploading an unfinished design file drawn through a user interface, means for selecting a design style and display settings as options, means for transmitting the uploaded design file and the selected options to the server, means for the server to process the design file using a machine learning model, means for transmitting a processing progress status from the server to the terminal, means for transmitting a processing completion notice and a processing result file from the server to the terminal, means for the terminal to display a processing completion notice to the user and allow the user to download the result file, and means for automatically uploading and displaying the completed design file to a virtual store, thereby enabling the user to efficiently complete a design and immediately display the design in the virtual store.

[0847] A "user interface" is a software component that provides a screen and input means for a user to access and operate a system.

[0848] A "design file" is a digital file representing an unfinished illustration or design drawn by a user.

[0849] "Options" are additional setting items such as design styles and display settings that can be selected by the user.

[0850] "Server" means a computer system that receives and processes design files and options uploaded by users.

[0851] "Machine Learning Model" means a computational model utilizing machine learning algorithms used to process design files.

[0852] "Processing progress" is information that the server uses to notify the user of the processing status of the design file in real time.

[0853] The "processing completion notification" is notification information that the server uses to notify the user that processing of the design file has been completed.

[0854] "Result File" means the final design file processed by the machine learning model.

[0855] A "virtual store" is an online platform such as a shop or exhibition hall that is virtually set up on the Internet.

[0856] A system for implementing the present invention comprises a user interface, a server, a machine learning model, a terminal, and a virtual store.

[0857] Users upload unfinished design files through their device's browser. The user interface displays a "File Upload" button, which when clicked displays a file selection dialog, allowing users to select and upload a design file from their local disk. In addition, the user interface provides options for "Design Style Selection" and "Display Settings," allowing users to select the style and effect they desire.

[0858] The selected design file and settings information are sent from the device to the server as an HTTP POST request. This request includes the file data and the style and display settings specified by the user. The server receives this data and temporarily stores it.

[0859] The server then launches an environment for the machine learning model (for example, one using TensorFlow or PyTorch) and provides the received data to the model, which uses deep learning techniques to analyze the design file and colorize it in the specified style.

[0860] The server manages the progress of this processing and periodically transmits progress information to the terminal in real time. The terminal displays the received progress information on a user interface, allowing the user to grasp the current progress.

[0861] When the processing is complete, the server generates a processing result file and sends a processing completion notification to the terminal. This completion notification includes a download link for the result file. Based on the received completion notification, the terminal displays a pop-up or dialog box on the user interface and generates a "Download" button. The user can click this to save the completed design file to their local disk.

[0862] In addition, the completed design file is automatically uploaded to the virtual store by the server and instantly displayed online, allowing users to instantly share and exhibit their created designs.

[0863] A concrete example scenario might be a scenario where a user uses smart glasses to take a photo of a sketch character they've drawn in the real world, then uploads it to an app and specifies a "fantasy style" coloring. An example prompt for this would be:

[0864] "Color the line art below in a vibrant, fantasy style."

[0865] Contains the user's line drawing illustration data.

[0866] This system allows users to easily complete designs and instantly display them in a virtual store.

[0867] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0868] Step 1:

[0869] The user opens the user interface via a browser on the device. As input, the user uses the browser on the device. The user clicks the "File Upload" button and selects and uploads an unfinished design file from the local disk in the file selection dialog. As output, the selected design file is obtained.

[0870] Step 2:

[0871] On the user interface, the user selects the desired style from the "Select Design Style" drop-down menu. As input, there is the style information selected by the user. As output, the design file together with the style information is sent to the server.

[0872] Step 3:

[0873] The device sends the uploaded design file and selected style information to the server as an HTTP POST request. The file data and style setting information from the device are used as input. The request arrives at the server as output.

[0874] Step 4:

[0875] The server receives and temporarily stores design files and style information. The input is the file and information sent from the device. The output is the file and information saved in the server's storage.

[0876] Step 5:

[0877] The server launches a machine learning model (e.g., a model using TensorFlow or PyTorch) and provides it with the incoming data. As input, it uses the saved design file and style information. The machine learning model processes the data and colorizes it with the specified style. As output, it generates the processed finished design file.

[0878] Step 6:

[0879] The server manages the processing progress and periodically sends progress information to the device. The input is the processing status of the machine learning model. The output is the progress information sent from the server to the device via data communication.

[0880] Step 7:

[0881] The progress information received by the terminal is displayed on the user interface in real time. As input, there is progress information sent from the server. As output, there is real-time progress displayed on the user interface.

[0882] Step 8:

[0883] When the server completes the process, it generates a result file and sends a notification of completion to the device. The input is the completed design file processed by the machine learning model. The output is a notification of completion and a download link provided by the server to the device.

[0884] Step 9:

[0885] Based on the completion notification received by the terminal, a pop-up or dialog box is displayed on the user interface and a "Download" button is generated. The input is a completion notification from the server. The output is a pop-up or dialog box displayed on the user interface.

[0886] Step 10:

[0887] The user clicks the "Download" button to save the completed design file to their local disk. As input, there is a "Download" button in a popup or dialog box. As output, there is a completed design file saved to their local disk.

[0888] Step 11:

[0889] The server automatically uploads the completed design file to the virtual store and displays it. The input is the processed completed design file. The output is the design file displayed in the virtual store.

[0890] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0891] This invention is a system that reduces the burden on users and improves efficiency in manga production, and in particular, it employs a configuration that combines an AI image processing engine and an emotion engine. Specifically, it is a system that improves the user experience by recognizing the user's emotions and providing dynamic responses accordingly.

[0892] User Interface Parts

[0893] Users operate the system through a dedicated user interface (UI), which can be accessed from a browser or a dedicated application. Users can access the system from a login screen and perform the necessary operations.

[0894] Example: A user opens a browser, accesses the system's URL, enters login information, and logs in.

[0895] Sending part of a user request

[0896] 1. The user uploads a line art file. The UI displays an "Upload Line Art" button, and clicking it displays a file selection dialog. The user selects an illustration file from their local disk and uploads it.

[0897] 2. The user selects the style and display settings. Select the desired option from the "Style Selection" and "Display Settings" options displayed on the UI.

[0898] 3. The device prepares to send the line drawing file and selected options to the server.

[0899] Example: A user uploads an unfinished line art file and selects a colorful style.

[0900] The working parts of the emotion engine

[0901] The emotion engine has the ability to analyze the user's facial expressions and voice. This engine works in conjunction with the user interface to recognize the user's emotions in real time.

[0902] Example: The user allows the camera and microphone, and the emotion engine recognizes joy or displeasure from the user's facial expressions and voice.

[0903] The emotion engine dynamically adjusts the UI based on the emotional data it recognizes. If the user is dissatisfied, the UI will respond by suggesting assistance.

[0904] Server-side processing

[0905] 1. The server receives the request sent from the device and temporarily stores it.

[0906] 2. The server starts the AI ​​image processing engine to process the received data, and selects and operates different AI models depending on the specified style.

[0907] Example: The server selects an AI model that corresponds to a colorful style and processes a line art file.

[0908] 3. The server periodically notifies the terminal of the progress of the processing, and the progress information is displayed to the user in real time.

[0909] Returning and downloading results

[0910] 1. After the processing is complete, the server generates a result file and sends a notification to the device along with a download link for the result file.

[0911] 2. The device notifies the user on the UI based on the received completion notification and generates a download button, which the user can click to download the result file.

[0912] Example: The user sees the completion notification and clicks the download button to receive the completed color illustration file.

[0913] The system's unique feature is its ability to improve the user experience by incorporating an emotion engine that dynamically responds to the user's emotional state. The emotion engine recognizes the user's emotional state and uses that information to change the screen display and operation guides, supporting efficient and smooth work progress. Furthermore, by linking with an AI image processing engine, the entire manga production process can be automated, significantly shortening production time.

[0914] This system reduces the amount of tedious manual work required for users, providing an environment where users can focus more on creative work. It also enables efficient workflow, allowing users to quickly produce high-quality works.

[0915] The processing flow will be explained below.

[0916] Step 1:

[0917] The user opens the user interface (UI) of the device, launches a browser or a dedicated application, and accesses the system login screen. The user enters the username and password to log in.

[0918] Step 2:

[0919] The user uploads an unfinished illustration file by clicking the "Upload Line Art" button on the UI, selecting a line art file saved on the local disk from the file selection dialog, and uploading it.

[0920] Step 3:

[0921] The user selects the style and display settings. Select the desired style or effect from the "Style Selection" drop-down menu or other options in the UI. The device records the selection.

[0922] Step 4:

[0923] The device sends the uploaded illustration file and the selected style settings to the server as an HTTP POST request, which includes the file data and style information.

[0924] Step 5:

[0925] The server receives the request and temporarily saves the file data. The server saves the received illustration file in a specified folder and records the style setting information.

[0926] Step 6:

[0927] The server activates the emotion engine and analyzes the user's facial expressions and voice data sent from the user interface. The server recognizes the user's emotional state in real time through the emotion engine.

[0928] Step 7:

[0929] The server dynamically adjusts the UI display based on the emotional data it recognizes. For example, if the user is frustrated, a support message or assistance options will be displayed.

[0930] Step 8:

[0931] The server processes the illustration file using an AI image processing engine, selects the corresponding AI model based on the specified style information, and applies coloring and effects to the line art file.

[0932] Step 9:

[0933] The server manages the progress of the process in real time and notifies the device. Each time the progress of the process is updated, the progress is sent to the device.

[0934] Step 10:

[0935] The terminal displays the received progress information on the user interface, allowing the user to check the progress of the processing in real time.

[0936] Step 11:

[0937] When the server completes the process, it generates a result file and sends a notification to the device, which includes a download link for the result file.

[0938] Step 12:

[0939] The terminal displays a completion notification to the user and provides a download button, after which the user can confirm the completion notification and click the download button to download the resulting file.

[0940] Step 13:

[0941] The user clicks the download button to save the completed color illustration file to their local disk. The user can then check the completed work and further process or use it as needed.

[0942] This system allows users to quickly and efficiently automate parts of manga production, significantly reducing production time and effort. Furthermore, by combining it with an emotion engine, it provides flexible support according to the user's state, improving the user experience.

[0943] Example 2

[0944] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0945] The manga production process is extremely time-consuming and labor-intensive, requiring a great deal of manual work. As a result, the burden on the workers increases, leading to problems with reduced production efficiency. Another issue is that the system cannot adequately respond to changes in the user's emotions that occur during the process, resulting in a poor user experience. Furthermore, efficient progress management is required to quickly produce high-quality works.

[0946] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0947] In this invention, the server includes means for uploading unfinished image files to be drawn through a user interface, means for selecting appearances and settings as options, means for transmitting the uploaded image files and the selected options to the server, means for processing the image files using an artificial intelligence image processing engine by the server, means for recognizing emotions from the user's facial expressions and voice using an emotion engine, means for dynamically adjusting the user interface based on the user's emotional state, means for transmitting a processing progress status from the server to the terminal, means for transmitting a processing completion notice and a processing result file from the server to the terminal, and means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file. This allows the system to dynamically respond to the user's emotional state, enabling efficient progress management and rapid high-quality manga production.

[0948] A "user interface" is a screen or application that allows a user to interact with a system and perform various operations.

[0949] An "unfinished image file" is an image that is being uploaded by a user and is in a state before processing or editing.

[0950] "Options" are appearance and setting choices that the user can specify, including image processing styles, display settings, and the like.

[0951] A "server" is a computer system that receives requests from users and processes images and stores and manages data.

[0952] An "artificial intelligence image processing engine" is software equipped with AI technology used to analyze and process image data.

[0953] An "emotion engine" is a technology that recognizes emotions from a user's facial expressions and voice, and adjusts the system's behavior based on that emotional state.

[0954] "Progress" is information indicating the progress of image processing, and displays in real time how far the processing has progressed.

[0955] The "processing completion notification" is a message for notifying the user that image processing has been completed.

[0956] "Processing result file" refers to the final image file that has been processed and edited by the AI ​​image processing engine.

[0957] A "terminal" is a device that a user uses to access the system and perform various operations and check the processing results.

[0958] "Downloading" refers to the act of saving a processing result file sent from a server to a terminal.

[0959] "Dynamic adjustment" means changing the system's display and operation methods in real time according to the user's emotions, progress, etc.

[0960] This system reduces the burden on users and improves efficiency in manga production, and in particular employs a configuration that combines an image processing engine and an emotion engine that utilize artificial intelligence. The entire system operates through a combination of a user interface, terminals, and a server.

[0961] First, the user operates the system through a dedicated user interface (UI). This UI can be accessed from a standard web browser or a dedicated application. The user enters their authentication information on the login screen and logs into the system. After logging in, the user is redirected to the dashboard, where they can click the "Upload Line Art" button to select and upload an unfinished image file from their local disk.

[0962] Next, the user selects style and display settings as options on the UI. Specifically, for example, they select "Watercolor" from the "Style Selection" menu, and then adjust "Brightness" and "Contrast" in the display settings options. This selected data is then prepared by the device to be sent to the server. For transmission, the HTTP protocol or other communication technology is used.

[0963] After receiving the request from the device, the server temporarily stores the data in storage. It then launches an artificial intelligence image processing engine using an AI framework such as TensorFlow or PyTorch to process the received data. The appropriate AI model is selected based on the specified style, and processing is performed. For example, if the user selects a "watercolor" style, the AI ​​model corresponding to the style is applied, converting the line art file into a beautiful watercolor-style color illustration.

[0964] The emotion engine analyzes the user's facial expressions and voice while they are operating the device to recognize their current emotions. This function collects data in real time through the camera and microphone and uses machine learning algorithms to determine their emotional state. For example, if the user allows the camera and microphone to be used, the emotion engine recognizes emotions such as "happiness" or "dissatisfaction" from the user's facial expressions. Based on the recognized emotional state, the system dynamically adjusts the UI. Specifically, if the user is dissatisfied, the UI will automatically suggest assistance.

[0965] The server periodically notifies the device of the progress of image processing, which is displayed in real time on the UI. This allows the user to check the progress of the processing. The progress information is displayed as a message such as "50% complete." Finally, after processing is complete, the server generates a result file and sends a processing completion notification to the device. This notification also includes a download link for the result file. The user can receive the notification and download the result file by clicking the "Download" button on the UI.

[0966] In this way, this system significantly reduces the user's manual work and supports high-quality manga production by utilizing generative AI models. Furthermore, the introduction of an emotion engine enables dynamic responses according to the user's emotional state, providing a better user experience.

[0967] Examples and prompts:

[0968] As a concrete example, we will describe how a user accesses the system, uploads a line drawing file, selects "watercolor" as the style, and authorizes the camera and microphone. We will also explain the process in which the emotion engine recognizes the user's frustration and the UI suggests assistance. Finally, we will explain the entire process in which the user downloads the watercolor color illustration file generated by the server after processing is complete.

[0969] Example prompt sentence:

[0970] "Please explain in detail the process by which a user opens a browser, logs into the system, uploads a line drawing file, and selects style and display settings. Also, please explain in detail the entire process by which the emotion engine recognizes the user's frustration, the server processes the image using an AI image processing engine, and finally the user downloads the resulting file."

[0971] This system reduces the amount of tedious manual work required for users, providing an environment where users can focus more on creative work. It also enables efficient workflow, allowing users to quickly produce high-quality works.

[0972] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0973] Step 1:

[0974] A user accesses the system from a browser or a dedicated application and enters their username and password on the login screen. The entered authentication information is sent to the server, and if authentication is successful, the dashboard screen is displayed. When login is successful, the output is the dashboard screen.

[0975] Specific operation: The user opens a browser, accesses the system's URL, enters login information, and clicks the "Login" button.

[0976] ---

[0977] Step 2:

[0978] The user clicks the "Upload Line Art" button on the dashboard, selects an unfinished image file from the local disk, and uploads it. At this time, the file path and file name are also sent to the server, which receives and temporarily saves the file. The input is the unfinished image file selected by the user, and the output is the unfinished image file saved on the server.

[0979] Specific operation: The user clicks the "Upload line art" button on the dashboard, selects the appropriate line art file from the file selection dialog, and uploads it.

[0980] ---

[0981] Step 3:

[0982] Select the style and display settings for the image file uploaded by the user. Select the desired style from the "Style Selection" menu and make detailed settings with the "Display Settings" option. The input is the style and display settings selected by the user, and the output is the style and display setting information.

[0983] Specific behavior: The user selects "Watercolor" from the "Style Selection" menu and adjusts "Brightness" and "Contrast" in the display settings options.

[0984] ---

[0985] Step 4:

[0986] The device sends the uploaded image file and the style and display settings selected by the user to the server. This sending process uses HTTP requests and API calls to securely transfer data. The input is the image file and style settings, and the output is the data sent to the server.

[0987] Specific operation: The device sends the line drawing file and the "watercolor" style setting to the server via an HTTP request.

[0988] ---

[0989] Step 5:

[0990] The emotion engine recognizes the user's facial expressions and voice and analyzes emotions in real time. It uses machine learning algorithms to determine the user's emotional state based on data collected from the camera and microphone. The input is data from the camera and microphone, and the output is the recognized emotional state.

[0991] Specific operation: The user allows the camera and microphone, and the emotion engine analyzes emotions such as "happiness" or "satisfaction" from the user's facial expression.

[0992] ---

[0993] Step 6:

[0994] The server launches an artificial intelligence image processing engine and processes the received image file in the specified style. It uses TensorFlow and PyTorch to analyze and process the image data. The input is the image file and style settings uploaded by the user, and the output is the processed image file.

[0995] Specific operation: The server applies the "watercolor style" model to process the unfinished image file into a color illustration.

[0996] ---

[0997] Step 7:

[0998] The server notifies the terminal of the progress of the processing. The terminal reflects the received progress status in the user interface in real time and displays it to the user. The input is the progress information calculated by the server, and the output is the progress status displayed on the terminal.

[0999] Specific operation: When the server has completed 50% of the processing, it notifies the terminal of the progress and displays the message "50% complete" on the UI.

[1000] ---

[1001] Step 8:

[1002] After the processing is complete, the server generates a result file and sends a processing completion notification to the device. This notification includes a download link for the result file. The input is the processed image file, and the output is the completion notification and the download link.

[1003] What it does: The server generates the final watercolor-style color illustration file and sends a notification saying "Processing complete. You can download it here" along with a link.

[1004] ---

[1005] Step 9:

[1006] The user receives a notification of the completion of the process and clicks the "Download" button on the UI to download the generated result file. The input is the completion notification from the server, and the output is the result file saved locally by the user.

[1007] Specific operation: The user confirms the notification and clicks the "Download" button on the UI to save the generated watercolor-style color illustration file.

[1008] (Application example 2)

[1009] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1010] In manga production, there is a need to reduce the burden on users and improve efficiency. In particular, there is a lack of systems that provide dynamic support according to the user's emotional state and enable smooth progress in the work, making it difficult to provide an environment where users can concentrate on creative work. Another issue is that existing systems only provide simple functions without taking the user's emotional state into consideration.

[1011] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for uploading an unfinished illustration file to be drawn through a user interface; means for selecting an optional style and display settings; means for analyzing the user's emotional state using an emotion recognition engine; means for transmitting the uploaded illustration file, the selected options, and the analyzed emotional state to the server; means for processing the illustration file using an AI image processing engine by the server; means for transmitting a processing progress status from the server to the terminal; means for transmitting a processing completion notice and a processing result file from the server to the terminal; means for dynamically adjusting the user interface according to the emotional state; and means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file. This provides interactive support according to the user's emotional state, improves the efficiency of creative work, and enables the user to engage in creative activities with a sense of satisfaction.

[1012] (definition statement)

[1013] A "user interface" is an interface that provides a screen and operating means for a user to access and operate a system.

[1014] An "illustration file" refers to image data that a user is currently creating or has completed, and is specifically a file that contains manga line drawings and sketches.

[1015] "Options" are choices that a user can set within the system, including style and display settings.

[1016] An "emotion recognition engine" is an engine that analyzes a user's facial expressions and voice to recognize their emotional state in real time.

[1017] The "AI image processing engine" is an engine that uses artificial intelligence to process uploaded illustration files based on a specific style.

[1018] "Progress" is information that indicates the progress of work while the system is processing it.

[1019] The "processing completion notification" is a notification that notifies the user that the server has completed processing of the illustration file.

[1020] "Result file" refers to the illustration file after it has been processed by the AI ​​image processing engine.

[1021] "Dynamic adjustment" means that the display content and operation guide of the user interface are changed according to the emotional state of the user.

[1022] A "terminal" is a device used by a user to access the system, including a smartphone, a personal computer, etc.

[1023] This invention is a system for reducing the user's burden and improving efficiency in manga production. In particular, it has a function to provide dynamic support according to the user's emotional state. This system consists of a user interface (UI), an emotion recognition engine, an AI image processing engine, a server, and a terminal.

[1024] Hardware and software used

[1025] Hardware: Smartphone, PC (with camera and microphone)

[1026] software:

[1027] AI image processing engine: TensorFlow

[1028] Emotion recognition engine: OpenCV (facial expression recognition), Google Cloud Speech-to-Text API (voice recognition)

[1029] Web server: Node.js

[1030] System configuration

[1031] 1. User Interface (UI):

[1032] Users can upload unfinished illustration files and select style and display settings through a dedicated UI, which can be accessed from a browser or a dedicated application.

[1033] 2. Emotion Recognition Engine:

[1034] The emotion recognition engine analyzes the user's facial expressions and voice in real time to recognize their emotional state. For example, when a user accesses the emotion recognition engine using a camera and microphone, the system can detect emotions such as joy or displeasure.

[1035] 3. AI image processing engine:

[1036] The AI ​​image processing engine running on the server processes the uploaded illustration file according to the selected style, allowing the illustration to be automatically colored in the style specified by the user.

[1037] 4. Server:

[1038] The server temporarily stores the illustration file and settings information received from the user, then starts the AI ​​image processing engine to process it. It also has a function to send progress and processing completion notifications to the terminal.

[1039] 5. Terminal:

[1040] The terminal is the device that the user uses to access the system, such as a smartphone or PC. The terminal displays the progress and processing completion notifications sent from the server to the user in real time, and allows the user to download the result file.

[1041] Specific examples

[1042] Consider a scenario where a user uploads a line art file and selects the "Realistic Style." The user enters the following prompt:

[1043] Example prompt sentence:

[1044] "User ID: example_user_id, Request type: Image processing, Style: Realistic, Image file path: / path / to / uploaded / line_art.png"

[1045] This prompt is an instruction for the system's AI image processing engine to select an appropriate model and color the illustration. Additionally, if the user allows the camera and microphone, the emotion recognition engine will recognize joy or displeasure from facial expressions and voice in real time and dynamically adjust the UI, significantly improving user efficiency.

[1046] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1047] (Processing steps of patent embodiment)

[1048] Step 1:

[1049] Users access the system through a dedicated user interface (UI) and upload unfinished illustration files.

[1050] Input: Unfinished illustration file

[1051] Operation: Click the "Upload Line Art" button on the user interface and select an illustration file from your local disk from the file selection dialog.

[1052] Output: Temporarily save uploaded illustration files

[1053] Step 2:

[1054] The user selects style and display settings from options in the UI.

[1055] Input: Style selection, display settings

[1056] Action: Select the desired option from the "Style Selection" or "Display Settings" drop-down menu.

[1057] Output: Information about selected styles and display settings

[1058] Step 3:

[1059] An emotion recognition engine analyzes the user's emotional state.

[1060] Input: User's facial and voice data

[1061] How it works: It uses a camera and microphone to collect and analyze the user's facial expressions and voice in real time.

[1062] Output: Analyzed emotional state data

[1063] Step 4:

[1064] The terminal prepares to send the uploaded illustration file, the selected options, and the analyzed emotional state to the server.

[1065] Input: illustration file, style settings, emotional state

[1066] Operation: This data is compiled into a prompt statement format and prepared to be sent to the server.

[1067] Output: Data in prompt format

[1068] Step 5:

[1069] The server temporarily stores the received illustration file, options, and emotional state data, and then activates an AI image processing engine to process the illustration file.

[1070] Input: Data in prompt format (illustration file, style settings, emotional state)

[1071] How it works: Analyzes the received data, selects the AI ​​image processing engine that corresponds to the specified style, and colors the line drawing.

[1072] Output: Colored illustration file

[1073] Step 6:

[1074] The server sends the progress of the illustration processing to the terminal.

[1075] Input: Processing progress information

[1076] How it works: The AI ​​image processing engine reports progress in real time and sends that information to your device.

[1077] Output: Progress notification

[1078] Step 7:

[1079] The server sends a processing completion notice and a result file to the terminal.

[1080] Input: Processing completion information, result file

[1081] Operation: Once processing is complete, a notification of completion will be sent to the device along with a download link for the generated color illustration file.

[1082] Output: Processing completion notification and download link

[1083] Step 8:

[1084] The terminal displays a notification to the user that the process is complete and allows the result file to be downloaded.

[1085] Input: Processing completion notification, download link

[1086] Behavior: Displays a notification in the UI and generates a "Download" button that the user clicks to download the resulting file.

[1087] Output: Downloaded color illustration file

[1088] In this way, specific actions are performed at each step, allowing users to create manga efficiently. Interactive support is also provided according to the user's emotional state, creating a more creative work environment.

[1089] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1090] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1091] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1092] [Fourth embodiment]

[1093] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1094] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1095] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1096] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1097] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1098] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1099] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1100] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1101] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1102] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1103] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1104] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1105] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1106] This invention provides a means to efficiently automate part of the manga production process through a system that operates via a server and terminals. Specifically, it is a system that automatically processes unfinished illustration files drawn by users using an AI image processing engine to complete them in a short amount of time.

[1107] User Interface Parts

[1108] Users operate the system through a dedicated user interface (UI). Users access the system from their device's browser and upload unfinished illustration files. The UI displays an "Upload Line Art" button, and clicking this displays a file selection dialog. Here, users can select an illustration file from their local disk and upload it.

[1109] Additionally, the UI provides "Style Selection" and "Display Settings" options, allowing users to select the style and effect they desire.

[1110] Sending part of the request

[1111] The selected illustration file and settings are sent from the device to the server as an HTTP POST request, which includes the file data and the user-specified style and display settings.

[1112] Server-side processing

[1113] The server receives the illustration file and setting information sent from the device and temporarily stores them. The server then starts the AI ​​image processing engine and provides the received data to the processing engine. The AI ​​image processing engine uses a deep learning model to analyze the line drawing of the illustration and color it in the specified style.

[1114] Progress management and notifications

[1115] The server manages the processing progress of the AI ​​image processing engine and periodically checks its progress. This progress information is sent from the server to the device in real time. The device displays the received progress information on the UI, allowing the user to understand the current progress.

[1116] Returning and downloading results

[1117] When the processing is complete, the server generates a processing result file and sends a processing completion notification to the device. This completion notification includes a download link for the result file. Based on the received completion notification, the device displays a pop-up or dialog box on the UI and generates a "Download" button.

[1118] The user confirms the completion notification and clicks the download button to save the resulting file to their local disk, allowing them to obtain the colored finished illustration in a short time.

[1119] Specific examples

[1120] For example, consider the case where a user uploads a monochrome line drawing illustration and chooses to colorize it in a vibrant style.

[1121] 1. The user uses the device to click the "Upload Line Art" button on the UI to upload an unfinished monochrome line art file.

[1122] 2. The user selects the "Colorful" style from the "Style Selection" drop-down menu.

[1123] 3. The device sends the file and style information to the server.

[1124] 4. The server processes the file using an AI image processing engine and periodically sends progress updates to the device.

[1125] 5. When the progress reaches 100%, the server will send a notification of completion and a download link for the result file to the device.

[1126] 6. The device displays a completion notice to the user and provides a "Download" button.

[1127] 7. The user clicks the download button and receives the completed color illustration file.

[1128] This invention significantly reduces the time and effort required to create manga, providing an environment in which creators can concentrate on creating their work. It also enhances creators' creativity by supporting a variety of styles and settings.

[1129] The processing flow will be explained below.

[1130] Step 1:

[1131] The user accesses the terminal's user interface. The user interface can be accessed from a browser or a dedicated application, and a login screen is displayed. The user enters login information and logs into the system.

[1132] Step 2:

[1133] The user uploads a line drawing file. The user clicks the "Upload Line Drawing" button on the UI and selects a line drawing file from the local disk in the file selection dialog that appears. The device prepares to send the selected file to the server.

[1134] Step 3:

[1135] The user selects a style or display setting. The UI displays a drop-down menu of style choices and other options. The user selects the style or display setting they want.

[1136] Step 4:

[1137] The device sends the uploaded line art file and selected options to the server as an HTTP POST request, which includes the file data and style information.

[1138] Step 5:

[1139] The server receives the request, temporarily stores the received data, and saves the received line drawing file in an appropriate folder, while also recording the style information specified by the user.

[1140] Step 6:

[1141] The server starts the AI ​​image processing engine and provides the received data to the processing engine. The server selects an AI model corresponding to the specified style and processes the line drawing file.

[1142] Step 7:

[1143] The server manages the progress of the processing and periodically notifies the device. As the AI ​​image processing progresses, the progress status is reported to the server, which then periodically sends it to the device.

[1144] Step 8:

[1145] The terminal displays the received progress information on the UI so that the user can check the current processing status in real time.

[1146] Step 9:

[1147] After the process is complete, the server generates a result file and sends a notification of the completion of the process to the device, along with a download link for the result file.

[1148] Step 10:

[1149] The device displays a pop-up or dialog box on the UI based on the received completion notification, and generates a "Download" button to allow the user to download the result file.

[1150] Step 11:

[1151] The user downloads the resulting file. After confirming the completion notification, they click the download button and save it to their local disk. The completed color illustration file is provided.

[1152] This series of processes allows users to quickly and efficiently automate parts of manga production, significantly reducing production time and effort.

[1153] Example 1

[1154] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1155] Traditional manga production has the problem of requiring a great deal of time and effort to color images and reflect styles. Manual coloring, in particular, is a factor that reduces productivity, and many creators are seeking more efficient methods. Furthermore, there is often a lack of smooth communication methods for checking progress and obtaining results. To solve these issues, a system is needed that automates image processing, allowing users to check progress in real time and obtain results quickly.

[1156] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1157] In this invention, the server includes a means for uploading an unfinished image file to be drawn through a user interface, a means for selecting an optional style and display setting, a means for transmitting the uploaded image file and the selected option to the server, a means for processing the image file using an artificial intelligence image processing engine by the server, a means for transmitting a processing progress report from the server to the terminal, a means for transmitting a processing completion notice and a processing result file from the server to the terminal, and a means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file, thereby enabling the user to quickly automatically colorize and style images, check the progress in real time, and reliably obtain the results.

[1158] A "user interface" is a dedicated interface for users to perform operations, and includes, for example, buttons and menus displayed on the screen.

[1159] "Image files" refers to digital files such as line drawings and illustrations, and includes formats such as JPEG, PNG, and BMP.

[1160] "Style" refers to the design and effect settings applied in image processing, and includes different methods of expression such as "vivid colors" and "monochrome."

[1161] "Display settings" refers to various settings related to the display of images and user interfaces, including, for example, brightness and contrast adjustments.

[1162] A "server" refers to a computer system that accepts requests from client terminals via a network and processes and stores data.

[1163] "Artificial intelligence image processing engine" refers to software that analyzes and processes images using deep learning models and other artificial intelligence technologies.

[1164] "Processing progress" refers to information indicating the progress of the processing currently being performed by the image processing engine.

[1165] A "processing completion notification" is a message from the system notifying you that image processing is complete, and typically includes a download link.

[1166] The "download link" indicates the URL for the user to obtain the result file.

[1167] "Terminal" refers to an electronic device such as a computer or smartphone that is operated by a user.

[1168] The present invention is a system that automatically processes unfinished image files drawn by users using an artificial intelligence image processing engine to complete them in a short time. The system is operated through a user interface and operates via a server and terminals.

[1169] First, the user operates the system through a dedicated user interface (UI). The user accesses the system from the device's browser and uploads an unfinished image file. The UI displays an "Upload Image" button, and clicking this displays a file selection dialog. The user selects an image file from the local disk and uploads it. Furthermore, the UI provides options for "Style Selection" and "Display Settings," allowing the user to select the desired style and effect.

[1170] The image file and settings information selected by the user are sent from the device to the server as an HTTP POST request. This request includes the file data and the style and display settings specified by the user. The server receives the image file and settings information sent from the device and temporarily stores them in a temporary directory on the server. The server then starts an artificial intelligence image processing engine and provides the received data to the processing engine. The artificial intelligence image processing engine uses deep learning models such as TensorFlow and PyTorch to analyze the image and apply styles.

[1171] As the processing progresses, the server monitors the progress of the artificial intelligence image processing engine and sends progress information to the device in real time at regular intervals. The device displays the received progress information on the UI, allowing the user to understand the current progress. After the processing is complete, the server generates a processing result file and sends a processing completion notification to the device. This completion notification includes a download link for the result file. Based on the received completion notification, the device displays a pop-up or dialog box on the UI and generates a "Download" button. The user can click this download button to save the result file to their local disk.

[1172] For example, suppose a user uploads a monochrome line drawing illustration and chooses to colorize it in a vibrant style. Here is an example prompt:

[1173] Example prompt sentence:

[1174] The user uses the device to click the "Upload Image" button on the UI to upload an unfinished monochrome line drawing file. The user selects the "Vivid" style from the "Style Selection" drop-down menu. The device sends the file and style information to the server. The server processes the file using an artificial intelligence image processing engine and periodically sends progress updates to the device. Once processing is complete, the server sends a processing completion notification and a download link for the resulting file to the device. The device displays a completion notification to the user and provides a "Download" button. The user clicks the download button to receive the completed color illustration file.

[1175] This invention will significantly reduce the time and effort required to create manga, allowing creators to focus on creating their works. It will also enhance creators' creativity by supporting a variety of styles and settings.

[1176] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1177] Step 1:

[1178] The user uses a terminal to access a dedicated user interface. The user clicks the "Upload Image" button and selects and uploads an unfinished image file from the local disk. Based on the input, the UI receives the selected file "ImageFile.png". This file is uploaded and ready for the next step.

[1179] Specific behavior: A user opens a browser, accesses the system's web page, clicks the "Upload Image" button, selects "Image File.png" in the file selection dialog, and uploads it.

[1180] Step 2:

[1181] The user selects the desired style and effect from the "Style Selection" and "Display Settings" options. Based on the input, the UI receives the selected style "Vivid" and display settings. The setting information is ready to be sent to the server.

[1182] What it does: The user opens the "Style Selection" drop-down menu on the UI, selects the "Vivid" style, and then adjusts other effects in the "Display Settings" options.

[1183] Step 3:

[1184] The device sends the image file and setting information selected by the user to the server as an HTTP POST request. Based on the input, the file data "Image file.png" and style information "Vivid" arrive at the server. The server receives this data and temporarily stores it.

[1185] Specific operation: The device generates a POST request and sends it to the server's image processing API endpoint, including the file "image file.png" and the "vivid" style information. The server receives it and saves it in the " / tmp / uploads / " directory.

[1186] Step 4:

[1187] The server launches an artificial intelligence image processing engine and provides the saved image file and configuration information to the processing engine. Based on the input, a deep learning model (e.g., TensorFlow) is used to analyze and style the image. As an output, a processed image file is generated.

[1188] Specific operation: The server runs the Python script "process_image.py" to process "image_file.png" using the TensorFlow model. The process takes a few seconds to a few minutes.

[1189] Step 5:

[1190] The server monitors the processing progress of the AI ​​image processing engine. The progress status is checked at regular intervals and the progress information is sent to the terminal in real time. Based on the input, processing progress information is generated and sent to the terminal.

[1191] Specific operation: The server checks the progress of the processing every 5 seconds and sends a message containing progress information (e.g., processing progress 50%) to the terminal via WebSocket communication.

[1192] Step 6:

[1193] Once the processing is complete, the server generates a processing result file. The server then sends a processing completion notification to the device, which includes a download link for the processing result file. Based on the input, the processing result "processed_image.png" is generated and a download link is created.

[1194] Specific operation: The server generates a file called "processed_image.png" and sends a message to the terminal containing a processing completion notification and a download link (e.g., "http: / / example.com / download / processed_image.png").

[1195] Step 7:

[1196] The device displays a pop-up or dialog box on the UI based on the received processing completion notification. It generates a "Download" button to allow the user to download the result file. Based on the input, the UI displays a download link so that the user can obtain the result file.

[1197] Specific operation: The device receives the completion notification and displays a "Download" button on the UI. The user clicks this button and saves the resulting file "processed_image.png" to the local disk.

[1198] By following these steps, users can obtain a finished image that is automatically processed in a short time.

[1199] (Application example 1)

[1200] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1201] There is a need for a system that can efficiently complete unfinished design files drawn by users and instantly display the completed designs in a virtual store. Conventional methods require specialized skills and a significant amount of time for users to complete the design. In addition, displaying the completed design must be done manually, which is a time-consuming process.

[1202] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1203] In this invention, the server includes means for uploading an unfinished design file drawn through a user interface, means for selecting a design style and display settings as options, means for transmitting the uploaded design file and the selected options to the server, means for the server to process the design file using a machine learning model, means for transmitting a processing progress status from the server to the terminal, means for transmitting a processing completion notice and a processing result file from the server to the terminal, means for the terminal to display a processing completion notice to the user and allow the user to download the result file, and means for automatically uploading and displaying the completed design file to a virtual store, thereby enabling the user to efficiently complete a design and immediately display the design in the virtual store.

[1204] A "user interface" is a software component that provides a screen and input means for a user to access and operate a system.

[1205] A "design file" is a digital file representing an unfinished illustration or design drawn by a user.

[1206] "Options" are additional setting items such as design styles and display settings that can be selected by the user.

[1207] "Server" means a computer system that receives and processes design files and options uploaded by users.

[1208] "Machine Learning Model" means a computational model utilizing machine learning algorithms used to process design files.

[1209] "Processing progress" is information that the server uses to notify the user of the processing status of the design file in real time.

[1210] The "processing completion notification" is notification information that the server uses to notify the user that processing of the design file has been completed.

[1211] "Result File" means the final design file processed by the machine learning model.

[1212] A "virtual store" is an online platform such as a shop or exhibition hall that is virtually set up on the Internet.

[1213] A system for implementing the present invention comprises a user interface, a server, a machine learning model, a terminal, and a virtual store.

[1214] Users upload unfinished design files through their device's browser. The user interface displays a "File Upload" button, which when clicked displays a file selection dialog, allowing users to select and upload a design file from their local disk. In addition, the user interface provides options for "Design Style Selection" and "Display Settings," allowing users to select the style and effect they desire.

[1215] The selected design file and settings information are sent from the device to the server as an HTTP POST request. This request includes the file data and the style and display settings specified by the user. The server receives this data and temporarily stores it.

[1216] The server then launches an environment for the machine learning model (for example, one using TensorFlow or PyTorch) and provides the received data to the model, which uses deep learning techniques to analyze the design file and colorize it in the specified style.

[1217] The server manages the progress of this processing and periodically transmits progress information to the terminal in real time. The terminal displays the received progress information on a user interface, allowing the user to grasp the current progress.

[1218] When the processing is complete, the server generates a processing result file and sends a processing completion notification to the terminal. This completion notification includes a download link for the result file. Based on the received completion notification, the terminal displays a pop-up or dialog box on the user interface and generates a "Download" button. The user can click this to save the completed design file to their local disk.

[1219] In addition, the completed design file is automatically uploaded to the virtual store by the server and instantly displayed online, allowing users to instantly share and exhibit their created designs.

[1220] A concrete example scenario might be a scenario where a user uses smart glasses to take a photo of a sketch character they've drawn in the real world, then uploads it to an app and specifies a "fantasy style" coloring. An example prompt for this would be:

[1221] "Color the line art below in a vibrant, fantasy style."

[1222] Contains the user's line drawing illustration data.

[1223] This system allows users to easily complete designs and instantly display them in a virtual store.

[1224] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1225] Step 1:

[1226] The user opens the user interface via a browser on the device. As input, the user uses the browser on the device. The user clicks the "File Upload" button and selects and uploads an unfinished design file from the local disk in the file selection dialog. As output, the selected design file is obtained.

[1227] Step 2:

[1228] On the user interface, the user selects the desired style from the "Select Design Style" drop-down menu. As input, there is the style information selected by the user. As output, the design file together with the style information is sent to the server.

[1229] Step 3:

[1230] The device sends the uploaded design file and selected style information to the server as an HTTP POST request. The file data and style setting information from the device are used as input. The request arrives at the server as output.

[1231] Step 4:

[1232] The server receives and temporarily stores design files and style information. The input is the file and information sent from the device. The output is the file and information saved in the server's storage.

[1233] Step 5:

[1234] The server launches a machine learning model (e.g., a model using TensorFlow or PyTorch) and provides it with the incoming data. As input, it uses the saved design file and style information. The machine learning model processes the data and colorizes it with the specified style. As output, it generates the processed finished design file.

[1235] Step 6:

[1236] The server manages the processing progress and periodically sends progress information to the device. The input is the processing status of the machine learning model. The output is the progress information sent from the server to the device via data communication.

[1237] Step 7:

[1238] The progress information received by the terminal is displayed on the user interface in real time. As input, there is progress information sent from the server. As output, there is real-time progress displayed on the user interface.

[1239] Step 8:

[1240] When the server completes the process, it generates a result file and sends a notification of completion to the device. The input is the completed design file processed by the machine learning model. The output is a notification of completion and a download link provided by the server to the device.

[1241] Step 9:

[1242] Based on the completion notification received by the terminal, a pop-up or dialog box is displayed on the user interface and a "Download" button is generated. The input is a completion notification from the server. The output is a pop-up or dialog box displayed on the user interface.

[1243] Step 10:

[1244] The user clicks the "Download" button to save the completed design file to their local disk. As input, there is a "Download" button in a popup or dialog box. As output, there is a completed design file saved to their local disk.

[1245] Step 11:

[1246] The server automatically uploads the completed design file to the virtual store and displays it. The input is the processed completed design file. The output is the design file displayed in the virtual store.

[1247] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1248] This invention is a system that reduces the burden on users and improves efficiency in manga production, and in particular, it employs a configuration that combines an AI image processing engine and an emotion engine. Specifically, it is a system that improves the user experience by recognizing the user's emotions and providing dynamic responses accordingly.

[1249] User Interface Parts

[1250] Users operate the system through a dedicated user interface (UI), which can be accessed from a browser or a dedicated application. Users can access the system from a login screen and perform the necessary operations.

[1251] Example: A user opens a browser, accesses the system's URL, enters login information, and logs in.

[1252] Sending part of a user request

[1253] 1. The user uploads a line art file. The UI displays an "Upload Line Art" button, and clicking it displays a file selection dialog. The user selects an illustration file from their local disk and uploads it.

[1254] 2. The user selects the style and display settings. Select the desired option from the "Style Selection" and "Display Settings" options displayed on the UI.

[1255] 3. The device prepares to send the line drawing file and selected options to the server.

[1256] Example: A user uploads an unfinished line art file and selects a colorful style.

[1257] The working parts of the emotion engine

[1258] The emotion engine has the ability to analyze the user's facial expressions and voice. This engine works in conjunction with the user interface to recognize the user's emotions in real time.

[1259] Example: The user allows the camera and microphone, and the emotion engine recognizes joy or displeasure from the user's facial expressions and voice.

[1260] The emotion engine dynamically adjusts the UI based on the emotional data it recognizes. If the user is dissatisfied, the UI will respond by suggesting assistance.

[1261] Server-side processing

[1262] 1. The server receives the request sent from the device and temporarily stores it.

[1263] 2. The server starts the AI ​​image processing engine to process the received data, and selects and operates different AI models depending on the specified style.

[1264] Example: The server selects an AI model that corresponds to a colorful style and processes a line art file.

[1265] 3. The server periodically notifies the terminal of the progress of the processing, and the progress information is displayed to the user in real time.

[1266] Returning and downloading results

[1267] 1. After the processing is complete, the server generates a result file and sends a notification to the device along with a download link for the result file.

[1268] 2. The device notifies the user on the UI based on the received completion notification and generates a download button, which the user can click to download the result file.

[1269] Example: The user sees the completion notification and clicks the download button to receive the completed color illustration file.

[1270] The system's unique feature is its ability to improve the user experience by incorporating an emotion engine that dynamically responds to the user's emotional state. The emotion engine recognizes the user's emotional state and uses that information to change the screen display and operation guides, supporting efficient and smooth work progress. Furthermore, by linking with an AI image processing engine, the entire manga production process can be automated, significantly shortening production time.

[1271] This system reduces the amount of tedious manual work required for users, providing an environment where users can focus more on creative work. It also enables efficient workflow, allowing users to quickly produce high-quality works.

[1272] The processing flow will be explained below.

[1273] Step 1:

[1274] The user opens the user interface (UI) of the device, launches a browser or a dedicated application, and accesses the system login screen. The user enters the username and password to log in.

[1275] Step 2:

[1276] The user uploads an unfinished illustration file by clicking the "Upload Line Art" button on the UI, selecting a line art file saved on the local disk from the file selection dialog, and uploading it.

[1277] Step 3:

[1278] The user selects the style and display settings. Select the desired style or effect from the "Style Selection" drop-down menu or other options in the UI. The device records the selection.

[1279] Step 4:

[1280] The device sends the uploaded illustration file and the selected style settings to the server as an HTTP POST request, which includes the file data and style information.

[1281] Step 5:

[1282] The server receives the request and temporarily saves the file data. The server saves the received illustration file in a specified folder and records the style setting information.

[1283] Step 6:

[1284] The server activates the emotion engine and analyzes the user's facial expressions and voice data sent from the user interface. The server recognizes the user's emotional state in real time through the emotion engine.

[1285] Step 7:

[1286] The server dynamically adjusts the UI display based on the emotional data it recognizes. For example, if the user is frustrated, a support message or assistance options will be displayed.

[1287] Step 8:

[1288] The server processes the illustration file using an AI image processing engine, selects the corresponding AI model based on the specified style information, and applies coloring and effects to the line art file.

[1289] Step 9:

[1290] The server manages the progress of the process in real time and notifies the device. Each time the progress of the process is updated, the progress is sent to the device.

[1291] Step 10:

[1292] The terminal displays the received progress information on the user interface, allowing the user to check the progress of the processing in real time.

[1293] Step 11:

[1294] When the server completes the process, it generates a result file and sends a notification to the device, which includes a download link for the result file.

[1295] Step 12:

[1296] The terminal displays a completion notification to the user and provides a download button, after which the user can confirm the completion notification and click the download button to download the resulting file.

[1297] Step 13:

[1298] The user clicks the download button to save the completed color illustration file to their local disk. The user can then check the completed work and further process or use it as needed.

[1299] This system allows users to quickly and efficiently automate parts of manga production, significantly reducing production time and effort. Furthermore, by combining it with an emotion engine, it provides flexible support according to the user's state, improving the user experience.

[1300] Example 2

[1301] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1302] The manga production process is extremely time-consuming and labor-intensive, requiring a great deal of manual work. As a result, the burden on the workers increases, leading to problems with reduced production efficiency. Another issue is that the system cannot adequately respond to changes in the user's emotions that occur during the process, resulting in a poor user experience. Furthermore, efficient progress management is required to quickly produce high-quality works.

[1303] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1304] In this invention, the server includes means for uploading unfinished image files to be drawn through a user interface, means for selecting appearances and settings as options, means for transmitting the uploaded image files and the selected options to the server, means for processing the image files using an artificial intelligence image processing engine by the server, means for recognizing emotions from the user's facial expressions and voice using an emotion engine, means for dynamically adjusting the user interface based on the user's emotional state, means for transmitting a processing progress status from the server to the terminal, means for transmitting a processing completion notice and a processing result file from the server to the terminal, and means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file. This allows the system to dynamically respond to the user's emotional state, enabling efficient progress management and rapid high-quality manga production.

[1305] A "user interface" is a screen or application that allows a user to interact with a system and perform various operations.

[1306] An "unfinished image file" is an image that is being uploaded by a user and is in a state before processing or editing.

[1307] "Options" are appearance and setting choices that the user can specify, including image processing styles, display settings, and the like.

[1308] A "server" is a computer system that receives requests from users and processes images and stores and manages data.

[1309] An "artificial intelligence image processing engine" is software equipped with AI technology used to analyze and process image data.

[1310] An "emotion engine" is a technology that recognizes emotions from a user's facial expressions and voice, and adjusts the system's behavior based on that emotional state.

[1311] "Progress" is information indicating the progress of image processing, and displays in real time how far the processing has progressed.

[1312] The "processing completion notification" is a message for notifying the user that image processing has been completed.

[1313] "Processing result file" refers to the final image file that has been processed and edited by the AI ​​image processing engine.

[1314] A "terminal" is a device that a user uses to access the system and perform various operations and check the processing results.

[1315] "Downloading" refers to the act of saving a processing result file sent from a server to a terminal.

[1316] "Dynamic adjustment" means changing the system's display and operation methods in real time according to the user's emotions, progress, etc.

[1317] This system reduces the burden on users and improves efficiency in manga production, and in particular employs a configuration that combines an image processing engine and an emotion engine that utilize artificial intelligence. The entire system operates through a combination of a user interface, terminals, and a server.

[1318] First, the user operates the system through a dedicated user interface (UI). This UI can be accessed from a standard web browser or a dedicated application. The user enters their authentication information on the login screen and logs into the system. After logging in, the user is redirected to the dashboard, where they can click the "Upload Line Art" button to select and upload an unfinished image file from their local disk.

[1319] Next, the user selects style and display settings as options on the UI. Specifically, for example, they select "Watercolor" from the "Style Selection" menu, and then adjust "Brightness" and "Contrast" in the display settings options. This selected data is then prepared by the device to be sent to the server. For transmission, the HTTP protocol or other communication technology is used.

[1320] After receiving the request from the device, the server temporarily stores the data in storage. It then launches an artificial intelligence image processing engine using an AI framework such as TensorFlow or PyTorch to process the received data. The appropriate AI model is selected based on the specified style, and processing is performed. For example, if the user selects a "watercolor" style, the AI ​​model corresponding to the style is applied, converting the line art file into a beautiful watercolor-style color illustration.

[1321] The emotion engine analyzes the user's facial expressions and voice while they are operating the device to recognize their current emotions. This function collects data in real time through the camera and microphone and uses machine learning algorithms to determine their emotional state. For example, if the user allows the camera and microphone to be used, the emotion engine recognizes emotions such as "happiness" or "dissatisfaction" from the user's facial expressions. Based on the recognized emotional state, the system dynamically adjusts the UI. Specifically, if the user is dissatisfied, the UI will automatically suggest assistance.

[1322] The server periodically notifies the device of the progress of image processing, which is displayed in real time on the UI. This allows the user to check the progress of the processing. The progress information is displayed as a message such as "50% complete." Finally, after processing is complete, the server generates a result file and sends a processing completion notification to the device. This notification also includes a download link for the result file. The user can receive the notification and download the result file by clicking the "Download" button on the UI.

[1323] In this way, this system significantly reduces the user's manual work and supports high-quality manga production by utilizing generative AI models. Furthermore, the introduction of an emotion engine enables dynamic responses according to the user's emotional state, providing a better user experience.

[1324] Examples and prompts:

[1325] As a concrete example, we will describe how a user accesses the system, uploads a line drawing file, selects "watercolor" as the style, and authorizes the camera and microphone. We will also explain the process in which the emotion engine recognizes the user's frustration and the UI suggests assistance. Finally, we will explain the entire process in which the user downloads the watercolor color illustration file generated by the server after processing is complete.

[1326] Example prompt sentence:

[1327] "Please explain in detail the process by which a user opens a browser, logs into the system, uploads a line drawing file, and selects style and display settings. Also, please explain in detail the entire process by which the emotion engine recognizes the user's frustration, the server processes the image using an AI image processing engine, and finally the user downloads the resulting file."

[1328] This system reduces the amount of tedious manual work required for users, providing an environment where users can focus more on creative work. It also enables efficient workflow, allowing users to quickly produce high-quality works.

[1329] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1330] Step 1:

[1331] A user accesses the system from a browser or a dedicated application and enters their username and password on the login screen. The entered authentication information is sent to the server, and if authentication is successful, the dashboard screen is displayed. When login is successful, the output is the dashboard screen.

[1332] Specific operation: The user opens a browser, accesses the system's URL, enters login information, and clicks the "Login" button.

[1333] ---

[1334] Step 2:

[1335] The user clicks the "Upload Line Art" button on the dashboard, selects an unfinished image file from the local disk, and uploads it. At this time, the file path and file name are also sent to the server, which receives and temporarily saves the file. The input is the unfinished image file selected by the user, and the output is the unfinished image file saved on the server.

[1336] Specific operation: The user clicks the "Upload line art" button on the dashboard, selects the appropriate line art file from the file selection dialog, and uploads it.

[1337] ---

[1338] Step 3:

[1339] Select the style and display settings for the image file uploaded by the user. Select the desired style from the "Style Selection" menu and make detailed settings with the "Display Settings" option. The input is the style and display settings selected by the user, and the output is the style and display setting information.

[1340] Specific behavior: The user selects "Watercolor" from the "Style Selection" menu and adjusts "Brightness" and "Contrast" in the display settings options.

[1341] ---

[1342] Step 4:

[1343] The device sends the uploaded image file and the style and display settings selected by the user to the server. This sending process uses HTTP requests and API calls to securely transfer data. The input is the image file and style settings, and the output is the data sent to the server.

[1344] Specific operation: The device sends the line drawing file and the "watercolor" style setting to the server via an HTTP request.

[1345] ---

[1346] Step 5:

[1347] The emotion engine recognizes the user's facial expressions and voice and analyzes emotions in real time. It uses machine learning algorithms to determine the user's emotional state based on data collected from the camera and microphone. The input is data from the camera and microphone, and the output is the recognized emotional state.

[1348] Specific operation: The user allows the camera and microphone, and the emotion engine analyzes emotions such as "happiness" or "satisfaction" from the user's facial expression.

[1349] ---

[1350] Step 6:

[1351] The server launches an artificial intelligence image processing engine and processes the received image file in the specified style. It uses TensorFlow and PyTorch to analyze and process the image data. The input is the image file and style settings uploaded by the user, and the output is the processed image file.

[1352] Specific operation: The server applies the "watercolor style" model to process the unfinished image file into a color illustration.

[1353] ---

[1354] Step 7:

[1355] The server notifies the terminal of the progress of the processing. The terminal reflects the received progress status in the user interface in real time and displays it to the user. The input is the progress information calculated by the server, and the output is the progress status displayed on the terminal.

[1356] Specific operation: When the server has completed 50% of the processing, it notifies the terminal of the progress and displays the message "50% complete" on the UI.

[1357] ---

[1358] Step 8:

[1359] After the processing is complete, the server generates a result file and sends a processing completion notification to the device. This notification includes a download link for the result file. The input is the processed image file, and the output is the completion notification and the download link.

[1360] What it does: The server generates the final watercolor-style color illustration file and sends a notification saying "Processing complete. You can download it here" along with a link.

[1361] ---

[1362] Step 9:

[1363] The user receives a notification of the completion of the process and clicks the "Download" button on the UI to download the generated result file. The input is the completion notification from the server, and the output is the result file saved locally by the user.

[1364] Specific operation: The user confirms the notification and clicks the "Download" button on the UI to save the generated watercolor-style color illustration file.

[1365] (Application example 2)

[1366] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1367] In manga production, there is a need to reduce the burden on users and improve efficiency. In particular, there is a lack of systems that provide dynamic support according to the user's emotional state and enable smooth progress in the work, making it difficult to provide an environment where users can concentrate on creative work. Another issue is that existing systems only provide simple functions without taking the user's emotional state into consideration.

[1368] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for uploading an unfinished illustration file to be drawn through a user interface; means for selecting an optional style and display settings; means for analyzing the user's emotional state using an emotion recognition engine; means for transmitting the uploaded illustration file, the selected options, and the analyzed emotional state to the server; means for processing the illustration file using an AI image processing engine by the server; means for transmitting a processing progress status from the server to the terminal; means for transmitting a processing completion notice and a processing result file from the server to the terminal; means for dynamically adjusting the user interface according to the emotional state; and means for displaying a processing completion notice to the user on the terminal and allowing the user to download the result file. This provides interactive support according to the user's emotional state, improves the efficiency of creative work, and enables the user to engage in creative activities with a sense of satisfaction.

[1369] (definition statement)

[1370] A "user interface" is an interface that provides a screen and operating means for a user to access and operate a system.

[1371] An "illustration file" refers to image data that a user is currently creating or has completed, and is specifically a file that contains manga line drawings and sketches.

[1372] "Options" are choices that a user can set within the system, including style and display settings.

[1373] An "emotion recognition engine" is an engine that analyzes a user's facial expressions and voice to recognize their emotional state in real time.

[1374] The "AI image processing engine" is an engine that uses artificial intelligence to process uploaded illustration files based on a specific style.

[1375] "Progress" is information that indicates the progress of work while the system is processing it.

[1376] The "processing completion notification" is a notification that notifies the user that the server has completed processing of the illustration file.

[1377] "Result file" refers to the illustration file after it has been processed by the AI ​​image processing engine.

[1378] "Dynamic adjustment" means that the display content and operation guide of the user interface are changed according to the emotional state of the user.

[1379] A "terminal" is a device used by a user to access the system, including a smartphone, a personal computer, etc.

[1380] This invention is a system for reducing the user's burden and improving efficiency in manga production. In particular, it has a function to provide dynamic support according to the user's emotional state. This system consists of a user interface (UI), an emotion recognition engine, an AI image processing engine, a server, and a terminal.

[1381] Hardware and software used

[1382] Hardware: Smartphone, PC (with camera and microphone)

[1383] software:

[1384] AI image processing engine: TensorFlow

[1385] Emotion recognition engine: OpenCV (facial expression recognition), Google Cloud Speech-to-Text API (voice recognition)

[1386] Web server: Node.js

[1387] System configuration

[1388] 1. User Interface (UI):

[1389] Users can upload unfinished illustration files and select style and display settings through a dedicated UI, which can be accessed from a browser or a dedicated application.

[1390] 2. Emotion Recognition Engine:

[1391] The emotion recognition engine analyzes the user's facial expressions and voice in real time to recognize their emotional state. For example, when a user accesses the emotion recognition engine using a camera and microphone, the system can detect emotions such as joy or displeasure.

[1392] 3. AI image processing engine:

[1393] The AI ​​image processing engine running on the server processes the uploaded illustration file according to the selected style, allowing the illustration to be automatically colored in the style specified by the user.

[1394] 4. Server:

[1395] The server temporarily stores the illustration file and settings information received from the user, then starts the AI ​​image processing engine to process it. It also has a function to send progress and processing completion notifications to the terminal.

[1396] 5. Terminal:

[1397] The terminal is the device that the user uses to access the system, such as a smartphone or PC. The terminal displays the progress and processing completion notifications sent from the server to the user in real time, and allows the user to download the result file.

[1398] Specific examples

[1399] Consider a scenario where a user uploads a line art file and selects the "Realistic Style." The user enters the following prompt:

[1400] Example prompt sentence:

[1401] "User ID: example_user_id, Request type: Image processing, Style: Realistic, Image file path: / path / to / uploaded / line_art.png"

[1402] This prompt is an instruction for the system's AI image processing engine to select an appropriate model and color the illustration. Additionally, if the user allows the camera and microphone, the emotion recognition engine will recognize joy or displeasure from facial expressions and voice in real time and dynamically adjust the UI, significantly improving user efficiency.

[1403] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1404] (Processing steps of patent embodiment)

[1405] Step 1:

[1406] Users access the system through a dedicated user interface (UI) and upload unfinished illustration files.

[1407] Input: Unfinished illustration file

[1408] Operation: Click the "Upload Line Art" button on the user interface and select an illustration file from your local disk from the file selection dialog.

[1409] Output: Temporarily save uploaded illustration files

[1410] Step 2:

[1411] The user selects style and display settings from options in the UI.

[1412] Input: Style selection, display settings

[1413] Action: Select the desired option from the "Style Selection" or "Display Settings" drop-down menu.

[1414] Output: Information about selected styles and display settings

[1415] Step 3:

[1416] An emotion recognition engine analyzes the user's emotional state.

[1417] Input: User's facial and voice data

[1418] How it works: It uses a camera and microphone to collect and analyze the user's facial expressions and voice in real time.

[1419] Output: Analyzed emotional state data

[1420] Step 4:

[1421] The terminal prepares to send the uploaded illustration file, the selected options, and the analyzed emotional state to the server.

[1422] Input: illustration file, style settings, emotional state

[1423] Operation: This data is compiled into a prompt statement format and prepared to be sent to the server.

[1424] Output: Data in prompt format

[1425] Step 5:

[1426] The server temporarily stores the received illustration file, options, and emotional state data, and then activates an AI image processing engine to process the illustration file.

[1427] Input: Data in prompt format (illustration file, style settings, emotional state)

[1428] How it works: Analyzes the received data, selects the AI ​​image processing engine that corresponds to the specified style, and colors the line drawing.

[1429] Output: Colored illustration file

[1430] Step 6:

[1431] The server sends the progress of the illustration processing to the terminal.

[1432] Input: Processing progress information

[1433] How it works: The AI ​​image processing engine reports progress in real time and sends that information to your device.

[1434] Output: Progress notification

[1435] Step 7:

[1436] The server sends a processing completion notice and a result file to the terminal.

[1437] Input: Processing completion information, result file

[1438] Operation: Once processing is complete, a notification of completion will be sent to the device along with a download link for the generated color illustration file.

[1439] Output: Processing completion notification and download link

[1440] Step 8:

[1441] The terminal displays a notification to the user that the process is complete and allows the result file to be downloaded.

[1442] Input: Processing completion notification, download link

[1443] Behavior: Displays a notification in the UI and generates a "Download" button that the user clicks to download the resulting file.

[1444] Output: Downloaded color illustration file

[1445] In this way, specific actions are performed at each step, allowing users to create manga efficiently. Interactive support is also provided according to the user's emotional state, creating a more creative work environment.

[1446] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1447] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1448] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1449] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1450] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1451] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1452] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1453] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1454] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1455] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1456] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1457] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1458] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1459] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1460] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1461] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1462] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1463] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1464] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1465] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1466] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1467] The following is further disclosed regarding the above embodiment.

[1468] (Claim 1)

[1469] a means for uploading unfinished illustration files to be drawn through a user interface;

[1470] Optionally, you can select styles and display settings,

[1471] means for transmitting the uploaded illustration file and the selected options to a server;

[1472] A means for processing illustration files using an AI image processing engine by a server;

[1473] A means for transmitting a progress status of the processing from the server to the terminal;

[1474] means for transmitting a processing completion notice and a processing result file from the server to the terminal;

[1475] means for displaying a notification to the user by the terminal that processing has been completed and allowing the user to download the result file;

[1476] A system including:

[1477] (Claim 2)

[1478] 10. The system of claim 1, wherein the server includes means for selecting and operating different AI image processing engines based on a style selection specified by a user.

[1479] (Claim 3)

[1480] 2. The system of claim 1, wherein the terminal includes means for displaying the progress transmitted from the server to the user in real time.

[1481] "Example 1"

[1482] (Claim 1)

[1483] means for uploading unfinished image files to be rendered through the user interface;

[1484] Optionally, you can select styles and display settings,

[1485] means for transmitting the uploaded image file and the selected options to a server;

[1486] means for processing the image file by the server using an artificial intelligence image processing engine;

[1487] A means for transmitting a progress status of the processing from the server to the terminal;

[1488] means for transmitting a processing completion notice and a processing result file from the server to the terminal;

[1489] means for displaying a notification to the user by the terminal that processing has been completed and allowing the user to download the result file;

[1490] A system including:

[1491] (Claim 2)

[1492] 10. The system of claim 1, wherein the server includes means for selecting and operating different artificial intelligence image processing engines based on style selections specified by a user.

[1493] (Claim 3)

[1494] 10. The system of claim 1, wherein the terminal includes means for displaying the progress transmitted from the server to the user in real time.

[1495] "Application Example 1"

[1496] Scope of the new claims

[1497] (Claim 1)

[1498] means for uploading unfinished design files to be drawn through the user interface;

[1499] Optionally, you can choose design styles and display settings.

[1500] means for transmitting the uploaded design file and selected options to a server;

[1501] means for processing, by a server, the design file using the machine learning model;

[1502] A means for transmitting a progress status of the processing from the server to the terminal;

[1503] means for transmitting a processing completion notice and a processing result file from the server to the terminal;

[1504] means for displaying a notification to the user by the terminal that processing has been completed and allowing the user to download the result file;

[1505] A means to automatically upload and display completed design files in a virtual store,

[1506] A system including:

[1507] (Claim 2)

[1508] 10. The system of claim 1, wherein the server includes means for selecting and running different machine learning models based on design style selections specified by a user.

[1509] (Claim 3)

[1510] 2. The system of claim 1, wherein the terminal includes means for displaying the progress transmitted from the server to the user in real time.

[1511] "Example 2: Combining Emotion Engines"

[1512] (Claim 1)

[1513] means for uploading unfinished image files to be rendered through the user interface;

[1514] Optional means to select appearance and settings,

[1515] means for transmitting the uploaded image file and the selected options to a server;

[1516] means for processing the image file by the server using an artificial intelligence image processing engine;

[1517] A means for recognizing emotions from a user's facial expressions and voice using an emotion engine;

[1518] means for dynamically adjusting a user interface based on the emotional state of a user;

[1519] A means for transmitting a progress status of the processing from the server to the terminal;

[1520] means for transmitting a processing completion notice and a processing result file from the server to the terminal;

[1521] means for displaying a notification to the user by the terminal that processing has been completed and allowing the user to download the result file;

[1522] A system including:

[1523] (Claim 2)

[1524] 10. The system of claim 1, wherein the server includes means for selecting and operating different artificial intelligence image processing engines based on style selections specified by a user.

[1525] (Claim 3)

[1526] 2. The system of claim 1, wherein the terminal includes means for displaying the progress transmitted from the server to the user in real time.

[1527] "Application example 2 when combining emotion engines"

[1528] Claims

[1529] (Claim 1)

[1530] a means for uploading unfinished illustration files to be drawn through a user interface;

[1531] Optionally, you can select styles and display settings,

[1532] means for analyzing the emotional state of a user by an emotion recognition engine;

[1533] means for transmitting the uploaded illustration file, the selected options, and the analyzed emotional state to a server;

[1534] A means for processing illustration files using an AI image processing engine by a server;

[1535] A means for transmitting a progress status of the processing from the server to the terminal;

[1536] means for transmitting a processing completion notice and a processing result file from the server to the terminal;

[1537] means for dynamically adjusting a user interface in response to an emotional state;

[1538] means for displaying a notification to the user by the terminal that processing has been completed and allowing the user to download the result file;

[1539] A system including:

[1540] (Claim 2)

[1541] 10. The system of claim 1, wherein the server includes means for selecting and operating different AI image processing engines based on a style selection specified by a user.

[1542] (Claim 3)

[1543] 2. The system of claim 1, wherein the terminal includes means for displaying the progress transmitted from the server to the user in real time. [Explanation of symbols]

[1544] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for uploading unfinished illustration files to be drawn through a user interface; Optionally, a means to select style and display settings, means for transmitting the uploaded illustration file and the selected options to a server; A means for processing illustration files using an AI image processing engine by a server; A means for transmitting a progress status of the processing from the server to the terminal; means for transmitting a processing completion notice and a processing result file from the server to the terminal; means for displaying a notification to the user by the terminal that processing has been completed and allowing the user to download the result file; A system including:

2. 10. The system of claim 1, wherein the server includes means for selecting and operating different AI image processing engines based on style selections specified by a user.

3. 2. The system of claim 1, wherein the terminal includes means for displaying to the user in real time the progress status transmitted from the server.

Images