System

Abstract

To provide a system.SOLUTION: A system contains: an emotion engine for recognizing an emotion of a user; learning means of learning a conversation style in a daily conversation of the user by using a deep learning model and the emotion engine; response generation means of generating a response to an utterance of the user on the basis of the learned conversation style and the emotion by inputting a specific prompt text into a generative AI model; and generation means of generating a content to be recommended to the user on the basis of the learned conversation style.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Although conventional AI has the ability to learn from users' daily conversations, chat content, and service logs and respond in the same way as the user would, it lacks the means to efficiently share the results of its learning, and also lacks the ability to generate images and videos that users desire. [Means for solving the problem]

[0005] The system of the present disclosure includes an emotion engine that recognizes a user's emotions, a learning means that learns the user's daily conversation style using a deep learning model and the emotion engine, a response generation means that generates a response to the user's utterances based on the learned conversation style and the emotions by inputting a specific prompt sentence into a generative AI model, and a generation means that generates recommended content for the user based on the learned conversation style. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12]FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. [Figure 19] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system according to the embodiment 2 of the second embodiment in the case where an emotion engine is combined. [Figure 20] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when combined with an emotion engine. [Figure 21] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the embodiment 3 of the form example 3 in the case where an emotion engine is combined. [Figure 22] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In the present invention, a deep learning model is used as a learning means for learning the contents of daily conversations and chats of users and service logs. Specifically, the user's conversation log is used as input data, and the model is trained to imitate the conversation style. This training is performed using a technique called fine-tuning.

[0029] "Example 2"

[0030] The trained deep learning model is used as a reply generator that replies like a user based on the learned content. Specifically, for new input, a reply is generated using the trained model. This reply is based on the trained conversation style of the user.

[0031] "Example 3"

[0032] A generative deep learning model is used as a means of generating desired images and videos. Specifically, specific images and videos are generated based on instructions from a user. For example, if a user instructs the model to "generate an image of a cat," the model generates an image of a cat.

[0033] "Example 4"

[0034] A cloud storage service is used as a means of sharing the training file in a general file format of several MB. Specifically, the parameters of the trained model are converted into a general file format and the file is uploaded to the cloud storage service. Users can use the training results by downloading the file and importing it into their own systems.

[0035] The process flow of each embodiment will be described below.

[0036] "Example 1"

[0037] Step 1: Collect users' daily conversations, chat content, and service logs.

[0038] Step 2: Use the collected data as input data for a deep learning model.

[0039] Step 3: Train the model to learn to mimic the user's speaking style. This training is done using a technique called fine-tuning.

[0040] "Example 2"

[0041] Step 1: Receive new input (e.g., a question or comment from the user).

[0042] Step 2: Use the trained deep learning model to generate a response to the input.

[0043] Step 3: Providing the generated response to the user, which is based on the learned conversation style of the user.

[0044] "Example 3"

[0045] Step 1: Receive a command from a user (e.g., "generate an image of a cat").

[0046] Step 2: Use a generative deep learning model to generate images and videos based on the instructions.

[0047] Step 3: Provide the generated images and videos to the user.

[0048] "Example 4"

[0049] Step 1: Convert the trained model parameters into a common file format.

[0050] Step 2: Upload the converted file to a cloud storage service.

[0051] Step 3: Users can use the learning results by downloading the uploaded files and importing them into their own systems.

[0052] Example 1

[0053] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0054] Conventional conversation generation systems have had difficulty generating natural responses that fully reflect the individual conversation style of the user. In addition, there was a lack of efficient means to carry out a series of processes such as preprocessing of training data, fine-tuning of the model, evaluation, and deployment. This limited the user experience and led to problems such as a decline in the quality of the system's responses.

[0055] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0056] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like a user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several MB, a preprocessing means for the learning means for preprocessing the collected conversation logs and deleting unnecessary information, a training means for the response generating means for training to imitate the user's conversation style using a deep learning model, a fine-tuning means for the training means for fine-tuning the model using a fine-tuning method, an evaluation means for the response generating means for evaluating the trained model, an accuracy confirmation means for the evaluation means for confirming the accuracy of the model using test data, a deployment means for the response generating means for deploying the model that has been evaluated, and an embedding means for the deployment means for incorporating the model into an actual service. This allows for generating natural responses that reflect the individual conversation style of the user, and enables efficient data processing and model training, evaluation, and deployment. The system includes a server including an emotion engine that recognizes the user's emotions, a learning means that learns the conversation style of the user's daily conversation using a deep learning model and the emotion engine, a response generating means that generates a response to the user's utterance based on the learned conversation style and the emotions by inputting a specific prompt sentence into a generation AI model, and a generating means that generates recommended content for the user based on the learned conversation style. The server may include an uploading means that uploads the content to a cloud storage service. When the emotion engine recognizes the user's emotion of joy from the user's daily conversation, the response generating means may generate a response to the user's emotion of joy.

[0057] "User" refers to an individual who uses the system to engage in everyday conversations and chats.

[0058] "Conversation log" refers to data that records the content of conversations and chats conducted by users.

[0059] "Service log" refers to the history and operation records of services used by a user.

[0060] The "learning means" refers to a method or device for learning the contents of users' daily conversations and chats, and service logs.

[0061] "Response generation means" refers to a method or device for generating responses based on what has been learned in the same way as a user would.

[0062] "Generator" refers to a method or device for generating a desired image or video.

[0063] "Sharing means" refers to a method or device for sharing learning files in a common file format of several MB.

[0064] "Preprocessing means" refers to a method or device for preprocessing the collected conversation logs and deleting unnecessary information.

[0065] "Training means" refers to a method or apparatus for training a deep learning model to mimic a user's speaking style.

[0066] "Fine-tuning means" refers to a method or device for fine-tuning a model using a fine-tuning technique.

[0067] "Evaluation means" refers to a method or device for evaluating a trained model.

[0068] "Accuracy verification means" refers to a method or device for verifying the accuracy of a model using test data.

[0069] "Deployment means" refers to a method or device for deploying a model after evaluation has been completed.

[0070] "Integration means" refers to a method or device for incorporating a deployed model into an actual service.

[0071] A "deep learning model" refers to a model that uses a multi-layer neural network to learn data and make predictions or generate results.

[0072] "Fine-tuning" refers to a technique that takes a pre-trained model and fine-tunes it to suit a specific task or dataset.

[0073] This invention is a system that uses a generative AI model that learns the contents of a user's daily conversations and chats, and service logs, and imitates the user's conversation style. A specific embodiment of this system is described below.

[0074] Hardware and Software Used

[0075] Hardware: A server equipped with a high-performance GPU (e.g. NVIDIA Tesla V100)

[0076] Software: Deep learning frameworks (e.g. TENSORFLOW(R)、PyTorch)

[0077] Data collection

[0078] User: For everyday conversations and chats.

[0079] Terminal: Logs user conversations and chat content.

[0080] Server: Receives the conversation logs sent from the devices and stores them in a database.

[0081] Specific operation: When a user sends a message such as "The weather is nice today" in a chat app, the content is recorded on the device and sent to the server.

[0082] Data Preprocessing

[0083] Server: Preprocesses the collected conversation logs, specifically removing unnecessary information (e.g. emojis, special characters) and cleaning the text data.

[0084] Specific operation: Remove emojis from the message "The weather is nice today ○ (○ represents a specific emoji)" and generate clean text data "The weather is nice today."

[0085] Training the model

[0086] Server: Train the deep learning model using the preprocessed data, using fine-tuning techniques.

[0087] How it works: Based on a pre-trained GPT-3 model, it fine-tunes the model to match the user's conversational style, for example by learning the user's frequently used phrases and expressions.

[0088] Evaluating the model

[0089] Server: Evaluates the trained model by checking the accuracy of the model using test data.

[0090] Specific behavior: Using the user's past conversation logs as test data, we evaluate how natural the generated responses are. For example, we check whether the response generated for the input "Where are you going today?" is "I'm going to see a movie today."

[0091] Deploying the model

[0092] Server: The model that has completed evaluation is deployed to the server and incorporated into the actual service.

[0093] Terminal: Uses the deployed model to generate real-time responses to user input.

[0094] How it works: A generative AI model built into a chat app responds to a user's input, "What are you planning to do today?" by generating a real-time response such as, "I'm planning to go to a cafe with my friends today."

[0095] Examples of prompt statements

[0096] "Learn the phrases that users use in conversations with their friends and generate responses that mimic that style."

[0097] The system aims to learn the user's conversation style and generate more natural responses, which will provide the user with a more comfortable communication experience.

[0098] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0099] Step 1: Data collection

[0100] User: For everyday conversations and chats.

[0101] Terminal: Logs user conversations and chat content.

[0102] Server: Receives the conversation logs sent from the devices and stores them in a database.

[0103] Input: A message sent by a user in a chat app (e.g., "The weather is nice today").

[0104] Output: Conversation logs stored on the server.

[0105] Specific operation: When a user sends a message such as "The weather is nice today" in a chat app, the content is recorded on the device and sent to the server.

[0106] Step 2: Data Preprocessing

[0107] Server: Preprocesses the collected conversation logs, specifically removing unnecessary information (e.g. emojis, special characters) and cleaning the text data.

[0108] Input: Conversation log stored on the server (e.g. "The weather is nice today ○ (○ represents a specific emoji)").

[0109] Output: Preprocessed, clean text data (e.g., "The weather is nice today").

[0110] Specific operation: Remove emojis from the message "The weather is nice today ○ (○ represents a specific emoji)" and generate clean text data "The weather is nice today."

[0111] Step 3: Training the model

[0112] Server: Train the deep learning model using the preprocessed data, using fine-tuning techniques.

[0113] Input: Preprocessed, clean text data (e.g., "The weather is nice today").

[0114] Output: A deep learning model fine-tuned to the user's speaking style.

[0115] How it works: It uses a pre-trained GPT-3 model to fine-tune the user's speaking style, for example by learning the user's most commonly used phrases and expressions.

[0116] Step 4: Evaluate the model

[0117] Server: Evaluates the trained model by checking the accuracy of the model using test data.

[0118] Input: A trained deep learning model and test data (e.g., a user's past conversation logs).

[0119] Output: The accuracy assessment of the model.

[0120] Specific behavior: Using the user's past conversation logs as test data, we evaluate how natural the generated responses are. For example, we check whether the response generated for the input "Where are you going today?" is "I'm going to see a movie today."

[0121] Step 5: Deploy the model

[0122] Server: The model that has completed evaluation is deployed to the server and incorporated into the actual service.

[0123] Terminal: Uses the deployed model to generate real-time responses to user input.

[0124] Input: A deep learning model that has completed evaluation.

[0125] Output: A generative AI model embedded into a real service.

[0126] How it works: A generative AI model built into a chat app responds to a user's input, "What are you planning to do today?" by generating a real-time response such as, "I'm planning to go to a cafe with my friends today."

[0127] (Application example 1)

[0128] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0129] Conventional content recommendation systems have difficulty in accurately grasping the user's interests and concerns, making it difficult to provide personalized content. In addition, content recommendations are not based on the user's conversation style or individual needs, which results in a lack of improvement in the user experience.

[0130] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0131] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding as the user would based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, and a recommending means for recommending personalized content based on the user's conversation log. This makes it possible to recommend personalized content based on the user's interests and concerns.

[0132] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[0133] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[0134] The "generation means" is a means for generating images and videos desired by the user.

[0135] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[0136] A "recommendation means" is a means for recommending personalized content based on a user's conversation log.

[0137] A system for implementing this invention includes a learning means for learning the contents of a user's daily conversations and chats and service logs, a response generation means for generating responses in the same way as the user would based on the learned contents, a generation means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, and a recommendation means for recommending personalized content based on the user's conversation logs.

[0138] Hardware and Software Configuration

[0139] The server uses the following hardware and software:

[0140] Hardware: A server with a powerful processor, sufficient memory, and storage.

[0141] Software: Python, OpenAI® API, Database Management System (e.g. MySQL®)

[0142] Data processing and calculation

[0143] The server stores the user's daily conversations, chat contents, and service logs in a database in JSON format. This data is used to fine-tune the deep learning model. Specifically, the model is trained to imitate the user's conversation style using the user's conversation log as input data.

[0144] Recommendation method operation

[0145] The server recommends personalized content based on the user's input. For example, if the user inputs "What are your recent interests?", the server infers the user's interests based on past conversation logs and recommends related content.

[0146] Examples

[0147] When a user uses his / her smartphone to input the question "What are your recent interests?" into the application, the server responds as follows:

[0148] Examples of prompt statements

[0149] User: What are your current interests?

[0150] AI: Based on your recent interests, we'd like to recommend the following content to you:

[0151] 1. Latest Technology News

[0152] 2. Popular Movie Reviews

[0153] 3. New music album release information

[0154] In this way, personalized content recommendations based on the user's interests become possible.

[0155] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0156] Step 1:

[0157] The server collects users' daily conversations, chat contents, and service logs.

[0158] Input: User conversation logs, chat contents, service logs

[0159] Data processing: Save the collected data in JSON format to the database.

[0160] Output: Conversation logs, chat contents, and service logs stored in the database

[0161] Step 2:

[0162] The server uses the collected data to fine-tune the deep learning model.

[0163] Input: Conversation logs, chat contents, and service logs stored in the database

[0164] Data Computing: Training a model to mimic a user's conversational style

[0165] Output: A fine-tuned deep learning model

[0166] Step 3:

[0167] A user inputs a question into the application using a smartphone.

[0168] Input: User question (e.g. "What are your current interests?")

[0169] Data processing: Sending user input to the server

[0170] Output: The user's question sent to the server.

[0171] Step 4:

[0172] The server uses the fine-tuned model to generate answers to users' questions.

[0173] Input: User question sent to the server, fine-tuned deep learning model

[0174] Data Computing: Using models to generate optimal answers to user questions

[0175] Output: The generated response

[0176] Step 5:

[0177] The server recommends personalized content based on the generated responses.

[0178] Input: Generated responses, user's past conversation logs

[0179] Data calculation: Select content that matches the user's interests based on generated responses and past conversation logs

[0180] Output: A personalized list of content

[0181] Step 6:

[0182] The server sends a personalized list of content to the user's smartphone.

[0183] Input: A list of personalized content

[0184] Data processing: Send the list of contents to the user's smartphone

[0185] Output: Personalized content displayed on the user's smartphone.

[0186] Step 7:

[0187] The user checks the personalized content displayed on the smartphone.

[0188] Input: Personalized content displayed on a smartphone

[0189] Data processing: User reviews the content and makes selections as necessary

[0190] Output: User selected content

[0191] In this way, personalized content recommendations based on the user's interests and concerns are realized.

[0192] Example 2

[0193] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0194] Conventional dialogue systems have difficulty generating natural responses based on the user's conversation style, which results in poor user experience. In addition, the quality of the responses generated is low, and they often fail to provide users with appropriate information. Furthermore, there has been a lack of systems that can efficiently process user input data and quickly generate appropriate responses.

[0195] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0196] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like a user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several MB, a receiving means for receiving input data from a user, a preprocessing means for preprocessing the received input data, an input means for inputting the preprocessed data into a generative AI model, a generating means for the generative AI model to generate a response, a postprocessing means for postprocessing the generated response, and a transmitting means for transmitting the postprocessed response to the user. This makes it possible to quickly generate natural responses based on the user's conversation style with high quality.

[0197] A "learning means" is a means for collecting the contents of users' daily conversations and chats, as well as service logs, and training a model based on them.

[0198] The "response generation means" is a means for generating natural responses to user inputs using the model trained by the learning means.

[0199] The "generation means" refers to a means for generating content such as images and videos according to the user's wishes.

[0200] The "sharing means" is a means for sharing the learning file with other users or systems in a general file format of several MB.

[0201] The "receiving means" is a means for receiving input data from a user.

[0202] The "preprocessing means" is a means for performing preprocessing such as normalizing received input data and deleting unnecessary spaces.

[0203] An "input means" is a means for inputting preprocessed data into a generative AI model.

[0204] A "generative AI model" is an artificial intelligence model that generates responses based on user input data.

[0205] "Post-processing means" refers to means for post-processing the response generated by the generative AI model, such as adjusting the format or deleting unnecessary characters.

[0206] A "transmitting means" is a means for transmitting a post-processed reply to a user.

[0207] This invention is a system that learns from a user's daily conversations, chat contents, and service logs, and responds like the user. This system includes a learning means, a response generating means, a generating means, a sharing means, a receiving means, a preprocessing means, an input means, a generating AI model, a postprocessing means, and a transmitting means.

[0208] The server uses a receiving means to receive input data from a user. The receiving means receives data via a communication protocol such as an HTTP request or WebSocket. For example, when a user inputs "What's the weather like today?", this data is received by the receiving means.

[0209] Next, the server preprocesses the received input data using a preprocessing means. For the preprocessing means, Python libraries such as NLTK and spaCy can be used. Preprocessing includes text normalization (e.g., converting uppercase to lowercase, removing unnecessary spaces, etc.). For example, "What's the weather like today?" is converted to "What's the weather like today?".

[0210] The preprocessed data is input to a generative AI model via an input means. For example, OpenAI's GPT-3 or Google's BERT is used as the generative AI model. The generative AI model generates a response based on the input data. For example, in response to the input "How is the weather today?", a response such as "Today's weather is sunny" is generated.

[0211] The generated response is post-processed using the post-processing means. The post-processing includes formatting the text and removing unnecessary characters. The post-processed response is transmitted to the user via the transmitting means. This allows the user to receive a natural response from the system.

[0212] As a concrete example, if a user inputs "What are your plans for tomorrow?", the server receives this input, performs preprocessing, and then inputs it into the generative AI model. The generative AI model generates a response such as "Tomorrow's plans are a meeting," and sends it to the user after post-processing.

[0213] An example of a prompt sentence is, "The following are examples of prompt sentences that a user can use to ask the system a question. For example, in response to the question, 'What are your plans for tomorrow?' the system will generate a response using the learned model."

[0214] In this way, the server can generate natural responses based on the user's input and provide them to the user.

[0215] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0216] Step 1:

[0217] The user submits the input data.

[0218] The user opens a chat application on the terminal, types "What's the weather like today?", and presses the send button. The input data is sent to the server in text format.

[0219] Step 2:

[0220] The server receives the input data.

[0221] The server receives the HTTP POST request from the user and obtains the text data "What's the weather like today?" contained in the request body. The input data is received in its original format.

[0222] Step 3:

[0223] The server pre-processes the input data.

[0224] The server normalizes the received text data using preprocessing measures. Specifically, it uses Python's NLTK or spaCy to convert the text to lower case and remove unnecessary spaces. For example, "What's the weather like today?" is converted to "What's the weather like today?". The preprocessed data is output in a state that is ready to be fed into a generative AI model.

[0225] Step 4:

[0226] The server inputs the preprocessed data into the generative AI model.

[0227] The server inputs the preprocessed text data into the generative AI model by sending an API request and passing the data to the model. The input data is converted into a format that the generative AI model can understand.

[0228] Step 5:

[0229] A generative AI model generates the response.

[0230] A generative AI model generates a response based on input data. For example, in response to the input "What's the weather like today?", it generates a response such as "Today's weather is sunny." The generated response is output in text format.

[0231] Step 6:

[0232] The server post-processes the generated reply.

[0233] The server uses post-processing means to format and remove unnecessary characters from the generated response. For example, if a response is generated that says "The weather is sunny today," it will be formatted appropriately. The post-processed data is output ready to be sent to the user.

[0234] Step 7:

[0235] The server sends a post-processed reply to the user.

[0236] The server sends the post-processed reply as an HTTP response to the user, who sees the reply "The weather is sunny today" in the chat application on his terminal. The output data is displayed in a format that the user can understand.

[0237] In this way, the server can generate natural responses based on the user's input and provide them to the user.

[0238] (Application example 2)

[0239] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0240] Conventional content delivery services have the problem that it is difficult to recommend personalized content based on a user's individual conversation style and preferences. It is also difficult to generate natural responses based on a user's conversation style. This can lead to a poor user experience and a decrease in the frequency of service use.

[0241] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0242] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of a user and service logs, a response generating means for responding like a user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several MB, a recommending means for recommending content based on the user's conversation style, and a prompt generating means for generating prompt sentences using a generative AI model. This makes it possible to recommend personalized content based on the individual conversation style and preferences of the user, and is expected to improve the user experience.

[0243] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[0244] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[0245] A "generation means" is a means for generating a desired image or video.

[0246] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[0247] A "recommendation means" is a means for recommending content based on a user's conversation style.

[0248] A "prompt generation means" is a means for generating a prompt sentence using a generative AI model.

[0249] A system for implementing this invention includes a learning means for learning the content of a user's daily conversations and chats and service logs, a response generation means for generating responses like the user's based on the learned content, a generation means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, a recommendation means for recommending content based on the user's conversation style, and a prompt generation means for generating prompt sentences using a generative AI model.

[0250] Hardware and Software Used

[0251] Hardware: Smartphones, servers

[0252] Software: Python, OpenAI API

[0253] Data processing and calculation

[0254] Learning medium

[0255] The server collects the user's daily conversations, chat contents, and service logs, and uses this data to learn the user's conversation style. For learning, fine-tuning is used to optimize the deep learning model.

[0256] Response Generation Method

[0257] The server uses the learned model to generate natural responses to user input. For example, if a user asks, "What are some recent movies you recommend?" the server responds, "The recent movie I recommend is 'Inception.' It's a thrilling film that depicts a dream within a dream!"

[0258] generation means

[0259] The server generates images and videos according to the user's request. For example, if the user requests "Show me an image of a sunset," the server generates and provides an appropriate image.

[0260] means of sharing

[0261] The server shares the training files in a common file format of a few MB, which allows easy sharing of data with other devices and users.

[0262] Recommendation method

[0263] The server recommends content based on the user's conversation style, for example, if the user has a casual conversation style, it recommends movies and music that match that style.

[0264] Prompt Generation Method

[0265] The server uses a generative AI model to generate prompts, such as the following:

[0266] User conversation style: Casual and friendly

[0267] User input: What are some recent movie recommendations?

[0268] reply:

[0269] In this way, personalized content recommendations based on a user's individual conversation style and preferences become possible, which is expected to improve the user experience.

[0270] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0271] Step 1:

[0272] The server collects users' daily conversations, chat contents, and service logs.

[0273] Input: User conversation data, chat logs, service logs

[0274] Output: Collected data

[0275] Specific operation: The server periodically collects logs of conversations and chats conducted by users on their smartphones and other devices and stores them in a database.

[0276] Step 2:

[0277] The server uses the collected data to learn the user's speaking style.

[0278] Input: Collected data

[0279] Output: The trained model

[0280] How it works: The server uses the collected data to fine-tune the deep learning model and learn the user's conversation style, allowing the model to reflect the user's specific conversation patterns and preferences.

[0281] Step 3:

[0282] The server generates natural responses to the user's input.

[0283] Input: User input, trained model

[0284] Output: The generated response

[0285] Specific operation: When a user types "What is a recommended recent movie?" into their smartphone, the server uses the trained model to generate a response such as "A recommended recent movie is 'Inception.' It's a thrilling film that depicts a dream within a dream!"

[0286] Step 4:

[0287] The server generates images and videos according to the user's wishes.

[0288] Input: User request

[0289] Output: Generated images and videos

[0290] Specific behavior: When a user requests "Show me an image of a sunset," the server generates an image of a sunset using an appropriate image generation algorithm and provides it to the user.

[0291] Step 5:

[0292] The server shares the training files in a common file format of several MB.

[0293] Input: Training file

[0294] Output: Shared files

[0295] Specific operation: The server converts the training file into a common file format (e.g., JSON or CSV) of several MB and shares it with other devices and users.

[0296] Step 6:

[0297] The server recommends content based on the user's conversation style.

[0298] Input: User conversation style, content database

[0299] Output: Recommended content

[0300] Specific operation: The server analyzes the user's conversation style and recommends movies, music, and other content that matches that style. For example, for a user with a casual conversation style, it recommends relaxing movies and music.

[0301] Step 7:

[0302] The server generates a prompt sentence using a generative AI model.

[0303] Input: User's speaking style, user input

[0304] Output: Generated prompt statement

[0305] Specific behavior: The server generates a prompt based on the user's speaking style and input. For example, it generates the following prompt:

[0306] User conversation style: Casual and friendly

[0307] User input: What are some recent movie recommendations?

[0308] reply:

[0309] Example 3

[0310] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0311] In conventional systems, when users wanted to generate images or videos, the quality and accuracy of the generated content was often low. In addition, there was a lack of a way to efficiently share the parameters of the trained model, making it difficult for users to easily use the training results. Furthermore, there was no infrastructure in place to provide content that was quickly generated based on user instructions.

[0312] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0313] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like the user based on the learned contents, a generating means using a generative deep learning model for generating desired images and videos, a sharing means using a cloud storage service for sharing learning files in a general file format of several MB, a means for receiving user instructions and inputting them into the generative deep learning model, and a means for displaying the generated images and videos to the user. This enables users to quickly generate high-quality images and videos and efficiently share and use the learning results.

[0314] The "learning means" refers to a means for collecting users' daily conversations, chat contents, and service logs, and training a model using a machine learning algorithm based on that data.

[0315] The "answer generation means" is a means for generating a reply to a user's input in a manner similar to that of the user, using the model trained by the learning means.

[0316] A "generative deep learning model" is a model that uses deep learning algorithms to generate specific images or videos based on user instructions.

[0317] A "generation means" is a means for generating images and videos based on user instructions using a generative deep learning model.

[0318] The "sharing means" refers to a means for converting the parameters of the trained model into a common file format of a few MB and sharing it via a cloud storage service.

[0319] A "cloud storage service" is an online service for storing and sharing data via the Internet.

[0320] A "server" is a computer system whose role is to receive instructions from a user, input them into a generative deep learning model, and return the generated images and videos to the user.

[0321] A "terminal" is a device through which a user inputs instructions and through which generated images and videos are displayed.

[0322] The present invention provides a system for generating images and videos that a user desires and efficiently sharing parameters of a trained model. A specific embodiment of this system will be described below.

[0323] Hardware and Software Used

[0324] 1. Server: A high-performance computer system that receives instructions from the user and inputs them into the generative deep learning model. The server also returns the generated images and videos to the user.

[0325] 2. Terminal: A device on which a user inputs instructions and displays the generated images and videos. Terminals include PCs, smartphones, tablets, etc.

[0326] 3. Generative deep learning model: A model that uses deep learning algorithms to generate specific images or videos based on user instructions. For example, GAN (Generative Adversarial Network) and VAE (Variational Autoencoder) are used.

[0327] 4. Cloud storage services: Online services for storing and sharing data over the Internet. Examples include Google Drive and Dropbox.

[0328] System Operation

[0329] 1. The user inputs instructions to generate the desired image or video. For example, the user might input "generate an image of a cat."

[0330] 2. The terminal sends the user's instructions to the server. The terminal sends the entered instructions to the server as an HTTP request.

[0331] 3. The server receives the instructions and inputs them into the generative deep learning model. The server receives the HTTP request, analyzes the instructions, and passes them to the generative deep learning model.

[0332] 4. The generative deep learning model generates images and videos based on instructions. Based on the instruction "generate an image of a cat," the generative deep learning model uses its internal algorithms to generate an image of a cat.

[0333] 5. The server receives the generated images and videos and sends them to the device. The server receives the generated images and sends them to the device as an HTTP response.

[0334] 6. The device displays the images and videos it has received to the user. The device displays the images it has received on its screen so that the user can view them.

[0335] Examples

[0336] For example, if a user instructs the server to "generate a video of a landscape," the server inputs the instruction into the generative deep learning model, which then generates a video of the landscape. The generated video is then provided to the user through the server.

[0337] Example prompt:

[0338] "Generate an image of a cat"

[0339] "Generate a video of the scenery"

[0340] Sharing study files

[0341] 1. The server converts the model parameters trained into a common file format. For example, the server converts the model parameters trained into a file called "model_parameters.bin".

[0342] 2. The server uploads the converted file to a cloud storage service. The server uploads "model_parameters.bin" to Google Drive.

[0343] 3. The user accesses the cloud storage service and downloads the necessary files. The user accesses Google Drive and downloads "model_parameters.bin".

[0344] 4. The user imports the downloaded file into their own system and uses the learning results. The user imports the downloaded "model_parameters.bin" into their own system and uses the learning results.

[0345] In this way, the user can quickly generate high-quality images and videos and efficiently share and use the learning results. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0346] Step 1:

[0347] The user inputs instructions to generate the desired image or video.

[0348] Input: The user types "generate a picture of a cat" into the input form on the terminal.

[0349] Specific operation: The user uses the device's keyboard or touch screen to input the content of the image or video to be generated in text format.

[0350] Output: Your terminal will display the instructions and you are ready to move on to the next step.

[0351] Step 2:

[0352] The terminal transmits the user's instructions to the server.

[0353] Input: Instructions entered by the user.

[0354] Specific operation: The terminal sends the input instructions to the server as an HTTP request.

[0355] Output: The server receives the HTTP request and parses the instructions.

[0356] Step 3:

[0357] The server receives the instructions and inputs them into the generative deep learning model.

[0358] Input: The HTTP request sent from the terminal.

[0359] Specific operation: The server receives an HTTP request, analyzes the instructions, and passes them to the generative deep learning model.

[0360] Output: The generative deep learning model receives the instructions.

[0361] Step 4:

[0362] Generative deep learning models generate images and videos based on instructions.

[0363] Input: Instructions passed from the server.

[0364] Specific operation: Based on the instruction "generate an image of a cat," the generative deep learning model uses an internal algorithm to generate an image of a cat.

[0365] Output: The generated image data.

[0366] Step 5:

[0367] The server receives the generated images and videos and sends them to the terminal.

[0368] Input: Generated image data from a generative deep learning model.

[0369] Specific operation: The server receives the generated image and sends it to the terminal as an HTTP response.

[0370] Output: The device receives the HTTP response.

[0371] Step 6:

[0372] The device displays the images and videos it receives to the user.

[0373] Input: The generated image data sent from the server.

[0374] Specific operation: The image received by the device is displayed on the screen so that the user can check it.

[0375] Output: The user visually checks the generated image.

[0376] Step 7:

[0377] The server converts the trained model parameters into a common file format.

[0378] Input: The parameters of the trained model.

[0379] Specific operation: The server converts the parameters of the trained model into a file called "model_parameters.bin".

[0380] Output: Model parameter files converted to a common file format.

[0381] Step 8:

[0382] The server uploads the converted files to a cloud storage service.

[0383] Input: Model parameter files converted to a common file format.

[0384] Specific operation: The server uploads "model_parameters.bin" to the cloud storage service.

[0385] Output: The file uploaded to the cloud storage service.

[0386] Step 9:

[0387] A user accesses a cloud storage service and downloads a required file.

[0388] Input: A file uploaded to a cloud storage service.

[0389] Specific operation: A user accesses a cloud storage service and downloads "model_parameters.bin".

[0390] Output: The files downloaded to the user's device.

[0391] Step 10:

[0392] The user imports the downloaded file into their own system and uses the learning results.

[0393] Input: The downloaded “model_parameters.bin” file.

[0394] Specific operation: The user imports the downloaded "model_parameters.bin" into their own system and uses the learning results.

[0395] Output: Training results available on the user's system.

[0396] (Application example 3)

[0397] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0398] In conventional image and video generation systems using generative deep learning models, it was difficult for users to quickly and efficiently generate and share the content they wanted. In addition, there was a lack of means to upload the generated content to cloud storage and share it with other users, which resulted in low user convenience. Furthermore, there was a lack of sharing of learning files tailored to individual needs.

[0399] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0400] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding as the user would based on the learned contents, a generating means for generating desired images and videos, an uploading means for uploading the generated images and videos to cloud storage, and a sharing means for sharing learning files in a general file format of several MB. This allows the user to quickly and efficiently generate desired content and share it with other users through cloud storage. It also allows sharing of learning files tailored to individual needs.

[0401] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[0402] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[0403] The "generation means" is a means for generating desired images and videos based on user instructions.

[0404] The "uploading means" is a means for uploading the generated images and videos to cloud storage.

[0405] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[0406] A system for implementing this invention includes a learning means for learning the contents of a user's daily conversations and chats and service logs, a response generation means for generating responses in the manner of the user based on the learned contents, a generation means for generating desired images and videos, an uploading means for uploading the generated images and videos to cloud storage, and a sharing means for sharing learning files in a common file format of several MB.

[0407] Explanation of system program processing

[0408] Hardware and Software

[0409] The server uses a high-performance computer equipped with a GPU. The software uses the transformers library to execute generative deep learning models, the requests library to process HTTP requests, and the PIL library for image processing.

[0410] Data processing and calculation

[0411] 1. Learning method: The server collects users' daily conversations, chat contents, and service logs, and uses them to train a generative deep learning model. Fine-tuning is performed using the transformers library for learning.

[0412] 2. Response generation: Using the trained model, we generate appropriate responses based on the user's input, enabling responses that sound more natural than the user would.

[0413] 3. Generation: A user inputs a prompt, and the generative deep learning model generates the desired image or video based on the prompt. For example, if a user inputs "generate an image of a beach at sunset," the model generates the image.

[0414] 4. Uploading method: The generated images and videos are uploaded to cloud storage by sending an HTTP request using the requests library.

[0415] 5. Sharing: The learning files are converted into a common file format of several MB and uploaded to cloud storage. Users can download the files and import them into their own systems.

[0416] Examples

[0417] When a user enters the prompt "Generate an image of a cat," the generative deep learning model generates an image of a cat and uploads it to cloud storage. The user can then download and use the uploaded image.

[0418] Example prompt:

[0419] Generate a cat image

[0420] Generate sunset beach images

[0421] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0422] Step 1:

[0423] The user enters a prompt sentence.

[0424] Input: A prompt describing the image or video the user is looking for (e.g., "Generate an image of a cat").

[0425] How it works: A user enters a prompt through a smartphone or computer interface.

[0426] Output: The prompt text is sent to the server.

[0427] Step 2:

[0428] The server receives the prompt sentence and inputs it into the generative deep learning model.

[0429] Input: The prompt text sent by the user.

[0430] How it works: The server receives a prompt text and uses the transformers library to feed the prompt text to a generative deep learning model.

[0431] Output: A generation request based on the prompt is passed to the model.

[0432] Step 3:

[0433] A generative deep learning model generates images and videos based on the prompt text.

[0434] Input: A generated request based on the prompt statement.

[0435] How it works: The model analyzes the prompt and generates an appropriate image or video. For example, in response to the prompt "generate an image of a cat," the model generates an image of a cat.

[0436] Output: The generated image and / or video data.

[0437] Step 4:

[0438] The server uploads the generated images and videos to cloud storage.

[0439] Input: Generated image and / or video data.

[0440] How it works: The server uses the requests library to upload the generated images and videos to cloud storage.

[0441] Output: URL of uploaded image or video to cloud storage.

[0442] Step 5:

[0443] The server returns the cloud storage URL to the user.

[0444] Input: The URL of the image or video uploaded to cloud storage.

[0445] How it works: The server sends the URL of the generated content back to the user.

[0446] Output: The URL where the user can access it.

[0447] Step 6:

[0448] The user downloads the generated images and videos from the cloud storage URL.

[0449] Input: The URL returned by the server.

[0450] How it works: The user visits a URL and downloads the generated image or video.

[0451] Output: Images and videos stored on the user's device.

[0452] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0453] "Example 1"

[0454] In one embodiment of the present invention, an emotion engine is used as a learning means for learning a user's daily conversations, chat contents, and service logs. The emotion engine recognizes emotions from the user's speech and incorporates the emotions as learning data. For example, if a user says, "Today was fun," the emotion engine recognizes the emotion of "joy" from this speech and incorporates the information as learning data.

[0455] "Example 2"

[0456] In addition, an emotion engine is used as a reply generating means that replies like a user based on the learned content. The emotion engine adapts to the user's emotions based on the learned emotion data.

[0457] For example, if a user says, "I had fun today," the emotion engine generates a joy-adapted response such as, "That's great."

[0458] "Example 3"

[0459] Furthermore, an emotion engine is used as a generating means for generating desired images and videos. The emotion engine generates images and videos adapted to the user's emotions based on learned emotion data. For example, if a user says, "Today was fun," and requests, "Show me an image of a happy scene," the emotion engine generates an image of a joyful scene.

[0460] "Example 4"

[0461] Finally, the emotion engine is used as a means of sharing the learning file in a general file format of several MB. The emotion engine converts the learned emotion data into a general file format and shares the file. This allows other systems and users to use the learning results of the emotion engine.

[0462] The process flow of each embodiment will be described below.

[0463] "Example 1"

[0464] Step 1: Receive the user's utterance.

[0465] Step 2: The emotion engine recognizes emotions from the user's speech.

[0466] Step 3: The recognized emotions are input as learning data.

[0467] "Example 2"

[0468] Step 1: Receive the user's utterance.

[0469] Step 2: Based on the emotion data learned by the emotion engine, a response is generated that is adapted to the user's emotions.

[0470] "Example 3"

[0471] Step 1: Receive user speech and a request for images or videos.

[0472] Step 2: Based on the emotional data learned by the emotion engine, images and videos are generated that are adapted to the user's emotions.

[0473] "Example 4"

[0474] Step 1: Convert the emotion data learned by the emotion engine into a common file format.

[0475] Step 2: Share the converted file.

[0476] Example 1

[0477] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0478] Conventional systems had difficulty effectively learning the content of users' daily conversations and chats and generating responses appropriate to the user. In addition, they lacked the ability to recognize users' emotions and generate responses based on them, limiting the user experience. Furthermore, when it came to sharing learning files, flexible responses tailored to individual needs were required.

[0479] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0480] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like the user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several megabytes, an emotion recognition means for recognizing emotions from the user's speech and incorporating the emotions as learning data, and a response generating means for generating responses suitable for the user using a generative AI model. This makes it possible to effectively learn the user's conversation style and generate responses based on emotions. It is also possible to share learning files tailored to individual needs.

[0481] A "learning means" is a means for collecting users' daily conversations, chat contents, and service logs, and training a deep learning model based on them.

[0482] The "response generating means" is a means for imitating the user's conversation style based on the learned content and generating an appropriate reply.

[0483] The "generation means" is a means for generating images and videos according to the user's wishes.

[0484] "Sharing means" refers to a means for sharing learning files in a common file format of several megabytes.

[0485] The "emotion recognition means" is a means for recognizing emotions from the user's speech and incorporating the emotions as learning data.

[0486] A "response generation means" is a means for generating a response appropriate to a user using a generative AI model.

[0487] "Fine-tuning" is a technique for optimizing an existing deep learning model to fit a user's data.

[0488] The present invention is a system that learns the contents of a user's daily conversations and chats, and service logs, and generates responses appropriate to the user. A specific embodiment of this system will be described below.

[0489] 1. Program Generation

[0490] The server generates a program to learn the user's daily conversations, chat contents, and service logs. The program is designed to imitate the user's conversation style using a deep learning model.

[0491] 2. Program Processing Explanation

[0492] The server receives the user's conversation log as input data and uses a deep learning model to train the model to imitate the user's conversation style. This training uses a technique called fine-tuning. Specifically, the following process is performed:

[0493] Data collection: The server collects the user's daily conversations, chat contents, and service logs. For example, if a user sends a message such as "I had fun today" in a chat app, the message is stored on the server.

[0494] Data preprocessing: The server preprocesses the collected data and converts it into a format suitable for the model. Specifically, it normalizes and tokenizes the text. For example, it tokenizes a statement such as "Today was fun" into "Today was fun."

[0495] Model training: The server trains a deep learning model using the preprocessed data, using fine-tuning techniques to fine-tune the existing model to the user's speaking style, using deep learning frameworks such as TensorFlow and PyTorch.

[0496] Emotion recognition: The server uses the emotion engine to recognize emotions from the user's speech. For example, if a user says, "Today was fun," the emotion engine recognizes the emotion of "joy" from this speech and incorporates that information into learning data.

[0497] Response generation: The server uses the generative AI model to generate a response appropriate for the user. Specifically, the server creates a prompt sentence based on the user's utterance, inputs it into the generative AI model, and generates an appropriate response.

[0498] 3. Examples of concrete examples and prompts

[0499] As a specific example, consider a case where a user says, "Today was fun." This utterance is processed as follows.

[0500] The server collects these comments and stores them as text data.

[0501] The emotion engine recognizes the emotion "joy" from this statement.

[0502] The recognized emotional information is incorporated into the deep learning model as training data.

[0503] The server uses this data to fine-tune the model, training it to mimic the user's speaking style.

[0504] The server generates a prompt sentence, "Tell me more about your fun events," and inputs it into the generative AI model to generate a response, "That's great! What did you enjoy?"

[0505] Examples of prompt sentences to be input to a generative AI model include the following:

[0506] "Please tell us about an event that you found enjoyable."

[0507] "Please tell us what your plans are for the day when you feel busy."

[0508] "Please tell us about movies that you found interesting."

[0509] In this way, the server learns the user's conversational style and uses a generative AI model to generate responses that are appropriate for the user.

[0510] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0511] Step 1: Data collection

[0512] The server collects the user's daily conversations, chat contents, and service logs. For example, when a user sends a message such as "I had fun today" in a chat app, the message is stored in the server. The input is the user's text data, and the output is the stored text data.

[0513] Step 2: Data Preprocessing

[0514] The server pre-processes the collected data. Specifically, it performs the following processes:

[0515] Text normalization: removing unnecessary spaces and special characters from what the user says.

[0516] Tokenization: Splitting text into words and phrases. For example, tokenizing an utterance like "I had fun today" into "I had fun today".

[0517] Labeling: Using the emotion engine, an emotion label is assigned to each utterance. For example, a utterance such as "Today was fun" is labeled as "Joy."

[0518] The input is stored text data, and the output is preprocessed text data and emotion labels.

[0519] Step 3: Model training

[0520] The server trains a deep learning model using the preprocessed data, specifically by following the steps below.

[0521] Dataset Creation: We create a dataset by combining the preprocessed text with the sentiment labels.

[0522] Fine-tuning: Optimizing an existing deep learning model to fit your data, for example by updating the model parameters using TensorFlow or PyTorch.

[0523] The input is preprocessed text data and sentiment labels, and the output is a trained deep learning model.

[0524] Step 4: Emotion Recognition

[0525] The server uses the emotion engine to recognize emotions from the user's speech. Specifically, it performs the following processes.

[0526] Text analysis: Analyze what the user says and input it into the emotion engine.

[0527] Emotion labeling: The emotion engine recognizes emotions from utterances and assigns corresponding emotion labels. For example, a statement like "Today was fun" is labeled as "Joy."

[0528] The input is the user's spoken text and the output is an emotion label.

[0529] Step 5: Response Generation

[0530] The server uses the generative AI model to generate a response appropriate for the user. Specifically, it follows the steps below.

[0531] Prompt creation: Create a prompt based on what the user says. For example, if the user says, "I had fun today," the prompt will be, "Tell me more about fun things that happened to you."

[0532] Response generation: A prompt is fed into the generative AI model to generate an appropriate response, for example, "That's great! What did you enjoy?"

[0533] The input is the prompt sentence and the output is the generated response.

[0534] In this way, the server learns the user's conversational style and uses a generative AI model to generate responses that are appropriate for the user.

[0535] (Application example 1)

[0536] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0537] Conventional content delivery services have the problem that it is difficult to recommend personalized content that takes into account the individual conversation style and emotions of each user. This makes it impossible to provide optimal content according to the user's interests and emotions, preventing an improvement in the user experience.

[0538] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a learning means for learning the contents of daily conversations and chats of the user and service logs, a response generating means for responding like the user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, a recommendation means for learning the user's conversation style and emotions and providing individually customized content, and a recommendation generating means for recommending customized content based on the user's input. This enables personalized content recommendation based on the user's individual conversation style and emotions.

[0539] The "contents of the user's daily conversations and chats" are records of the user's daily conversations and message exchanges.

[0540] A "service log" is a record of operation history and usage status regarding a service used by a user.

[0541] A "learning means" is a method or device for analyzing user conversations and service logs, extracting patterns and features, and training a model.

[0542] A "response generating means" is a method or device for generating a response based on what has been learned, just like a user would.

[0543] A "generation means" is a method or device for generating content such as images and videos in accordance with a user's wishes.

[0544] A "sharing means" is a method or device for sharing a learning file with other users or systems in a general file format of several MB.

[0545] A "recommender" is a method or device for learning a user's speech style and emotions and providing individually customized content.

[0546] A "recommendation generator" is a method or apparatus for recommending customized content based on user input.

[0547] "Fine-tuning" is a technique that involves additional training of an existing deep learning model to adapt it to a specific task or dataset.

[0548] As an embodiment of the present invention, a system is constructed that learns the contents of daily conversations and chats of users and service logs, and provides individually customized content. A specific implementation method thereof is shown below.

[0549] System Configuration

[0550] 1. Hardware:

[0551] Server: Use a server with high-performance computing power to train and infer deep learning models.

[0552] Terminal: Interacts with the user using a user terminal such as a smartphone or smart glasses.

[0553] 2. Software:

[0554] TensorFlow: Used for training and inference of deep learning models.

[0555] Transformers library: Used to leverage generative AI models such as GPT-2.

[0556] Emotion recognition pipeline: Used to recognize emotions from user speech.

[0557] Data processing and calculation

[0558] 1. Data collection:

[0559] The system collects users' daily conversations, chat content, and service logs. This data is sent from the user's device to the server.

[0560] 2. Data Analysis:

[0561] The server analyzes the collected data to extract the user's speaking style and emotions. An emotion recognition pipeline is used to recognize emotions from the user's speech.

[0562] 3. Train the model:

[0563] The server uses the analyzed data to fine-tune the deep learning model, which generates a model based on the user's conversation style and emotions.

[0564] 4. Content Recommendation:

[0565] The server recommends customized content based on user input, using generative AI models to generate content according to the user's interests and emotions.

[0566] Examples

[0567] For example, when a user inputs "What movie should I watch today?", the server performs the following process.

[0568] 1. Parse the user input and recognize emotions using the emotion recognition pipeline.

[0569] 2. Using a deep learning model, we recommend the best movies based on the recognized emotions and past conversation logs.

[0570] 3. The list of recommended movies is sent to the user's device and displayed.

[0571] Examples of prompt statements

[0572] Using the user's conversation log as input data, the emotion engine recognizes emotions and incorporates those emotions as training data. For example, if a user says, "Today was fun," the emotion engine recognizes the emotion of "joy" from this statement and incorporates that information as training data. This data is used to fine-tune the deep learning model to mimic the user's conversation style.

[0573] In this way, personalized content recommendations based on a user's individual speaking style and emotions are possible.

[0574] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0575] Step 1:

[0576] Users input the contents of their daily conversations and chats into their terminals. The terminals collect this data and send it to the server. The input data is a conversation log or chat log in text format.

[0577] Step 2:

[0578] The server analyzes the received conversation logs and chat logs. Specifically, it uses an emotion recognition pipeline to recognize emotions from each utterance. The input is the conversation log, and the output is the emotion label corresponding to each utterance.

[0579] Step 3:

[0580] The server uses the analyzed data to fine-tune a deep learning model. Specifically, it uses TensorFlow and the Transformers library to train a model based on the user's conversation style and emotions. The input is the conversation log with emotion labels, and the output is a fine-tuned generative AI model.

[0581] Step 4:

[0582] The user inputs a new question to the terminal. For example, the user inputs a question such as "What movie should I watch today?" The input data is the user's question.

[0583] Step 5:

[0584] The server analyzes the user's input and generates the best content using a generative AI model. Specifically, it tokenizes the user's question, inputs it into the model, and decodes the generated text. The input is the user's question, and the output is a list of recommended content.

[0585] Step 6:

[0586] The server sends the generated list of contents to the terminal. The terminal displays the received contents to the user. The input is the list of recommended contents, and the output is the contents displayed on the terminal.

[0587] Step 7:

[0588] The user checks the recommended content and selects or views it as necessary, which provides the user with the most suitable content according to their interests and emotions.

[0589] Example 2

[0590] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0591] Conventional systems have had difficulty generating responses that adapt to the user's conversation style and emotions. They also lacked the means to generate appropriate responses to new input based on what they had learned. Furthermore, sharing of learning files was not tailored to individual needs, making it difficult to share information efficiently.

[0592] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0593] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding as the user would based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, a means for generating responses using the learned model for new inputs, and a means for generating responses adapted to the user's emotions based on the learned emotion data. This makes it possible to generate responses adapted to the user's conversation style and emotions, and realizes efficient information sharing.

[0594] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[0595] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[0596] A "generation means" is a means for generating a desired image or video.

[0597] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[0598] "Means for generating a response using a model trained for a new input" refers to means for generating an appropriate response for a new input using a model trained in advance.

[0599] The "means for generating a response adapted to the user's emotions based on learned emotion data" refers to a means for generating a response adapted to the user's emotions based on learned emotion data.

[0600] This invention is a system that learns the contents of daily conversations and chats of users and service logs, and generates replies based on the learned contents as if the user were responding. Furthermore, it can generate desired images and videos, and share the learning files in a general file format of several MB.

[0601] Hardware and Software Used

[0602] Hardware: Servers, terminals

[0603] Software: Deep learning model, emotion engine

[0604] Program Processing

[0605] 1. The user provides input

[0606] A user enters text into an input field on a terminal.

[0607] Example: A user types, "How was your day?"

[0608] 2. The device sends input to the server

[0609] The terminal transmits the user's input to the server.

[0610] Example: A device sends text data such as "How was your day?" to a server.

[0611] 3. The server receives the input and passes it to the deep learning model.

[0612] The server analyzes the input received from the device and passes it to the deep learning model.

[0613] Example: A server inputs the text "How was your day?" into a deep learning model.

[0614] 4. The deep learning model generates a response

[0615] Deep learning models generate responses based on the data they have learned from.

[0616] Example: A deep learning model generates the response "Today was a very good day."

[0617] 5. The server uses an emotion engine to tailor the response

[0618] The server passes the generated response to an emotion engine, which tailors the response to the user's emotion.

[0619] Example: An emotion engine refines "Today was a great day" to "That's wonderful."

[0620] 6. The server sends the final response to the device.

[0621] The server sends the final adjusted response to the terminal.

[0622] Example: The server sends the text "That's great" to the device.

[0623] 7. The device displays the reply to the user

[0624] The terminal displays the received response to the user.

[0625] Example: A device displays the text "That's great" on the screen.

[0626] Examples and prompts

[0627] Examples

[0628] User: "How was your day?"

[0629] Server: "Today was a great day."

[0630] User: "Today was fun."

[0631] Server: "That's great."

[0632] Examples of prompt statements

[0633] "If a user types 'How was your day?', how should the server respond?"

[0634] "If a user says, 'I had fun today,' what response do we generate using the emotion engine?"

[0635] In this way, the server is a system that uses deep learning models and emotion engines to generate responses that adapt to the user's conversation style and emotions.

[0636] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0637] Step 1:

[0638] The user provides input

[0639] Input: The user types text into an input field on the terminal.

[0640] What happens: A user types, "How was your day?"

[0641] Output: Text data entered into the terminal.

[0642] Step 2:

[0643] The terminal sends input to the server

[0644] Input: Text data entered into the terminal.

[0645] Specific operation: The device sends text data such as "How was your day?" to the server.

[0646] Output: The text data sent to the server.

[0647] Step 3:

[0648] The server receives the input and passes it to the deep learning model.

[0649] Input: The text data sent to the server.

[0650] What happens: The server receives the text "How was your day?" and inputs it into the deep learning model.

[0651] Output: The text data passed to the deep learning model.

[0652] Step 4:

[0653] The deep learning model generates the answer

[0654] Input: Text data passed to the deep learning model.

[0655] What it does: A deep learning model generates a response based on the data it has learned from.

[0656] Output: The response text "Today was a great day."

[0657] Step 5:

[0658] The server uses an emotion engine to tailor the response

[0659] Input: The response text generated by the deep learning model.

[0660] What happens: The server passes the generated response to the emotion engine, which adjusts the response to suit the user's emotions.

[0661] Output: The tailored response text: "That's great."

[0662] Step 6:

[0663] The server sends the final response to the device.

[0664] Input: The adjusted response text.

[0665] Specific action: The server sends the text "That's great" to the device.

[0666] Output: The reply text sent to the terminal.

[0667] Step 7:

[0668] The device displays the reply to the user

[0669] Input: The reply text sent to the terminal.

[0670] What happens: The device will display the text "That's great" on the screen.

[0671] Output: The response text displayed to the user.

[0672] (Application example 2)

[0673] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0674] Conventional content distribution services do not adequately recommend content personalized based on the user's viewing history or emotions. In addition, they lack the ability to generate responses that adapt to the user's emotions through dialogue with the user, limiting the user experience. This leads to problems such as a decline in user satisfaction and engagement.

[0675] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0676] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like the user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, a recommendation means for recommending content based on the user's viewing history and emotion data, and an emotion-adaptive response generating means for generating responses adapted to the emotions of users' comments. This makes it possible to provide personalized content recommendations based on the user's viewing history and emotions, and an emotion-adaptive interactive interface.

[0677] The "learning means" is a means for collecting the contents of users' daily conversations and chats, as well as service logs, and training a model based on these.

[0678] The "reply generating means" is a means for generating a reply that resembles the user's conversation style based on the learned content.

[0679] The "generation means" is a means for generating images and videos according to the user's wishes.

[0680] "Sharing means" refers to a means for sharing learning files with other systems or users in a general file format of several MB.

[0681] A "recommendation means" is a means for recommending appropriate content based on a user's viewing history and emotional data.

[0682] The "emotion-adaptive response generating means" is a means for generating a response to a user's comment that is adapted to the emotion of the user.

[0683] A system for implementing this invention includes a learning means for learning the contents of a user's daily conversations and chats and service logs, a response generation means for responding like the user would based on the learned contents, a generation means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, a recommendation means for recommending content based on the user's viewing history and emotional data, and an emotion-adaptive response generation means for generating responses to user comments that are emotion-adapted.

[0684] Hardware and Software Used

[0685] Hardware: Smartphone, head-mounted display

[0686] Software: Python, OpenAI API

[0687] Data processing and calculation

[0688] Learning medium

[0689] The server collects the user's daily conversations, chat contents, and service logs, and trains a deep learning model based on this data. Fine-tuning technology is used for training to generate a model that is adapted to the individual conversation style of the user.

[0690] Response Generation Method

[0691] The server uses the trained model to generate appropriate responses to user comments. For example, if a user says, "I had fun today," the server uses the emotion engine to generate a response such as, "That's great."

[0692] generation means

[0693] The server generates images and videos based on the user's preferences, using a generative AI model to provide high-quality content.

[0694] means of sharing

[0695] The server shares the learning files with other systems and users in a common file format of several MB, ensuring data compatibility across different devices.

[0696] Recommendation method

[0697] The server recommends appropriate content based on the user's viewing history and emotion data. For example, if the user says, "Today was fun," the server recommends comedy movies based on the emotion data.

[0698] Emotion-adaptive response generation method

[0699] The server generates a response that is adapted to the user's emotions in response to the user's utterances, using the OpenAI API to realize a natural dialogue that is adapted to the user's emotions.

[0700] Examples

[0701] User says: "Today was fun"

[0702] Assistant replies: "That's great! If you're in a fun mood, this comedy movie is a great choice."

[0703] Examples of prompt statements

[0704] User said: Today was fun

[0705] Generate responses that are emotionally appropriate.

[0706] In this way, it is possible to provide personalized content recommendations based on a user's viewing history and emotions, as well as an emotionally adaptive interactive interface.

[0707] The flow of the identification process in the application example 2 will be described with reference to FIG.

[0708] Step 1:

[0709] The server collects the user's daily conversations, chat contents, and service logs. These data are input as the user's remarks and behavior history. The collected data is stored in a database and used in the later learning process.

[0710] Step 2:

[0711] The server trains a deep learning model based on the collected data. Fine-tuning technology is used for this training. User conversation data and service logs are used as input, and a model adapted to the user's conversation style is generated as output.

[0712] Step 3:

[0713] The server uses the trained model to generate appropriate responses to user utterances. The input is the user's current utterance, and the output is a response based on the user's conversational style. For example, if the user says "I had fun today," the emotion engine generates a response such as "That's great."

[0714] Step 4:

[0715] The server generates images and videos according to the user's wishes: as input, the user's wishes and requests are used, and as output, images and videos generated using a generative AI model are provided.

[0716] Step 5:

[0717] The server shares the training files with other systems and users in a common file format of several MB. The training files are used as input, and data converted into a shareable file format is provided as output.

[0718] Step 6:

[0719] The server recommends appropriate content based on the user's viewing history and emotion data. The user's viewing history and emotion data are used as input, and content recommendations adapted to the user's emotion are output. For example, if the user says, "Today was fun," comedy movies are recommended based on the emotion data.

[0720] Step 7:

[0721] The server generates a response that is adapted to the user's emotions in response to the user's utterances. The input is the user's utterance and emotion data, and the output is a response that is adapted to the emotion. This uses the OpenAI API to realize a natural dialogue that is adapted to the user's emotions.

[0722] In this way, it is possible to provide personalized content recommendations based on a user's viewing history and emotions, as well as an emotionally adaptive interactive interface.

[0723] Example 3

[0724] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0725] Conventional systems have had problems with generating images and videos that correspond to the user's emotions and specific requests, and with efficiently sharing trained model parameters and emotion data. For this reason, there is a demand for a system that can respond to the diverse needs of users and efficiently share learning results.

[0726] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0727] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding as if the user were to respond based on the learned contents, a generating means for generating desired images and videos using a generative deep learning model, a sharing means for uploading the learning file to a cloud storage service in a general file format of several MB and sharing it, a generating means for generating images and videos adapted to the user's emotions using an emotion engine, and a sharing means for converting the emotion data learned using the emotion engine into a general file format and sharing it. This makes it possible to generate images and videos according to the various needs of users, and to efficiently share the learning results.

[0728] A "user" is an individual or group that uses the system to perform operations such as generating images and videos and sharing learning files.

[0729] The "learning means" is a function for analyzing the contents of users' daily conversations and chats, as well as service logs, and learning the data.

[0730] The "response generating means" is a function for generating responses based on learned content in the same way as a user would.

[0731] A "generative deep learning model" is a deep learning algorithm for generating specific images or videos based on user instructions.

[0732] "Generation means" refers to a function for generating desired images or videos using a generative deep learning model or emotion engine.

[0733] "Sharing method" is a function for converting learning files into a common file format of several MB and uploading them to a cloud storage service for sharing.

[0734] A "cloud storage service" is an online service for storing and sharing data via the Internet.

[0735] An "emotion engine" is an algorithm for generating adaptive images and videos based on a user's emotions.

[0736] "Common file formats" are data formats that are widely used, including JSON and CSV.

[0737] "Fine-tuning" is a technique in which an existing learning model is subjected to additional training in order to adapt it to a specific task.

[0738] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0739] This invention is a system that learns the contents of daily conversations and chats of users and service logs, and generates images and videos based on the user's instructions. It also has a function to efficiently share the parameters and emotion data of the learned model.

[0740] Hardware and Software Used

[0741] The server uses the following hardware and software:

[0742] Hardware: A server equipped with a high-performance GPU (e.g. NVIDIA GPU)

[0743] Software: Generative deep learning models (e.g. GAN, VQ-VAE), emotion engines, cloud storage services (e.g. Google Drive, Dropbox)

[0744] The terminal is a device on which the user inputs the prompt text and checks the generated images and videos. Specifically, it includes a PC, a smartphone, a tablet, etc.

[0745] Program Processing

[0746] The user inputs the content of the image or video they want to generate as a prompt using the terminal. For example, they input prompts such as "Generate an image of a cat" or "Today was fun. Show me an image of a fun scene."

[0747] The server analyzes the prompt received from the user and inputs it into the generative deep learning model and emotion engine. The generative deep learning model generates images and videos according to the user's instructions based on pre-trained data. The emotion engine generates images and videos that adapt to the user's emotions.

[0748] The generated images and videos are sent from the server to the user's device, where they can be viewed by the user.

[0749] Sharing study files

[0750] The server converts the trained model parameters into a common file format of several MB (e.g., JSON, HDF5) and uploads it to a cloud storage service. Users can use the trained results by downloading the file from the cloud storage service and importing it into their own systems.

[0751] In addition, the emotion data learned using the emotion engine is also converted into a common file format (e.g., CSV, JSON) and a shared link is generated. Users can download the file from the shared link and import it into their own systems.

[0752] Examples

[0753] Example prompt:

[0754] "Generate an image of a cat"

[0755] "Generate a video of dogs playing under a blue sky"

[0756] "Today was fun. Show me a picture of a fun moment."

[0757] "I'm feeling sad, generate a comforting image for me"

[0758] In this way, a system is provided that enables users to generate images and videos that meet their various needs and efficiently share their learning results. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0759] Step 1:

[0760] The user inputs the content of the image or video they want to generate as a prompt using the terminal. For example, they input prompts such as "Generate an image of a cat" or "Today was fun. Show me an image of a fun scene."

[0761] Input: prompt statement

[0762] Output: The prompt is sent to the server.

[0763] Specific action: The user enters a prompt sentence into the input field of the terminal and presses the send button.

[0764] Step 2:

[0765] The server analyzes the prompt received from the user and inputs it into a generative deep learning model and emotion engine.

[0766] Input: prompt statement

[0767] Output: Parsed prompt statement

[0768] What it does: The server parses the prompt using natural language processing algorithms and converts it into a format suitable for generative deep learning models and emotion engines.

[0769] Step 3:

[0770] The server generates images and videos using a generative deep learning model.

[0771] Input: Parsed prompt statement

[0772] Output: Generated images and videos

[0773] Specific operation: The server runs a generative deep learning model (e.g., GAN, VQ-VAE) to generate images and videos based on the prompt text.

[0774] Step 4:

[0775] The server uses an emotion engine to generate images and videos that are adapted to the user's emotions.

[0776] Input: Parsed prompt statement

[0777] Output: Emotion-adaptive images and videos

[0778] What it does: The server runs the emotion engine and generates images and videos based on the user's emotions.

[0779] Step 5:

[0780] The server transmits the generated images and videos to the user's device.

[0781] Input: Generated images and videos

[0782] Output: Images and videos sent to the user's device

[0783] Specific operation: The server sends the generated images and videos to the user's device so that the user can view them.

[0784] Step 6:

[0785] The server converts the trained model parameters into a common file format of a few MB and uploads them to a cloud storage service.

[0786] Input: trained model parameters

[0787] Output: A file uploaded to a cloud storage service.

[0788] Specific operation: The server converts the trained model parameters into JSON format and uploads them to Google Drive or Dropbox.

[0789] Step 7:

[0790] A user downloads a file from a cloud storage service and imports it into their system.

[0791] Input: Shared link of cloud storage service

[0792] Output: The downloaded file.

[0793] Specific behavior: A user accesses a cloud storage service and downloads a file from a shared link.

[0794] Step 8:

[0795] The server converts the learned emotion data using the emotion engine into a general file format and generates a sharing link.

[0796] Input: Trained emotion data

[0797] Output: Shared link

[0798] Specific operation: The server converts the learned emotion data into CSV format and generates a shared link.

[0799] Step 9:

[0800] Users download the files from the shared link and import them into their systems.

[0801] Input: Shared Link

[0802] Output: The downloaded file.

[0803] Specific behavior: A user accesses a shared link and downloads a file.

[0804] (Application example 3)

[0805] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0806] Conventional image and video generation systems have difficulty generating content according to the user's emotions, and have been unable to provide content that is adapted to the user's emotions. In addition, there was a lack of means to efficiently share the generated content, limiting the user experience. Furthermore, there was a lack of functionality to upload the generated content to cloud storage services, making it difficult to easily share it with other devices and users.

[0807] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0808] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like the user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, an emotion adaptive generating means for analyzing the emotions of the user and generating images and videos adapted to the emotions, and an uploading means for uploading the generated images and videos to a cloud storage service. This enables the generation of content according to the emotions of the user and efficient sharing.

[0809] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[0810] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[0811] The "generation means" is a means for generating desired images and videos based on user instructions.

[0812] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[0813] The "emotion-adaptive generation means" is a means for analyzing the user's emotions and generating images and videos adapted to those emotions.

[0814] The "uploading means" is a means for uploading the generated images and videos to a cloud storage service.

[0815] The present invention is a system that learns the contents of a user's daily conversations and chats, as well as service logs, generates images and videos that correspond to the user's emotions, and uploads them to a cloud storage service. A specific embodiment of this system is described below.

[0816] System Configuration

[0817] The system consists of the following main components:

[0818] 1. Learning method: A method for learning from users' daily conversations, chat contents, and service logs. Specifically, natural language processing technology is used to analyze users' text data and learn the users' language patterns and emotions.

[0819] 2. Response generation means: A means for generating responses in the same way as a user would, based on what has been learned. An appropriate response to the user's input is generated using a generative deep learning model.

[0820] 3. Generation means: A means for generating desired images and videos based on user instructions. Using a generative deep learning model, images and videos are generated based on the user's prompt sentences.

[0821] 4. Sharing method: A method for sharing training files in a common file format of several MB. Use a cloud storage service to share the parameters of the trained model.

[0822] 5. Emotion-adaptive generation means: A means for analyzing the user's emotions and generating images and videos that adapt to those emotions. The emotion engine is used to analyze the user's emotions and generate content based on the results.

[0823] 6. Uploading means: A means for uploading the generated images and videos to a cloud storage service. The generated content is stored in cloud storage and shared with other devices and users.

[0824] Program Processing

[0825] The server receives input from the user and analyzes the emotion using an emotion engine. Based on the analysis result, it passes the prompt text to a generative deep learning model to generate adapted images and videos. The generated content is uploaded to a cloud storage service and made accessible to the user.

[0826] Hardware and Software Used

[0827] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[0828] Software: Natural language processing libraries (e.g., transformers), generative deep learning models (e.g., Stable Diffusion), cloud storage services (e.g., AWS S3)

[0829] Examples

[0830] When a user uses a smartphone app to input "Today was fun," the server uses an emotion engine to parse it as "joy." Next, the server passes the prompt "a joyful scene" to a generative deep learning model, which generates an image of a joyful scene. The generated image is uploaded to a cloud storage service, and the user can view it through the app.

[0831] Examples of prompt statements

[0832] Today was fun

[0833] Based on this prompt sentence, the emotion engine parses it as "joy," and the generative deep learning model generates an image of "a joyful scene."

[0834] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0835] Step 1:

[0836] A user uses a terminal (such as a smartphone, tablet, or PC) to input text such as "Today was fun" into an application. The input text is sent to a server.

[0837] Input: User text input

[0838] Output: Text data sent to the server

[0839] Step 2:

[0840] The server passes the received text data to the emotion engine, which analyzes the user's emotion. The emotion engine uses natural language processing technology to analyze the text data and output an emotion label (e.g., joy).

[0841] Input: User's text data

[0842] Output: Emotion label (e.g. joy)

[0843] Step 3:

[0844] The server generates a prompt sentence to be passed to the generative deep learning model based on the emotion label. For example, if the emotion label is "joy", the server generates the prompt sentence "a joyful scene".

[0845] Input: emotion label

[0846] Output: prompt sentence (e.g. a joyful scene)

[0847] Step 4:

[0848] The server passes the prompt sentence to the generative deep learning model to generate images and videos. The generative deep learning model generates images and videos adapted based on the prompt sentence and returns them to the server.

[0849] Input: Prompt statement

[0850] Output: Generated images and videos

[0851] Step 5:

[0852] The server uploads the generated images and videos to a cloud storage service. Once the upload is complete, a URL for the cloud storage service is generated and returned to the server.

[0853] Input: Generated images and videos

[0854] Output: URL of the cloud storage service

[0855] Step 6:

[0856] The server sends the URL of the cloud storage service to the user's device, and the user can view the images and videos generated through the application.

[0857] Input: URL of cloud storage service

[0858] Output: The URL displayed on the user's device and any generated images or videos.

[0859] Through the above processing steps, images and videos can be generated and shared in response to the user's emotions.

[0860] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0861] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0862] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0863] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0864] [Second embodiment]

[0865] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0866] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0867] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0868] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0869] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0870] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0871] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0872] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0873] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0874] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0875] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0876] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0877] "Example 1"

[0878] In the present invention, a deep learning model is used as a learning means for learning the contents of daily conversations and chats of users and service logs. Specifically, the user's conversation log is used as input data, and the model is trained to imitate the conversation style. This training is performed using a technique called fine-tuning.

[0879] "Example 2"

[0880] The trained deep learning model is used as a reply generator that replies like a user based on the learned content. Specifically, for new input, a reply is generated using the trained model. This reply is based on the trained conversation style of the user.

[0881] "Example 3"

[0882] A generative deep learning model is used as a means of generating desired images and videos. Specifically, specific images and videos are generated based on instructions from a user. For example, if a user instructs the model to "generate an image of a cat," the model generates an image of a cat.

[0883] "Example 4"

[0884] A cloud storage service is used as a means of sharing the training file in a general file format of several MB. Specifically, the parameters of the trained model are converted into a general file format and the file is uploaded to the cloud storage service. Users can use the training results by downloading the file and importing it into their own systems.

[0885] The process flow of each embodiment will be described below.

[0886] "Example 1"

[0887] Step 1: Collect users' daily conversations, chat content, and service logs.

[0888] Step 2: Use the collected data as input data for a deep learning model.

[0889] Step 3: Train the model to learn to mimic the user's speaking style. This training is done using a technique called fine-tuning.

[0890] "Example 2"

[0891] Step 1: Receive new input (e.g., a question or comment from the user).

[0892] Step 2: Use the trained deep learning model to generate a response to the input.

[0893] Step 3: Providing the generated response to the user, which is based on the learned conversation style of the user.

[0894] "Example 3"

[0895] Step 1: Receive a command from a user (e.g., "generate an image of a cat").

[0896] Step 2: Use a generative deep learning model to generate images and videos based on the instructions.

[0897] Step 3: Provide the generated images and videos to the user.

[0898] "Example 4"

[0899] Step 1: Convert the trained model parameters into a common file format.

[0900] Step 2: Upload the converted file to a cloud storage service.

[0901] Step 3: Users can use the learning results by downloading the uploaded files and importing them into their own systems.

[0902] Example 1

[0903] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0904] Conventional conversation generation systems have had difficulty generating natural responses that fully reflect the individual conversation style of the user. In addition, there was a lack of efficient means to carry out a series of processes such as preprocessing of training data, fine-tuning of the model, evaluation, and deployment. This limited the user experience and led to problems such as a decline in the quality of the system's responses.

[0905] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0906] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like a user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several MB, a preprocessing means for the learning means for preprocessing the collected conversation logs and deleting unnecessary information, a training means for the response generating means for training to imitate the user's conversation style using a deep learning model, a fine-tuning means for the training means for fine-tuning the model using a fine-tuning method, an evaluation means for the response generating means for evaluating the trained model, an accuracy confirmation means for the evaluation means for confirming the accuracy of the model using test data, a deployment means for the response generating means for deploying the model that has been evaluated, and an embedding means for the deployment means for incorporating the model into an actual service. This allows for generating natural responses that reflect the individual conversation style of the user, and enables efficient data processing and model training, evaluation, and deployment.

[0907] "User" refers to an individual who uses the system to engage in everyday conversations and chats.

[0908] "Conversation log" refers to data that records the content of conversations and chats conducted by users.

[0909] "Service log" refers to the history and operation records of services used by a user.

[0910] The "learning means" refers to a method or device for learning the contents of users' daily conversations and chats, and service logs.

[0911] "Response generation means" refers to a method or device for generating responses based on what has been learned in the same way as a user would.

[0912] "Generator" refers to a method or device for generating a desired image or video.

[0913] "Sharing means" refers to a method or device for sharing learning files in a common file format of several MB.

[0914] "Preprocessing means" refers to a method or device for preprocessing the collected conversation logs and deleting unnecessary information.

[0915] "Training means" refers to a method or apparatus for training a deep learning model to mimic a user's speaking style.

[0916] "Fine-tuning means" refers to a method or device for fine-tuning a model using a fine-tuning technique.

[0917] "Evaluation means" refers to a method or device for evaluating a trained model.

[0918] "Accuracy verification means" refers to a method or device for verifying the accuracy of a model using test data.

[0919] "Deployment means" refers to a method or device for deploying a model after evaluation has been completed.

[0920] "Integration means" refers to a method or device for incorporating a deployed model into an actual service.

[0921] A "deep learning model" refers to a model that uses a multi-layer neural network to learn data and make predictions or generate results.

[0922] "Fine-tuning" refers to a technique that takes a pre-trained model and fine-tunes it to suit a specific task or dataset.

[0923] This invention is a system that uses a generative AI model that learns the contents of a user's daily conversations and chats, and service logs, and imitates the user's conversation style. A specific embodiment of this system is described below.

[0924] Hardware and Software Used

[0925] Hardware: A server equipped with a high-performance GPU (e.g. NVIDIA Tesla V100)

[0926] Software: Deep learning frameworks (e.g. TensorFlow, PyTorch)

[0927] Data collection

[0928] User: For everyday conversations and chats.

[0929] Terminal: Logs user conversations and chat content.

[0930] Server: Receives the conversation logs sent from the devices and stores them in a database.

[0931] Specific operation: When a user sends a message such as "The weather is nice today" in a chat app, the content is recorded on the device and sent to the server.

[0932] Data Preprocessing

[0933] Server: Preprocesses the collected conversation logs, specifically removing unnecessary information (e.g. emojis, special characters) and cleaning the text data.

[0934] Specific operation: Remove emojis from the message "The weather is nice today ○ (○ represents a specific emoji)" and generate clean text data "The weather is nice today."

[0935] Training the model

[0936] Server: Train the deep learning model using the preprocessed data, using fine-tuning techniques.

[0937] How it works: It uses a pre-trained GPT-3 model to fine-tune the user's speaking style, for example by learning the user's most commonly used phrases and expressions.

[0938] Evaluating the model

[0939] Server: Evaluates the trained model by checking the accuracy of the model using test data.

[0940] Specific behavior: Using the user's past conversation logs as test data, we evaluate how natural the generated responses are. For example, we check whether the response generated for the input "Where are you going today?" is "I'm going to see a movie today."

[0941] Deploying the model

[0942] Server: The model that has completed evaluation is deployed to the server and incorporated into the actual service.

[0943] Terminal: Uses the deployed model to generate real-time responses to user input.

[0944] How it works: A generative AI model built into a chat app responds to a user's input, "What are you planning to do today?" by generating a real-time response such as, "I'm planning to go to a cafe with my friends today."

[0945] Examples of prompt statements

[0946] "Learn the phrases that users use in conversations with their friends and generate responses that mimic that style."

[0947] The system aims to learn the user's conversation style and generate more natural responses, which will provide the user with a more comfortable communication experience.

[0948] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0949] Step 1: Data collection

[0950] User: For everyday conversations and chats.

[0951] Terminal: Logs user conversations and chat content.

[0952] Server: Receives the conversation logs sent from the devices and stores them in a database.

[0953] Input: A message sent by a user in a chat app (e.g., "The weather is nice today").

[0954] Output: Conversation logs stored on the server.

[0955] Specific operation: When a user sends a message such as "The weather is nice today" in a chat app, the content is recorded on the device and sent to the server.

[0956] Step 2: Data Preprocessing

[0957] Server: Preprocesses the collected conversation logs, specifically removing unnecessary information (e.g. emojis, special characters) and cleaning the text data.

[0958] Input: Conversation log stored on the server (e.g. "The weather is nice today ○ (○ represents a specific emoji)").

[0959] Output: Preprocessed, clean text data (e.g., "The weather is nice today").

[0960] Specific operation: Remove emojis from the message "The weather is nice today ○ (○ represents a specific emoji)" and generate clean text data "The weather is nice today."

[0961] Step 3: Training the model

[0962] Server: Train the deep learning model using the preprocessed data, using fine-tuning techniques.

[0963] Input: Preprocessed, clean text data (e.g., "The weather is nice today").

[0964] Output: A deep learning model fine-tuned to the user's speaking style.

[0965] How it works: It uses a pre-trained GPT-3 model to fine-tune the user's speaking style, for example by learning the user's most commonly used phrases and expressions.

[0966] Step 4: Evaluate the model

[0967] Server: Evaluates the trained model by checking the accuracy of the model using test data.

[0968] Input: A trained deep learning model and test data (e.g., a user's past conversation logs).

[0969] Output: The accuracy assessment of the model.

[0970] Specific behavior: Using the user's past conversation logs as test data, we evaluate how natural the generated responses are. For example, we check whether the response generated for the input "Where are you going today?" is "I'm going to see a movie today."

[0971] Step 5: Deploy the model

[0972] Server: The model that has completed evaluation is deployed to the server and incorporated into the actual service.

[0973] Terminal: Uses the deployed model to generate real-time responses to user input.

[0974] Input: A deep learning model that has completed evaluation.

[0975] Output: A generative AI model embedded into a real service.

[0976] How it works: A generative AI model built into a chat app responds to a user's input, "What are you planning to do today?" by generating a real-time response such as, "I'm planning to go to a cafe with my friends today."

[0977] (Application example 1)

[0978] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0979] Conventional content recommendation systems have difficulty in accurately grasping the user's interests and concerns, making it difficult to provide personalized content. In addition, content recommendations are not based on the user's conversation style or individual needs, which results in a lack of improvement in the user experience.

[0980] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0981] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding as the user would based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, and a recommending means for recommending personalized content based on the user's conversation log. This makes it possible to recommend personalized content based on the user's interests and concerns.

[0982] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[0983] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[0984] The "generation means" is a means for generating images and videos desired by the user.

[0985] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[0986] A "recommendation means" is a means for recommending personalized content based on a user's conversation log.

[0987] A system for implementing this invention includes a learning means for learning the contents of a user's daily conversations and chats and service logs, a response generation means for generating responses in the same way as the user would based on the learned contents, a generation means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, and a recommendation means for recommending personalized content based on the user's conversation logs.

[0988] Hardware and Software Configuration

[0989] The server uses the following hardware and software:

[0990] Hardware: A server with a powerful processor, sufficient memory, and storage.

[0991] Software: Python, OpenAI API, Database Management System (e.g. MySQL)

[0992] Data processing and calculation

[0993] The server stores the user's daily conversations, chat contents, and service logs in a database in JSON format. This data is used to fine-tune the deep learning model. Specifically, the model is trained to imitate the user's conversation style using the user's conversation log as input data.

[0994] Recommendation method operation

[0995] The server recommends personalized content based on the user's input. For example, if the user inputs "What are your recent interests?", the server infers the user's interests based on past conversation logs and recommends related content.

[0996] Examples

[0997] When a user uses his / her smartphone to input the question "What are your recent interests?" into the application, the server responds as follows:

[0998] Examples of prompt statements

[0999] User: What are your current interests?

[1000] AI: Based on your recent interests, we'd like to recommend the following content to you:

[1001] 1. Latest Technology News

[1002] 2. Popular Movie Reviews

[1003] 3. New music album release information

[1004] In this way, personalized content recommendations based on the user's interests become possible.

[1005] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1006] Step 1:

[1007] The server collects users' daily conversations, chat contents, and service logs.

[1008] Input: User conversation logs, chat contents, service logs

[1009] Data processing: Save the collected data in JSON format to the database.

[1010] Output: Conversation logs, chat contents, and service logs stored in the database

[1011] Step 2:

[1012] The server uses the collected data to fine-tune the deep learning model.

[1013] Input: Conversation logs, chat contents, and service logs stored in the database

[1014] Data Computing: Training a model to mimic a user's conversational style

[1015] Output: A fine-tuned deep learning model

[1016] Step 3:

[1017] A user inputs a question into the application using a smartphone.

[1018] Input: User question (e.g. "What are your current interests?")

[1019] Data processing: Sending user input to the server

[1020] Output: The user's question sent to the server.

[1021] Step 4:

[1022] The server uses the fine-tuned model to generate answers to users' questions.

[1023] Input: User question sent to the server, fine-tuned deep learning model

[1024] Data Computing: Using models to generate optimal answers to user questions

[1025] Output: The generated response

[1026] Step 5:

[1027] The server recommends personalized content based on the generated responses.

[1028] Input: Generated responses, user's past conversation logs

[1029] Data calculation: Select content that matches the user's interests based on generated responses and past conversation logs

[1030] Output: A personalized list of content

[1031] Step 6:

[1032] The server sends a personalized list of content to the user's smartphone.

[1033] Input: A list of personalized content

[1034] Data processing: Send the list of contents to the user's smartphone

[1035] Output: Personalized content displayed on the user's smartphone.

[1036] Step 7:

[1037] The user checks the personalized content displayed on the smartphone.

[1038] Input: Personalized content displayed on a smartphone

[1039] Data processing: User reviews the content and makes selections as necessary

[1040] Output: User selected content

[1041] In this way, personalized content recommendations based on the user's interests and concerns are realized.

[1042] Example 2

[1043] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1044] Conventional dialogue systems have difficulty generating natural responses based on the user's conversation style, which results in poor user experience. In addition, the quality of the responses generated is low, and they often fail to provide users with appropriate information. Furthermore, there has been a lack of systems that can efficiently process user input data and quickly generate appropriate responses.

[1045] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1046] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like a user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several MB, a receiving means for receiving input data from a user, a preprocessing means for preprocessing the received input data, an input means for inputting the preprocessed data into a generative AI model, a generating means for the generative AI model to generate a response, a postprocessing means for postprocessing the generated response, and a transmitting means for transmitting the postprocessed response to the user. This makes it possible to quickly generate natural responses based on the user's conversation style with high quality.

[1047] A "learning means" is a means for collecting the contents of users' daily conversations and chats, as well as service logs, and training a model based on them.

[1048] The "response generation means" is a means for generating natural responses to user inputs using the model trained by the learning means.

[1049] The "generation means" refers to a means for generating content such as images and videos according to the user's wishes.

[1050] The "sharing means" is a means for sharing the learning file with other users or systems in a general file format of several MB.

[1051] The "receiving means" is a means for receiving input data from a user.

[1052] The "preprocessing means" is a means for performing preprocessing such as normalizing received input data and deleting unnecessary spaces.

[1053] An "input means" is a means for inputting preprocessed data into a generative AI model.

[1054] A "generative AI model" is an artificial intelligence model that generates responses based on user input data.

[1055] "Post-processing means" refers to means for post-processing the response generated by the generative AI model, such as adjusting the format or deleting unnecessary characters.

[1056] A "transmitting means" is a means for transmitting a post-processed reply to a user.

[1057] This invention is a system that learns from a user's daily conversations, chat contents, and service logs, and responds like the user. This system includes a learning means, a response generating means, a generating means, a sharing means, a receiving means, a preprocessing means, an input means, a generating AI model, a postprocessing means, and a transmitting means.

[1058] The server uses a receiving means to receive input data from a user. The receiving means receives data via a communication protocol such as an HTTP request or WebSocket. For example, when a user inputs "What's the weather like today?", this data is received by the receiving means.

[1059] Next, the server preprocesses the received input data using a preprocessing means. For the preprocessing means, Python libraries such as NLTK and spaCy can be used. Preprocessing includes text normalization (e.g., converting uppercase to lowercase, removing unnecessary spaces, etc.). For example, "What's the weather like today?" is converted to "What's the weather like today?".

[1060] The preprocessed data is input to a generative AI model via an input means. Examples of generative AI models that can be used include OpenAI's GPT-3 and Google's BERT. The generative AI model generates a response based on the input data. For example, in response to the input "How's the weather today?", a response such as "Today's weather is sunny" is generated.

[1061] The generated response is post-processed using the post-processing means. The post-processing includes formatting the text and removing unnecessary characters. The post-processed response is transmitted to the user via the transmitting means. This allows the user to receive a natural response from the system.

[1062] As a concrete example, if a user inputs "What are your plans for tomorrow?", the server receives this input, performs preprocessing, and then inputs it into the generative AI model. The generative AI model generates a response such as "Tomorrow's plans are a meeting," and sends it to the user after post-processing.

[1063] An example of a prompt sentence is, "The following are examples of prompt sentences that a user can use to ask the system a question. For example, in response to the question, 'What are your plans for tomorrow?' the system will generate a response using the learned model."

[1064] In this way, the server can generate natural responses based on the user's input and provide them to the user.

[1065] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1066] Step 1:

[1067] The user submits the input data.

[1068] The user opens a chat application on the terminal, types "What's the weather like today?", and presses the send button. The input data is sent to the server in text format.

[1069] Step 2:

[1070] The server receives the input data.

[1071] The server receives the HTTP POST request from the user and obtains the text data "What's the weather like today?" contained in the request body. The input data is received in its original format.

[1072] Step 3:

[1073] The server pre-processes the input data.

[1074] The server normalizes the received text data using preprocessing measures. Specifically, it uses Python's NLTK or spaCy to convert the text to lower case and remove unnecessary spaces. For example, "What's the weather like today?" is converted to "What's the weather like today?". The preprocessed data is output in a state that is ready to be fed into a generative AI model.

[1075] Step 4:

[1076] The server inputs the preprocessed data into the generative AI model.

[1077] The server inputs the preprocessed text data into the generative AI model by sending an API request and passing the data to the model. The input data is converted into a format that the generative AI model can understand.

[1078] Step 5:

[1079] A generative AI model generates the response.

[1080] A generative AI model generates a response based on input data. For example, in response to the input "What's the weather like today?", it generates a response such as "Today's weather is sunny." The generated response is output in text format.

[1081] Step 6:

[1082] The server post-processes the generated reply.

[1083] The server uses post-processing means to format and remove unnecessary characters from the generated response. For example, if a response is generated that says "The weather is sunny today," it will be formatted appropriately. The post-processed data is output ready to be sent to the user.

[1084] Step 7:

[1085] The server sends a post-processed reply to the user.

[1086] The server sends the post-processed reply as an HTTP response to the user, who sees the reply "The weather is sunny today" in the chat application on his terminal. The output data is displayed in a format that the user can understand.

[1087] In this way, the server can generate natural responses based on the user's input and provide them to the user.

[1088] (Application example 2)

[1089] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1090] Conventional content delivery services have the problem that it is difficult to recommend personalized content based on a user's individual conversation style and preferences. It is also difficult to generate natural responses based on a user's conversation style. This can lead to a poor user experience and a decrease in the frequency of service use.

[1091] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1092] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of a user and service logs, a response generating means for responding like a user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several MB, a recommending means for recommending content based on the user's conversation style, and a prompt generating means for generating prompt sentences using a generative AI model. This makes it possible to recommend personalized content based on the individual conversation style and preferences of the user, and is expected to improve the user experience.

[1093] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[1094] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[1095] A "generation means" is a means for generating a desired image or video.

[1096] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[1097] A "recommendation means" is a means for recommending content based on a user's conversation style.

[1098] A "prompt generation means" is a means for generating a prompt sentence using a generative AI model.

[1099] A system for implementing this invention includes a learning means for learning the content of a user's daily conversations and chats and service logs, a response generation means for generating responses like the user's based on the learned content, a generation means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, a recommendation means for recommending content based on the user's conversation style, and a prompt generation means for generating prompt sentences using a generative AI model.

[1100] Hardware and Software Used

[1101] Hardware: Smartphones, servers

[1102] Software: Python, OpenAI API

[1103] Data processing and calculation

[1104] Learning medium

[1105] The server collects the user's daily conversations, chat contents, and service logs, and uses this data to learn the user's conversation style. For learning, fine-tuning is used to optimize the deep learning model.

[1106] Response Generation Method

[1107] The server uses the learned model to generate natural responses to user input. For example, if a user asks, "What are some recent movies you recommend?" the server responds, "The recent movie I recommend is 'Inception.' It's a thrilling film that depicts a dream within a dream!"

[1108] generation means

[1109] The server generates images and videos according to the user's request. For example, if the user requests "Show me an image of a sunset," the server generates and provides an appropriate image.

[1110] means of sharing

[1111] The server shares the training files in a common file format of a few MB, which allows easy sharing of data with other devices and users.

[1112] Recommendation method

[1113] The server recommends content based on the user's conversation style, for example, if the user has a casual conversation style, it recommends movies and music that match that style.

[1114] Prompt Generation Method

[1115] The server uses a generative AI model to generate prompts, such as the following:

[1116] User conversation style: Casual and friendly

[1117] User input: What are some recent movie recommendations?

[1118] reply:

[1119] In this way, personalized content recommendations based on a user's individual conversation style and preferences become possible, which is expected to improve the user experience.

[1120] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1121] Step 1:

[1122] The server collects users' daily conversations, chat contents, and service logs.

[1123] Input: User conversation data, chat logs, service logs

[1124] Output: Collected data

[1125] Specific operation: The server periodically collects logs of conversations and chats conducted by users on their smartphones and other devices and stores them in a database.

[1126] Step 2:

[1127] The server uses the collected data to learn the user's speaking style.

[1128] Input: Collected data

[1129] Output: The trained model

[1130] How it works: The server uses the collected data to fine-tune the deep learning model and learn the user's conversation style, allowing the model to reflect the user's specific conversation patterns and preferences.

[1131] Step 3:

[1132] The server generates natural responses to the user's input.

[1133] Input: User input, trained model

[1134] Output: The generated response

[1135] Specific operation: When a user types "What is a recommended recent movie?" into their smartphone, the server uses the trained model to generate a response such as "A recommended recent movie is 'Inception.' It's a thrilling film that depicts a dream within a dream!"

[1136] Step 4:

[1137] The server generates images and videos according to the user's wishes.

[1138] Input: User request

[1139] Output: Generated images and videos

[1140] Specific behavior: When a user requests "Show me an image of a sunset," the server generates an image of a sunset using an appropriate image generation algorithm and provides it to the user.

[1141] Step 5:

[1142] The server shares the training files in a common file format of several MB.

[1143] Input: Training file

[1144] Output: Shared files

[1145] Specific operation: The server converts the training file into a common file format (e.g., JSON or CSV) of several MB and shares it with other devices and users.

[1146] Step 6:

[1147] The server recommends content based on the user's conversation style.

[1148] Input: User conversation style, content database

[1149] Output: Recommended content

[1150] Specific operation: The server analyzes the user's conversation style and recommends movies, music, and other content that matches that style. For example, for a user with a casual conversation style, it recommends relaxing movies and music.

[1151] Step 7:

[1152] The server generates a prompt sentence using a generative AI model.

[1153] Input: User's speaking style, user input

[1154] Output: Generated prompt statement

[1155] Specific behavior: The server generates a prompt based on the user's speaking style and input. For example, it generates the following prompt:

[1156] User conversation style: Casual and friendly

[1157] User input: What are some recent movie recommendations?

[1158] reply:

[1159] Example 3

[1160] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1161] In conventional systems, when users wanted to generate images or videos, the quality and accuracy of the generated content was often low. In addition, there was a lack of a way to efficiently share the parameters of the trained model, making it difficult for users to easily use the training results. Furthermore, there was no infrastructure in place to provide content that was quickly generated based on user instructions.

[1162] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1163] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like the user based on the learned contents, a generating means using a generative deep learning model for generating desired images and videos, a sharing means using a cloud storage service for sharing learning files in a general file format of several MB, a means for receiving user instructions and inputting them into the generative deep learning model, and a means for displaying the generated images and videos to the user. This enables users to quickly generate high-quality images and videos and efficiently share and use the learning results.

[1164] The "learning means" refers to a means for collecting users' daily conversations, chat contents, and service logs, and training a model using a machine learning algorithm based on that data.

[1165] The "answer generation means" is a means for generating a reply to a user's input in a manner similar to that of the user, using the model trained by the learning means.

[1166] A "generative deep learning model" is a model that uses deep learning algorithms to generate specific images or videos based on user instructions.

[1167] A "generation means" is a means for generating images and videos based on user instructions using a generative deep learning model.

[1168] The "sharing means" refers to a means for converting the parameters of the trained model into a common file format of a few MB and sharing it via a cloud storage service.

[1169] A "cloud storage service" is an online service for storing and sharing data via the Internet.

[1170] A "server" is a computer system whose role is to receive instructions from a user, input them into a generative deep learning model, and return the generated images and videos to the user.

[1171] A "terminal" is a device through which a user inputs instructions and through which generated images and videos are displayed.

[1172] The present invention provides a system for generating images and videos that a user desires and efficiently sharing parameters of a trained model. A specific embodiment of this system will be described below.

[1173] Hardware and Software Used

[1174] 1. Server: A high-performance computer system that receives instructions from the user and inputs them into the generative deep learning model. The server also returns the generated images and videos to the user.

[1175] 2. Terminal: A device on which a user inputs instructions and displays the generated images and videos. Terminals include PCs, smartphones, tablets, etc.

[1176] 3. Generative deep learning model: A model that uses deep learning algorithms to generate specific images or videos based on user instructions. For example, GAN (Generative Adversarial Network) and VAE (Variational Autoencoder) are used.

[1177] 4. Cloud storage services: Online services for storing and sharing data over the Internet. Examples include Google Drive and Dropbox.

[1178] System Operation

[1179] 1. The user inputs instructions to generate the desired image or video. For example, the user might input "generate an image of a cat."

[1180] 2. The terminal sends the user's instructions to the server. The terminal sends the entered instructions to the server as an HTTP request.

[1181] 3. The server receives the instructions and inputs them into the generative deep learning model. The server receives the HTTP request, analyzes the instructions, and passes them to the generative deep learning model.

[1182] 4. The generative deep learning model generates images and videos based on instructions. Based on the instruction "generate an image of a cat," the generative deep learning model uses its internal algorithms to generate an image of a cat.

[1183] 5. The server receives the generated images and videos and sends them to the device. The server receives the generated images and sends them to the device as an HTTP response.

[1184] 6. The device displays the images and videos it has received to the user. The device displays the images it has received on its screen so that the user can view them.

[1185] Examples

[1186] For example, if a user instructs the server to "generate a video of a landscape," the server inputs the instruction into the generative deep learning model, which then generates a video of the landscape. The generated video is then provided to the user through the server.

[1187] Example prompt:

[1188] "Generate an image of a cat"

[1189] "Generate a video of the scenery"

[1190] Sharing study files

[1191] 1. The server converts the model parameters trained into a common file format. For example, the server converts the model parameters trained into a file called "model_parameters.bin".

[1192] 2. The server uploads the converted file to a cloud storage service. The server uploads "model_parameters.bin" to Google Drive.

[1193] 3. The user accesses the cloud storage service and downloads the necessary files. The user accesses Google Drive and downloads "model_parameters.bin".

[1194] 4. The user imports the downloaded file into their own system and uses the learning results. The user imports the downloaded "model_parameters.bin" into their own system and uses the learning results.

[1195] In this way, the user can quickly generate high-quality images and videos and efficiently share and use the learning results. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1196] Step 1:

[1197] The user inputs instructions to generate the desired image or video.

[1198] Input: The user types "generate a picture of a cat" into the input form on the terminal.

[1199] Specific operation: The user uses the device's keyboard or touch screen to input the content of the image or video to be generated in text format.

[1200] Output: Your terminal will display the instructions and you are ready to move on to the next step.

[1201] Step 2:

[1202] The terminal transmits the user's instructions to the server.

[1203] Input: Instructions entered by the user.

[1204] Specific operation: The terminal sends the input instructions to the server as an HTTP request.

[1205] Output: The server receives the HTTP request and parses the instructions.

[1206] Step 3:

[1207] The server receives the instructions and inputs them into the generative deep learning model.

[1208] Input: The HTTP request sent from the terminal.

[1209] Specific operation: The server receives an HTTP request, analyzes the instructions, and passes them to the generative deep learning model.

[1210] Output: The generative deep learning model receives the instructions.

[1211] Step 4:

[1212] Generative deep learning models generate images and videos based on instructions.

[1213] Input: Instructions passed from the server.

[1214] Specific operation: Based on the instruction "generate an image of a cat," the generative deep learning model uses an internal algorithm to generate an image of a cat.

[1215] Output: The generated image data.

[1216] Step 5:

[1217] The server receives the generated images and videos and sends them to the terminal.

[1218] Input: Generated image data from a generative deep learning model.

[1219] Specific operation: The server receives the generated image and sends it to the terminal as an HTTP response.

[1220] Output: The device receives the HTTP response.

[1221] Step 6:

[1222] The device displays the images and videos it receives to the user.

[1223] Input: The generated image data sent from the server.

[1224] Specific operation: The image received by the device is displayed on the screen so that the user can check it.

[1225] Output: The user visually checks the generated image.

[1226] Step 7:

[1227] The server converts the trained model parameters into a common file format.

[1228] Input: The parameters of the trained model.

[1229] Specific operation: The server converts the parameters of the trained model into a file called "model_parameters.bin".

[1230] Output: Model parameter files converted to a common file format.

[1231] Step 8:

[1232] The server uploads the converted files to a cloud storage service.

[1233] Input: Model parameter files converted to a common file format.

[1234] Specific operation: The server uploads "model_parameters.bin" to the cloud storage service.

[1235] Output: The file uploaded to the cloud storage service.

[1236] Step 9:

[1237] A user accesses a cloud storage service and downloads a required file.

[1238] Input: A file uploaded to a cloud storage service.

[1239] Specific operation: A user accesses a cloud storage service and downloads "model_parameters.bin".

[1240] Output: The files downloaded to the user's device.

[1241] Step 10:

[1242] The user imports the downloaded file into their own system and uses the learning results.

[1243] Input: The downloaded “model_parameters.bin” file.

[1244] Specific operation: The user imports the downloaded "model_parameters.bin" into their own system and uses the learning results.

[1245] Output: Training results available on the user's system.

[1246] (Application example 3)

[1247] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1248] In conventional image and video generation systems using generative deep learning models, it was difficult for users to quickly and efficiently generate and share the content they wanted. In addition, there was a lack of means to upload the generated content to cloud storage and share it with other users, which resulted in low user convenience. Furthermore, there was a lack of sharing of learning files tailored to individual needs.

[1249] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1250] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding as the user would based on the learned contents, a generating means for generating desired images and videos, an uploading means for uploading the generated images and videos to cloud storage, and a sharing means for sharing learning files in a general file format of several MB. This allows the user to quickly and efficiently generate desired content and share it with other users through cloud storage. It also allows sharing of learning files tailored to individual needs.

[1251] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[1252] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[1253] The "generation means" is a means for generating desired images and videos based on user instructions.

[1254] The "uploading means" is a means for uploading the generated images and videos to cloud storage.

[1255] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[1256] A system for implementing this invention includes a learning means for learning the contents of a user's daily conversations and chats and service logs, a response generation means for generating responses in the manner of the user based on the learned contents, a generation means for generating desired images and videos, an uploading means for uploading the generated images and videos to cloud storage, and a sharing means for sharing learning files in a common file format of several MB.

[1257] Explanation of system program processing

[1258] Hardware and Software

[1259] The server uses a high-performance computer equipped with a GPU. The software uses the transformers library to execute generative deep learning models, the requests library to process HTTP requests, and the PIL library for image processing.

[1260] Data processing and calculation

[1261] 1. Learning method: The server collects users' daily conversations, chat contents, and service logs, and uses them to train a generative deep learning model. Fine-tuning is performed using the transformers library for learning.

[1262] 2. Response generation: Using the trained model, we generate appropriate responses based on the user's input, enabling responses that sound more natural than the user would.

[1263] 3. Generation: A user inputs a prompt, and the generative deep learning model generates the desired image or video based on the prompt. For example, if a user inputs "generate an image of a beach at sunset," the model generates the image.

[1264] 4. Uploading method: The generated images and videos are uploaded to cloud storage by sending an HTTP request using the requests library.

[1265] 5. Sharing: The learning files are converted into a common file format of several MB and uploaded to cloud storage. Users can download the files and import them into their own systems.

[1266] Examples

[1267] When a user enters the prompt "Generate an image of a cat," the generative deep learning model generates an image of a cat and uploads it to cloud storage. The user can then download and use the uploaded image.

[1268] Example prompt:

[1269] Generate a cat image

[1270] Generate sunset beach images

[1271] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1272] Step 1:

[1273] The user enters a prompt sentence.

[1274] Input: A prompt describing the image or video the user is looking for (e.g., "Generate an image of a cat").

[1275] How it works: A user enters a prompt through a smartphone or computer interface.

[1276] Output: The prompt text is sent to the server.

[1277] Step 2:

[1278] The server receives the prompt sentence and inputs it into the generative deep learning model.

[1279] Input: The prompt text sent by the user.

[1280] How it works: The server receives a prompt text and uses the transformers library to feed the prompt text to a generative deep learning model.

[1281] Output: A generation request based on the prompt is passed to the model.

[1282] Step 3:

[1283] A generative deep learning model generates images and videos based on the prompt text.

[1284] Input: A generated request based on the prompt statement.

[1285] How it works: The model analyzes the prompt and generates an appropriate image or video. For example, in response to the prompt "generate an image of a cat," the model generates an image of a cat.

[1286] Output: The generated image and / or video data.

[1287] Step 4:

[1288] The server uploads the generated images and videos to cloud storage.

[1289] Input: Generated image and / or video data.

[1290] How it works: The server uses the requests library to upload the generated images and videos to cloud storage.

[1291] Output: URL of uploaded image or video to cloud storage.

[1292] Step 5:

[1293] The server returns the cloud storage URL to the user.

[1294] Input: The URL of the image or video uploaded to cloud storage.

[1295] How it works: The server sends the URL of the generated content back to the user.

[1296] Output: The URL where the user can access it.

[1297] Step 6:

[1298] The user downloads the generated images and videos from the cloud storage URL.

[1299] Input: The URL returned by the server.

[1300] How it works: The user visits a URL and downloads the generated image or video.

[1301] Output: Images and videos stored on the user's device.

[1302] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1303] "Example 1"

[1304] In one embodiment of the present invention, an emotion engine is used as a learning means for learning a user's daily conversations, chat contents, and service logs. The emotion engine recognizes emotions from the user's speech and incorporates the emotions as learning data. For example, if a user says, "Today was fun," the emotion engine recognizes the emotion of "joy" from this speech and incorporates the information as learning data.

[1305] "Example 2"

[1306] In addition, an emotion engine is used as a reply generating means that replies like the user based on the learned content. The emotion engine generates a reply adapted to the user's emotions based on the learned emotion data. For example, if the user says, "Today was fun," the emotion engine generates a reply adapted to joy, such as "That's wonderful."

[1307] "Example 3"

[1308] Furthermore, an emotion engine is used as a generating means for generating desired images and videos. The emotion engine generates images and videos adapted to the user's emotions based on learned emotion data. For example, if a user says, "Today was fun," and requests, "Show me an image of a happy scene," the emotion engine generates an image of a joyful scene.

[1309] "Example 4"

[1310] Finally, the emotion engine is used as a means of sharing the learning file in a general file format of several MB. The emotion engine converts the learned emotion data into a general file format and shares the file. This allows other systems and users to use the learning results of the emotion engine.

[1311] The process flow of each embodiment will be described below.

[1312] "Example 1"

[1313] Step 1: Receive the user's utterance.

[1314] Step 2: The emotion engine recognizes emotions from the user's speech.

[1315] Step 3: The recognized emotions are input as learning data.

[1316] "Example 2"

[1317] Step 1: Receive the user's utterance.

[1318] Step 2: Based on the emotion data learned by the emotion engine, a response is generated that is adapted to the user's emotions.

[1319] "Example 3"

[1320] Step 1: Receive user speech and a request for images or videos.

[1321] Step 2: Based on the emotional data learned by the emotion engine, images and videos are generated that are adapted to the user's emotions.

[1322] "Example 4"

[1323] Step 1: Convert the emotion data learned by the emotion engine into a common file format.

[1324] Step 2: Share the converted file.

[1325] Example 1

[1326] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1327] Conventional systems had difficulty effectively learning the content of users' daily conversations and chats and generating responses appropriate to the user. In addition, they lacked the ability to recognize users' emotions and generate responses based on them, limiting the user experience. Furthermore, when it came to sharing learning files, flexible responses tailored to individual needs were required.

[1328] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1329] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like the user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several megabytes, an emotion recognition means for recognizing emotions from the user's speech and incorporating the emotions as learning data, and a response generating means for generating responses suitable for the user using a generative AI model. This makes it possible to effectively learn the user's conversation style and generate responses based on emotions. It is also possible to share learning files tailored to individual needs.

[1330] A "learning means" is a means for collecting users' daily conversations, chat contents, and service logs, and training a deep learning model based on them.

[1331] The "response generating means" is a means for imitating the user's conversation style based on the learned content and generating an appropriate reply.

[1332] The "generation means" is a means for generating images and videos according to the user's wishes.

[1333] "Sharing means" refers to a means for sharing learning files in a common file format of several megabytes.

[1334] The "emotion recognition means" is a means for recognizing emotions from the user's speech and incorporating the emotions as learning data.

[1335] A "response generation means" is a means for generating a response appropriate to a user using a generative AI model.

[1336] "Fine-tuning" is a technique for optimizing an existing deep learning model to fit a user's data.

[1337] The present invention is a system that learns the contents of a user's daily conversations and chats, and service logs, and generates responses appropriate to the user. A specific embodiment of this system will be described below.

[1338] 1. Program Generation

[1339] The server generates a program to learn the user's daily conversations, chat contents, and service logs. The program is designed to imitate the user's conversation style using a deep learning model.

[1340] 2. Program Processing Explanation

[1341] The server receives the user's conversation log as input data and uses a deep learning model to train the model to imitate the user's conversation style. This training uses a technique called fine-tuning. Specifically, the following process is performed:

[1342] Data collection: The server collects the user's daily conversations, chat contents, and service logs. For example, if a user sends a message such as "I had fun today" in a chat app, the message is stored on the server.

[1343] Data preprocessing: The server preprocesses the collected data and converts it into a format suitable for the model. Specifically, it normalizes and tokenizes the text. For example, it tokenizes a statement such as "Today was fun" into "Today was fun."

[1344] Model training: The server trains a deep learning model using the preprocessed data, using fine-tuning techniques to fine-tune the existing model to the user's speaking style, using deep learning frameworks such as TensorFlow and PyTorch.

[1345] Emotion recognition: The server uses the emotion engine to recognize emotions from the user's speech. For example, if a user says, "Today was fun," the emotion engine recognizes the emotion of "joy" from this speech and incorporates that information into learning data.

[1346] Response generation: The server uses the generative AI model to generate a response appropriate for the user. Specifically, the server creates a prompt sentence based on the user's utterance, inputs it into the generative AI model, and generates an appropriate response.

[1347] 3. Examples of concrete examples and prompts

[1348] As a specific example, consider a case where a user says, "Today was fun." This utterance is processed as follows.

[1349] The server collects these comments and stores them as text data.

[1350] The emotion engine recognizes the emotion "joy" from this statement.

[1351] The recognized emotional information is incorporated into the deep learning model as training data.

[1352] The server uses this data to fine-tune the model, training it to mimic the user's speaking style.

[1353] The server generates a prompt sentence, "Tell me more about your fun events," and inputs it into the generative AI model to generate a response, "That's great! What did you enjoy?"

[1354] Examples of prompt sentences to be input to a generative AI model include the following:

[1355] "Please tell us about an event that you found enjoyable."

[1356] "Please tell us what your plans are for the day when you feel busy."

[1357] "Please tell us about movies that you found interesting."

[1358] In this way, the server learns the user's conversational style and uses a generative AI model to generate responses that are appropriate for the user.

[1359] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1360] Step 1: Data collection

[1361] The server collects the user's daily conversations, chat contents, and service logs. For example, when a user sends a message such as "I had fun today" in a chat app, the message is stored in the server. The input is the user's text data, and the output is the stored text data.

[1362] Step 2: Data Preprocessing

[1363] The server pre-processes the collected data. Specifically, it performs the following processes:

[1364] Text normalization: removing unnecessary spaces and special characters from what the user says.

[1365] Tokenization: Splitting text into words and phrases. For example, tokenizing an utterance like "I had fun today" into "I had fun today".

[1366] Labeling: Using the emotion engine, an emotion label is assigned to each utterance. For example, a utterance such as "Today was fun" is labeled as "Joy."

[1367] The input is stored text data, and the output is preprocessed text data and emotion labels.

[1368] Step 3: Model training

[1369] The server trains a deep learning model using the preprocessed data, specifically by following the steps below.

[1370] Dataset Creation: We create a dataset by combining the preprocessed text with the sentiment labels.

[1371] Fine-tuning: Optimizing an existing deep learning model to fit your data, for example by updating the model parameters using TensorFlow or PyTorch.

[1372] The input is preprocessed text data and sentiment labels, and the output is a trained deep learning model.

[1373] Step 4: Emotion Recognition

[1374] The server uses the emotion engine to recognize emotions from the user's speech. Specifically, it performs the following processes.

[1375] Text analysis: Analyze what the user says and input it into the emotion engine.

[1376] Emotion labeling: The emotion engine recognizes emotions from utterances and assigns corresponding emotion labels. For example, a statement like "Today was fun" is labeled as "Joy."

[1377] The input is the user's spoken text and the output is an emotion label.

[1378] Step 5: Response Generation

[1379] The server uses the generative AI model to generate a response appropriate for the user. Specifically, it follows the steps below.

[1380] Prompt creation: Create a prompt based on what the user says. For example, if the user says, "I had fun today," the prompt will be, "Tell me more about fun things that happened to you."

[1381] Response generation: A prompt is fed into the generative AI model to generate an appropriate response, for example, "That's great! What did you enjoy?"

[1382] The input is the prompt sentence and the output is the generated response.

[1383] In this way, the server learns the user's conversational style and uses a generative AI model to generate responses that are appropriate for the user.

[1384] (Application example 1)

[1385] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1386] Conventional content delivery services have the problem that it is difficult to recommend personalized content that takes into account the individual conversation style and emotions of each user. This makes it impossible to provide optimal content according to the user's interests and emotions, preventing an improvement in the user experience.

[1387] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a learning means for learning the contents of daily conversations and chats of the user and service logs, a response generating means for responding like the user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, a recommendation means for learning the user's conversation style and emotions and providing individually customized content, and a recommendation generating means for recommending customized content based on the user's input. This enables personalized content recommendation based on the user's individual conversation style and emotions.

[1388] The "contents of the user's daily conversations and chats" are records of the user's daily conversations and message exchanges.

[1389] A "service log" is a record of operation history and usage status regarding a service used by a user.

[1390] A "learning means" is a method or device for analyzing user conversations and service logs, extracting patterns and features, and training a model.

[1391] A "response generating means" is a method or device for generating a response based on what has been learned, just like a user would.

[1392] A "generation means" is a method or device for generating content such as images and videos in accordance with a user's wishes.

[1393] A "sharing means" is a method or device for sharing a learning file with other users or systems in a general file format of several MB.

[1394] A "recommender" is a method or device for learning a user's speech style and emotions and providing individually customized content.

[1395] A "recommendation generator" is a method or apparatus for recommending customized content based on user input.

[1396] "Fine-tuning" is a technique that involves additional training of an existing deep learning model to adapt it to a specific task or dataset.

[1397] As an embodiment of the present invention, a system is constructed that learns the contents of daily conversations and chats of users and service logs, and provides individually customized content. A specific implementation method thereof is shown below.

[1398] System Configuration

[1399] 1. Hardware:

[1400] Server: Use a server with high-performance computing power to train and infer deep learning models.

[1401] Terminal: Interacts with the user using a user terminal such as a smartphone or smart glasses.

[1402] 2. Software:

[1403] TensorFlow: Used for training and inference of deep learning models.

[1404] Transformers library: Used to leverage generative AI models such as GPT-2.

[1405] Emotion recognition pipeline: Used to recognize emotions from user speech.

[1406] Data processing and calculation

[1407] 1. Data collection:

[1408] The system collects users' daily conversations, chat content, and service logs. This data is sent from the user's device to the server.

[1409] 2. Data Analysis:

[1410] The server analyzes the collected data to extract the user's speaking style and emotions. An emotion recognition pipeline is used to recognize emotions from the user's speech.

[1411] 3. Train the model:

[1412] The server uses the analyzed data to fine-tune the deep learning model, which generates a model based on the user's conversation style and emotions.

[1413] 4. Content Recommendation:

[1414] The server recommends customized content based on user input, using generative AI models to generate content according to the user's interests and emotions.

[1415] Examples

[1416] For example, when a user inputs "What movie should I watch today?", the server performs the following process.

[1417] 1. Parse the user input and recognize emotions using the emotion recognition pipeline.

[1418] 2. Using a deep learning model, we recommend the best movies based on the recognized emotions and past conversation logs.

[1419] 3. The list of recommended movies is sent to the user's device and displayed.

[1420] Examples of prompt statements

[1421] Using the user's conversation log as input data, the emotion engine recognizes emotions and incorporates those emotions as training data. For example, if a user says, "Today was fun," the emotion engine recognizes the emotion of "joy" from this statement and incorporates that information as training data. This data is used to fine-tune the deep learning model to mimic the user's conversation style.

[1422] In this way, personalized content recommendations based on a user's individual speaking style and emotions are possible.

[1423] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1424] Step 1:

[1425] Users input the contents of their daily conversations and chats into their terminals. The terminals collect this data and send it to the server. The input data is a conversation log or chat log in text format.

[1426] Step 2:

[1427] The server analyzes the received conversation logs and chat logs. Specifically, it uses an emotion recognition pipeline to recognize emotions from each utterance. The input is the conversation log, and the output is the emotion label corresponding to each utterance.

[1428] Step 3:

[1429] The server uses the analyzed data to fine-tune a deep learning model. Specifically, it uses TensorFlow and the Transformers library to train a model based on the user's conversation style and emotions. The input is the conversation log with emotion labels, and the output is a fine-tuned generative AI model.

[1430] Step 4:

[1431] The user inputs a new question to the terminal. For example, the user inputs a question such as "What movie should I watch today?" The input data is the user's question.

[1432] Step 5:

[1433] The server analyzes the user's input and generates the best content using a generative AI model. Specifically, it tokenizes the user's question, inputs it into the model, and decodes the generated text. The input is the user's question, and the output is a list of recommended content.

[1434] Step 6:

[1435] The server sends the generated list of contents to the terminal. The terminal displays the received contents to the user. The input is the list of recommended contents, and the output is the contents displayed on the terminal.

[1436] Step 7:

[1437] The user checks the recommended content and selects or views it as necessary, which provides the user with the most suitable content according to their interests and emotions.

[1438] Example 2

[1439] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1440] Conventional systems have had difficulty generating responses that adapt to the user's conversation style and emotions. They also lacked the means to generate appropriate responses to new input based on what they had learned. Furthermore, sharing of learning files was not tailored to individual needs, making it difficult to share information efficiently.

[1441] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1442] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding as the user would based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, a means for generating responses using the learned model for new inputs, and a means for generating responses adapted to the user's emotions based on the learned emotion data. This makes it possible to generate responses adapted to the user's conversation style and emotions, and realizes efficient information sharing.

[1443] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[1444] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[1445] A "generation means" is a means for generating a desired image or video.

[1446] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[1447] "Means for generating a response using a model trained for a new input" refers to means for generating an appropriate response for a new input using a model trained in advance.

[1448] The "means for generating a response adapted to the user's emotions based on learned emotion data" refers to a means for generating a response adapted to the user's emotions based on learned emotion data.

[1449] This invention is a system that learns the contents of daily conversations and chats of users and service logs, and generates replies based on the learned contents as if the user were responding. Furthermore, it can generate desired images and videos, and share the learning files in a general file format of several MB.

[1450] Hardware and Software Used

[1451] Hardware: Servers, terminals

[1452] Software: Deep learning model, emotion engine

[1453] Program Processing

[1454] 1. The user provides input

[1455] A user enters text into an input field on a terminal.

[1456] Example: A user types, "How was your day?"

[1457] 2. The device sends input to the server

[1458] The terminal transmits the user's input to the server.

[1459] Example: A device sends text data such as "How was your day?" to a server.

[1460] 3. The server receives the input and passes it to the deep learning model.

[1461] The server analyzes the input received from the device and passes it to the deep learning model.

[1462] Example: A server inputs the text "How was your day?" into a deep learning model.

[1463] 4. The deep learning model generates a response

[1464] Deep learning models generate responses based on the data they have learned from.

[1465] Example: A deep learning model generates the response "Today was a very good day."

[1466] 5. The server uses an emotion engine to tailor the response

[1467] The server passes the generated response to an emotion engine, which tailors the response to the user's emotion.

[1468] Example: An emotion engine refines "Today was a great day" to "That's wonderful."

[1469] 6. The server sends the final response to the device.

[1470] The server sends the final adjusted response to the terminal.

[1471] Example: The server sends the text "That's great" to the device.

[1472] 7. The device displays the reply to the user

[1473] The terminal displays the received response to the user.

[1474] Example: A device displays the text "That's great" on the screen.

[1475] Examples and prompts

[1476] Examples

[1477] User: "How was your day?"

[1478] Server: "Today was a great day."

[1479] User: "Today was fun."

[1480] Server: "That's great."

[1481] Examples of prompt statements

[1482] "If a user types 'How was your day?', how should the server respond?"

[1483] "If a user says, 'I had fun today,' what response do we generate using the emotion engine?"

[1484] In this way, the server is a system that uses deep learning models and emotion engines to generate responses that adapt to the user's conversation style and emotions.

[1485] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1486] Step 1:

[1487] The user provides input

[1488] Input: The user types text into an input field on the terminal.

[1489] What happens: A user types, "How was your day?"

[1490] Output: Text data entered into the terminal.

[1491] Step 2:

[1492] The terminal sends input to the server

[1493] Input: Text data entered into the terminal.

[1494] Specific operation: The device sends text data such as "How was your day?" to the server.

[1495] Output: The text data sent to the server.

[1496] Step 3:

[1497] The server receives the input and passes it to the deep learning model.

[1498] Input: The text data sent to the server.

[1499] What happens: The server receives the text "How was your day?" and inputs it into the deep learning model.

[1500] Output: The text data passed to the deep learning model.

[1501] Step 4:

[1502] The deep learning model generates the answer

[1503] Input: Text data passed to the deep learning model.

[1504] What it does: A deep learning model generates a response based on the data it has learned from.

[1505] Output: The response text "Today was a great day."

[1506] Step 5:

[1507] The server uses an emotion engine to tailor the response

[1508] Input: The response text generated by the deep learning model.

[1509] What happens: The server passes the generated response to the emotion engine, which adjusts the response to suit the user's emotions.

[1510] Output: The tailored response text: "That's great."

[1511] Step 6:

[1512] The server sends the final response to the device.

[1513] Input: The adjusted response text.

[1514] Specific action: The server sends the text "That's great" to the device.

[1515] Output: The reply text sent to the terminal.

[1516] Step 7:

[1517] The device displays the reply to the user

[1518] Input: The reply text sent to the terminal.

[1519] What happens: The device will display the text "That's great" on the screen.

[1520] Output: The response text displayed to the user.

[1521] (Application example 2)

[1522] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1523] Conventional content distribution services do not adequately recommend content personalized based on the user's viewing history or emotions. In addition, they lack the ability to generate responses that adapt to the user's emotions through dialogue with the user, limiting the user experience. This leads to problems such as a decline in user satisfaction and engagement.

[1524] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1525] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like the user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, a recommendation means for recommending content based on the user's viewing history and emotion data, and an emotion-adaptive response generating means for generating responses adapted to the emotions of users' comments. This makes it possible to provide personalized content recommendations based on the user's viewing history and emotions, and an emotion-adaptive interactive interface.

[1526] The "learning means" is a means for collecting the contents of users' daily conversations and chats, as well as service logs, and training a model based on these.

[1527] The "reply generating means" is a means for generating a reply that resembles the user's conversation style based on the learned content.

[1528] The "generation means" is a means for generating images and videos according to the user's wishes.

[1529] "Sharing means" refers to a means for sharing learning files with other systems or users in a general file format of several MB.

[1530] A "recommendation means" is a means for recommending appropriate content based on a user's viewing history and emotional data.

[1531] The "emotion-adaptive response generating means" is a means for generating a response to a user's comment that is adapted to the emotion of the user.

[1532] A system for implementing this invention includes a learning means for learning the contents of a user's daily conversations and chats and service logs, a response generation means for responding like the user would based on the learned contents, a generation means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, a recommendation means for recommending content based on the user's viewing history and emotional data, and an emotion-adaptive response generation means for generating responses to user comments that are emotion-adapted.

[1533] Hardware and Software Used

[1534] Hardware: Smartphone, head-mounted display

[1535] Software: Python, OpenAI API

[1536] Data processing and calculation

[1537] Learning medium

[1538] The server collects the user's daily conversations, chat contents, and service logs, and trains a deep learning model based on this data. Fine-tuning technology is used for training to generate a model that is adapted to the individual conversation style of the user.

[1539] Response Generation Method

[1540] The server uses the trained model to generate appropriate responses to user comments. For example, if a user says, "I had fun today," the server uses the emotion engine to generate a response such as, "That's great."

[1541] generation means

[1542] The server generates images and videos based on the user's preferences, using a generative AI model to provide high-quality content.

[1543] means of sharing

[1544] The server shares the learning files with other systems and users in a common file format of several MB, ensuring data compatibility across different devices.

[1545] Recommendation method

[1546] The server recommends appropriate content based on the user's viewing history and emotion data. For example, if the user says, "Today was fun," the server recommends comedy movies based on the emotion data.

[1547] Emotion-adaptive response generation method

[1548] The server generates a response that is adapted to the user's emotions in response to the user's utterances, using the OpenAI API to realize a natural dialogue that is adapted to the user's emotions.

[1549] Examples

[1550] User says: "Today was fun"

[1551] Assistant replies: "That's great! If you're in a fun mood, this comedy movie is a great choice."

[1552] Examples of prompt statements

[1553] User said: Today was fun

[1554] Generate responses that are emotionally appropriate.

[1555] In this way, it is possible to provide personalized content recommendations based on a user's viewing history and emotions, as well as an emotionally adaptive interactive interface.

[1556] The flow of the identification process in the application example 2 will be described with reference to FIG.

[1557] Step 1:

[1558] The server collects the user's daily conversations, chat contents, and service logs. These data are input as the user's remarks and behavior history. The collected data is stored in a database and used in the later learning process.

[1559] Step 2:

[1560] The server trains a deep learning model based on the collected data. Fine-tuning technology is used for this training. User conversation data and service logs are used as input, and a model adapted to the user's conversation style is generated as output.

[1561] Step 3:

[1562] The server uses the trained model to generate appropriate responses to user utterances. The input is the user's current utterance, and the output is a response based on the user's conversational style. For example, if the user says "I had fun today," the emotion engine generates a response such as "That's great."

[1563] Step 4:

[1564] The server generates images and videos according to the user's wishes: as input, the user's wishes and requests are used, and as output, images and videos generated using a generative AI model are provided.

[1565] Step 5:

[1566] The server shares the training files with other systems and users in a common file format of several MB. The training files are used as input, and data converted into a shareable file format is provided as output.

[1567] Step 6:

[1568] The server recommends appropriate content based on the user's viewing history and emotion data. The user's viewing history and emotion data are used as input, and content recommendations adapted to the user's emotion are output. For example, if the user says, "Today was fun," comedy movies are recommended based on the emotion data.

[1569] Step 7:

[1570] The server generates a response that is adapted to the user's emotions in response to the user's utterances. The input is the user's utterance and emotion data, and the output is a response that is adapted to the emotion. This uses the OpenAI API to realize a natural dialogue that is adapted to the user's emotions.

[1571] In this way, it is possible to provide personalized content recommendations based on a user's viewing history and emotions, as well as an emotionally adaptive interactive interface.

[1572] Example 3

[1573] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1574] Conventional systems have had problems with generating images and videos that correspond to the user's emotions and specific requests, and with efficiently sharing trained model parameters and emotion data. For this reason, there is a demand for a system that can respond to the diverse needs of users and efficiently share learning results.

[1575] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1576] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding as if the user were to respond based on the learned contents, a generating means for generating desired images and videos using a generative deep learning model, a sharing means for uploading the learning file to a cloud storage service in a general file format of several MB and sharing it, a generating means for generating images and videos adapted to the user's emotions using an emotion engine, and a sharing means for converting the emotion data learned using the emotion engine into a general file format and sharing it. This makes it possible to generate images and videos according to the various needs of users, and to efficiently share the learning results.

[1577] A "user" is an individual or group that uses the system to perform operations such as generating images and videos and sharing learning files.

[1578] The "learning means" is a function for analyzing the contents of users' daily conversations and chats, as well as service logs, and learning the data.

[1579] The "response generating means" is a function for generating responses based on learned content in the same way as a user would.

[1580] A "generative deep learning model" is a deep learning algorithm for generating specific images or videos based on user instructions.

[1581] "Generation means" refers to a function for generating desired images or videos using a generative deep learning model or emotion engine.

[1582] "Sharing method" is a function for converting learning files into a common file format of several MB and uploading them to a cloud storage service for sharing.

[1583] A "cloud storage service" is an online service for storing and sharing data via the Internet.

[1584] An "emotion engine" is an algorithm for generating adaptive images and videos based on a user's emotions.

[1585] "Common file formats" are data formats that are widely used, including JSON and CSV.

[1586] "Fine-tuning" is a technique in which an existing learning model is subjected to additional training in order to adapt it to a specific task.

[1587] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1588] This invention is a system that learns the contents of daily conversations and chats of users and service logs, and generates images and videos based on the user's instructions. It also has a function to efficiently share the parameters and emotion data of the learned model.

[1589] Hardware and Software Used

[1590] The server uses the following hardware and software:

[1591] Hardware: A server equipped with a high-performance GPU (e.g. NVIDIA GPU)

[1592] Software: Generative deep learning models (e.g. GAN, VQ-VAE), emotion engines, cloud storage services (e.g. Google Drive, Dropbox)

[1593] The terminal is a device on which the user inputs the prompt text and checks the generated images and videos. Specifically, it includes a PC, a smartphone, a tablet, etc.

[1594] Program Processing

[1595] The user inputs the content of the image or video they want to generate as a prompt using the terminal. For example, they input prompts such as "Generate an image of a cat" or "Today was fun. Show me an image of a fun scene."

[1596] The server analyzes the prompt received from the user and inputs it into the generative deep learning model and emotion engine. The generative deep learning model generates images and videos according to the user's instructions based on pre-trained data. The emotion engine generates images and videos that adapt to the user's emotions.

[1597] The generated images and videos are sent from the server to the user's device, where they can be viewed by the user.

[1598] Sharing study files

[1599] The server converts the trained model parameters into a common file format of several MB (e.g., JSON, HDF5) and uploads it to a cloud storage service. Users can use the trained results by downloading the file from the cloud storage service and importing it into their own systems.

[1600] In addition, the emotion data learned using the emotion engine is also converted into a common file format (e.g., CSV, JSON) and a shared link is generated. Users can download the file from the shared link and import it into their own systems.

[1601] Examples

[1602] Example prompt:

[1603] "Generate an image of a cat"

[1604] "Generate a video of dogs playing under a blue sky"

[1605] "Today was fun. Show me a picture of a fun moment."

[1606] "I'm feeling sad, generate a comforting image for me"

[1607] In this way, a system is provided that enables users to generate images and videos that meet their various needs and efficiently share their learning results. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1608] Step 1:

[1609] The user inputs the content of the image or video they want to generate as a prompt using the terminal. For example, they input prompts such as "Generate an image of a cat" or "Today was fun. Show me an image of a fun scene."

[1610] Input: prompt statement

[1611] Output: The prompt is sent to the server.

[1612] Specific action: The user enters a prompt sentence into the input field of the terminal and presses the send button.

[1613] Step 2:

[1614] The server analyzes the prompt received from the user and inputs it into a generative deep learning model and emotion engine.

[1615] Input: prompt statement

[1616] Output: Parsed prompt statement

[1617] What it does: The server parses the prompt using natural language processing algorithms and converts it into a format suitable for generative deep learning models and emotion engines.

[1618] Step 3:

[1619] The server generates images and videos using a generative deep learning model.

[1620] Input: Parsed prompt statement

[1621] Output: Generated images and videos

[1622] Specific operation: The server runs a generative deep learning model (e.g., GAN, VQ-VAE) to generate images and videos based on the prompt text.

[1623] Step 4:

[1624] The server uses an emotion engine to generate images and videos that are adapted to the user's emotions.

[1625] Input: Parsed prompt statement

[1626] Output: Emotion-adaptive images and videos

[1627] What it does: The server runs the emotion engine and generates images and videos based on the user's emotions.

[1628] Step 5:

[1629] The server transmits the generated images and videos to the user's device.

[1630] Input: Generated images and videos

[1631] Output: Images and videos sent to the user's device

[1632] Specific operation: The server sends the generated images and videos to the user's device so that the user can view them.

[1633] Step 6:

[1634] The server converts the trained model parameters into a common file format of a few MB and uploads them to a cloud storage service.

[1635] Input: trained model parameters

[1636] Output: A file uploaded to a cloud storage service.

[1637] Specific operation: The server converts the trained model parameters into JSON format and uploads them to Google Drive or Dropbox.

[1638] Step 7:

[1639] A user downloads a file from a cloud storage service and imports it into their system.

[1640] Input: Shared link of cloud storage service

[1641] Output: The downloaded file.

[1642] Specific behavior: A user accesses a cloud storage service and downloads a file from a shared link.

[1643] Step 8:

[1644] The server converts the learned emotion data using the emotion engine into a general file format and generates a sharing link.

[1645] Input: Trained emotion data

[1646] Output: Shared link

[1647] Specific operation: The server converts the learned emotion data into CSV format and generates a shared link.

[1648] Step 9:

[1649] Users download the files from the shared link and import them into their systems.

[1650] Input: Shared Link

[1651] Output: The downloaded file.

[1652] Specific behavior: A user accesses a shared link and downloads a file.

[1653] (Application example 3)

[1654] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1655] Conventional image and video generation systems have difficulty generating content according to the user's emotions, and have been unable to provide content that is adapted to the user's emotions. In addition, there was a lack of means to efficiently share the generated content, limiting the user experience. Furthermore, there was a lack of functionality to upload the generated content to cloud storage services, making it difficult to easily share it with other devices and users.

[1656] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1657] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like the user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, an emotion adaptive generating means for analyzing the emotions of the user and generating images and videos adapted to the emotions, and an uploading means for uploading the generated images and videos to a cloud storage service. This enables the generation of content according to the emotions of the user and efficient sharing.

[1658] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[1659] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[1660] The "generation means" is a means for generating desired images and videos based on user instructions.

[1661] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[1662] The "emotion-adaptive generation means" is a means for analyzing the user's emotions and generating images and videos adapted to those emotions.

[1663] The "uploading means" is a means for uploading the generated images and videos to a cloud storage service.

[1664] The present invention is a system that learns the contents of a user's daily conversations and chats, as well as service logs, generates images and videos that correspond to the user's emotions, and uploads them to a cloud storage service. A specific embodiment of this system is described below.

[1665] System Configuration

[1666] The system consists of the following main components:

[1667] 1. Learning method: A method for learning from users' daily conversations, chat contents, and service logs. Specifically, natural language processing technology is used to analyze users' text data and learn the users' language patterns and emotions.

[1668] 2. Response generation means: A means for generating responses in the same way as a user would, based on what has been learned. An appropriate response to the user's input is generated using a generative deep learning model.

[1669] 3. Generation means: A means for generating desired images and videos based on user instructions. Using a generative deep learning model, images and videos are generated based on the user's prompt sentences.

[1670] 4. Sharing method: A method for sharing training files in a common file format of several MB. Use a cloud storage service to share the parameters of the trained model.

[1671] 5. Emotion-adaptive generation means: A means for analyzing the user's emotions and generating images and videos that adapt to those emotions. The emotion engine is used to analyze the user's emotions and generate content based on the results.

[1672] 6. Uploading means: A means for uploading the generated images and videos to a cloud storage service. The generated content is stored in cloud storage and shared with other devices and users.

[1673] Program Processing

[1674] The server receives input from the user and analyzes the emotion using an emotion engine. Based on the analysis result, it passes the prompt text to a generative deep learning model to generate adapted images and videos. The generated content is uploaded to a cloud storage service and made accessible to the user.

[1675] Hardware and Software Used

[1676] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[1677] Software: Natural language processing libraries (e.g. transformers), generative deep learning models (e.g. Stable Diffusion), cloud storage services (e.g. AWS S3)

[1678] Examples

[1679] When a user uses a smartphone app to input "Today was fun," the server uses an emotion engine to parse it as "joy." Next, the server passes the prompt "a joyful scene" to a generative deep learning model, which generates an image of a joyful scene. The generated image is uploaded to a cloud storage service, and the user can view it through the app.

[1680] Examples of prompt statements

[1681] Today was fun

[1682] Based on this prompt sentence, the emotion engine parses it as "joy," and the generative deep learning model generates an image of "a joyful scene."

[1683] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1684] Step 1:

[1685] A user uses a terminal (such as a smartphone, tablet, or PC) to input text such as "Today was fun" into an application. The input text is sent to a server.

[1686] Input: User text input

[1687] Output: Text data sent to the server

[1688] Step 2:

[1689] The server passes the received text data to the emotion engine, which analyzes the user's emotion. The emotion engine uses natural language processing technology to analyze the text data and output an emotion label (e.g., joy).

[1690] Input: User's text data

[1691] Output: Emotion label (e.g. joy)

[1692] Step 3:

[1693] The server generates a prompt sentence to be passed to the generative deep learning model based on the emotion label. For example, if the emotion label is "joy", the server generates the prompt sentence "a joyful scene".

[1694] Input: emotion label

[1695] Output: prompt sentence (e.g. a joyful scene)

[1696] Step 4:

[1697] The server passes the prompt sentence to the generative deep learning model to generate images and videos. The generative deep learning model generates images and videos adapted based on the prompt sentence and returns them to the server.

[1698] Input: Prompt statement

[1699] Output: Generated images and videos

[1700] Step 5:

[1701] The server uploads the generated images and videos to a cloud storage service. Once the upload is complete, a URL for the cloud storage service is generated and returned to the server.

[1702] Input: Generated images and videos

[1703] Output: URL of the cloud storage service

[1704] Step 6:

[1705] The server sends the URL of the cloud storage service to the user's device, and the user can view the images and videos generated through the application.

[1706] Input: URL of cloud storage service

[1707] Output: The URL displayed on the user's device and any generated images or videos.

[1708] Through the above processing steps, images and videos can be generated and shared in response to the user's emotions.

[1709] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1710] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1711] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1712] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1713] [Third embodiment]

[1714] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1715] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1716] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1717] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1718] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1719] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1720] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1721] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1722] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1723] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1724] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1725] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1726] "Example 1"

[1727] In the present invention, a deep learning model is used as a learning means for learning the contents of daily conversations and chats of users and service logs. Specifically, the user's conversation log is used as input data, and the model is trained to imitate the conversation style. This training is performed using a technique called fine-tuning.

[1728] "Example 2"

[1729] The trained deep learning model is used as a reply generator that replies like a user based on the learned content. Specifically, for new input, a reply is generated using the trained model. This reply is based on the trained conversation style of the user.

[1730] "Example 3"

[1731] A generative deep learning model is used as a means of generating desired images and videos. Specifically, specific images and videos are generated based on instructions from a user. For example, if a user instructs the model to "generate an image of a cat," the model generates an image of a cat.

[1732] "Example 4"

[1733] A cloud storage service is used as a means of sharing the training file in a general file format of several MB. Specifically, the parameters of the trained model are converted into a general file format and the file is uploaded to the cloud storage service. Users can use the training results by downloading the file and importing it into their own systems.

[1734] The process flow of each embodiment will be described below.

[1735] "Example 1"

[1736] Step 1: Collect users' daily conversations, chat content, and service logs.

[1737] Step 2: Use the collected data as input data for a deep learning model.

[1738] Step 3: Train the model to learn to mimic the user's speaking style. This training is done using a technique called fine-tuning.

[1739] "Example 2"

[1740] Step 1: Receive new input (e.g., a question or comment from the user).

[1741] Step 2: Use the trained deep learning model to generate a response to the input.

[1742] Step 3: Providing the generated response to the user, which is based on the learned conversation style of the user.

[1743] "Example 3"

[1744] Step 1: Receive a command from a user (e.g., "generate an image of a cat").

[1745] Step 2: Use a generative deep learning model to generate images and videos based on the instructions.

[1746] Step 3: Provide the generated images and videos to the user.

[1747] "Example 4"

[1748] Step 1: Convert the trained model parameters into a common file format.

[1749] Step 2: Upload the converted file to a cloud storage service.

[1750] Step 3: Users can use the learning results by downloading the uploaded files and importing them into their own systems.

[1751] Example 1

[1752] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1753] Conventional conversation generation systems have had difficulty generating natural responses that fully reflect the individual conversation style of the user. In addition, there was a lack of efficient means to carry out a series of processes such as preprocessing of training data, fine-tuning of the model, evaluation, and deployment. This limited the user experience and led to problems such as a decline in the quality of the system's responses.

[1754] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1755] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like a user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several MB, a preprocessing means for the learning means for preprocessing the collected conversation logs and deleting unnecessary information, a training means for the response generating means for training to imitate the user's conversation style using a deep learning model, a fine-tuning means for the training means for fine-tuning the model using a fine-tuning method, an evaluation means for the response generating means for evaluating the trained model, an accuracy confirmation means for the evaluation means for confirming the accuracy of the model using test data, a deployment means for the response generating means for deploying the model that has been evaluated, and an embedding means for the deployment means for incorporating the model into an actual service. This allows for generating natural responses that reflect the individual conversation style of the user, and enables efficient data processing and model training, evaluation, and deployment.

[1756] "User" refers to an individual who uses the system to engage in everyday conversations and chats.

[1757] "Conversation log" refers to data that records the content of conversations and chats conducted by users.

[1758] "Service log" refers to the history and operation records of services used by a user.

[1759] The "learning means" refers to a method or device for learning the contents of users' daily conversations and chats, and service logs.

[1760] "Response generation means" refers to a method or device for generating responses based on what has been learned in the same way as a user would.

[1761] "Generator" refers to a method or device for generating a desired image or video.

[1762] "Sharing means" refers to a method or device for sharing learning files in a common file format of several MB.

[1763] "Preprocessing means" refers to a method or device for preprocessing the collected conversation logs and deleting unnecessary information.

[1764] "Training means" refers to a method or apparatus for training a deep learning model to mimic a user's speaking style.

[1765] "Fine-tuning means" refers to a method or device for fine-tuning a model using a fine-tuning technique.

[1766] "Evaluation means" refers to a method or device for evaluating a trained model.

[1767] "Accuracy verification means" refers to a method or device for verifying the accuracy of a model using test data.

[1768] "Deployment means" refers to a method or device for deploying a model after evaluation has been completed.

[1769] "Integration means" refers to a method or device for incorporating a deployed model into an actual service.

[1770] A "deep learning model" refers to a model that uses a multi-layer neural network to learn data and make predictions or generate results.

[1771] "Fine-tuning" refers to a technique that takes a pre-trained model and fine-tunes it to suit a specific task or dataset.

[1772] This invention is a system that uses a generative AI model that learns the contents of a user's daily conversations and chats, and service logs, and imitates the user's conversation style. A specific embodiment of this system is described below.

[1773] Hardware and Software Used

[1774] Hardware: A server equipped with a high-performance GPU (e.g. NVIDIA Tesla V100)

[1775] Software: Deep learning frameworks (e.g. TensorFlow, PyTorch)

[1776] Data collection

[1777] User: For everyday conversations and chats.

[1778] Terminal: Logs user conversations and chat content.

[1779] Server: Receives the conversation logs sent from the devices and stores them in a database.

[1780] Specific operation: When a user sends a message such as "The weather is nice today" in a chat app, the content is recorded on the device and sent to the server.

[1781] Data Preprocessing

[1782] Server: Preprocesses the collected conversation logs, specifically removing unnecessary information (e.g. emojis, special characters) and cleaning the text data.

[1783] Specific operation: Remove emojis from the message "The weather is nice today ○ (○ represents a specific emoji)" and generate clean text data "The weather is nice today."

[1784] Training the model

[1785] Server: Train the deep learning model using the preprocessed data, using fine-tuning techniques.

[1786] How it works: It uses a pre-trained GPT-3 model to fine-tune the user's speaking style, for example by learning the user's most commonly used phrases and expressions.

[1787] Evaluating the model

[1788] Server: Evaluates the trained model by checking the accuracy of the model using test data.

[1789] Specific behavior: Using the user's past conversation logs as test data, we evaluate how natural the generated responses are. For example, we check whether the response generated for the input "Where are you going today?" is "I'm going to see a movie today."

[1790] Deploying the model

[1791] Server: The model that has completed evaluation is deployed to the server and incorporated into the actual service.

[1792] Terminal: Uses the deployed model to generate real-time responses to user input.

[1793] How it works: A generative AI model built into a chat app responds to a user's input, "What are you planning to do today?" by generating a real-time response such as, "I'm planning to go to a cafe with my friends today."

[1794] Examples of prompt statements

[1795] "Learn the phrases that users use in conversations with their friends and generate responses that mimic that style."

[1796] The system aims to learn the user's conversation style and generate more natural responses, which will provide the user with a more comfortable communication experience.

[1797] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1798] Step 1: Data collection

[1799] User: For everyday conversations and chats.

[1800] Terminal: Logs user conversations and chat content.

[1801] Server: Receives the conversation logs sent from the devices and stores them in a database.

[1802] Input: A message sent by a user in a chat app (e.g., "The weather is nice today").

[1803] Output: Conversation logs stored on the server.

[1804] Specific operation: When a user sends a message such as "The weather is nice today" in a chat app, the content is recorded on the device and sent to the server.

[1805] Step 2: Data Preprocessing

[1806] Server: Preprocesses the collected conversation logs, specifically removing unnecessary information (e.g. emojis, special characters) and cleaning the text data.

[1807] Input: Conversation log stored on the server (e.g. "The weather is nice today ○ (○ represents a specific emoji)").

[1808] Output: Preprocessed, clean text data (e.g., "The weather is nice today").

[1809] Specific operation: Remove emojis from the message "The weather is nice today ○ (○ represents a specific emoji)" and generate clean text data "The weather is nice today."

[1810] Step 3: Training the model

[1811] Server: Train the deep learning model using the preprocessed data, using fine-tuning techniques.

[1812] Input: Preprocessed, clean text data (e.g., "The weather is nice today").

[1813] Output: A deep learning model fine-tuned to the user's speaking style.

[1814] How it works: It uses a pre-trained GPT-3 model to fine-tune the user's speaking style, for example by learning the user's most commonly used phrases and expressions.

[1815] Step 4: Evaluate the model

[1816] Server: Evaluates the trained model by checking the accuracy of the model using test data.

[1817] Input: A trained deep learning model and test data (e.g., a user's past conversation logs).

[1818] Output: The accuracy assessment of the model.

[1819] Specific behavior: Using the user's past conversation logs as test data, we evaluate how natural the generated responses are. For example, we check whether the response generated for the input "Where are you going today?" is "I'm going to see a movie today."

[1820] Step 5: Deploy the model

[1821] Server: The model that has completed evaluation is deployed to the server and incorporated into the actual service.

[1822] Terminal: Uses the deployed model to generate real-time responses to user input.

[1823] Input: A deep learning model that has completed evaluation.

[1824] Output: A generative AI model embedded into a real service.

[1825] How it works: A generative AI model built into a chat app responds to a user's input, "What are you planning to do today?" by generating a real-time response such as, "I'm planning to go to a cafe with my friends today."

[1826] (Application example 1)

[1827] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1828] Conventional content recommendation systems have difficulty in accurately grasping the user's interests and concerns, making it difficult to provide personalized content. In addition, content recommendations are not based on the user's conversation style or individual needs, which results in a lack of improvement in the user experience.

[1829] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1830] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding as the user would based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, and a recommending means for recommending personalized content based on the user's conversation log. This makes it possible to recommend personalized content based on the user's interests and concerns.

[1831] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[1832] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[1833] The "generation means" is a means for generating images and videos desired by the user.

[1834] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[1835] A "recommendation means" is a means for recommending personalized content based on a user's conversation log.

[1836] A system for implementing this invention includes a learning means for learning the contents of a user's daily conversations and chats and service logs, a response generation means for generating responses in the same way as the user would based on the learned contents, a generation means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, and a recommendation means for recommending personalized content based on the user's conversation logs.

[1837] Hardware and Software Configuration

[1838] The server uses the following hardware and software:

[1839] Hardware: A server with a powerful processor, sufficient memory, and storage.

[1840] Software: Python, OpenAI API, Database Management System (e.g. MySQL)

[1841] Data processing and calculation

[1842] The server stores the user's daily conversations, chat contents, and service logs in a database in JSON format. This data is used to fine-tune the deep learning model. Specifically, the model is trained to imitate the user's conversation style using the user's conversation log as input data.

[1843] Recommendation method operation

[1844] The server recommends personalized content based on the user's input. For example, if the user inputs "What are your recent interests?", the server infers the user's interests based on past conversation logs and recommends related content.

[1845] Examples

[1846] When a user uses his / her smartphone to input the question "What are your recent interests?" into the application, the server responds as follows:

[1847] Examples of prompt statements

[1848] User: What are your current interests?

[1849] AI: Based on your recent interests, we'd like to recommend the following content to you:

[1850] 1. Latest Technology News

[1851] 2. Popular Movie Reviews

[1852] 3. New music album release information

[1853] In this way, personalized content recommendations based on the user's interests become possible.

[1854] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1855] Step 1:

[1856] The server collects users' daily conversations, chat contents, and service logs.

[1857] Input: User conversation logs, chat contents, service logs

[1858] Data processing: Save the collected data in JSON format to the database.

[1859] Output: Conversation logs, chat contents, and service logs stored in the database

[1860] Step 2:

[1861] The server uses the collected data to fine-tune the deep learning model.

[1862] Input: Conversation logs, chat contents, and service logs stored in the database

[1863] Data Computing: Training a model to mimic a user's conversational style

[1864] Output: A fine-tuned deep learning model

[1865] Step 3:

[1866] A user inputs a question into the application using a smartphone.

[1867] Input: User question (e.g. "What are your current interests?")

[1868] Data processing: Sending user input to the server

[1869] Output: The user's question sent to the server.

[1870] Step 4:

[1871] The server uses the fine-tuned model to generate answers to users' questions.

[1872] Input: User question sent to the server, fine-tuned deep learning model

[1873] Data Computing: Using models to generate optimal answers to user questions

[1874] Output: The generated response

[1875] Step 5:

[1876] The server recommends personalized content based on the generated responses.

[1877] Input: Generated responses, user's past conversation logs

[1878] Data calculation: Select content that matches the user's interests based on generated responses and past conversation logs

[1879] Output: A personalized list of content

[1880] Step 6:

[1881] The server sends a personalized list of content to the user's smartphone.

[1882] Input: A list of personalized content

[1883] Data processing: Send the list of contents to the user's smartphone

[1884] Output: Personalized content displayed on the user's smartphone.

[1885] Step 7:

[1886] The user checks the personalized content displayed on the smartphone.

[1887] Input: Personalized content displayed on a smartphone

[1888] Data processing: User reviews the content and makes selections as necessary

[1889] Output: User selected content

[1890] In this way, personalized content recommendations based on the user's interests and concerns are realized.

[1891] Example 2

[1892] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1893] Conventional dialogue systems have difficulty generating natural responses based on the user's conversation style, which results in poor user experience. In addition, the quality of the responses generated is low, and they often fail to provide users with appropriate information. Furthermore, there has been a lack of systems that can efficiently process user input data and quickly generate appropriate responses.

[1894] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1895] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like a user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several MB, a receiving means for receiving input data from a user, a preprocessing means for preprocessing the received input data, an input means for inputting the preprocessed data into a generative AI model, a generating means for the generative AI model to generate a response, a postprocessing means for postprocessing the generated response, and a transmitting means for transmitting the postprocessed response to the user. This makes it possible to quickly generate natural responses based on the user's conversation style with high quality.

[1896] A "learning means" is a means for collecting the contents of users' daily conversations and chats, as well as service logs, and training a model based on them.

[1897] The "response generation means" is a means for generating natural responses to user inputs using the model trained by the learning means.

[1898] The "generation means" refers to a means for generating content such as images and videos according to the user's wishes.

[1899] The "sharing means" is a means for sharing the learning file with other users or systems in a general file format of several MB.

[1900] The "receiving means" is a means for receiving input data from a user.

[1901] The "preprocessing means" is a means for performing preprocessing such as normalizing received input data and deleting unnecessary spaces.

[1902] An "input means" is a means for inputting preprocessed data into a generative AI model.

[1903] A "generative AI model" is an artificial intelligence model that generates responses based on user input data.

[1904] "Post-processing means" refers to means for post-processing the response generated by the generative AI model, such as adjusting the format or deleting unnecessary characters.

[1905] A "transmitting means" is a means for transmitting a post-processed reply to a user.

[1906] This invention is a system that learns from a user's daily conversations, chat contents, and service logs, and responds like the user. This system includes a learning means, a response generating means, a generating means, a sharing means, a receiving means, a preprocessing means, an input means, a generating AI model, a postprocessing means, and a transmitting means.

[1907] The server uses a receiving means to receive input data from a user. The receiving means receives data via a communication protocol such as an HTTP request or WebSocket. For example, when a user inputs "What's the weather like today?", this data is received by the receiving means.

[1908] Next, the server preprocesses the received input data using a preprocessing means. For the preprocessing means, Python libraries such as NLTK and spaCy can be used. Preprocessing includes text normalization (e.g., converting uppercase to lowercase, removing unnecessary spaces, etc.). For example, "What's the weather like today?" is converted to "What's the weather like today?".

[1909] The preprocessed data is input to a generative AI model via an input means. Examples of generative AI models that can be used include OpenAI's GPT-3 and Google's BERT. The generative AI model generates a response based on the input data. For example, in response to the input "How's the weather today?", a response such as "Today's weather is sunny" is generated.

[1910] The generated response is post-processed using the post-processing means. The post-processing includes formatting the text and removing unnecessary characters. The post-processed response is transmitted to the user via the transmitting means. This allows the user to receive a natural response from the system.

[1911] As a concrete example, if a user inputs "What are your plans for tomorrow?", the server receives this input, performs preprocessing, and then inputs it into the generative AI model. The generative AI model generates a response such as "Tomorrow's plans are a meeting," and sends it to the user after post-processing.

[1912] An example of a prompt sentence is, "The following are examples of prompt sentences that a user can use to ask the system a question. For example, in response to the question, 'What are your plans for tomorrow?' the system will generate a response using the learned model."

[1913] In this way, the server can generate natural responses based on the user's input and provide them to the user.

[1914] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1915] Step 1:

[1916] The user submits the input data.

[1917] The user opens a chat application on the terminal, types "What's the weather like today?", and presses the send button. The input data is sent to the server in text format.

[1918] Step 2:

[1919] The server receives the input data.

[1920] The server receives the HTTP POST request from the user and obtains the text data "What's the weather like today?" contained in the request body. The input data is received in its original format.

[1921] Step 3:

[1922] The server pre-processes the input data.

[1923] The server normalizes the received text data using preprocessing measures. Specifically, it uses Python's NLTK or spaCy to convert the text to lower case and remove unnecessary spaces. For example, "What's the weather like today?" is converted to "What's the weather like today?". The preprocessed data is output in a state that is ready to be fed into a generative AI model.

[1924] Step 4:

[1925] The server inputs the preprocessed data into the generative AI model.

[1926] The server inputs the preprocessed text data into the generative AI model by sending an API request and passing the data to the model. The input data is converted into a format that the generative AI model can understand.

[1927] Step 5:

[1928] A generative AI model generates the response.

[1929] A generative AI model generates a response based on input data. For example, in response to the input "What's the weather like today?", it generates a response such as "Today's weather is sunny." The generated response is output in text format.

[1930] Step 6:

[1931] The server post-processes the generated reply.

[1932] The server uses post-processing means to format and remove unnecessary characters from the generated response. For example, if a response is generated that says "The weather is sunny today," it will be formatted appropriately. The post-processed data is output ready to be sent to the user.

[1933] Step 7:

[1934] The server sends a post-processed reply to the user.

[1935] The server sends the post-processed reply as an HTTP response to the user, who sees the reply "The weather is sunny today" in the chat application on his terminal. The output data is displayed in a format that the user can understand.

[1936] In this way, the server can generate natural responses based on the user's input and provide them to the user.

[1937] (Application example 2)

[1938] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1939] Conventional content delivery services have the problem that it is difficult to recommend personalized content based on a user's individual conversation style and preferences. It is also difficult to generate natural responses based on a user's conversation style. This can lead to a poor user experience and a decrease in the frequency of service use.

[1940] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1941] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of a user and service logs, a response generating means for responding like a user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several MB, a recommending means for recommending content based on the user's conversation style, and a prompt generating means for generating prompt sentences using a generative AI model. This makes it possible to recommend personalized content based on the individual conversation style and preferences of the user, and is expected to improve the user experience.

[1942] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[1943] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[1944] A "generation means" is a means for generating a desired image or video.

[1945] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[1946] A "recommendation means" is a means for recommending content based on a user's conversation style.

[1947] A "prompt generation means" is a means for generating a prompt sentence using a generative AI model.

[1948] A system for implementing this invention includes a learning means for learning the content of a user's daily conversations and chats and service logs, a response generation means for generating responses like the user's based on the learned content, a generation means for generating desired images and videos, a sharing means for sharing learning files in a general file format of several MB, a recommendation means for recommending content based on the user's conversation style, and a prompt generation means for generating prompt sentences using a generative AI model.

[1949] Hardware and Software Used

[1950] Hardware: Smartphones, servers

[1951] Software: Python, OpenAI API

[1952] Data processing and calculation

[1953] Learning medium

[1954] The server collects the user's daily conversations, chat contents, and service logs, and uses this data to learn the user's conversation style. For learning, fine-tuning is used to optimize the deep learning model.

[1955] Response Generation Method

[1956] The server uses the learned model to generate natural responses to user input. For example, if a user asks, "What are some recent movies you recommend?" the server responds, "The recent movie I recommend is 'Inception.' It's a thrilling film that depicts a dream within a dream!"

[1957] generation means

[1958] The server generates images and videos according to the user's request. For example, if the user requests "Show me an image of a sunset," the server generates and provides an appropriate image.

[1959] means of sharing

[1960] The server shares the training files in a common file format of a few MB, which allows easy sharing of data with other devices and users.

[1961] Recommendation method

[1962] The server recommends content based on the user's conversation style, for example, if the user has a casual conversation style, it recommends movies and music that match that style.

[1963] Prompt Generation Method

[1964] The server uses a generative AI model to generate prompts, such as the following:

[1965] User conversation style: Casual and friendly

[1966] User input: What are some recent movie recommendations?

[1967] reply:

[1968] In this way, personalized content recommendations based on a user's individual conversation style and preferences become possible, which is expected to improve the user experience.

[1969] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1970] Step 1:

[1971] The server collects users' daily conversations, chat contents, and service logs.

[1972] Input: User conversation data, chat logs, service logs

[1973] Output: Collected data

[1974] Specific operation: The server periodically collects logs of conversations and chats conducted by users on their smartphones and other devices and stores them in a database.

[1975] Step 2:

[1976] The server uses the collected data to learn the user's speaking style.

[1977] Input: Collected data

[1978] Output: The trained model

[1979] How it works: The server uses the collected data to fine-tune the deep learning model and learn the user's conversation style, allowing the model to reflect the user's specific conversation patterns and preferences.

[1980] Step 3:

[1981] The server generates natural responses to the user's input.

[1982] Input: User input, trained model

[1983] Output: The generated response

[1984] Specific operation: When a user types "What is a recommended recent movie?" into their smartphone, the server uses the trained model to generate a response such as "A recommended recent movie is 'Inception.' It's a thrilling film that depicts a dream within a dream!"

[1985] Step 4:

[1986] The server generates images and videos according to the user's wishes.

[1987] Input: User request

[1988] Output: Generated images and videos

[1989] Specific behavior: When a user requests "Show me an image of a sunset," the server generates an image of a sunset using an appropriate image generation algorithm and provides it to the user.

[1990] Step 5:

[1991] The server shares the training files in a common file format of several MB.

[1992] Input: Training file

[1993] Output: Shared files

[1994] Specific operation: The server converts the training file into a common file format (e.g., JSON or CSV) of several MB and shares it with other devices and users.

[1995] Step 6:

[1996] The server recommends content based on the user's conversation style.

[1997] Input: User conversation style, content database

[1998] Output: Recommended content

[1999] Specific operation: The server analyzes the user's conversation style and recommends movies, music, and other content that matches that style. For example, for a user with a casual conversation style, it recommends relaxing movies and music.

[2000] Step 7:

[2001] The server generates a prompt sentence using a generative AI model.

[2002] Input: User's speaking style, user input

[2003] Output: Generated prompt statement

[2004] Specific behavior: The server generates a prompt based on the user's speaking style and input. For example, it generates the following prompt:

[2005] User conversation style: Casual and friendly

[2006] User input: What are some recent movie recommendations?

[2007] reply:

[2008] Example 3

[2009] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[2010] In conventional systems, when users wanted to generate images or videos, the quality and accuracy of the generated content was often low. In addition, there was a lack of a way to efficiently share the parameters of the trained model, making it difficult for users to easily use the training results. Furthermore, there was no infrastructure in place to provide content that was quickly generated based on user instructions.

[2011] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[2012] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like the user based on the learned contents, a generating means using a generative deep learning model for generating desired images and videos, a sharing means using a cloud storage service for sharing learning files in a general file format of several MB, a means for receiving user instructions and inputting them into the generative deep learning model, and a means for displaying the generated images and videos to the user. This enables users to quickly generate high-quality images and videos and efficiently share and use the learning results.

[2013] The "learning means" refers to a means for collecting users' daily conversations, chat contents, and service logs, and training a model using a machine learning algorithm based on that data.

[2014] The "answer generation means" is a means for generating a reply to a user's input in a manner similar to that of the user, using the model trained by the learning means.

[2015] A "generative deep learning model" is a model that uses deep learning algorithms to generate specific images or videos based on user instructions.

[2016] A "generation means" is a means for generating images and videos based on user instructions using a generative deep learning model.

[2017] The "sharing means" refers to a means for converting the parameters of the trained model into a common file format of a few MB and sharing it via a cloud storage service.

[2018] A "cloud storage service" is an online service for storing and sharing data via the Internet.

[2019] A "server" is a computer system whose role is to receive instructions from a user, input them into a generative deep learning model, and return the generated images and videos to the user.

[2020] A "terminal" is a device through which a user inputs instructions and through which generated images and videos are displayed.

[2021] The present invention provides a system for generating images and videos that a user desires and efficiently sharing parameters of a trained model. A specific embodiment of this system will be described below.

[2022] Hardware and Software Used

[2023] 1. Server: A high-performance computer system that receives instructions from the user and inputs them into the generative deep learning model. The server also returns the generated images and videos to the user.

[2024] 2. Terminal: A device on which a user inputs instructions and displays the generated images and videos. Terminals include PCs, smartphones, tablets, etc.

[2025] 3. Generative deep learning model: A model that uses deep learning algorithms to generate specific images or videos based on user instructions. For example, GAN (Generative Adversarial Network) and VAE (Variational Autoencoder) are used.

[2026] 4. Cloud storage services: Online services for storing and sharing data over the Internet. Examples include Google Drive and Dropbox.

[2027] System Operation

[2028] 1. The user inputs instructions to generate the desired image or video. For example, the user might input "generate an image of a cat."

[2029] 2. The terminal sends the user's instructions to the server. The terminal sends the entered instructions to the server as an HTTP request.

[2030] 3. The server receives the instructions and inputs them into the generative deep learning model. The server receives the HTTP request, analyzes the instructions, and passes them to the generative deep learning model.

[2031] 4. The generative deep learning model generates images and videos based on instructions. Based on the instruction "generate an image of a cat," the generative deep learning model uses its internal algorithms to generate an image of a cat.

[2032] 5. The server receives the generated images and videos and sends them to the device. The server receives the generated images and sends them to the device as an HTTP response.

[2033] 6. The device displays the images and videos it has received to the user. The device displays the images it has received on its screen so that the user can view them.

[2034] Examples

[2035] For example, if a user instructs the server to "generate a video of a landscape," the server inputs the instruction into the generative deep learning model, which then generates a video of the landscape. The generated video is then provided to the user through the server.

[2036] Example prompt:

[2037] "Generate an image of a cat"

[2038] "Generate a video of the scenery"

[2039] Sharing study files

[2040] 1. The server converts the model parameters trained into a common file format. For example, the server converts the model parameters trained into a file called "model_parameters.bin".

[2041] 2. The server uploads the converted file to a cloud storage service. The server uploads "model_parameters.bin" to Google Drive.

[2042] 3. The user accesses the cloud storage service and downloads the necessary files. The user accesses Google Drive and downloads "model_parameters.bin".

[2043] 4. The user imports the downloaded file into their own system and uses the learning results. The user imports the downloaded "model_parameters.bin" into their own system and uses the learning results.

[2044] In this way, the user can quickly generate high-quality images and videos and efficiently share and use the learning results. The flow of the identification process in the third embodiment will be described with reference to FIG.

[2045] Step 1:

[2046] The user inputs instructions to generate the desired image or video.

[2047] Input: The user types "generate a picture of a cat" into the input form on the terminal.

[2048] Specific operation: The user uses the device's keyboard or touch screen to input the content of the image or video to be generated in text format.

[2049] Output: Your terminal will display the instructions and you are ready to move on to the next step.

[2050] Step 2:

[2051] The terminal transmits the user's instructions to the server.

[2052] Input: Instructions entered by the user.

[2053] Specific operation: The terminal sends the input instructions to the server as an HTTP request.

[2054] Output: The server receives the HTTP request and parses the instructions.

[2055] Step 3:

[2056] The server receives the instructions and inputs them into the generative deep learning model.

[2057] Input: The HTTP request sent from the terminal.

[2058] Specific operation: The server receives an HTTP request, analyzes the instructions, and passes them to the generative deep learning model.

[2059] Output: The generative deep learning model receives the instructions.

[2060] Step 4:

[2061] Generative deep learning models generate images and videos based on instructions.

[2062] Input: Instructions passed from the server.

[2063] Specific operation: Based on the instruction "generate an image of a cat," the generative deep learning model uses an internal algorithm to generate an image of a cat.

[2064] Output: The generated image data.

[2065] Step 5:

[2066] The server receives the generated images and videos and sends them to the terminal.

[2067] Input: Generated image data from a generative deep learning model.

[2068] Specific operation: The server receives the generated image and sends it to the terminal as an HTTP response.

[2069] Output: The device receives the HTTP response.

[2070] Step 6:

[2071] The device displays the images and videos it receives to the user.

[2072] Input: The generated image data sent from the server.

[2073] Specific operation: The image received by the device is displayed on the screen so that the user can check it.

[2074] Output: The user visually checks the generated image.

[2075] Step 7:

[2076] The server converts the trained model parameters into a common file format.

[2077] Input: The parameters of the trained model.

[2078] Specific operation: The server converts the parameters of the trained model into a file called "model_parameters.bin".

[2079] Output: Model parameter files converted to a common file format.

[2080] Step 8:

[2081] The server uploads the converted files to a cloud storage service.

[2082] Input: Model parameter files converted to a common file format.

[2083] Specific operation: The server uploads "model_parameters.bin" to the cloud storage service.

[2084] Output: The file uploaded to the cloud storage service.

[2085] Step 9:

[2086] A user accesses a cloud storage service and downloads a required file.

[2087] Input: A file uploaded to a cloud storage service.

[2088] Specific operation: A user accesses a cloud storage service and downloads "model_parameters.bin".

[2089] Output: The files downloaded to the user's device.

[2090] Step 10:

[2091] The user imports the downloaded file into their own system and uses the learning results.

[2092] Input: The downloaded “model_parameters.bin” file.

[2093] Specific operation: The user imports the downloaded "model_parameters.bin" into their own system and uses the learning results.

[2094] Output: Training results available on the user's system.

[2095] (Application example 3)

[2096] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[2097] In conventional image and video generation systems using generative deep learning models, it was difficult for users to quickly and efficiently generate and share the content they wanted. In addition, there was a lack of means to upload the generated content to cloud storage and share it with other users, which resulted in low user convenience. Furthermore, there was a lack of sharing of learning files tailored to individual needs.

[2098] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[2099] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding as the user would based on the learned contents, a generating means for generating desired images and videos, an uploading means for uploading the generated images and videos to cloud storage, and a sharing means for sharing learning files in a general file format of several MB. This allows the user to quickly and efficiently generate desired content and share it with other users through cloud storage. It also allows sharing of learning files tailored to individual needs.

[2100] The "learning means" is a means for learning the contents of users' daily conversations and chats, and service logs.

[2101] The "response generating means" is a means for generating a response based on the learned content in the same way as a user would.

[2102] The "generation means" is a means for generating desired images and videos based on user instructions.

[2103] The "uploading means" is a means for uploading the generated images and videos to cloud storage.

[2104] "Sharing means" refers to a means for sharing learning files in a common file format of several MB.

[2105] A system for implementing this invention includes a learning means for learning the contents of a user's daily conversations and chats and service logs, a response generation means for generating responses in the manner of the user based on the learned contents, a generation means for generating desired images and videos, an uploading means for uploading the generated images and videos to cloud storage, and a sharing means for sharing learning files in a common file format of several MB.

[2106] Explanation of system program processing

[2107] Hardware and Software

[2108] The server uses a high-performance computer equipped with a GPU. The software uses the transformers library to execute generative deep learning models, the requests library to process HTTP requests, and the PIL library for image processing.

[2109] Data processing and calculation

[2110] 1. Learning method: The server collects users' daily conversations, chat contents, and service logs, and uses them to train a generative deep learning model. Fine-tuning is performed using the transformers library for learning.

[2111] 2. Response generation: Using the trained model, we generate appropriate responses based on the user's input, enabling responses that sound more natural than the user would.

[2112] 3. Generation: A user inputs a prompt, and the generative deep learning model generates the desired image or video based on the prompt. For example, if a user inputs "generate an image of a beach at sunset," the model generates the image.

[2113] 4. Uploading method: The generated images and videos are uploaded to cloud storage by sending an HTTP request using the requests library.

[2114] 5. Sharing: The learning files are converted into a common file format of several MB and uploaded to cloud storage. Users can download the files and import them into their own systems.

[2115] Examples

[2116] When a user enters the prompt "Generate an image of a cat," the generative deep learning model generates an image of a cat and uploads it to cloud storage. The user can then download and use the uploaded image.

[2117] Example prompt:

[2118] Generate a cat image

[2119] Generate sunset beach images

[2120] The flow of the specific process in the application example 3 will be described with reference to FIG.

[2121] Step 1:

[2122] The user enters a prompt sentence.

[2123] Input: A prompt describing the image or video the user is looking for (e.g., "Generate an image of a cat").

[2124] How it works: A user enters a prompt through a smartphone or computer interface.

[2125] Output: The prompt text is sent to the server.

[2126] Step 2:

[2127] The server receives the prompt sentence and inputs it into the generative deep learning model.

[2128] Input: The prompt text sent by the user.

[2129] How it works: The server receives a prompt text and uses the transformers library to feed the prompt text to a generative deep learning model.

[2130] Output: A generation request based on the prompt is passed to the model.

[2131] Step 3:

[2132] A generative deep learning model generates images and videos based on the prompt text.

[2133] Input: A generated request based on the prompt statement.

[2134] How it works: The model analyzes the prompt and generates an appropriate image or video. For example, in response to the prompt "generate an image of a cat," the model generates an image of a cat.

[2135] Output: The generated image and / or video data.

[2136] Step 4:

[2137] The server uploads the generated images and videos to cloud storage.

[2138] Input: Generated image and / or video data.

[2139] How it works: The server uses the requests library to upload the generated images and videos to cloud storage.

[2140] Output: URL of uploaded image or video to cloud storage.

[2141] Step 5:

[2142] The server returns the cloud storage URL to the user.

[2143] Input: The URL of the image or video uploaded to cloud storage.

[2144] How it works: The server sends the URL of the generated content back to the user.

[2145] Output: The URL where the user can access it.

[2146] Step 6:

[2147] The user downloads the generated images and videos from the cloud storage URL.

[2148] Input: The URL returned by the server.

[2149] How it works: The user visits a URL and downloads the generated image or video.

[2150] Output: Images and videos stored on the user's device.

[2151] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2152] "Example 1"

[2153] In one embodiment of the present invention, an emotion engine is used as a learning means for learning a user's daily conversations, chat contents, and service logs. The emotion engine recognizes emotions from the user's speech and incorporates the emotions as learning data. For example, if a user says, "Today was fun," the emotion engine recognizes the emotion of "joy" from this speech and incorporates the information as learning data.

[2154] "Example 2"

[2155] In addition, an emotion engine is used as a reply generating means that replies like the user based on the learned content. The emotion engine generates a reply adapted to the user's emotions based on the learned emotion data. For example, if the user says, "Today was fun," the emotion engine generates a reply adapted to joy, such as "That's wonderful."

[2156] "Example 3"

[2157] Furthermore, an emotion engine is used as a generating means for generating desired images and videos. The emotion engine generates images and videos adapted to the user's emotions based on learned emotion data. For example, if a user says, "Today was fun," and requests, "Show me an image of a happy scene," the emotion engine generates an image of a joyful scene.

[2158] "Example 4"

[2159] Finally, the emotion engine is used as a means of sharing the learning file in a general file format of several MB. The emotion engine converts the learned emotion data into a general file format and shares the file. This allows other systems and users to use the learning results of the emotion engine.

[2160] The process flow of each embodiment will be described below.

[2161] "Example 1"

[2162] Step 1: Receive the user's utterance.

[2163] Step 2: The emotion engine recognizes emotions from the user's speech.

[2164] Step 3: The recognized emotions are input as learning data.

[2165] "Example 2"

[2166] Step 1: Receive the user's utterance.

[2167] Step 2: Based on the emotion data learned by the emotion engine, a response is generated that is adapted to the user's emotions.

[2168] "Example 3"

[2169] Step 1: Receive user speech and a request for images or videos.

[2170] Step 2: Based on the emotional data learned by the emotion engine, images and videos are generated that are adapted to the user's emotions.

[2171] "Example 4"

[2172] Step 1: Convert the emotion data learned by the emotion engine into a common file format.

[2173] Step 2: Share the converted file.

[2174] Example 1

[2175] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[2176] Conventional systems had difficulty effectively learning the content of users' daily conversations and chats and generating responses appropriate to the user. In addition, they lacked the ability to recognize users' emotions and generate responses based on them, limiting the user experience. Furthermore, when it came to sharing learning files, flexible responses tailored to individual needs were required.

[2177] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2178] In this invention, the server includes a learning means for learning the contents of daily conversations and chats of users and service logs, a response generating means for responding like the user based on the learned contents, a generating means for generating desired images and videos, a sharing means for sharing a learning file in a general file format of several megabytes, an emotion recognition means for recognizing emotions from the user's speech and incorporating the emotions as learning data, and a response generating means for generating responses suitable for the user using a generative AI model. This makes it possible to effectively learn the user's conversation style and generate responses based on emotions. It is also possible to share learning files tailored to individual needs.

[2179] A "learning means" is a means for collecting users' daily conversations, chat contents, and service logs, and training a deep learning model based on them.

[2180] The "response generating means" is a means for imitating the user's conversation style based on the learned content and generating an appropriate reply.

[2181] The "generation means" is a means for generating images and videos according to the user's wishes.

[2182] "Sharing means" refers to a means for sharing learning files in a common file format of several megabytes.

[2183] The "emotion recognition means" is a means for recognizing emotions from the user's speech and incorporating the emotions as learning data.

[2184] A "response generation means" is a means for generating a response appropriate to a user using a generative AI model.

[2185] "Fine-tuning" is a technique for optimizing an existing deep learning model to fit a user's data.

[2186] The present invention is a system that learns the contents of a user's daily conversations and chats, and service logs, and generates responses appropriate to the user. A specific embodiment of this system will be described below.

[2187] 1. Program Generation

[2188] The server generates a program to learn the user's daily conversations, chat contents, and service logs. The program is designed to imitate the user's conversation style using a deep learning model.

[2189] 2. Program Processing Explanation

[2190] The server receives the user's conversation log as input data and uses a deep learning model to train the model to imitate the user's conversation style. This training uses a technique called fine-tuning. Specifically, the following process is performed:

[2191] Data collection: The server collects the user's daily conversations, chat contents, and service logs. For example, if a user sends a message such as "I had fun today" in a chat app, the message is stored on the server.

[2192] Data preprocessing: The server preprocesses the collected data and converts it into a format suitable for the model. Specifically, it normalizes and tokenizes the text. For example, it tokenizes a statement such as "Today was fun" into "Today was fun."

[2193] Model training: The server trains a deep learning model using the preprocessed data, using fine-tuning techniques to fine-tune the existing model to the user's speaking style, using deep learning frameworks such as TensorFlow and PyTorch.

[2194] Emotion recognition: The server uses the emotion engine to recognize emotions from the user's speech. For example, if a user says, "Today was fun," the emotion engine recognizes the emotion of "joy" from this speech and incorporates that information into learning data.

[2195] Response generation: The server uses the generative AI model to generate a response appropriate for the user. Specifically, the server creates a prompt sentence based on the user's utterance, inputs it into the generative AI model, and generates an appropriate response.

[2196] 3. Examples of concrete examples and prompts

[2197] As a specific example, consider a case where a user says, "Today was fun." This utterance is processed as follows.

[2198] The server collects these comments and stores them as text data.

[2199] The emotion engine recognizes the emotion "joy" from this statement.

[2200] The recognized emotional information is incorporated into the deep learning model as training data.

[2201] The server uses this data to fine-tune the model, training it to mimic the user's speaking style.

[2202] The server generates a prompt sentence, "Tell me more about your fun events," and inputs it into the generative AI model to generate a response, "That's great! What did you enjoy?"

[2203] Examples of prompt sentences to be input to a generative AI model include the following:

[2204] "Please tell us about an event that you found enjoyable."

[2205] "Please tell us what your plans are for the day when you feel busy."

[2206] "Please tell us about movies that you found interesting."

[2207] In this way, the server learns the user's conversational style and uses a generative AI model to generate responses that are appropriate for the user.

[2208] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2209] Step 1: Data collection

[2210] The server collects the user's daily conversations, chat contents, and service logs. For example, when a user sends a message such as "I had fun today" in a chat app, the message is stored in the server. The input is the user's text data, and the output is the stored text data.

[2211] Step 2: Data Preprocessing

[2212] The server pre-processes the collected data. Specifically, it performs the following processes:

[2213] Text normalization: removing unnecessary spaces and special characters from what the user says.

[2214] Tokenization: Splitting text into words and phrases. For example, tokenizing an utterance like "I had fun today" into "I had fun today".

[2215] Labeling: Using the emotion engine, an emotion label is assigned to each utterance. For example, a utterance such as "Today was fun" is labeled as "Joy."

[2216] The input is stored text data, and the output is preprocessed text data and emotion labels.

[2217] Step 3: Model training

[2218] The server trains a deep learning model using the preprocessed data, specifically by following the steps below.

[2219] Dataset Creation: We create a dataset by combining the preprocessed text with the sentiment labels.

[2220] Fine-tuning: Optimizing an existing deep learning model to fit your data, for example by updating the model parameters using TensorFlow or PyTorch.

[2221] The input is preprocessed text data and sentiment labels, and the output is a trained deep learning model.

[2222] Step 4: Emotion Recognition

[2223] The server uses the emotion engine to recognize emotions from the user's speech. Specifically, it performs the following processes.

[2224] Text analysis: Analyze what the user says and input it into the emotion engine.

[2225] Emotion labeling: The emotion engine recognizes emotions from utterances and assigns corresponding emotion labels. For example, a statement like "Today was fun" is labeled as "Joy."

[2226] The input is the user's spoken text and the output is an emotion label.

[2227] Step 5: Response Generation

[2228] The server uses the generative AI model to generate a response appropriate for the user. Specifically, it follows the steps below.

[2229] Prompt creation: Create a prompt based on what the user says. For example, if the user says, "I had fun today," the prompt will be, "Tell me more about fun things that happened to you."

[2230] Response generation: A prompt is fed into the generative AI model to generate an appropriate ...

Claims

1. an emotion engine that recognizes the user's emotions; A learning means for learning a conversation style of the user's daily conversation using a deep learning model and the emotion engine; A response generating means for generating a response to the user's utterance based on the learned conversation style and the emotion by inputting a specific prompt sentence into a generative AI model; a generating means for generating recommended content for the user based on the learned conversation style; A system including:

2. The system of claim 1 , further comprising an uploading means for uploading the content to a cloud storage service.

3. The system according to claim 1 , wherein when the emotion engine recognizes an emotion of joy of the user from the user's daily conversation, the response generating means generates a response to the emotion of joy of the user.

Images