System

Abstract

To provide a system with which it is possible to quickly and efficiently generate a personalized digital gift based on face images uploaded by a user.SOLUTION: Provided is a system including: means for receiving a face photograph uploaded by a user; means for extracting the features of face from the face photograph by using a machine learning algorithm; means for generating a prompt sentence that directs generation of a digital gift in which the face of a messenger application stamp is converted into the face of the face photograph on the basis of the extracted features of face or a digital gift in which the face of a person in a designated scene of video work is converted into the face of the face photograph; means for generating an individualized digital gift on the basis of the face of the received face photograph by using the generated prompt sentence and a generative AI model; and means for presenting the generated digital gift as a personalized digital gift.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional digital gifts are generic and difficult to personalize to the tastes and needs of individual users. Although there are technologies that use photos of friends and family to convert the faces and dialogue of people in messenger app stamps, comic book and movie scenes, there is no system that provides this as a digital gift. [Means for solving the problem]

[0005] The present invention receives photos uploaded by users and converts the faces and dialogue of people in messenger app stamps, cartoons, and movie scenes based on the photos. The conversion results are then provided as personalized digital gifts. This system makes it possible to provide personalized digital gifts tailored to the tastes and needs of individual users. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13]FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 11 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The system of the present invention includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a scene in a cartoon or movie based on the photo, and a means for providing the conversion result as a personalized digital gift. Specifically, a user uploads his / her own photo through a website or application. Next, the system uses a machine learning algorithm to extract features from the photo and converts the face or dialogue of the person based on the features. Finally, the generated personalized digital gift is sent to the user via email or a messenger app.

[0029] "Example 2"

[0030] As a specific example, consider a case where a user uploads a photo of a friend and converts the friend's face into a stamp for a messenger app. First, the user uploads the friend's photo to the system. Next, the system uses a machine learning algorithm to extract the friend's facial features and converts the face of the person in the stamp into the friend's face based on the features. Finally, the system sends the generated personalized digital gift to the user via email or a messenger app. In this way, the user can get an original stamp with the friend's face drawn on it.

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: The user uploads a photo of themselves via a website or application.

[0034] Step 2: The system uses machine learning algorithms to extract features from the photo.

[0035] Step 3: The system transforms the person's face and dialogue based on the extracted features.

[0036] Step 4: The generated personalized digital gift is sent to the user via email or messenger app.

[0037] "Example 2"

[0038] Step 1: A user uploads a photo of their friend into the system.

[0039] Step 2: The system uses machine learning algorithms to extract your friend's facial features.

[0040] Step 3: The system converts the face of the person in the stamp into the face of a friend based on the extracted features.

[0041] Step 4: Send the generated personalized digital gift to the designated user via email or messenger app.

[0042] Example 1

[0043] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0044] In the conventional digital gift generation system, when generating personalized digital gifts based on photos uploaded by users, the conversion of faces and dialogues is done manually, which is time-consuming and labor-intensive. In addition, there are also issues that the quality of the generated digital gifts is inconsistent and may not meet the user's expectations. Furthermore, there is a lack of a means to quickly and reliably provide the generated digital gifts to the users.

[0045] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0046] In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a stamp of a communication application or a face or dialogue of a person in a scene of a cartoon or a video work based on the image, a means for providing the conversion result as a personalized digital gift, a means for using a machine learning algorithm to extract features from the image, a means for using a generative AI model to generate a face or dialogue of a person based on the features, and a means for transmitting the generated personalized digital gift to the user via an electronic communication means. This makes it possible to quickly and efficiently generate a high-quality, consistent personalized digital gift based on an image uploaded by a user and provide it to the user.

[0047] "User" means an individual or entity that utilizes the System to upload an image and receive a personalized digital gift.

[0048] "Images" are visual data such as photographs and illustrations that users upload to the system.

[0049] A "communication application" is software for sending and receiving messages, and has the function of sending stamps and images.

[0050] A "stamp" is a small image or icon used within a communication application to express an emotion or message.

[0051] "Manga" is a series of pictures or illustrations intended to visually represent a story.

[0052] A "visual work" is any medium that contains a visually presented story or information, such as a film or television program.

[0053] A "machine learning algorithm" is a computational technique for learning patterns and features from data and making predictions and classifications.

[0054] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate new data.

[0055] A "personalized digital gift" is a digital gift that is customized for a particular user based on the user's image and information.

[0056] "Electronic communication means" refers to means for sending and receiving data via the Internet, email, messenger apps, etc.

[0057] The present invention is a system for generating and providing personalized digital gifts based on images uploaded by users. A specific embodiment of the system will be described below.

[0058] System Configuration

[0059] Hardware and Software

[0060] The server uses the following hardware and software:

[0061] Hardware: High-performance server machine (e.g. cloud server with CPU, GPU, and memory)

[0062] Software: Machine learning algorithms (e.g., TENSORFLOW®), generative AI models (e.g., GPT-4®, DALL-E), databases (e.g., Amazon® S3), communication methods (e.g., SMTP servers, messenger APIs)

[0063] Processing flow

[0064] 1. Upload your photo

[0065] A user accesses a website or application using a smartphone or PC. For example, the user opens an application on the smartphone, takes a photo of his / her face using the camera function, and presses the upload button. This sends the photo data to the application.

[0066] 2. Receiving photos

[0067] The server receives the images uploaded by the user. Specifically, the server uses an Amazon S3 bucket to store the image data. The server receives the HTTP request, analyzes the image data, and stores it.

[0068] 3. Feature Extraction

[0069] The server uses machine learning algorithms to extract features from the images. It uses TensorFlow for this process. The server feeds the image data into a TensorFlow model to identify features such as the contours of the face and the location of the eyes, nose, and mouth. For example, it uses a facial recognition model to obtain the coordinates of each part of the face.

[0070] 4. Face and dialogue transformation

[0071] Based on the extracted features, the server converts the faces and dialogue of people in stamps for communication applications, comics, and scenes in video works. Here, generative AI models (GPT-4, DALL-E) are used. The server uses DALL-E to synthesize the user's face into a specific scene and uses GPT-4 to generate the dialogue content. For example, the server synthesizes the user's face into a scene from a famous science fiction movie, and changes the dialogue in that scene to the user's name or a specific phrase.

[0072] 5. Create personalized digital gifts

[0073] The server generates a personalized digital gift based on the conversion result. The server combines the generated images and text to create a digital gift for the user. For example, it creates a digital stamp with the user's face or a comic strip featuring the user.

[0074] 6. Sending Digital Gifts

[0075] The server transmits the generated personalized digital gift to the user via electronic communication means. The server transmits a download link for the digital gift using the user's email address or messenger app ID. For example, the server transmits an email containing a download link for the digital gift to the user's email address.

[0076] Examples and prompts

[0077] Examples

[0078] Users upload a photo of their face through the app and receive a digital gift that features their face superimposed onto a scene from the famous sci-fi movie "XXX" based on the photo.

[0079] Examples of prompt statements

[0080] "Based on the face photo uploaded by the user, please composite the face into a scene from the famous science fiction movie "XXX" and generate a digital gift that includes a specific dialogue."

[0081] In this way, users can easily create and enjoy their own special digital gifts.

[0082] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0083] Step 1:

[0084] A user accesses a website or application using a smartphone or PC. The user takes a photo of his / her face using the camera function and presses the upload button. This sends the photo data to the application.

[0085] Input: A photo of the user's face

[0086] Output: The photo data sent to the application.

[0087] Step 2:

[0088] The server receives the images uploaded by the user: the server receives the HTTP request, parses the image data, and stores it in an Amazon S3 bucket.

[0089] Input: Photo data sent from the application

[0090] Output: Image data stored in an Amazon S3 bucket

[0091] Step 3:

[0092] The server uses machine learning algorithms to extract facial features from the images, using TensorFlow. The server feeds the image data into a TensorFlow model to identify features such as the facial contours and the location of the eyes, nose, and mouth.

[0093] Input: Image data stored in an Amazon S3 bucket

[0094] Output: Feature data such as facial contours and the positions of eyes, nose, and mouth

[0095] Step 4:

[0096] Based on the extracted features, the server converts the faces and dialogue of people in stamps for communication applications, comics, and scenes in video works. Here, generative AI models (GPT-4, DALL-E) are used. The server uses DALL-E to synthesize the user's face into a specific scene and uses GPT-4 to generate the dialogue content.

[0097] Input: Facial feature data

[0098] Output: Synthesized face image and generated dialogue content

[0099] Step 5:

[0100] The server generates a personalized digital gift based on the conversion result. The server combines the generated images and text to create a digital gift exclusive to the user.

[0101] Input: Synthetic face image and generated dialogue

[0102] Output: personalized digital gift (images and text)

[0103] Step 6:

[0104] The server transmits the generated personalized digital gift to the user via electronic communication means. The server uses the user's email address or messenger app ID to send a download link for the digital gift.

[0105] Input: Personalized digital gift (image and / or text)

[0106] Output: Download link for the digital gift sent to the user

[0107] (Application example 1)

[0108] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] Conventional advertising systems have the problem that it is difficult to provide users with personalized advertisements, and the effectiveness of advertisements is limited. In addition, there is a lack of a way for users to easily generate personalized advertisements based on their own photos, so the degree of personalization of advertisements cannot be improved.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a cartoon or movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, a means for generating the conversion result as an advertising material, and a means for providing the generated advertising material to a user. This makes it possible to generate a personalized advertisement based on the user's photo.

[0111] "Means for receiving photos uploaded by users" refers to a function that enables users to send their own photos to a server via the Internet and receive the photo data.

[0112] "Means for changing the faces and dialogue of people appearing in messenger app stamps, comics, or movie scenes based on the photograph" refers to a function for changing the faces and dialogue of people appearing in messenger app stamps, comics, or movie scenes using features extracted from a photograph uploaded by a user.

[0113] The "means for providing the conversion result as a personalized digital gift" is a function for individually providing digital content including the converted face and dialogue to the user.

[0114] The "means for generating the conversion result as advertising material" is a function for creating advertising material such as advertising banners and video advertisements using the converted faces and dialogue.

[0115] The "means for providing the created advertising material to the user" is a function for providing the created advertising material to the user so that the user can use it.

[0116] A "machine learning algorithm" is an algorithm that enables a computer to learn patterns from data and then use the results of that learning to make predictions and classify new data.

[0117] "Feature extraction" is the process of extracting important information or patterns from an image or data.

[0118] A "personalized digital gift" is a digital gift that is customized based on a user's individual information and preferences.

[0119] "Advertising Materials" refers to content such as images, videos, and text used in an advertising campaign.

[0120] A system for implementing this invention includes means for receiving photos uploaded by a user, means for converting messenger app stamps or the faces and dialogue of people in cartoon or movie scenes based on the photos, means for providing the results of the conversion as a personalized digital gift, means for generating the results of the conversion as advertising material, and means for providing the generated advertising material to a user.

[0121] The server receives photos uploaded by users over the Internet. The received photos are pre-processed using image processing libraries such as OpenCV and PIL. Then, machine learning algorithms (e.g., face recognition model using Keras) are used to extract facial features from the photos. This feature extraction process provides important information about the user's face.

[0122] Based on the extracted features, a generative AI model is used to transform the faces and dialogue of people in messenger app stickers, comics, and movie scenes. The transformed results are then generated as personalized digital gifts. Furthermore, the same transformed results are used to generate advertising materials (e.g., advertising banners and video ads).

[0123] The generated personalized digital gifts and advertising materials are then sent to the user via email or messenger apps, allowing users to easily generate personalized ads based on their own photos and display them in a special way for friends and family.

[0124] As a concrete example, consider a case where a user uploads a photo of themselves and generates a personalized ad by superimposing their face onto an advertising banner for a specific product (e.g. a new smartphone), which can then be shared with the user's friends and family via social media or messaging apps.

[0125] An example of a prompt is as follows:

[0126] "Upload a photo of yourself and we'll generate personalized ads for you that will be shown specially to your friends and family."

[0127] In this way, it becomes possible to generate personalized advertisements based on the user's photographs, thereby improving the effectiveness of the advertisements.

[0128] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0129] Step 1:

[0130] A user uploads a photo. The user sends his / her photo to the server via the Internet. The input is the user's photo data, and the output is the photo data stored on the server.

[0131] Step 2:

[0132] The server receives the photos and performs pre-processing. The server uses image processing libraries such as OpenCV and PIL to convert the photos to the appropriate format and adjust the size. The input is the user's photo data and the output is the pre-processed photo data.

[0133] Step 3:

[0134] The server uses machine learning algorithms to extract facial features from photos. The server uses a Keras facial recognition model to extract facial features from the preprocessed photo data. The input is the preprocessed photo data, and the output is the facial feature data.

[0135] Step 4:

[0136] The server uses a generative AI model to convert the faces and dialogue of people in messenger app stamps, comics, and movie scenes based on facial feature data. The input is facial feature data, and the output is the converted face and dialogue data.

[0137] Step 5:

[0138] The server generates the transformed data as a personalized digital gift. The server uses the transformed facial and / or interaction data to create a digital gift. The input is the transformed facial and / or interaction data, and the output is a personalized digital gift.

[0139] Step 6:

[0140] The server generates the converted data as advertising material. The server uses the converted face and dialogue data to create advertising material such as advertising banners and video ads. The input is the converted face and dialogue data, and the output is advertising material.

[0141] Step 7:

[0142] The server provides the generated personalized digital gift and advertising material to the user. The server sends the generated digital gift and advertising material to the user via email or a messenger app. The input is the personalized digital gift and advertising material, and the output is the digital gift and advertising material sent to the user.

[0143] Example 2

[0144] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0145] Conventional messenger apps and digital gift delivery methods make it difficult for users to easily create individually customized content. In addition, converting friends' faces into stamps and other digital content requires specialized knowledge and skills, making it difficult for general users to use. Furthermore, there is a lack of a way to quickly and easily share the generated content.

[0146] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0147] In this invention, the server includes a means for receiving an image uploaded by a user, a means for extracting facial features based on the image using a machine learning algorithm, a means for generating a facial image for a stamp based on the features using a generative AI model, and a means for providing the generated stamp image to the user. This allows the user to convert the face of a friend into a stamp or other digital content and share it quickly and easily without requiring specialized knowledge or skills.

[0148] "User" means an individual or organization that utilizes the system to upload images and receive generated stamp images.

[0149] "Image" refers to visual data such as photographs or drawings uploaded by users.

[0150] A "server" is a computer system for processing images received from a user and providing a generated stamp image.

[0151] "Machine learning algorithms" are computational techniques used to extract facial features from images, and include face detection algorithms and facial landmark detection algorithms.

[0152] A "face detection algorithm" is a computational method for locating faces within an image.

[0153] A "facial landmark detection algorithm" is a computational method for identifying facial feature points (landmark points).

[0154] A "generative AI model" is an artificial intelligence model that generates facial images for stamps based on extracted facial features.

[0155] A "stamp image" is a facial image created by a generative AI model that can be used in messenger apps, etc.

[0156] The "means for providing" is a method for sending the generated stamp image to the user via email or a messenger app.

[0157] This invention is a system that generates face images for stamps using a machine learning algorithm and a generative AI model based on an image uploaded by a user, and provides the image to the user. A specific embodiment of this system will be described below.

[0158] First, the user uses a terminal (smartphone or PC) to access the system's web interface or a dedicated application. The user clicks the photo upload button and selects the friend's photo file. After selecting, the user presses the upload button to send the photo to the server.

[0159] The server receives the image sent by the user. It temporarily stores the received image data and then applies a face detection algorithm. Specifically, it uses OpenCV's Haar Cascade or Dlib's face detection function to detect faces in the image. Once face detection is complete, the server obtains the coordinate information of the face.

[0160] Next, the server uses Dlib's facial landmark detection feature to extract facial features. By extracting facial landmark points, detailed facial features can be captured. The extracted landmark points are used in the next processing step.

[0161] The server uses a generative AI model (e.g., GAN: Generative Adversarial Network) to generate a face image for the stamp based on the extracted facial features. The GAN model uses the extracted landmark points as input to generate a face image for the stamp. The generated face image is saved in a format that the user can use in messenger apps, etc.

[0162] Finally, the server provides the generated stamp image to the user via email or a messenger app (e.g., LINE or WhatsApp). The user can download the received stamp image and use it in the messenger app.

[0163] As a concrete example, consider the case where a user uploads a photo of a friend and converts the friend's face into a LINE stamp. The user selects the photo in the LINE app on their smartphone and uploads it to the system. The server uses OpenCV to detect faces in the received photo and Dlib to extract facial features. It then uses GAN to convert the friend's face into a stamp and sends the generated stamp to the user as a LINE message. The user can use the received stamp in LINE chats.

[0164] An example of a prompt is, "Upload a photo of a friend and turn their face into a LINE stamp."

[0165] The system allows users, without any specialist knowledge or skills, to convert their friends' faces into stamps and other digital content that can be shared quickly and easily.

[0166] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0167] Step 1:

[0168] A user uploads a photo of a friend.

[0169] Specifically, the user uses a terminal (smartphone or PC) to access the system's web interface or a dedicated application. The user clicks the photo upload button and selects a friend's photo file. After selecting, the user presses the upload button, and the photo is sent to the server.

[0170] Input: A photo file selected by the user

[0171] Output: Photo data sent to the server

[0172] Step 2:

[0173] The server receives the photo and performs face detection.

[0174] Specifically, the server receives the photo sent by the user and temporarily stores it. Then, it detects the face in the photo using OpenCV's Haar Cascade and Dlib's face detection function. Once face detection is complete, the server obtains the face coordinate information.

[0175] Input: Photo data sent by the user

[0176] Output: Face coordinate information

[0177] Step 3:

[0178] The server extracts facial features.

[0179] Specifically, the server uses Dlib's facial landmark detection feature to extract facial landmark points, which allow detailed facial features to be identified, and these points are used in the next processing step.

[0180] Input: Face coordinate information

[0181] Output: Facial landmark points

[0182] Step 4:

[0183] The server generates a face image for the stamp.

[0184] Specifically, the server uses a generative AI model (for example, a Generative Adversarial Network (GAN)) to generate a facial image for the stamp. The GAN model uses the extracted landmark points as input to generate a facial image for the stamp. The generated facial image is saved in a format that the user can use in messenger apps, etc.

[0185] Input: Facial landmark points

[0186] Output: Face image for stamp

[0187] Step 5:

[0188] The server provides the generated stamp image to the user.

[0189] Specifically, the server sends the generated stamp image to the user via email or a messenger app (e.g., LINE or WhatsApp). The user can download the received stamp image and use it in the messenger app.

[0190] Input: Generated stamp image

[0191] Output: Stamp image sent to user

[0192] (Application example 2)

[0193] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0194] In autonomous vehicles, there is a demand for providing passengers with personalized entertainment and information. However, with conventional systems, it is difficult to provide content that reflects the individual characteristics of each passenger, which makes it difficult to increase passenger satisfaction.

[0195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0196] In this invention, the server includes a means for receiving an image uploaded by a user, a means for generating an avatar for the infotainment system of the autonomous vehicle based on the image, and a means for displaying the generated avatar on a display inside the vehicle, thereby generating a personalized avatar based on the passenger's face, and providing customized entertainment and information to the passenger.

[0197] "User" refers to an individual or organization that uses the System.

[0198] "Upload" refers to the act of a user sending data from their device to a server.

[0199] "Image" refers to visual information stored in digital format.

[0200] An "autonomous vehicle" refers to a vehicle that drives autonomously using artificial intelligence and sensor technology.

[0201] An "infotainment system" refers to an in-car system that integrates information and entertainment.

[0202] An "avatar" refers to a digital character that reflects the characteristics of a user.

[0203] "Generation" refers to the process by which a system creates new data based on input data.

[0204] "Display" refers to a device for displaying visual information.

[0205] A "machine learning algorithm" refers to a computational technique for learning patterns from data and making predictions or classifications.

[0206] "Features" refer to important information extracted from images or data.

[0207] "Entertainment" refers to content that provides enjoyment and entertainment to users.

[0208] "Information" refers to data and knowledge that is useful to the user.

[0209] "Customization" refers to adjustment to the needs and preferences of an individual user.

[0210] A system for implementing the invention includes means for receiving an image uploaded by a user, means for generating an avatar for an infotainment system of an autonomous vehicle based on the image, and means for displaying the generated avatar on an in-vehicle display.

[0211] System Programming

[0212] The system operates as follows.

[0213] 1. User uploads an image:

[0214] Users upload images from their devices to the system, which transmits them to a server and stores them for processing.

[0215] 2. The server processes the image:

[0216] The server uses machine learning algorithms to extract facial features from the received images: it uses OpenCV to detect faces and Keras to extract features using a pre-trained facial feature extraction model.

[0217] 3. Generate an avatar:

[0218] Based on the extracted features, a generative AI model is used to generate an avatar, which is a digital character that reflects the user's facial features.

[0219] 4. Display on the display:

[0220] The generated avatar will be displayed on the autonomous vehicle's infotainment system display, allowing passengers to enjoy a personalized avatar based on their own face.

[0221] Hardware and Software Used

[0222] Hardware:

[0223] Cameras in self-driving vehicles: Used to capture passengers' faces.

[0224] Display: Used to display the generated avatar.

[0225] software:

[0226] OpenCV: Used for face detection.

[0227] Keras: Used to run the face feature extraction model.

[0228] PIL (Python Imaging Library): Used to process the generated avatar as an image.

[0229] Examples

[0230] For example, when a passenger enters the vehicle, a camera will automatically capture their face and the system will generate an avatar based on that image, which will then be displayed on the vehicle's display to provide customized entertainment and information to the passenger.

[0231] Examples of prompt statements

[0232] "Develop a system that takes a picture of a passenger's face and generates a personalized avatar based on that face. The avatar will be used by the infotainment system inside the car."

[0233] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0234] Step 1:

[0235] A user uploads an image.

[0236] A user selects an image from his / her device and uploads it to the system. The input is the image file stored on the user's device, and the output is the image data sent to the server. The server temporarily stores the received image data.

[0237] Step 2:

[0238] The server processes the images.

[0239] The server performs face detection on the received image data using OpenCV. The input is the uploaded image data, and the output is the area information of the detected face. Specifically, the image is converted to grayscale and a face detection algorithm is applied to identify the position of the face.

[0240] Step 3:

[0241] The server extracts facial features.

[0242] The server extracts features from the face detection results using a face feature extraction model pre-trained with Keras. The input is face region information, and the output is a face feature vector. Specifically, the detected face region is resized and input to the model to obtain a feature vector.

[0243] Step 4:

[0244] The server generates the avatar.

[0245] The server generates an avatar using a generative AI model based on the extracted facial feature vector. The input is the facial feature vector, and the output is the generated avatar image. Specifically, the feature vector is input into the generative AI model to generate the avatar image.

[0246] Step 5:

[0247] The server displays the avatar on the display.

[0248] The server transmits the generated avatar image to the display of the infotainment system of the autonomous vehicle. The input is the generated avatar image, and the output is the avatar displayed on the display. Specifically, the server converts the avatar image into a format suitable for the display and transmits a signal to display it on the display in the car.

[0249] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0250] "Example 1"

[0251] In one embodiment of the present invention, a system is provided that includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp, a cartoon or a movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, and an emotion engine for recognizing the emotion of the user. The emotion engine recognizes the emotion from the user's facial expression, tone of voice, text input, etc., and adjusts the generation of the personalized digital gift based on the emotion. For example, if it is recognized that the user is happy, the system generates a personalized digital gift that expresses joy.

[0252] "Example 2"

[0253] In another embodiment of the present invention, a user uploads a photo of a friend to the system, and the system uses a machine learning algorithm to extract the friend's facial features. The system then converts the person's face on the stamp into the friend's face based on the extracted features. Furthermore, the system recognizes the user's emotion with an emotion engine and adjusts the generated personalized digital gift based on the emotion. For example, if the user is recognized as sad, the system generates a personalized digital gift with a comforting message.

[0254] The process flow of each embodiment will be described below.

[0255] "Example 1"

[0256] Step 1: A user uploads a photo to the system.

[0257] Step 2: The system receives the uploaded photo and extracts the person's facial features from the photo.

[0258] Step 3: Based on the extracted features, the system converts the faces and dialogue of people in messenger app stickers, comic book and movie scenes.

[0259] Step 4: The system uses the emotion engine to recognize the user's emotion.

[0260] Step 5: The system tailors the generated personalized digital gift based on the recognized emotion and provides it to the user.

[0261] "Example 2"

[0262] Step 1: A user uploads a photo of their friend into the system.

[0263] Step 2: The system receives the uploaded photo and uses machine learning algorithms to extract your friend's facial features.

[0264] Step 3: The system converts the face of the person in the stamp into the face of a friend based on the extracted features. Step 4: The system uses an emotion engine to recognize the user's emotion.

[0265] Step 5: The system tailors the generated personalized digital gift based on the recognized emotion and provides it to the user.

[0266] Example 1

[0267] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0268] Conventional digital gift generation systems provide uniform content without considering the user's emotions, making it difficult to provide a personalized experience. In addition, conversion processing based on images uploaded by users was limited, making it difficult to convert into various formats such as messenger app stamps, comics, and video scenes.

[0269] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a face or dialogue of a person in a communication application stamp, a cartoon or a video scene based on the image, a means for providing the conversion result as a personalized digital gift, and an emotion engine for recognizing the user's emotion. This makes it possible to generate a personalized digital gift based on the user's emotion.

[0270] "User" refers to any person who utilizes the System to upload an image and receive a personalized digital gift.

[0271] "Image" refers to visual data such as photographs or drawings uploaded by users.

[0272] "Communication application" refers to software that enables users to send and receive messages, such as a messenger application or chat application.

[0273] "Stamps" refer to small images or icons used within communication applications.

[0274] "Manga" refers to a series of pictures or illustrations that visually depict a story.

[0275] A "film scene" refers to a portion of a visual medium such as a film or video.

[0276] "Conversion" refers to the process of changing original images or data into a different format or content.

[0277] "Personalized digital gifts" refers to individually customized digital content generated based on a user's image and emotions.

[0278] An "emotion engine" refers to software or algorithms that recognize emotions from a user's facial expressions, tone of voice, text input, etc.

[0279] A "machine learning algorithm" is a mathematical model that learns patterns from data and makes predictions or classifications.

[0280] The present invention is a system for generating and providing a personalized digital gift based on an image uploaded by a user. The system includes a means for a user to upload an image, a means for analyzing and transforming the image, a means for providing the transformation result as a personalized digital gift, and an emotion engine for recognizing the user's emotion.

[0281] Hardware and Software Used

[0282] Hardware

[0283] Server: Receives, analyzes, transforms images, recognizes emotions, and generates and delivers digital gifts.

[0284] Device: The device (computer, smartphone, tablet, etc.) through which a user uploads images.

[0285] software

[0286] Website or application: Provides an interface for users to upload images.

[0287] Machine learning algorithms used to extract features from images and perform transformations (e.g. OpenCV, Dlib, TensorFlow, PyTorch).

[0288] Emotion engine: Software for recognizing user emotions (e.g. Emotion API).

[0289] Generative AI models: Models for converting images and generating dialogue (e.g., StyleGAN, GPT-3 (registered trademark)).

[0290] Mail Server and Messenger API: Used to send the generated digital gifts to users.

[0291] Examples

[0292] A user accesses a website and uploads an image of their own smiling face. The server receives the image and uses a machine learning algorithm to extract facial features. Next, the emotion engine recognizes the user's smile as "happiness." Based on this information, the server generates a stamp for a communication application that uses the user's face to express happiness. Finally, the generated stamp is sent to the user's email address.

[0293] Examples of prompt statements

[0294] "Generate stickers for a communication application that express joy based on images uploaded by users. The user's emotions are recognized from their smiles."

[0295] In this way, it is possible to generate and offer personalized digital gifts based on the user's emotions.

[0296] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0297] Step 1: User uploads an image.

[0298] A user uploads his / her own image through a website or application by clicking an upload button and selecting an image file from a file selection dialog. The input is the user's image file, and the output is the image data sent to the server.

[0299] Step 2: The server receives the image.

[0300] The server receives the image sent by the user and saves it in the database. Specifically, it receives the HTTP request, saves the image file in a temporary directory, and then saves it in the database. The input is the image data sent by the user, and the output is the path of the saved image file.

[0301] Step 3: The server analyzes the image.

[0302] The server uses machine learning algorithms to extract facial features from images. Specifically, it uses OpenCV and Dlib to perform face detection and identify the coordinates of facial landmarks (eyes, nose, mouth, etc.). The input is the path to the saved image file, and the output is the facial feature data.

[0303] Step 4: The server recognizes the emotion.

[0304] The server uses the emotion engine to recognize the user's emotion from the image. Specifically, it sends facial feature data to the Emotion API and classifies the emotion. The input is the facial feature data, and the output is the recognized emotion data.

[0305] Step 5: The server performs the conversion process.

[0306] The server performs conversion processing using a generative AI model based on the extracted facial features and recognized emotions. Specifically, it uses StyleGAN to convert the user's face into a stamp for a communication application, and uses GPT-3 to generate dialogue that matches the emotion. The input is facial feature data and emotion data, and the output is the converted stamp and dialogue data.

[0307] Step 6: The server generates the personalized digital gift.

[0308] The server generates a personalized digital gift based on the converted results. Specifically, it creates a digital card that includes a stamp with the user's face and a message that matches the user's emotion. The input is the converted stamp and conversation data, and the output is the generated digital gift.

[0309] Step 7: The server sends the digital gift.

[0310] The server sends the generated personalized digital gift to the user via email or a communication application. Specifically, the server uses the SMTP protocol or a messenger API to send the digital gift. The input is the generated digital gift and the output is the digital gift sent to the user.

[0311] (Application example 1)

[0312] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0313] In the conventional personalized digital gift generation system, since the conversion is performed uniformly without considering the user's emotions, it is not possible to provide an appropriate digital gift according to the user's emotions. In addition, if the generated digital gift does not match the user's emotions, there is a problem that the user's satisfaction decreases. Furthermore, since there is a lack of means to effectively deliver the generated digital gift, it is difficult to deliver it to the user quickly and reliably.

[0314] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0315] In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a cartoon or movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, a means including an emotion engine for recognizing the emotion of the user, a means for adjusting the generation of the personalized digital gift based on the emotion, and a means for sending the generated personalized digital gift to the user via email or a messenger app. This makes it possible to generate and quickly deliver a personalized digital gift appropriate to the emotion of the user.

[0316] "Means for receiving photos uploaded by users" refers to a function that enables users to send their own photos to the system via the Internet and for the server to receive those photos.

[0317] "Messenger app stamps" are images or icons used within messenger applications to allow users to express emotions or messages.

[0318] "Means of transforming the faces and dialogue of characters in manga or movie scenes" refers to technology that replaces the faces and dialogue of characters appearing in manga or movie scenes with the user's face and dialogue based on photos uploaded by the user.

[0319] A "personalized digital gift" is a digital gift that is customized based on a user's individual characteristics and emotions.

[0320] An "emotion engine" is software that recognizes emotions from a user's facial expressions, tone of voice, text input, etc., and adjusts the system's operation based on those emotions.

[0321] A "machine learning algorithm" is a computational technique for learning patterns based on large amounts of data and making predictions and classifications.

[0322] "Means of sending via email or messenger app" refers to a function for sending the generated personalized digital gift to a user via email or messenger application.

[0323] An embodiment of the present invention will be described. First, a user uses a terminal such as a smartphone or a personal computer to access a dedicated application or website. The user sends a photo through an interface for uploading the user's own photo. The photo is received by a server.

[0324] The server uses a machine learning algorithm to extract features from the received photos. The algorithm uses image processing libraries such as OpenCV to detect facial feature points and analyze the shape and expression of the face. Then, based on the extracted features, it converts the faces and dialogue of people in messenger app stamps, comics, and movie scenes. For the conversion, it uses technologies such as FaceSwap and DeepFake.

[0325] In addition, the server recognizes the user's emotions using an emotion engine. The emotion engine uses software such as EmotionRecognizer to analyze emotions from the user's facial expressions, tone of voice, text input, etc. Based on the recognized emotions, the server adjusts the generation of personalized digital gifts. For example, if the user is happy, the server generates a digital gift that expresses joy.

[0326] The generated personalized digital gift is then sent to the user from the server via email or a messenger app, using a communication library such as MessagingService.

[0327] For example, if a user uploads a photo of themselves feeling happy, the system will recognize that emotion and edit the character in the movie scene to make them look happy. The generated video will be sent to the user's email address.

[0328] An example of a prompt sentence might be:

[0329] "Create an application that converts a character's face in a movie scene into the user's face based on a photo uploaded by the user, and edits the scene based on the user's emotions. Please also include a function to send the generated video to the user."

[0330] In this way, personalized digital gifts appropriate to the user's emotions can be generated and quickly delivered.

[0331] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0332] Step 1:

[0333] The user uses the device to access a dedicated application or website and uploads his or her own photograph.

[0334] Input: User photo

[0335] Output: The user's photo sent to the server.

[0336] How it works: A user selects a photo through the application interface and clicks the upload button, and the photo is sent to the server over the Internet.

[0337] Step 2:

[0338] Based on the photos received by the server, a machine learning algorithm is used to extract features from the photos.

[0339] Input: The user's photo sent to the server

[0340] Output: Extracted facial feature data

[0341] Specific operation: The server uses image processing libraries such as OpenCV to detect facial feature points from photos and analyze facial shape and facial expressions.

[0342] Step 3:

[0343] Based on the extracted features, the server converts the faces and dialogue of people in messenger app stamps, manga, and movie scenes.

[0344] Input: Extracted facial feature data

[0345] Output: Converted character faces and dialogue

[0346] How it works: The server uses techniques such as FaceSwap and DeepFake to swap the user's face with that of a character in a movie scene and adapts the dialogue to match the user's characteristics.

[0347] Step 4:

[0348] The server recognizes the user's emotions using an emotion engine.

[0349] Input: User photo

[0350] Output: Recognized emotion data

[0351] How it works: The server uses software such as EmotionRecognizer to analyze emotions from the user's facial expressions, tone of voice, text input, etc.

[0352] Step 5:

[0353] The server coordinates the generation of the personalized digital gift based on the recognized emotion.

[0354] Input: Recognized emotion data, transformed character faces and dialogue

[0355] Output: Personalized digital gifts tailored based on emotions

[0356] Specific operation: The server generates digital gifts that reflect emotions such as joy or sadness according to the recognized emotions.

[0357] Step 6:

[0358] The server then sends the generated personalized digital gift to the user via email or a messenger app.

[0359] Input: Personalized digital gifts tailored based on emotions

[0360] Output: A personalized digital gift sent to the user.

[0361] Specific operation: The server uses a communication library such as MessagingService to send the generated digital gift to the user's email address or messenger app.

[0362] Example 2

[0363] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0364] In conventional systems, it was difficult to provide personalized digital gifts that took into account the user's emotions when converting stamps in messenger apps and faces and dialogues in visual media based on photos uploaded by the user. In addition, there was a lack of means to appropriately send the generated digital gifts to the user. This limited the user experience and made it difficult to provide services that met individual needs.

[0365] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0366] In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or visual media based on the image, a means for providing the conversion result as a personalized digital gift, a means for recognizing a user's emotion and adjusting the personalized digital gift based on the emotion, and a means for sending the personalized digital gift to the user, thereby enabling the generation and sending of a personalized digital gift according to the user's emotion.

[0367] "User" means any individual or entity that utilizes the System to upload images and receive personalized digital gifts.

[0368] An "image" is data containing visual information, such as a photograph or drawing, that a user uploads to the system.

[0369] A "messenger app" is a software application that enables users to exchange messages, including text, images, and audio, with other users.

[0370] "Stamps" are small images used within messenger apps that express specific facial expressions or characters.

[0371] "Visual media" refers to media content that includes visual elements, such as manga and movies.

[0372] A "machine learning algorithm" is a computational technique for learning patterns from data and making predictions or classifications.

[0373] A "feature" is an identifiable element in an image, such as a facial contour, eyes, nose, or mouth.

[0374] A "personalized digital gift" is a digital gift that is individually generated based on a user's input and emotions.

[0375] An "emotion engine" is software or algorithms for analyzing and recognizing a user's emotions.

[0376] An "electronic means of communication" is any means for sending or receiving information in digital form, such as email or a messaging app.

[0377] This invention is a system that converts the faces and dialogues of people in stamps on messenger apps and visual media based on images uploaded by users and provides them as personalized digital gifts. A specific embodiment of this system will be described below.

[0378] System Configuration

[0379] Hardware

[0380] Server: A computer system with powerful processors and large memory capacity that runs image processing and machine learning algorithms.

[0381] Terminal: A smartphone or PC used by a user to upload images and receive generated digital gifts.

[0382] software

[0383] Image processing library: Software for pre-processing and transforming images, such as OpenCV or Adobe® Photoshop® API.

[0384] Machine learning algorithms: Algorithms for extracting features from images and performing transformations, such as TensorFlow.

[0385] Emotion engine: Software for recognizing user emotions, such as IBM Watson® or Microsoft® Azure® Emotion API.

[0386] Electronic communication means: means for sending the generated digital gift to the user, such as SMTP protocol or messenger API (e.g., Facebook® Messenger API).

[0387] System Operation

[0388] Uploading an image

[0389] Users use their devices to upload pictures of their friends to the system, either through the system's web interface or a dedicated app.

[0390] Image preprocessing and feature extraction

[0391] The server receives the uploaded image, first pre-processes the image using OpenCV, then uses TensorFlow's face recognition model to extract the facial features of the friend. In this feature extraction process, the main feature points such as the facial contour, eyes, nose, and mouth are identified.

[0392] Stamp Generation

[0393] The server uses Adobe Photoshop API to generate stamps based on the extracted facial features, adjusting the facial contours and facial expressions to fit the stamp design.

[0394] Emotion recognition and digital gift adjustment

[0395] The server uses IBM Watson to analyze the user's emotions, analyzing facial expressions and text inputs while the user is using the system. For example, if the server detects that the user is sad, it will generate a personalized digital gift with a comforting message.

[0396] Sending digital gifts

[0397] The server sends the generated personalized digital gift to the user via email or messenger app using SMTP protocol or messenger API. The user can view the received digital gift and share it with friends.

[0398] Examples

[0399] As a concrete example, consider a case where a user uploads a photo of a friend and converts the friend's face into a sticker in a messenger app.

[0400] 1. The user opens the dedicated app on their smartphone and selects and uploads photos of their friends.

[0401] 2. The server receives the uploaded photo, pre-processes the image using OpenCV, and then uses TensorFlow's face recognition model to extract the friend's facial features.

[0402] 3. The server uses Adobe Photoshop API to generate stamps based on the extracted facial features, adjusting the facial contours and facial expressions to fit the stamp design.

[0403] 4. The server uses IBM Watson to analyze the user's emotions and, if the user is recognized as sad, generates a personalized digital gift with a comforting message.

[0404] 5. The server sends the generated personalized digital gift to the user through the Facebook Messenger API. The user can view the received digital gift and share it with their friends.

[0405] Examples of prompt statements

[0406] Upload a photo of your friend and turn their face into a sticker. Generate and send personalized digital gifts based on your user's emotions.

[0407] By inputting this prompt sentence into the generative AI model, the system will carry out the above process and provide the user with an original stamp.

[0408] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0409] Step 1:

[0410] A user uses a terminal to upload a friend's image to the system. The user opens the system's web interface or a dedicated app, selects an image file, and presses the upload button. The input is the image file selected by the user, and the output is the image data sent to the server.

[0411] Step 2:

[0412] The server receives the uploaded image. The received image data is first preprocessed using an image processing library (e.g., OpenCV). Preprocessing includes image resizing, noise removal, color correction, etc. The input is the image data sent by the user, and the output is the preprocessed image data.

[0413] Step 3:

[0414] The server inputs the preprocessed image data into a machine learning algorithm (e.g., TensorFlow's facial recognition model) to extract facial features. The feature extraction process identifies key features such as the facial contour, eyes, nose, and mouth. The input is the preprocessed image data, and the output is the extracted facial feature data.

[0415] Step 4:

[0416] The server generates a stamp using an image processing library (e.g., Adobe Photoshop API) based on the extracted facial feature data. Specifically, it adjusts the facial contours and facial expressions to fit the stamp design. The input is the facial feature data, and the output is the generated stamp image.

[0417] Step 5:

[0418] The server recognizes the user's emotions using an emotion engine (e.g., IBM Watson). It analyzes emotions from facial expressions and text inputs while the user is using the system. The input is the user's facial expression data and text data, and the output is the recognized emotion data.

[0419] Step 6:

[0420] The server tailors the personalized digital gift including the generated stamp based on the recognized emotion data, e.g., if the user is recognized as sad, it generates a digital gift including a comforting message, where the input is the emotion data and the stamp image, and the output is the tailored personalized digital gift.

[0421] Step 7:

[0422] The server sends the generated personalized digital gift to the user via email or a messenger app using the SMTP protocol or a messenger API (e.g., Facebook Messenger API). The input is the personalized digital gift and the output is the digital gift sent to the user.

[0423] (Application example 2)

[0424] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0425] In conventional messenger apps and video content, it was difficult for users to generate personalized characters and stamps based on their friends' photos. In addition, there was a lack of functionality to adjust content according to the user's emotions, so there was a need to improve the user experience. Furthermore, there were limited means to effectively provide the generated content.

[0426] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0427] In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting the face and dialogue of a character of a messenger app stamp or video content based on the photo, a means for providing the conversion result as a personalized digital gift, and a means for recognizing the emotion of the user and adjusting the personalized digital gift based on the emotion. This allows the user to easily generate a personalized character or stamp based on a friend's photo, and further allows the content to be adjusted according to the user's emotion. In addition, the generated content can be effectively provided.

[0428] "User" means an individual who utilizes the System to upload photos and receive personalized digital gifts.

[0429] A "photo" is image data uploaded by a user, which includes features such as a friend's face.

[0430] "Messenger app stamps" are digital stamps used within messenger apps and are generated based on photos uploaded by users.

[0431] "Video content characters" are people or characters that appear in videos, whose faces and dialogue are transformed based on photos uploaded by users.

[0432] "Dialogue" refers to conversations or statements made by characters in video content or messenger apps.

[0433] "Personalized Digital Gifts" are individualized digital content that is generated based on photos and emotions uploaded by users.

[0434] A "machine learning algorithm" is an algorithm for extracting features from photographs and transforming people's faces and dialogue based on those features.

[0435] "Means for recognizing emotions" refers to technology for analyzing a user's emotions and adjusting content based on those emotions.

[0436] The "means of provision" refers to the means of sending the generated personalized digital gift to the user via email or a messenger app.

[0437] The system for implementing the present invention receives a photo uploaded by a user, converts the face and dialogue of a character in a messenger app stamp or video content based on the photo, and provides a generated personalized digital gift. It also includes a function of recognizing the user's emotions and adjusting the personalized digital gift based on the emotions.

[0438] Hardware and Software Used

[0439] Hardware: Smartphone, head-mounted display

[0440] Software: OpenCV, Dlib, EmotionRecognizer (emotion recognition library), VideoEditor (video editing library)

[0441] System Configuration

[0442] 1. User terminal: A device that users use to upload photos. This includes smartphones and head-mounted displays.

[0443] 2. Server: Receives the uploaded photos, uses machine learning algorithms to extract facial features, and processes them to transform the faces and dialogue of characters in the video content.

[0444] 3. Emotion Recognition Engine: Analyzes user emotions and tailors personalized digital gifts based on those emotions.

[0445] 4. Delivery method: The generated personalized digital gift is sent to the user via email or messenger app.

[0446] Data processing and calculation

[0447] 1. Facial feature extraction: The server extracts facial features from the uploaded photo using OpenCV and Dlib, which obtains the facial landmark points in the photo.

[0448] 2. Emotion Recognition: The server recognizes the user's emotion using EmotionRecognizer, which analyzes the user's emotional state when he uploads a photo.

[0449] 3. Video Editing: The server uses VideoEditor to edit the video content based on the extracted facial features and the recognized emotions, so that the faces and dialogues of the characters in the video are transformed into the faces of the user's friends and the content is adjusted according to their emotions.

[0450] Examples

[0451] Consider a case where a user uploads a photo of a friend and converts the friend's face into a character in a video content. For example, if the user wants to "make the friend's face into an animated character and create a touching scene," the system converts the friend's face into an animated character and generates a video including the touching scene.

[0452] Examples of prompt statements

[0453] Upload a photo of your friend, transform their face into a cartoon character, and generate a video with some touching moments.

[0454] In this way, users can enjoy personalized video content.

[0455] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0456] Step 1:

[0457] A user uploads a photo of a friend.

[0458] Input: A friend's photo file selected by the user.

[0459] Output: The photo data uploaded to the server.

[0460] Specific operation: A user selects a photo of a friend using a smartphone or a head-mounted display and uploads it to the system. The server receives the photo data.

[0461] Step 2:

[0462] The server extracts facial features from the photo.

[0463] Input: The uploaded photo data.

[0464] Output: Extracted facial feature data (landmark points).

[0465] What it does: The server uses OpenCV and Dlib to detect faces in photos and extract facial landmark points, which gives us facial feature data.

[0466] Step 3:

[0467] The server recognizes the user's emotions.

[0468] Input: The uploaded photo data.

[0469] Output: The recognized emotion data.

[0470] What it does: The server uses EmotionRecognizer to analyze the photo and recognize the user's emotion, which analyzes the user's emotional state when uploading the photo.

[0471] Step 4:

[0472] The server edits the video content based on facial features and emotions.

[0473] Input: extracted facial feature data, recognized emotion data, original video content.

[0474] Output: The edited video content.

[0475] How it works: The server uses VideoEditor to convert the character's face in the video into the friend's face based on the extracted facial feature data, and adjusts the video content based on the recognized emotion data, thereby generating personalized video content.

[0476] Step 5:

[0477] The server provides the generated personalized digital gift.

[0478] Input: The edited video content.

[0479] Output: A personalized digital gift provided to the user.

[0480] How it works: The server sends the generated video content to the user via email or messenger app, so that the user can receive personalized video content.

[0481] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0482] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0483] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0484] In the above embodiment, an example has been given in which the specific process is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0485] [Second embodiment]

[0486] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0487] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0488] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure.

[0489] The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0490] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0491] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0492] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0493] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0494] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0495] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0496] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0497] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0498] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0499] "Example 1"

[0500] The system of the present invention includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a scene in a cartoon or movie based on the photo, and a means for providing the conversion result as a personalized digital gift. Specifically, a user uploads his / her own photo through a website or application. Next, the system uses a machine learning algorithm to extract features from the photo and converts the face or dialogue of the person based on the features. Finally, the generated personalized digital gift is sent to the user via email or a messenger app.

[0501] "Example 2"

[0502] As a specific example, consider a case where a user uploads a photo of a friend and converts the friend's face into a stamp for a messenger app. First, the user uploads the friend's photo to the system. Next, the system uses a machine learning algorithm to extract the friend's facial features and converts the face of the person in the stamp into the friend's face based on the features. Finally, the system sends the generated personalized digital gift to the user via email or a messenger app. In this way, the user can get an original stamp with the friend's face drawn on it.

[0503] The process flow of each embodiment will be described below.

[0504] "Example 1"

[0505] Step 1: The user uploads a photo of themselves via a website or application.

[0506] Step 2: The system uses machine learning algorithms to extract features from the photo.

[0507] Step 3: The system transforms the person's face and dialogue based on the extracted features.

[0508] Step 4: The generated personalized digital gift is sent to the user via email or messenger app.

[0509] "Example 2"

[0510] Step 1: A user uploads a photo of their friend into the system.

[0511] Step 2: The system uses machine learning algorithms to extract your friend's facial features.

[0512] Step 3: The system converts the face of the person on the stamp into the face of a friend based on the extracted features. Step 4: The system sends the generated personalized digital gift to the user via email or messenger app.

[0513] Example 1

[0514] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0515] In the conventional digital gift generation system, when generating personalized digital gifts based on photos uploaded by users, the conversion of faces and dialogues is done manually, which is time-consuming and labor-intensive. In addition, there are also issues that the quality of the generated digital gifts is inconsistent and may not meet the user's expectations. Furthermore, there is a lack of a means to quickly and reliably provide the generated digital gifts to the users.

[0516] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0517] In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a stamp of a communication application or a face or dialogue of a person in a scene of a cartoon or a video work based on the image, a means for providing the conversion result as a personalized digital gift, a means for using a machine learning algorithm to extract features from the image, a means for using a generative AI model to generate a face or dialogue of a person based on the features, and a means for transmitting the generated personalized digital gift to the user via an electronic communication means. This makes it possible to quickly and efficiently generate a high-quality, consistent personalized digital gift based on an image uploaded by a user and provide it to the user.

[0518] "User" means an individual or entity that utilizes the System to upload an image and receive a personalized digital gift.

[0519] "Images" are visual data such as photographs and illustrations that users upload to the system.

[0520] A "communication application" is software for sending and receiving messages, and has the function of sending stamps and images.

[0521] A "stamp" is a small image or icon used within a communication application to express an emotion or message.

[0522] "Manga" is a series of pictures or illustrations intended to visually represent a story.

[0523] A "visual work" is any medium that contains a visually presented story or information, such as a film or television program.

[0524] A "machine learning algorithm" is a computational technique for learning patterns and features from data and making predictions and classifications.

[0525] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate new data.

[0526] A "personalized digital gift" is a digital gift that is customized for a particular user based on the user's image and information.

[0527] "Electronic communication means" refers to means for sending and receiving data via the Internet, email, messenger apps, etc.

[0528] The present invention is a system for generating and providing personalized digital gifts based on images uploaded by users. A specific embodiment of the system will be described below.

[0529] System Configuration

[0530] Hardware and Software

[0531] The server uses the following hardware and software:

[0532] Hardware: High-performance server machine (e.g. cloud server with CPU, GPU, and memory)

[0533] Software: Machine learning algorithms (e.g. TensorFlow), generative AI models (e.g. GPT-4, DALL-E), databases (e.g. Amazon S3), communications (e.g. SMTP servers, messenger APIs)

[0534] Processing flow

[0535] 1. Upload your photo

[0536] A user accesses a website or application using a smartphone or PC. For example, the user opens an application on the smartphone, takes a photo of his / her face using the camera function, and presses the upload button. This sends the photo data to the application.

[0537] 2. Receiving photos

[0538] The server receives the images uploaded by the user. Specifically, the server uses an Amazon S3 bucket to store the image data. The server receives the HTTP request, analyzes the image data, and stores it.

[0539] 3. Feature Extraction

[0540] The server uses machine learning algorithms to extract features from the images. It uses TensorFlow for this process. The server feeds the image data into a TensorFlow model to identify features such as the contours of the face and the location of the eyes, nose, and mouth. For example, it uses a facial recognition model to obtain the coordinates of each part of the face.

[0541] 4. Face and dialogue transformation

[0542] Based on the extracted features, the server converts the faces and dialogue of people in stamps for communication applications, comics, and scenes in video works. Here, generative AI models (GPT-4, DALL-E) are used. The server uses DALL-E to synthesize the user's face into a specific scene and uses GPT-4 to generate the dialogue content. For example, the server synthesizes the user's face into a scene from the famous science fiction movie "XXX" and changes the dialogue in that scene to the user's name or a specific phrase.

[0543] 5. Create personalized digital gifts

[0544] The server generates a personalized digital gift based on the conversion result. The server combines the generated images and text to create a digital gift for the user. For example, it creates a digital stamp with the user's face or a comic strip featuring the user.

[0545] 6. Sending Digital Gifts

[0546] The server transmits the generated personalized digital gift to the user via electronic communication means. The server transmits a download link for the digital gift using the user's email address or messenger app ID. For example, the server transmits an email containing a download link for the digital gift to the user's email address.

[0547] Examples and prompts

[0548] Examples

[0549] Users upload a photo of their face through the app and receive a digital gift that combines their face with a scene from the famous sci-fi movie "XXX" based on the photo.

[0550] Examples of prompt statements

[0551] "Based on the face photo uploaded by the user, please composite the face into a scene from the famous science fiction movie "XXX" and generate a digital gift that includes a specific dialogue."

[0552] In this way, users can easily create and enjoy their own special digital gifts.

[0553] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0554] Step 1:

[0555] A user accesses a website or application using a smartphone or PC. The user takes a photo of his or her face using the camera function and presses the upload button. This sends the photo data to the application.

[0556] Input: A photo of the user's face

[0557] Output: The photo data sent to the application.

[0558] Step 2:

[0559] The server receives the images uploaded by the user: the server receives the HTTP request, parses the image data, and stores it in an Amazon S3 bucket.

[0560] Input: Photo data sent from the application

[0561] Output: Image data stored in an Amazon S3 bucket

[0562] Step 3:

[0563] The server uses machine learning algorithms to extract features from the images, using TensorFlow. The server feeds the image data into a TensorFlow model to identify features such as the facial contours and the location of the eyes, nose, and mouth.

[0564] Input: Image data stored in an Amazon S3 bucket

[0565] Output: Feature data such as facial contours and the positions of eyes, nose, and mouth

[0566] Step 4:

[0567] Based on the extracted features, the server converts the faces and dialogue of people in stamps for communication applications, comics, and scenes in video works. Here, generative AI models (GPT-4, DALL-E) are used. The server uses DALL-E to synthesize the user's face into a specific scene and uses GPT-4 to generate the dialogue content.

[0568] Input: Facial feature data

[0569] Output: Synthesized face image and generated dialogue content

[0570] Step 5:

[0571] The server generates a personalized digital gift based on the conversion result. The server combines the generated images and text to create a digital gift exclusive to the user.

[0572] Input: Synthetic face image and generated dialogue

[0573] Output: personalized digital gift (images and text)

[0574] Step 6:

[0575] The server transmits the generated personalized digital gift to the user via electronic communication means. The server uses the user's email address or messenger app ID to send a download link for the digital gift.

[0576] Input: Personalized digital gift (image and / or text)

[0577] Output: Download link for the digital gift sent to the user

[0578] (Application example 1)

[0579] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0580] Conventional advertising systems have had the problem that it is difficult to provide users with personalized advertisements, and the effectiveness of advertisements is limited. In addition, there is a lack of a way for users to easily generate personalized advertisements based on their own photos, so it is not possible to improve the degree of personalization of advertisements.

[0581] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a cartoon or movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, a means for generating the conversion result as an advertising material, and a means for providing the generated advertising material to a user. This makes it possible to generate a personalized advertisement based on the user's photo.

[0582] "Means for receiving photos uploaded by users" refers to a function that enables users to send their own photos to a server via the Internet and receive the photo data.

[0583] "Means for changing the faces and dialogue of people appearing in messenger app stamps, comics, or movie scenes based on the photograph" refers to a function for changing the faces and dialogue of people appearing in messenger app stamps, comics, or movie scenes using features extracted from a photograph uploaded by a user.

[0584] The "means for providing the conversion result as a personalized digital gift" is a function for individually providing digital content including the converted face and dialogue to the user.

[0585] The "means for generating the conversion result as advertising material" is a function for creating advertising material such as advertising banners and video advertisements using the converted faces and dialogue.

[0586] The "means for providing the created advertising material to the user" is a function for providing the created advertising material to the user so that the user can use it.

[0587] A "machine learning algorithm" is an algorithm that enables a computer to learn patterns from data and then use the results of that learning to make predictions and classify new data.

[0588] "Feature extraction" is the process of extracting important information or patterns from an image or data.

[0589] A "personalized digital gift" is a digital gift that is customized based on a user's individual information and preferences.

[0590] "Advertising Materials" refers to content such as images, videos, and text used in an advertising campaign.

[0591] A system for implementing this invention includes means for receiving photos uploaded by a user, means for converting messenger app stamps or the faces and dialogue of people in cartoon or movie scenes based on the photos, means for providing the results of the conversion as a personalized digital gift, means for generating the results of the conversion as advertising material, and means for providing the generated advertising material to a user.

[0592] The server receives photos uploaded by users over the Internet. The received photos are pre-processed using image processing libraries such as OpenCV and PIL. Then, machine learning algorithms (e.g., face recognition model using Keras) are used to extract facial features from the photos. This feature extraction process provides important information about the user's face.

[0593] Based on the extracted features, a generative AI model is used to transform the faces and dialogue of people in messenger app stickers, comics, and movie scenes. The transformed results are then generated as personalized digital gifts. Furthermore, the same transformed results are used to generate advertising materials (e.g., advertising banners and video ads).

[0594] The generated personalized digital gifts and advertising materials are then sent to the user via email or messenger apps, allowing users to easily generate personalized ads based on their own photos and display them in a special way for friends and family.

[0595] As a concrete example, consider a case where a user uploads a photo of themselves and generates a personalized ad by superimposing their face onto an advertising banner for a specific product (e.g. a new smartphone), which can then be shared with the user's friends and family via social media or messaging apps.

[0596] An example of a prompt is as follows:

[0597] "Upload a photo of yourself and we'll generate personalized ads for you that will be shown specially to your friends and family."

[0598] In this way, it becomes possible to generate personalized advertisements based on the user's photographs, thereby improving the effectiveness of the advertisements.

[0599] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0600] Step 1:

[0601] A user uploads a photo. The user sends his / her photo to the server via the Internet. The input is the user's photo data, and the output is the photo data stored on the server.

[0602] Step 2:

[0603] The server receives the photos and performs pre-processing. The server uses image processing libraries such as OpenCV and PIL to convert the photos to the appropriate format and adjust the size. The input is the user's photo data and the output is the pre-processed photo data.

[0604] Step 3:

[0605] The server uses machine learning algorithms to extract facial features from photos. The server uses a Keras facial recognition model to extract facial features from the preprocessed photo data. The input is the preprocessed photo data, and the output is the facial feature data.

[0606] Step 4:

[0607] The server uses a generative AI model to convert the faces and dialogue of people in messenger app stamps, comics, and movie scenes based on facial feature data. The input is facial feature data, and the output is the converted face and dialogue data.

[0608] Step 5:

[0609] The server generates the transformed data as a personalized digital gift. The server uses the transformed facial and / or interaction data to create a digital gift. The input is the transformed facial and / or interaction data, and the output is a personalized digital gift.

[0610] Step 6:

[0611] The server generates the converted data as advertising material. The server uses the converted face and dialogue data to create advertising material such as advertising banners and video ads. The input is the converted face and dialogue data, and the output is advertising material.

[0612] Step 7:

[0613] The server provides the generated personalized digital gift and advertising material to the user. The server sends the generated digital gift and advertising material to the user via email or a messenger app. The input is the personalized digital gift and advertising material, and the output is the digital gift and advertising material sent to the user.

[0614] Example 2

[0615] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0616] Conventional messenger apps and digital gift delivery methods make it difficult for users to easily create individually customized content. In addition, converting friends' faces into stamps and other digital content requires specialized knowledge and skills, making it difficult for general users to use. Furthermore, there is a lack of a way to quickly and easily share the generated content.

[0617] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0618] In this invention, the server includes a means for receiving an image uploaded by a user, a means for extracting facial features based on the image using a machine learning algorithm, a means for generating a facial image for a stamp based on the features using a generative AI model, and a means for providing the generated stamp image to the user. This allows the user to convert the face of a friend into a stamp or other digital content and share it quickly and easily without requiring specialized knowledge or skills.

[0619] "User" means an individual or organization that utilizes the system to upload images and receive generated stamp images.

[0620] "Image" refers to visual data such as photographs or drawings uploaded by users.

[0621] A "server" is a computer system for processing images received from a user and providing a generated stamp image.

[0622] "Machine learning algorithms" are computational techniques used to extract facial features from images, and include face detection algorithms and facial landmark detection algorithms.

[0623] A "face detection algorithm" is a computational method for locating faces within an image.

[0624] A "facial landmark detection algorithm" is a computational method for identifying facial feature points (landmark points).

[0625] A "generative AI model" is an artificial intelligence model that generates facial images for stamps based on extracted facial features.

[0626] A "stamp image" is a facial image created by a generative AI model that can be used in messenger apps, etc.

[0627] The "means for providing" is a method for sending the generated stamp image to the user via email or a messenger app.

[0628] This invention is a system that generates face images for stamps using a machine learning algorithm and a generative AI model based on an image uploaded by a user, and provides the image to the user. A specific embodiment of this system will be described below.

[0629] First, the user uses a terminal (smartphone or PC) to access the system's web interface or a dedicated application. The user clicks the photo upload button and selects the friend's photo file. After selecting, the user presses the upload button to send the photo to the server.

[0630] The server receives the image sent by the user. It temporarily stores the received image data and then applies a face detection algorithm. Specifically, it uses OpenCV's Haar Cascade or Dlib's face detection function to detect faces in the image. Once face detection is complete, the server obtains the coordinate information of the face.

[0631] Next, the server uses Dlib's facial landmark detection feature to extract facial features. By extracting facial landmark points, detailed facial features can be captured. The extracted landmark points are used in the next processing step.

[0632] The server uses a generative AI model (e.g., GAN: Generative Adversarial Network) to generate a face image for the stamp based on the extracted facial features. The GAN model uses the extracted landmark points as input to generate a face image for the stamp. The generated face image is saved in a format that the user can use in messenger apps, etc.

[0633] Finally, the server provides the generated stamp image to the user via email or a messenger app (e.g., LINE or WhatsApp). The user can download the received stamp image and use it in the messenger app.

[0634] As a concrete example, consider the case where a user uploads a photo of a friend and converts the friend's face into a LINE stamp. The user selects the photo in the LINE app on their smartphone and uploads it to the system. The server uses OpenCV to detect faces in the received photo and Dlib to extract facial features. It then uses GAN to convert the friend's face into a stamp and sends the generated stamp to the user as a LINE message. The user can use the received stamp in LINE chats.

[0635] An example of a prompt is, "Upload a photo of a friend and turn their face into a LINE stamp."

[0636] The system allows users, without any specialist knowledge or skills, to convert their friends' faces into stamps and other digital content that can be shared quickly and easily.

[0637] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0638] Step 1:

[0639] A user uploads a photo of a friend.

[0640] Specifically, the user uses a terminal (smartphone or PC) to access the system's web interface or a dedicated application. The user clicks the photo upload button and selects a friend's photo file. After selecting, the user presses the upload button, and the photo is sent to the server.

[0641] Input: A photo file selected by the user

[0642] Output: Photo data sent to the server

[0643] Step 2:

[0644] The server receives the photo and performs face detection.

[0645] Specifically, the server receives the photo sent by the user and temporarily stores it. Then, it detects the face in the photo using OpenCV's Haar Cascade and Dlib's face detection function. Once face detection is complete, the server obtains the face coordinate information.

[0646] Input: Photo data sent by the user

[0647] Output: Face coordinate information

[0648] Step 3:

[0649] The server extracts facial features.

[0650] Specifically, the server uses Dlib's facial landmark detection feature to extract facial landmark points, which allow detailed facial features to be identified, and these points are used in the next processing step.

[0651] Input: Face coordinate information

[0652] Output: Facial landmark points

[0653] Step 4:

[0654] The server generates a face image for the stamp.

[0655] Specifically, the server uses a generative AI model (for example, a Generative Adversarial Network (GAN)) to generate a facial image for the stamp. The GAN model uses the extracted landmark points as input to generate a facial image for the stamp. The generated facial image is saved in a format that the user can use in messenger apps, etc.

[0656] Input: Facial landmark points

[0657] Output: Face image for stamp

[0658] Step 5:

[0659] The server provides the generated stamp image to the user.

[0660] Specifically, the server sends the generated stamp image to the user via email or a messenger app (e.g., LINE or WhatsApp). The user can download the received stamp image and use it in the messenger app.

[0661] Input: Generated stamp image

[0662] Output: Stamp image sent to user

[0663] (Application example 2)

[0664] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0665] In autonomous vehicles, there is a demand for providing passengers with personalized entertainment and information. However, with conventional systems, it is difficult to provide content that reflects the individual characteristics of each passenger, which makes it difficult to increase passenger satisfaction.

[0666] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0667] In this invention, the server includes a means for receiving an image uploaded by a user, a means for generating an avatar for the infotainment system of the autonomous vehicle based on the image, and a means for displaying the generated avatar on a display inside the vehicle, thereby generating a personalized avatar based on the passenger's face, and providing customized entertainment and information to the passenger.

[0668] "User" refers to an individual or organization that uses the System.

[0669] "Upload" refers to the act of a user sending data from their device to a server.

[0670] "Image" refers to visual information stored in digital format.

[0671] An "autonomous vehicle" refers to a vehicle that drives autonomously using artificial intelligence and sensor technology.

[0672] An "infotainment system" refers to an in-car system that integrates information and entertainment.

[0673] An "avatar" refers to a digital character that reflects the characteristics of a user.

[0674] "Generation" refers to the process by which a system creates new data based on input data.

[0675] "Display" refers to a device for displaying visual information.

[0676] A "machine learning algorithm" refers to a computational technique for learning patterns from data and making predictions or classifications.

[0677] "Features" refer to important information extracted from images or data.

[0678] "Entertainment" refers to content that provides enjoyment and entertainment to users.

[0679] "Information" refers to data and knowledge that is useful to the user.

[0680] "Customization" refers to adjustment to the needs and preferences of an individual user.

[0681] A system for implementing the invention includes means for receiving an image uploaded by a user, means for generating an avatar for an infotainment system of an autonomous vehicle based on the image, and means for displaying the generated avatar on an in-vehicle display.

[0682] System Programming

[0683] The system operates as follows.

[0684] 1. User uploads an image:

[0685] Users upload images from their devices to the system, which transmits them to a server and stores them for processing.

[0686] 2. The server processes the image:

[0687] The server uses machine learning algorithms to extract facial features from the received images: it uses OpenCV to detect faces and Keras to extract features using a pre-trained facial feature extraction model.

[0688] 3. Generate an avatar:

[0689] Based on the extracted features, a generative AI model is used to generate an avatar, which is a digital character that reflects the user's facial features.

[0690] 4. Display on the display:

[0691] The generated avatar will be displayed on the autonomous vehicle's infotainment system display, allowing passengers to enjoy a personalized avatar based on their own face.

[0692] Hardware and Software Used

[0693] Hardware:

[0694] Cameras in self-driving vehicles: Used to capture passengers' faces.

[0695] Display: Used to display the generated avatar.

[0696] software:

[0697] OpenCV: Used for face detection.

[0698] Keras: Used to run the face feature extraction model.

[0699] PIL (Python Imaging Library): Used to process the generated avatar as an image.

[0700] Examples

[0701] For example, when a passenger enters the vehicle, a camera will automatically capture their face and the system will generate an avatar based on that image, which will then be displayed on the vehicle's display to provide customized entertainment and information to the passenger.

[0702] Examples of prompt statements

[0703] "Develop a system that takes a picture of a passenger's face and generates a personalized avatar based on that face. The avatar will be used by the infotainment system inside the car."

[0704] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0705] Step 1:

[0706] A user uploads an image.

[0707] A user selects an image from his / her device and uploads it to the system. The input is the image file stored on the user's device, and the output is the image data sent to the server. The server temporarily stores the received image data.

[0708] Step 2:

[0709] The server processes the images.

[0710] The server performs face detection on the received image data using OpenCV. The input is the uploaded image data, and the output is the area information of the detected face. Specifically, the image is converted to grayscale and a face detection algorithm is applied to identify the position of the face.

[0711] Step 3:

[0712] The server extracts facial features.

[0713] The server extracts features from the face detection results using a face feature extraction model pre-trained with Keras. The input is face region information, and the output is a face feature vector. Specifically, the detected face region is resized and input to the model to obtain a feature vector.

[0714] Step 4:

[0715] The server generates the avatar.

[0716] The server generates an avatar using a generative AI model based on the extracted facial feature vector. The input is the facial feature vector, and the output is the generated avatar image. Specifically, the feature vector is input into the generative AI model to generate the avatar image.

[0717] Step 5:

[0718] The server displays the avatar on the display.

[0719] The server transmits the generated avatar image to the display of the infotainment system of the autonomous vehicle. The input is the generated avatar image, and the output is the avatar displayed on the display. Specifically, the server converts the avatar image into a format suitable for the display and transmits a signal to display it on the display in the car.

[0720] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0721] "Example 1"

[0722] In one embodiment of the present invention, a system is provided that includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp, a cartoon or a movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, and an emotion engine for recognizing the emotion of the user. The emotion engine recognizes the emotion from the user's facial expression, tone of voice, text input, etc., and adjusts the generation of the personalized digital gift based on the emotion. For example, if it is recognized that the user is happy, the system generates a personalized digital gift that expresses joy.

[0723] "Example 2"

[0724] In another embodiment of the present invention, a user uploads a photo of a friend to the system, and the system uses a machine learning algorithm to extract the friend's facial features. The system then converts the person's face on the stamp into the friend's face based on the extracted features. Furthermore, the system recognizes the user's emotion with an emotion engine and adjusts the generated personalized digital gift based on the emotion. For example, if the user is recognized as sad, the system generates a personalized digital gift with a comforting message.

[0725] The process flow of each embodiment will be described below.

[0726] "Example 1"

[0727] Step 1: A user uploads a photo to the system.

[0728] Step 2: The system receives the uploaded photo and extracts the person's facial features from the photo.

[0729] Step 3: Based on the extracted features, the system converts the faces and dialogue of people in messenger app stickers, comic book and movie scenes.

[0730] Step 4: The system uses the emotion engine to recognize the user's emotion.

[0731] Step 5: The system tailors the generated personalized digital gift based on the recognized emotion and provides it to the user.

[0732] "Example 2"

[0733] Step 1: A user uploads a photo of their friend into the system.

[0734] Step 2: The system receives the uploaded photo and uses machine learning algorithms to extract your friend's facial features.

[0735] Step 3: The system converts the face of the person in the stamp into the face of a friend based on the extracted features. Step 4: The system uses an emotion engine to recognize the user's emotion.

[0736] Step 5: The system tailors the generated personalized digital gift based on the recognized emotion and provides it to the user.

[0737] Example 1

[0738] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0739] Conventional digital gift generation systems provide uniform content without considering the user's emotions, making it difficult to provide a personalized experience. In addition, conversion processing based on images uploaded by users was limited, making it difficult to convert into various formats such as messenger app stamps, comics, and video scenes.

[0740] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a face or dialogue of a person in a communication application stamp, a cartoon or a video scene based on the image, a means for providing the conversion result as a personalized digital gift, and an emotion engine for recognizing the user's emotion. This makes it possible to generate a personalized digital gift based on the user's emotion.

[0741] "User" refers to any person who utilizes the System to upload an image and receive a personalized digital gift.

[0742] "Image" refers to visual data such as photographs or drawings uploaded by users.

[0743] "Communication application" refers to software that enables users to send and receive messages, such as a messenger application or chat application.

[0744] "Stamps" refer to small images or icons used within communication applications.

[0745] "Manga" refers to a series of pictures or illustrations that visually depict a story.

[0746] A "film scene" refers to a portion of a visual medium such as a film or video.

[0747] "Conversion" refers to the process of changing original images or data into a different format or content.

[0748] "Personalized digital gifts" refers to individually customized digital content generated based on a user's image and emotions.

[0749] An "emotion engine" refers to software or algorithms that recognize emotions from a user's facial expressions, tone of voice, text input, etc.

[0750] A "machine learning algorithm" is a mathematical model that learns patterns from data and makes predictions or classifications.

[0751] The present invention is a system for generating and providing a personalized digital gift based on an image uploaded by a user. The system includes a means for a user to upload an image, a means for analyzing and transforming the image, a means for providing the transformation result as a personalized digital gift, and an emotion engine for recognizing the user's emotion.

[0752] Hardware and Software Used

[0753] Hardware

[0754] Server: Receives, analyzes, transforms images, recognizes emotions, and generates and delivers digital gifts.

[0755] Device: The device (computer, smartphone, tablet, etc.) through which a user uploads images.

[0756] software

[0757] Website or application: Provides an interface for users to upload images.

[0758] Machine learning algorithms used to extract features from images and perform transformations (e.g. OpenCV, Dlib, TensorFlow, PyTorch).

[0759] Emotion engine: Software for recognizing user emotions (e.g. Emotion API).

[0760] Generative AI models: Models for image transformation and dialogue generation (e.g., StyleGAN, GPT-3).

[0761] Mail Server and Messenger API: Used to send the generated digital gifts to users.

[0762] Examples

[0763] A user accesses a website and uploads an image of their own smiling face. The server receives the image and uses a machine learning algorithm to extract facial features. Next, the emotion engine recognizes the user's smile as "happiness." Based on this information, the server generates a stamp for a communication application that uses the user's face to express happiness. Finally, the generated stamp is sent to the user's email address.

[0764] Examples of prompt statements

[0765] "Generate stickers for a communication application that express joy based on images uploaded by users. The user's emotions are recognized from their smiles."

[0766] In this way, it is possible to generate and offer personalized digital gifts based on the user's emotions.

[0767] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0768] Step 1: User uploads an image.

[0769] A user uploads his / her own image through a website or application by clicking an upload button and selecting an image file from a file selection dialog. The input is the user's image file, and the output is the image data sent to the server.

[0770] Step 2: The server receives the image.

[0771] The server receives the image sent by the user and saves it in the database. Specifically, it receives the HTTP request, saves the image file in a temporary directory, and then saves it in the database. The input is the image data sent by the user, and the output is the path of the saved image file.

[0772] Step 3: The server analyzes the image.

[0773] The server uses machine learning algorithms to extract facial features from images. Specifically, it uses OpenCV and Dlib to perform face detection and identify the coordinates of facial landmarks (eyes, nose, mouth, etc.). The input is the path to the saved image file, and the output is the facial feature data.

[0774] Step 4: The server recognizes the emotion.

[0775] The server uses the emotion engine to recognize the user's emotion from the image. Specifically, it sends facial feature data to the Emotion API and classifies the emotion. The input is the facial feature data, and the output is the recognized emotion data.

[0776] Step 5: The server performs the conversion process.

[0777] The server performs conversion processing using a generative AI model based on the extracted facial features and recognized emotions. Specifically, it uses StyleGAN to convert the user's face into a stamp for a communication application, and uses GPT-3 to generate dialogue that matches the emotion. The input is facial feature data and emotion data, and the output is the converted stamp and dialogue data.

[0778] Step 6: The server generates the personalized digital gift.

[0779] The server generates a personalized digital gift based on the converted results. Specifically, it creates a digital card that includes a stamp with the user's face and a message that matches the user's emotion. The input is the converted stamp and conversation data, and the output is the generated digital gift.

[0780] Step 7: The server sends the digital gift.

[0781] The server sends the generated personalized digital gift to the user via email or a communication application. Specifically, the server uses the SMTP protocol or a messenger API to send the digital gift. The input is the generated digital gift and the output is the digital gift sent to the user.

[0782] (Application example 1)

[0783] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0784] In the conventional personalized digital gift generation system, since the conversion is performed uniformly without considering the user's emotions, it is not possible to provide an appropriate digital gift according to the user's emotions. In addition, if the generated digital gift does not match the user's emotions, there is a problem that the user's satisfaction decreases. Furthermore, since there is a lack of means to effectively deliver the generated digital gift, it is difficult to deliver it to the user quickly and reliably.

[0785] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0786] In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a cartoon or movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, a means including an emotion engine for recognizing the emotion of the user, a means for adjusting the generation of the personalized digital gift based on the emotion, and a means for sending the generated personalized digital gift to the user via email or a messenger app. This makes it possible to generate and quickly deliver a personalized digital gift appropriate to the emotion of the user.

[0787] "Means for receiving photos uploaded by users" refers to a function that enables users to send their own photos to the system via the Internet and for the server to receive those photos.

[0788] "Messenger app stamps" are images or icons used within messenger applications to allow users to express emotions or messages.

[0789] "Means of transforming the faces and dialogue of characters in manga or movie scenes" refers to technology that replaces the faces and dialogue of characters appearing in manga or movie scenes with the user's face and dialogue based on photos uploaded by the user.

[0790] A "personalized digital gift" is a digital gift that is customized based on a user's individual characteristics and emotions.

[0791] An "emotion engine" is software that recognizes emotions from a user's facial expressions, tone of voice, text input, etc., and adjusts the system's operation based on those emotions.

[0792] A "machine learning algorithm" is a computational technique for learning patterns based on large amounts of data and making predictions and classifications.

[0793] "Means of sending via email or messenger app" refers to a function for sending the generated personalized digital gift to a user via email or messenger application.

[0794] An embodiment of the present invention will be described. First, a user uses a terminal such as a smartphone or a personal computer to access a dedicated application or website. The user sends a photo through an interface for uploading the user's own photo. The photo is received by a server.

[0795] The server uses a machine learning algorithm to extract features from the received photos. The algorithm uses image processing libraries such as OpenCV to detect facial feature points and analyze the shape and expression of the face. Then, based on the extracted features, it converts the faces and dialogue of people in messenger app stamps, comics, and movie scenes. For the conversion, it uses technologies such as FaceSwap and DeepFake.

[0796] In addition, the server recognizes the user's emotions using an emotion engine. The emotion engine uses software such as EmotionRecognizer to analyze emotions from the user's facial expressions, tone of voice, text input, etc. Based on the recognized emotions, the server adjusts the generation of personalized digital gifts. For example, if the user is happy, the server generates a digital gift that expresses joy.

[0797] The generated personalized digital gift is then sent to the user from the server via email or a messenger app, using a communication library such as MessagingService.

[0798] For example, if a user uploads a photo of themselves feeling happy, the system will recognize that emotion and edit the character in the movie scene to make them look happy. The generated video will be sent to the user's email address.

[0799] An example of a prompt sentence might be:

[0800] "Create an application that converts a character's face in a movie scene into the user's face based on a photo uploaded by the user, and edits the scene based on the user's emotions. Please also include a function to send the generated video to the user."

[0801] In this way, personalized digital gifts appropriate to the user's emotions can be generated and quickly delivered.

[0802] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0803] Step 1:

[0804] The user uses the device to access a dedicated application or website and uploads their own photo.

[0805] Input: User photo

[0806] Output: The user's photo sent to the server.

[0807] How it works: A user selects a photo through the application interface and clicks the upload button, and the photo is sent to the server over the Internet.

[0808] Step 2:

[0809] Based on the photos received by the server, a machine learning algorithm is used to extract features from the photos.

[0810] Input: The user's photo sent to the server

[0811] Output: Extracted facial feature data

[0812] Specific operation: The server uses image processing libraries such as OpenCV to detect facial feature points from photos and analyze facial shape and facial expressions.

[0813] Step 3:

[0814] Based on the extracted features, the server converts the faces and dialogue of people in messenger app stamps, manga, and movie scenes.

[0815] Input: Extracted facial feature data

[0816] Output: Converted character faces and dialogue

[0817] How it works: The server uses techniques such as FaceSwap and DeepFake to swap the user's face with that of a character in a movie scene and adapts the dialogue to match the user's characteristics.

[0818] Step 4:

[0819] The server recognizes the user's emotions using an emotion engine.

[0820] Input: User photo

[0821] Output: Recognized emotion data

[0822] How it works: The server uses software such as EmotionRecognizer to analyze emotions from the user's facial expressions, tone of voice, text input, etc.

[0823] Step 5:

[0824] The server coordinates the generation of the personalized digital gift based on the recognized emotion.

[0825] Input: Recognized emotion data, transformed character faces and dialogue

[0826] Output: Personalized digital gifts tailored based on emotions

[0827] Specific operation: The server generates digital gifts that reflect emotions such as joy or sadness according to the recognized emotions.

[0828] Step 6:

[0829] The server then sends the generated personalized digital gift to the user via email or a messenger app.

[0830] Input: Personalized digital gifts tailored based on emotions

[0831] Output: A personalized digital gift sent to the user.

[0832] Specific operation: The server uses a communication library such as MessagingService to send the generated digital gift to the user's email address or messenger app.

[0833] Example 2

[0834] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0835] In conventional systems, it was difficult to provide personalized digital gifts that took into account the user's emotions when converting stamps in messenger apps and faces and dialogues in visual media based on photos uploaded by the user. In addition, there was a lack of means to appropriately send the generated digital gifts to the user. This limited the user experience and made it difficult to provide services that met individual needs.

[0836] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0837] In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or visual media based on the image, a means for providing the conversion result as a personalized digital gift, a means for recognizing a user's emotion and adjusting the personalized digital gift based on the emotion, and a means for sending the personalized digital gift to the user, thereby enabling the generation and sending of a personalized digital gift according to the user's emotion.

[0838] "User" means any individual or entity that utilizes the System to upload images and receive personalized digital gifts.

[0839] An "image" is data containing visual information, such as a photograph or drawing, that a user uploads to the system.

[0840] A "messenger app" is a software application that enables users to exchange messages, including text, images, and audio, with other users.

[0841] "Stamps" are small images used within messenger apps that express specific facial expressions or characters.

[0842] "Visual media" refers to media content that includes visual elements, such as manga and movies.

[0843] A "machine learning algorithm" is a computational technique for learning patterns from data and making predictions or classifications.

[0844] A "feature" is an identifiable element in an image, such as a facial contour, eyes, nose, or mouth.

[0845] A "personalized digital gift" is a digital gift that is individually generated based on a user's input and emotions.

[0846] An "emotion engine" is software or algorithms for analyzing and recognizing a user's emotions.

[0847] An "electronic means of communication" is any means for sending or receiving information in digital form, such as email or a messaging app.

[0848] This invention is a system that converts the faces and dialogues of people in stamps and visual media of messenger apps based on images uploaded by users and provides them as personalized digital gifts. A specific embodiment of this system will be described below.

[0849] System Configuration

[0850] Hardware

[0851] Server: A computer system with powerful processors and large memory capacity that runs image processing and machine learning algorithms.

[0852] Terminal: A smartphone or PC used by a user to upload images and receive generated digital gifts.

[0853] software

[0854] Image processing library: Software for pre-processing and transforming images, such as OpenCV or Adobe Photoshop API.

[0855] Machine learning algorithms: Algorithms for extracting features from images and performing transformations, such as TensorFlow.

[0856] Emotion engine: Software for recognizing user emotions, such as IBM Watson or Microsoft Azure Emotion API.

[0857] Electronic communication means: Means for sending the generated digital gift to the user, such as SMTP protocol or messenger API (e.g. Facebook Messenger API).

[0858] System Operation

[0859] Uploading an image

[0860] Users use their devices to upload pictures of their friends to the system, either through the system's web interface or a dedicated app.

[0861] Image preprocessing and feature extraction

[0862] The server receives the uploaded image, first pre-processes the image using OpenCV, then uses TensorFlow's face recognition model to extract the facial features of the friend. In this feature extraction process, the main feature points such as the facial contour, eyes, nose, and mouth are identified.

[0863] Stamp Generation

[0864] The server uses Adobe Photoshop API to generate stamps based on the extracted facial features, adjusting the facial contours and facial expressions to fit the stamp design.

[0865] Emotion recognition and digital gift adjustment

[0866] The server uses IBM Watson to analyze the user's emotions, analyzing facial expressions and text inputs while the user is using the system. For example, if the server detects that the user is sad, it will generate a personalized digital gift with a comforting message.

[0867] Sending digital gifts

[0868] The server sends the generated personalized digital gift to the user via email or messenger app using SMTP protocol or messenger API. The user can view the received digital gift and share it with friends.

[0869] Examples

[0870] As a concrete example, consider a case where a user uploads a photo of a friend and converts the friend's face into a sticker in a messenger app.

[0871] 1. The user opens the dedicated app on their smartphone and selects and uploads photos of their friends.

[0872] 2. The server receives the uploaded photo, pre-processes the image using OpenCV, and then uses TensorFlow's face recognition model to extract the friend's facial features.

[0873] 3. The server uses Adobe Photoshop API to generate stamps based on the extracted facial features, adjusting the facial contours and facial expressions to fit the stamp design.

[0874] 4. The server uses IBM Watson to analyze the user's emotions and, if the user is recognized as sad, generates a personalized digital gift with a comforting message.

[0875] 5. The server sends the generated personalized digital gift to the user through the Facebook Messenger API. The user can view the received digital gift and share it with their friends.

[0876] Examples of prompt statements

[0877] Upload a photo of your friend and turn their face into a sticker. Generate and send personalized digital gifts based on your user's emotions.

[0878] By inputting this prompt sentence into the generative AI model, the system will carry out the above process and provide the user with an original stamp.

[0879] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0880] Step 1:

[0881] A user uses a terminal to upload a friend's image to the system. The user opens the system's web interface or a dedicated app, selects an image file, and presses the upload button. The input is the image file selected by the user, and the output is the image data sent to the server.

[0882] Step 2:

[0883] The server receives the uploaded image. The received image data is first preprocessed using an image processing library (e.g., OpenCV). Preprocessing includes image resizing, noise removal, color correction, etc. The input is the image data sent by the user, and the output is the preprocessed image data.

[0884] Step 3:

[0885] The server inputs the preprocessed image data into a machine learning algorithm (e.g., TensorFlow's facial recognition model) to extract facial features. The feature extraction process identifies key features such as the facial contour, eyes, nose, and mouth. The input is the preprocessed image data, and the output is the extracted facial feature data.

[0886] Step 4:

[0887] The server generates a stamp using an image processing library (e.g., Adobe Photoshop API) based on the extracted facial feature data. Specifically, it adjusts the facial contours and facial expressions to fit the stamp design. The input is the facial feature data, and the output is the generated stamp image.

[0888] Step 5:

[0889] The server recognizes the user's emotions using an emotion engine (e.g., IBM Watson). It analyzes emotions from facial expressions and text inputs while the user is using the system. The input is the user's facial expression data and text data, and the output is the recognized emotion data.

[0890] Step 6:

[0891] The server tailors the personalized digital gift including the generated stamp based on the recognized emotion data, e.g., if the user is recognized as sad, it generates a digital gift including a comforting message, where the input is the emotion data and the stamp image, and the output is the tailored personalized digital gift.

[0892] Step 7:

[0893] The server sends the generated personalized digital gift to the user via email or a messenger app using the SMTP protocol or a messenger API (e.g., Facebook Messenger API). The input is the personalized digital gift and the output is the digital gift sent to the user.

[0894] (Application example 2)

[0895] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0896] In conventional messenger apps and video content, it was difficult for users to generate personalized characters and stamps based on their friends' photos. In addition, there was a lack of functionality to adjust content according to the user's emotions, so there was a need to improve the user experience. Furthermore, there were limited means to effectively provide the generated content.

[0897] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0898] In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting the face and dialogue of a character of a messenger app stamp or video content based on the photo, a means for providing the conversion result as a personalized digital gift, and a means for recognizing the emotion of the user and adjusting the personalized digital gift based on the emotion. This allows the user to easily generate a personalized character or stamp based on a friend's photo, and further allows the content to be adjusted according to the user's emotion. In addition, the generated content can be effectively provided.

[0899] "User" means an individual who utilizes the System to upload photos and receive personalized digital gifts.

[0900] A "photo" is image data uploaded by a user, which includes features such as a friend's face.

[0901] "Messenger app stamps" are digital stamps used within messenger apps and are generated based on photos uploaded by users.

[0902] "Video content characters" are people or characters that appear in videos, whose faces and dialogue are transformed based on photos uploaded by users.

[0903] "Dialogue" refers to conversations or statements made by characters in video content or messenger apps.

[0904] "Personalized Digital Gifts" are individualized digital content that is generated based on photos and emotions uploaded by users.

[0905] A "machine learning algorithm" is an algorithm for extracting features from photographs and transforming people's faces and dialogue based on those features.

[0906] "Means for recognizing emotions" refers to technology for analyzing a user's emotions and adjusting content based on those emotions.

[0907] The "means of provision" refers to the means of sending the generated personalized digital gift to the user via email or a messenger app.

[0908] The system for implementing the present invention receives a photo uploaded by a user, converts the face and dialogue of a character in a messenger app stamp or video content based on the photo, and provides a generated personalized digital gift. It also includes a function of recognizing the user's emotions and adjusting the personalized digital gift based on the emotions.

[0909] Hardware and Software Used

[0910] Hardware: Smartphone, head-mounted display

[0911] Software: OpenCV, Dlib, EmotionRecognizer (emotion recognition library), VideoEditor (video editing library)

[0912] System Configuration

[0913] 1. User terminal: A device that users use to upload photos. This includes smartphones and head-mounted displays.

[0914] 2. Server: Receives the uploaded photos, uses machine learning algorithms to extract facial features, and processes them to transform the faces and dialogue of characters in the video content.

[0915] 3. Emotion Recognition Engine: Analyzes user emotions and tailors personalized digital gifts based on those emotions.

[0916] 4. Delivery method: The generated personalized digital gift is sent to the user via email or messenger app.

[0917] Data processing and calculation

[0918] 1. Facial feature extraction: The server extracts facial features from the uploaded photo using OpenCV and Dlib, which obtains the facial landmark points in the photo.

[0919] 2. Emotion Recognition: The server recognizes the user's emotion using EmotionRecognizer, which analyzes the user's emotional state when he uploads a photo.

[0920] 3. Video Editing: The server uses VideoEditor to edit the video content based on the extracted facial features and the recognized emotions, so that the faces and dialogues of the characters in the video are transformed into the faces of the user's friends and the content is adjusted according to their emotions.

[0921] Examples

[0922] Consider a case where a user uploads a photo of a friend and converts the friend's face into a character in a video content. For example, if the user wants to "make the friend's face into an animated character and create a touching scene," the system converts the friend's face into an animated character and generates a video including the touching scene.

[0923] Examples of prompt statements

[0924] Upload a photo of your friend, transform their face into a cartoon character, and generate a video with some touching moments.

[0925] In this way, users can enjoy personalized video content.

[0926] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0927] Step 1:

[0928] A user uploads a photo of a friend.

[0929] Input: A friend's photo file selected by the user.

[0930] Output: The photo data uploaded to the server.

[0931] Specific operation: A user selects a photo of a friend using a smartphone or a head-mounted display and uploads it to the system. The server receives the photo data.

[0932] Step 2:

[0933] The server extracts facial features from the photo.

[0934] Input: The uploaded photo data.

[0935] Output: Extracted facial feature data (landmark points).

[0936] What it does: The server uses OpenCV and Dlib to detect faces in photos and extract facial landmark points, which gives us facial feature data.

[0937] Step 3:

[0938] The server recognizes the user's emotions.

[0939] Input: The uploaded photo data.

[0940] Output: The recognized emotion data.

[0941] What it does: The server uses EmotionRecognizer to analyze the photo and recognize the user's emotion, which analyzes the user's emotional state when uploading the photo.

[0942] Step 4:

[0943] The server edits the video content based on facial features and emotions.

[0944] Input: extracted facial feature data, recognized emotion data, original video content.

[0945] Output: The edited video content.

[0946] How it works: The server uses VideoEditor to convert the character's face in the video into the friend's face based on the extracted facial feature data, and adjusts the video content based on the recognized emotion data, thereby generating personalized video content.

[0947] Step 5:

[0948] The server provides the generated personalized digital gift.

[0949] Input: The edited video content.

[0950] Output: A personalized digital gift provided to the user.

[0951] How it works: The server sends the generated video content to the user via email or messenger app, so that the user can receive personalized video content.

[0952] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0953] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0954] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0955] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0956] [Third embodiment]

[0957] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0958] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0959] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0960] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0961] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0962] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0963] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0964] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0965] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0966] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0967] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0968] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0969] "Example 1"

[0970] The system of the present invention includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a scene in a cartoon or movie based on the photo, and a means for providing the conversion result as a personalized digital gift. Specifically, a user uploads his / her own photo through a website or application. Next, the system uses a machine learning algorithm to extract features from the photo and converts the face or dialogue of the person based on the features. Finally, the generated personalized digital gift is sent to the user via email or a messenger app.

[0971] "Example 2"

[0972] As a specific example, consider a case where a user uploads a photo of a friend and converts the friend's face into a stamp for a messenger app. First, the user uploads the friend's photo to the system. Next, the system uses a machine learning algorithm to extract the friend's facial features and converts the face of the person in the stamp into the friend's face based on the features. Finally, the system sends the generated personalized digital gift to the user via email or a messenger app. In this way, the user can get an original stamp with the friend's face drawn on it.

[0973] The process flow of each embodiment will be described below.

[0974] "Example 1"

[0975] Step 1: The user uploads a photo of themselves via a website or application.

[0976] Step 2: The system uses machine learning algorithms to extract features from the photo.

[0977] Step 3: The system transforms the person's face and dialogue based on the extracted features.

[0978] Step 4: The generated personalized digital gift is sent to the user via email or messenger app.

[0979] "Example 2"

[0980] Step 1: A user uploads a photo of their friend into the system.

[0981] Step 2: The system uses machine learning algorithms to extract your friend's facial features.

[0982] Step 3: The system converts the face of the person on the stamp into the face of a friend based on the extracted features. Step 4: The system sends the generated personalized digital gift to the user via email or messenger app.

[0983] Example 1

[0984] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0985] In the conventional digital gift generation system, when generating personalized digital gifts based on photos uploaded by users, the conversion of faces and dialogues is done manually, which is time-consuming and labor-intensive. In addition, there are also issues that the quality of the generated digital gifts is inconsistent and may not meet the user's expectations. Furthermore, there is a lack of a means to quickly and reliably provide the generated digital gifts to the users.

[0986] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0987] In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a stamp of a communication application or a face or dialogue of a person in a scene of a cartoon or a video work based on the image, a means for providing the conversion result as a personalized digital gift, a means for using a machine learning algorithm to extract features from the image, a means for using a generative AI model to generate a face or dialogue of a person based on the features, and a means for transmitting the generated personalized digital gift to the user via an electronic communication means. This makes it possible to quickly and efficiently generate a high-quality, consistent personalized digital gift based on an image uploaded by a user and provide it to the user.

[0988] "User" means an individual or entity that utilizes the System to upload an image and receive a personalized digital gift.

[0989] "Images" are visual data such as photographs and illustrations that users upload to the system.

[0990] A "communication application" is software for sending and receiving messages, and has the function of sending stamps and images.

[0991] A "stamp" is a small image or icon used within a communication application to express an emotion or message.

[0992] "Manga" is a series of pictures or illustrations intended to visually represent a story.

[0993] A "visual work" is any medium that contains a visually presented story or information, such as a film or television program.

[0994] A "machine learning algorithm" is a computational technique for learning patterns and features from data and making predictions and classifications.

[0995] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate new data.

[0996] A "personalized digital gift" is a digital gift that is customized for a particular user based on the user's image and information.

[0997] "Electronic communication means" refers to means for sending and receiving data via the Internet, email, messenger apps, etc.

[0998] The present invention is a system for generating and providing personalized digital gifts based on images uploaded by users. A specific embodiment of the system will be described below.

[0999] System Configuration

[1000] Hardware and Software

[1001] The server uses the following hardware and software:

[1002] Hardware: High-performance server machine (e.g. cloud server with CPU, GPU, and memory)

[1003] Software: Machine learning algorithms (e.g. TensorFlow), generative AI models (e.g. GPT-4, DALL-E), databases (e.g. Amazon S3), communications (e.g. SMTP servers, messenger APIs)

[1004] Processing flow

[1005] 1. Upload your photo

[1006] A user accesses a website or application using a smartphone or PC. For example, the user opens an application on the smartphone, takes a photo of his / her face using the camera function, and presses the upload button. This sends the photo data to the application.

[1007] 2. Receiving photos

[1008] The server receives the images uploaded by the user. Specifically, the server uses an Amazon S3 bucket to store the image data. The server receives the HTTP request, analyzes the image data, and stores it.

[1009] 3. Feature Extraction

[1010] The server uses machine learning algorithms to extract features from the images. It uses TensorFlow for this process. The server feeds the image data into a TensorFlow model to identify features such as the contours of the face and the location of the eyes, nose, and mouth. For example, it uses a facial recognition model to obtain the coordinates of each part of the face.

[1011] 4. Face and dialogue transformation

[1012] Based on the extracted features, the server converts the faces and dialogue of people in stamps for communication applications, comics, and scenes in video works. Here, generative AI models (GPT-4, DALL-E) are used. The server uses DALL-E to synthesize the user's face into a specific scene and uses GPT-4 to generate the dialogue content. For example, the server synthesizes the user's face into a scene from the famous science fiction movie "XXX" and changes the dialogue in that scene to the user's name or a specific phrase.

[1013] 5. Create personalized digital gifts

[1014] The server generates a personalized digital gift based on the conversion result. The server combines the generated images and text to create a digital gift for the user. For example, it creates a digital stamp with the user's face or a comic strip featuring the user.

[1015] 6. Sending Digital Gifts

[1016] The server transmits the generated personalized digital gift to the user via electronic communication means. The server transmits a download link for the digital gift using the user's email address or messenger app ID. For example, the server transmits an email containing a download link for the digital gift to the user's email address.

[1017] Examples and prompts

[1018] Examples

[1019] Users upload a photo of their face through the app and receive a digital gift that features their face superimposed onto a scene from the famous sci-fi movie "XXX" based on the photo.

[1020] Examples of prompt statements

[1021] "Based on the face photo uploaded by the user, please composite the face into a scene from the famous science fiction movie "XXX" and generate a digital gift that includes a specific dialogue."

[1022] In this way, users can easily create and enjoy their own special digital gifts.

[1023] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1024] Step 1:

[1025] A user accesses a website or application using a smartphone or PC. The user takes a photo of his / her face using the camera function and presses the upload button. This sends the photo data to the application.

[1026] Input: A photo of the user's face

[1027] Output: The photo data sent to the application.

[1028] Step 2:

[1029] The server receives the images uploaded by the user: the server receives the HTTP request, parses the image data, and stores it in an Amazon S3 bucket.

[1030] Input: Photo data sent from the application

[1031] Output: Image data stored in an Amazon S3 bucket

[1032] Step 3:

[1033] The server uses machine learning algorithms to extract features from the images, using TensorFlow. The server feeds the image data into a TensorFlow model to identify features such as the facial contours and the location of the eyes, nose, and mouth.

[1034] Input: Image data stored in an Amazon S3 bucket

[1035] Output: Feature data such as facial contours and the positions of eyes, nose, and mouth

[1036] Step 4:

[1037] Based on the extracted features, the server converts the faces and dialogue of people in stamps for communication applications, comics, and scenes in video works. Here, generative AI models (GPT-4, DALL-E) are used. The server uses DALL-E to synthesize the user's face into a specific scene and uses GPT-4 to generate the dialogue content.

[1038] Input: Facial feature data

[1039] Output: Synthesized face image and generated dialogue content

[1040] Step 5:

[1041] The server generates a personalized digital gift based on the conversion result. The server combines the generated images and text to create a digital gift exclusive to the user.

[1042] Input: Synthetic face image and generated dialogue

[1043] Output: personalized digital gift (images and text)

[1044] Step 6:

[1045] The server transmits the generated personalized digital gift to the user via electronic communication means. The server uses the user's email address or messenger app ID to send a download link for the digital gift.

[1046] Input: Personalized digital gift (image and / or text)

[1047] Output: Download link for the digital gift sent to the user

[1048] (Application example 1)

[1049] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1050] Conventional advertising systems have the problem that it is difficult to provide users with personalized advertisements, and the effectiveness of advertisements is limited. In addition, there is a lack of a way for users to easily generate personalized advertisements based on their own photos, so the degree of personalization of advertisements cannot be improved.

[1051] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a cartoon or movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, a means for generating the conversion result as an advertising material, and a means for providing the generated advertising material to a user. This makes it possible to generate a personalized advertisement based on the user's photo.

[1052] "Means for receiving photos uploaded by users" refers to a function that enables users to send their own photos to a server via the Internet and receive the photo data.

[1053] "Means for changing the faces and dialogue of people appearing in messenger app stamps, comics, or movie scenes based on the photograph" refers to a function for changing the faces and dialogue of people appearing in messenger app stamps, comics, or movie scenes using features extracted from a photograph uploaded by a user.

[1054] The "means for providing the conversion result as a personalized digital gift" is a function for individually providing digital content including the converted face and dialogue to the user.

[1055] The "means for generating the conversion result as advertising material" is a function for creating advertising material such as advertising banners and video advertisements using the converted faces and dialogue.

[1056] The "means for providing the created advertising material to the user" is a function for providing the created advertising material to the user so that the user can use it.

[1057] A "machine learning algorithm" is an algorithm that enables a computer to learn patterns from data and then use the results of that learning to make predictions and classify new data.

[1058] "Feature extraction" is the process of extracting important information or patterns from an image or data.

[1059] A "personalized digital gift" is a digital gift that is customized based on a user's individual information and preferences.

[1060] "Advertising Materials" refers to content such as images, videos, and text used in an advertising campaign.

[1061] A system for implementing this invention includes means for receiving photos uploaded by a user, means for converting messenger app stamps or the faces and dialogue of people in cartoon or movie scenes based on the photos, means for providing the results of the conversion as a personalized digital gift, means for generating the results of the conversion as advertising material, and means for providing the generated advertising material to a user.

[1062] The server receives photos uploaded by users over the Internet. The received photos are pre-processed using image processing libraries such as OpenCV and PIL. Then, machine learning algorithms (e.g., face recognition model using Keras) are used to extract facial features from the photos. This feature extraction process provides important information about the user's face.

[1063] Based on the extracted features, a generative AI model is used to transform the faces and dialogue of people in messenger app stickers, comics, and movie scenes. The transformed results are then generated as personalized digital gifts. Furthermore, the same transformed results are used to generate advertising materials (e.g., advertising banners and video ads).

[1064] The generated personalized digital gifts and advertising materials are then sent to the user via email or messenger apps, allowing users to easily generate personalized ads based on their own photos and display them in a special way for friends and family.

[1065] As a concrete example, consider a case where a user uploads a photo of themselves and generates a personalized ad by superimposing their face onto an advertising banner for a specific product (e.g. a new smartphone), which can then be shared with the user's friends and family via social media or messaging apps.

[1066] An example of a prompt is as follows:

[1067] "Upload a photo of yourself and we'll generate personalized ads for you that will be shown specially to your friends and family."

[1068] In this way, it becomes possible to generate personalized advertisements based on the user's photographs, thereby improving the effectiveness of the advertisements.

[1069] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1070] Step 1:

[1071] A user uploads a photo. The user sends his / her photo to the server via the Internet. The input is the user's photo data, and the output is the photo data stored on the server.

[1072] Step 2:

[1073] The server receives the photos and performs pre-processing. The server uses image processing libraries such as OpenCV and PIL to convert the photos to the appropriate format and adjust the size. The input is the user's photo data and the output is the pre-processed photo data.

[1074] Step 3:

[1075] The server uses machine learning algorithms to extract facial features from photos. The server uses a Keras facial recognition model to extract facial features from the preprocessed photo data. The input is the preprocessed photo data, and the output is the facial feature data.

[1076] Step 4:

[1077] The server uses a generative AI model to convert the faces and dialogue of people in messenger app stamps, comics, and movie scenes based on facial feature data. The input is facial feature data, and the output is the converted face and dialogue data.

[1078] Step 5:

[1079] The server generates the transformed data as a personalized digital gift. The server uses the transformed facial and / or interaction data to create a digital gift. The input is the transformed facial and / or interaction data, and the output is a personalized digital gift.

[1080] Step 6:

[1081] The server generates the converted data as advertising material. The server uses the converted face and dialogue data to create advertising material such as advertising banners and video ads. The input is the converted face and dialogue data, and the output is advertising material.

[1082] Step 7:

[1083] The server provides the generated personalized digital gift and advertising material to the user. The server sends the generated digital gift and advertising material to the user via email or a messenger app. The input is the personalized digital gift and advertising material, and the output is the digital gift and advertising material sent to the user.

[1084] Example 2

[1085] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1086] Conventional messenger apps and digital gift delivery methods make it difficult for users to easily create individually customized content. In addition, converting friends' faces into stamps and other digital content requires specialized knowledge and skills, making it difficult for general users to use. Furthermore, there is a lack of a way to quickly and easily share the generated content.

[1087] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1088] In this invention, the server includes a means for receiving an image uploaded by a user, a means for extracting facial features based on the image using a machine learning algorithm, a means for generating a facial image for a stamp based on the features using a generative AI model, and a means for providing the generated stamp image to the user. This allows the user to convert the face of a friend into a stamp or other digital content and share it quickly and easily without requiring specialized knowledge or skills.

[1089] "User" means an individual or organization that utilizes the system to upload images and receive generated stamp images.

[1090] "Image" refers to visual data such as photographs or drawings uploaded by users.

[1091] A "server" is a computer system for processing images received from a user and providing a generated stamp image.

[1092] "Machine learning algorithms" are computational techniques used to extract facial features from images, and include face detection algorithms and facial landmark detection algorithms.

[1093] A "face detection algorithm" is a computational method for locating faces within an image.

[1094] A "facial landmark detection algorithm" is a computational method for identifying facial feature points (landmark points).

[1095] A "generative AI model" is an artificial intelligence model that generates facial images for stamps based on extracted facial features.

[1096] A "stamp image" is a facial image created by a generative AI model that can be used in messenger apps, etc.

[1097] The "means for providing" is a method for sending the generated stamp image to the user via email or a messenger app.

[1098] This invention is a system that generates face images for stamps using a machine learning algorithm and a generative AI model based on an image uploaded by a user, and provides the image to the user. A specific embodiment of this system will be described below.

[1099] First, the user uses a terminal (smartphone or PC) to access the system's web interface or a dedicated application. The user clicks the photo upload button and selects the friend's photo file. After selecting, the user presses the upload button to send the photo to the server.

[1100] The server receives the image sent by the user. It temporarily stores the received image data and then applies a face detection algorithm. Specifically, it uses OpenCV's Haar Cascade or Dlib's face detection function to detect faces in the image. Once face detection is complete, the server obtains the coordinate information of the face.

[1101] Next, the server uses Dlib's facial landmark detection feature to extract facial features. By extracting facial landmark points, detailed facial features can be captured. The extracted landmark points are used in the next processing step.

[1102] The server uses a generative AI model (e.g., GAN: Generative Adversarial Network) to generate a face image for the stamp based on the extracted facial features. The GAN model uses the extracted landmark points as input to generate a face image for the stamp. The generated face image is saved in a format that the user can use in messenger apps, etc.

[1103] Finally, the server provides the generated stamp image to the user via email or a messenger app (e.g., LINE or WhatsApp). The user can download the received stamp image and use it in the messenger app.

[1104] As a concrete example, consider the case where a user uploads a photo of a friend and converts the friend's face into a LINE stamp. The user selects the photo in the LINE app on their smartphone and uploads it to the system. The server uses OpenCV to detect faces in the received photo and Dlib to extract facial features. It then uses GAN to convert the friend's face into a stamp and sends the generated stamp to the user as a LINE message. The user can use the received stamp in LINE chats.

[1105] An example of a prompt is, "Upload a photo of a friend and turn their face into a LINE stamp."

[1106] The system allows users, without any specialist knowledge or skills, to convert their friends' faces into stamps and other digital content that can be shared quickly and easily.

[1107] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1108] Step 1:

[1109] A user uploads a photo of a friend.

[1110] Specifically, the user uses a terminal (smartphone or PC) to access the system's web interface or a dedicated application. The user clicks the photo upload button and selects a friend's photo file. After selecting, the user presses the upload button, and the photo is sent to the server.

[1111] Input: A photo file selected by the user

[1112] Output: Photo data sent to the server

[1113] Step 2:

[1114] The server receives the photo and performs face detection.

[1115] Specifically, the server receives the photo sent by the user and temporarily stores it. Then, it detects the face in the photo using OpenCV's Haar Cascade and Dlib's face detection function. Once face detection is complete, the server obtains the face coordinate information.

[1116] Input: Photo data sent by the user

[1117] Output: Face coordinate information

[1118] Step 3:

[1119] The server extracts facial features.

[1120] Specifically, the server uses Dlib's facial landmark detection feature to extract facial landmark points, which allow detailed facial features to be identified, and these points are used in the next processing step.

[1121] Input: Face coordinate information

[1122] Output: Facial landmark points

[1123] Step 4:

[1124] The server generates a face image for the stamp.

[1125] Specifically, the server uses a generative AI model (for example, a Generative Adversarial Network (GAN)) to generate a facial image for the stamp. The GAN model uses the extracted landmark points as input to generate a facial image for the stamp. The generated facial image is saved in a format that the user can use in messenger apps, etc.

[1126] Input: Facial landmark points

[1127] Output: Face image for stamp

[1128] Step 5:

[1129] The server provides the generated stamp image to the user.

[1130] Specifically, the server sends the generated stamp image to the user via email or a messenger app (e.g., LINE or WhatsApp). The user can download the received stamp image and use it in the messenger app.

[1131] Input: Generated stamp image

[1132] Output: Stamp image sent to user

[1133] (Application example 2)

[1134] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1135] In autonomous vehicles, there is a demand for providing passengers with personalized entertainment and information. However, with conventional systems, it is difficult to provide content that reflects the individual characteristics of each passenger, which makes it difficult to increase passenger satisfaction.

[1136] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1137] In this invention, the server includes a means for receiving an image uploaded by a user, a means for generating an avatar for the infotainment system of the autonomous vehicle based on the image, and a means for displaying the generated avatar on a display inside the vehicle, thereby generating a personalized avatar based on the passenger's face, and providing customized entertainment and information to the passenger.

[1138] "User" refers to an individual or organization that uses the System.

[1139] "Upload" refers to the act of a user sending data from their device to a server.

[1140] "Image" refers to visual information stored in digital format.

[1141] An "autonomous vehicle" refers to a vehicle that drives autonomously using artificial intelligence and sensor technology.

[1142] An "infotainment system" refers to an in-car system that integrates information and entertainment.

[1143] An "avatar" refers to a digital character that reflects the characteristics of a user.

[1144] "Generation" refers to the process by which a system creates new data based on input data.

[1145] "Display" refers to a device for displaying visual information.

[1146] A "machine learning algorithm" refers to a computational technique for learning patterns from data and making predictions or classifications.

[1147] "Features" refer to important information extracted from images or data.

[1148] "Entertainment" refers to content that provides enjoyment and entertainment to users.

[1149] "Information" refers to data and knowledge that is useful to the user.

[1150] "Customization" refers to adjustment to the needs and preferences of an individual user.

[1151] A system for implementing the invention includes means for receiving an image uploaded by a user, means for generating an avatar for an infotainment system of an autonomous vehicle based on the image, and means for displaying the generated avatar on an in-vehicle display.

[1152] System Programming

[1153] The system operates as follows.

[1154] 1. User uploads an image:

[1155] Users upload images from their devices to the system, which transmits them to a server and stores them for processing.

[1156] 2. The server processes the image:

[1157] The server uses machine learning algorithms to extract facial features from the received images: it uses OpenCV to detect faces and Keras to extract features using a pre-trained facial feature extraction model.

[1158] 3. Create an avatar:

[1159] Based on the extracted features, a generative AI model is used to generate an avatar, which is a digital character that reflects the user's facial features.

[1160] 4. Display on the display:

[1161] The generated avatar will be displayed on the autonomous vehicle's infotainment system display, allowing passengers to enjoy a personalized avatar based on their own face.

[1162] Hardware and Software Used

[1163] Hardware:

[1164] Cameras in self-driving vehicles: Used to capture passengers' faces.

[1165] Display: Used to display the generated avatar.

[1166] software:

[1167] OpenCV: Used for face detection.

[1168] Keras: Used to run the face feature extraction model.

[1169] PIL (Python Imaging Library): Used to process the generated avatar as an image.

[1170] Examples

[1171] For example, when a passenger enters the vehicle, a camera will automatically capture their face and the system will generate an avatar based on that image, which will then be displayed on the vehicle's display to provide customized entertainment and information to the passenger.

[1172] Examples of prompt statements

[1173] "Develop a system that takes a picture of a passenger's face and generates a personalized avatar based on that face. The avatar will be used by the infotainment system inside the car."

[1174] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1175] Step 1:

[1176] A user uploads an image.

[1177] A user selects an image from his / her device and uploads it to the system. The input is the image file stored on the user's device, and the output is the image data sent to the server. The server temporarily stores the received image data.

[1178] Step 2:

[1179] The server processes the images.

[1180] The server performs face detection on the received image data using OpenCV. The input is the uploaded image data, and the output is the area information of the detected face. Specifically, the image is converted to grayscale and a face detection algorithm is applied to identify the position of the face.

[1181] Step 3:

[1182] The server extracts facial features.

[1183] The server extracts features from the face detection results using a face feature extraction model pre-trained with Keras. The input is face region information, and the output is a face feature vector. Specifically, the detected face region is resized and input to the model to obtain a feature vector.

[1184] Step 4:

[1185] The server generates the avatar.

[1186] The server generates an avatar using a generative AI model based on the extracted facial feature vector. The input is the facial feature vector, and the output is the generated avatar image. Specifically, the feature vector is input into the generative AI model to generate the avatar image.

[1187] Step 5:

[1188] The server displays the avatar on the display.

[1189] The server transmits the generated avatar image to the display of the infotainment system of the autonomous vehicle. The input is the generated avatar image, and the output is the avatar displayed on the display. Specifically, the server converts the avatar image into a format suitable for the display and transmits a signal to display it on the display in the car.

[1190] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1191] "Example 1"

[1192] In one embodiment of the present invention, a system is provided that includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp, a cartoon or a movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, and an emotion engine for recognizing the emotion of the user. The emotion engine recognizes the emotion from the user's facial expression, tone of voice, text input, etc., and adjusts the generation of the personalized digital gift based on the emotion. For example, if it is recognized that the user is happy, the system generates a personalized digital gift that expresses joy.

[1193] "Example 2"

[1194] In another embodiment of the present invention, a user uploads a photo of a friend to the system, and the system uses a machine learning algorithm to extract the friend's facial features. The system then converts the person's face on the stamp into the friend's face based on the extracted features. Furthermore, the system recognizes the user's emotion with an emotion engine and adjusts the generated personalized digital gift based on the emotion. For example, if the user is recognized as sad, the system generates a personalized digital gift with a comforting message.

[1195] The process flow of each embodiment will be described below.

[1196] "Example 1"

[1197] Step 1: A user uploads a photo to the system.

[1198] Step 2: The system receives the uploaded photo and extracts the person's facial features from the photo.

[1199] Step 3: Based on the extracted features, the system converts the faces and dialogue of people in messenger app stickers, comic book and movie scenes.

[1200] Step 4: The system uses the emotion engine to recognize the user's emotion.

[1201] Step 5: The system tailors the generated personalized digital gift based on the recognized emotion and provides it to the user.

[1202] "Example 2"

[1203] Step 1: A user uploads a photo of their friend into the system.

[1204] Step 2: The system receives the uploaded photo and uses machine learning algorithms to extract your friend's facial features.

[1205] Step 3: The system converts the face of the person in the stamp into the face of a friend based on the extracted features. Step 4: The system uses an emotion engine to recognize the user's emotion.

[1206] Step 5: The system tailors the generated personalized digital gift based on the recognized emotion and provides it to the user.

[1207] Example 1

[1208] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1209] Conventional digital gift generation systems provide uniform content without considering the user's emotions, making it difficult to provide a personalized experience. In addition, conversion processing based on images uploaded by users was limited, making it difficult to convert into various formats such as messenger app stamps, comics, and video scenes.

[1210] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a face or dialogue of a person in a communication application stamp, a cartoon or a video scene based on the image, a means for providing the conversion result as a personalized digital gift, and an emotion engine for recognizing the user's emotion. This makes it possible to generate a personalized digital gift based on the user's emotion.

[1211] "User" refers to any person who utilizes the System to upload an image and receive a personalized digital gift.

[1212] "Image" refers to visual data such as photographs or drawings uploaded by users.

[1213] "Communication application" refers to software that enables users to send and receive messages, such as a messenger application or chat application.

[1214] "Stamps" refer to small images or icons used within communication applications.

[1215] "Manga" refers to a series of pictures or illustrations that visually depict a story.

[1216] A "film scene" refers to a portion of a visual medium such as a film or video.

[1217] "Conversion" refers to the process of changing original images or data into a different format or content.

[1218] "Personalized digital gifts" refers to individually customized digital content generated based on a user's image and emotions.

[1219] An "emotion engine" refers to software or algorithms that recognize emotions from a user's facial expressions, tone of voice, text input, etc.

[1220] A "machine learning algorithm" is a mathematical model that learns patterns from data and makes predictions or classifications.

[1221] The present invention is a system for generating and providing a personalized digital gift based on an image uploaded by a user. The system includes a means for a user to upload an image, a means for analyzing and transforming the image, a means for providing the transformation result as a personalized digital gift, and an emotion engine for recognizing the user's emotion.

[1222] Hardware and Software Used

[1223] Hardware

[1224] Server: Receives, analyzes, transforms images, recognizes emotions, and generates and delivers digital gifts.

[1225] Device: The device (computer, smartphone, tablet, etc.) through which a user uploads images.

[1226] software

[1227] Website or application: Provides an interface for users to upload images.

[1228] Machine learning algorithms used to extract features from images and perform transformations (e.g. OpenCV, Dlib, TensorFlow, PyTorch).

[1229] Emotion engine: Software for recognizing user emotions (e.g. Emotion API).

[1230] Generative AI models: Models for image transformation and dialogue generation (e.g., StyleGAN, GPT-3).

[1231] Mail Server and Messenger API: Used to send the generated digital gifts to users.

[1232] Examples

[1233] A user accesses a website and uploads an image of their own smiling face. The server receives the image and uses a machine learning algorithm to extract facial features. Next, the emotion engine recognizes the user's smile as "happiness." Based on this information, the server generates a stamp for a communication application that uses the user's face to express happiness. Finally, the generated stamp is sent to the user's email address.

[1234] Examples of prompt statements

[1235] "Generate stickers for a communication application that express joy based on images uploaded by users. The user's emotions are recognized from their smiles."

[1236] In this way, it is possible to generate and offer personalized digital gifts based on the user's emotions.

[1237] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1238] Step 1: User uploads an image.

[1239] A user uploads his / her own image through a website or application by clicking an upload button and selecting an image file from a file selection dialog. The input is the user's image file, and the output is the image data sent to the server.

[1240] Step 2: The server receives the image.

[1241] The server receives the image sent by the user and saves it in the database. Specifically, it receives the HTTP request, saves the image file in a temporary directory, and then saves it in the database. The input is the image data sent by the user, and the output is the path of the saved image file.

[1242] Step 3: The server analyzes the image.

[1243] The server uses machine learning algorithms to extract facial features from images. Specifically, it uses OpenCV and Dlib to perform face detection and identify the coordinates of facial landmarks (eyes, nose, mouth, etc.). The input is the path to the saved image file, and the output is the facial feature data.

[1244] Step 4: The server recognizes the emotion.

[1245] The server uses the emotion engine to recognize the user's emotion from the image. Specifically, it sends facial feature data to the Emotion API and classifies the emotion. The input is the facial feature data, and the output is the recognized emotion data.

[1246] Step 5: The server performs the conversion process.

[1247] The server performs conversion processing using a generative AI model based on the extracted facial features and recognized emotions. Specifically, it uses StyleGAN to convert the user's face into a stamp for a communication application, and uses GPT-3 to generate dialogue that matches the emotion. The input is facial feature data and emotion data, and the output is the converted stamp and dialogue data.

[1248] Step 6: The server generates the personalized digital gift.

[1249] The server generates a personalized digital gift based on the converted results. Specifically, it creates a digital card that includes a stamp with the user's face and a message that matches the user's emotion. The input is the converted stamp and conversation data, and the output is the generated digital gift.

[1250] Step 7: The server sends the digital gift.

[1251] The server sends the generated personalized digital gift to the user via email or a communication application. Specifically, the server uses the SMTP protocol or a messenger API to send the digital gift. The input is the generated digital gift and the output is the digital gift sent to the user.

[1252] (Application example 1)

[1253] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1254] In the conventional personalized digital gift generation system, since the conversion is performed uniformly without considering the user's emotions, it is not possible to provide an appropriate digital gift according to the user's emotions. In addition, if the generated digital gift does not match the user's emotions, there is a problem that the user's satisfaction decreases. Furthermore, since there is a lack of means to effectively deliver the generated digital gift, it is difficult to deliver it to the user quickly and reliably.

[1255] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1256] In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a cartoon or movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, a means including an emotion engine for recognizing the emotion of the user, a means for adjusting the generation of the personalized digital gift based on the emotion, and a means for sending the generated personalized digital gift to the user via email or a messenger app. This makes it possible to generate and quickly deliver a personalized digital gift appropriate to the emotion of the user.

[1257] "Means for receiving photos uploaded by users" refers to a function that enables users to send their own photos to the system via the Internet and for the server to receive those photos.

[1258] "Messenger app stamps" are images or icons used within messenger applications to allow users to express emotions or messages.

[1259] "Means of transforming the faces and dialogue of characters in manga or movie scenes" refers to technology that replaces the faces and dialogue of characters appearing in manga or movie scenes with the user's face and dialogue based on photos uploaded by the user.

[1260] A "personalized digital gift" is a digital gift that is customized based on a user's individual characteristics and emotions.

[1261] An "emotion engine" is software that recognizes emotions from a user's facial expressions, tone of voice, text input, etc., and adjusts the system's operation based on those emotions.

[1262] A "machine learning algorithm" is a computational technique for learning patterns based on large amounts of data and making predictions and classifications.

[1263] "Means of sending via email or messenger app" refers to a function for sending the generated personalized digital gift to a user via email or messenger application.

[1264] An embodiment of the present invention will be described. First, a user uses a terminal such as a smartphone or a personal computer to access a dedicated application or website. The user sends a photo through an interface for uploading the user's own photo. The photo is received by a server.

[1265] The server uses a machine learning algorithm to extract features from the received photos. The algorithm uses image processing libraries such as OpenCV to detect facial feature points and analyze the shape and expression of the face. Then, based on the extracted features, it converts the faces and dialogue of people in messenger app stamps, comics, and movie scenes. For the conversion, it uses technologies such as FaceSwap and DeepFake.

[1266] In addition, the server recognizes the user's emotions using an emotion engine. The emotion engine uses software such as EmotionRecognizer to analyze emotions from the user's facial expressions, tone of voice, text input, etc. Based on the recognized emotions, the server adjusts the generation of personalized digital gifts. For example, if the user is happy, the server generates a digital gift that expresses joy.

[1267] The generated personalized digital gift is then sent to the user from the server via email or a messenger app, using a communication library such as MessagingService.

[1268] For example, if a user uploads a photo of themselves feeling happy, the system will recognize that emotion and edit the character in the movie scene to make them look happy. The generated video will be sent to the user's email address.

[1269] An example of a prompt sentence might be:

[1270] "Create an application that converts a character's face in a movie scene into the user's face based on a photo uploaded by the user, and edits the scene based on the user's emotions. Please also include a function to send the generated video to the user."

[1271] In this way, personalized digital gifts appropriate to the user's emotions can be generated and quickly delivered.

[1272] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1273] Step 1:

[1274] The user uses the device to access a dedicated application or website and uploads their own photo.

[1275] Input: User photo

[1276] Output: The user's photo sent to the server.

[1277] How it works: A user selects a photo through the application interface and clicks the upload button, and the photo is sent to the server over the Internet.

[1278] Step 2:

[1279] Based on the photos received by the server, a machine learning algorithm is used to extract features from the photos.

[1280] Input: The user's photo sent to the server

[1281] Output: Extracted facial feature data

[1282] Specific operation: The server uses image processing libraries such as OpenCV to detect facial feature points from photos and analyze facial shape and facial expressions.

[1283] Step 3:

[1284] Based on the extracted features, the server converts the faces and dialogue of people in messenger app stamps, manga, and movie scenes.

[1285] Input: Extracted facial feature data

[1286] Output: Converted character faces and dialogue

[1287] How it works: The server uses techniques such as FaceSwap and DeepFake to swap the user's face with that of a character in a movie scene and adapts the dialogue to match the user's characteristics.

[1288] Step 4:

[1289] The server recognizes the user's emotions using an emotion engine.

[1290] Input: User photo

[1291] Output: Recognized emotion data

[1292] How it works: The server uses software such as EmotionRecognizer to analyze emotions from the user's facial expressions, tone of voice, text input, etc.

[1293] Step 5:

[1294] The server coordinates the generation of the personalized digital gift based on the recognized emotion.

[1295] Input: Recognized emotion data, transformed character faces and dialogue

[1296] Output: Personalized digital gifts tailored based on emotions

[1297] Specific operation: The server generates digital gifts that reflect emotions such as joy or sadness according to the recognized emotions.

[1298] Step 6:

[1299] The server then sends the generated personalized digital gift to the user via email or a messenger app.

[1300] Input: Personalized digital gifts tailored based on emotions

[1301] Output: A personalized digital gift sent to the user.

[1302] Specific operation: The server uses a communication library such as MessagingService to send the generated digital gift to the user's email address or messenger app.

[1303] Example 2

[1304] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1305] In conventional systems, it was difficult to provide personalized digital gifts that took into account the user's emotions when converting stamps in messenger apps and faces and dialogues in visual media based on photos uploaded by the user. In addition, there was a lack of means to appropriately send the generated digital gifts to the user. This limited the user experience and made it difficult to provide services that met individual needs.

[1306] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1307] In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or visual media based on the image, a means for providing the conversion result as a personalized digital gift, a means for recognizing a user's emotion and adjusting the personalized digital gift based on the emotion, and a means for sending the personalized digital gift to the user, thereby enabling the generation and sending of a personalized digital gift according to the user's emotion.

[1308] "User" means any individual or entity that utilizes the System to upload images and receive personalized digital gifts.

[1309] An "image" is data containing visual information, such as a photograph or drawing, that a user uploads to the system.

[1310] A "messenger app" is a software application that enables users to exchange messages, including text, images, and audio, with other users.

[1311] "Stamps" are small images used within messenger apps that express specific facial expressions or characters.

[1312] "Visual media" refers to media content that includes visual elements, such as manga and movies.

[1313] A "machine learning algorithm" is a computational technique for learning patterns from data and making predictions or classifications.

[1314] A "feature" is an identifiable element in an image, such as a facial contour, eyes, nose, or mouth.

[1315] A "personalized digital gift" is a digital gift that is individually generated based on a user's input and emotions.

[1316] An "emotion engine" is software or algorithms for analyzing and recognizing a user's emotions.

[1317] An "electronic means of communication" is any means for sending or receiving information in digital form, such as email or a messaging app.

[1318] This invention is a system that converts the faces and dialogues of people in stamps and visual media of messenger apps based on images uploaded by users and provides them as personalized digital gifts. A specific embodiment of this system will be described below.

[1319] System Configuration

[1320] Hardware

[1321] Server: A computer system with powerful processors and large memory capacity that runs image processing and machine learning algorithms.

[1322] Terminal: A smartphone or PC used by a user to upload images and receive generated digital gifts.

[1323] software

[1324] Image processing library: Software for pre-processing and transforming images, such as OpenCV or Adobe Photoshop API.

[1325] Machine learning algorithms: Algorithms for extracting features from images and performing transformations, such as TensorFlow.

[1326] Emotion engine: Software for recognizing user emotions, such as IBM Watson or Microsoft Azure Emotion API.

[1327] Electronic communication means: Means for sending the generated digital gift to the user, such as SMTP protocol or messenger API (e.g. Facebook Messenger API).

[1328] System Operation

[1329] Uploading an image

[1330] Users use their devices to upload pictures of their friends to the system, either through the system's web interface or a dedicated app.

[1331] Image preprocessing and feature extraction

[1332] The server receives the uploaded image, first pre-processes the image using OpenCV, then uses TensorFlow's face recognition model to extract the facial features of the friend. In this feature extraction process, the main feature points such as the facial contour, eyes, nose, and mouth are identified.

[1333] Stamp Generation

[1334] The server uses Adobe Photoshop API to generate stamps based on the extracted facial features, adjusting the facial contours and facial expressions to fit the stamp design.

[1335] Emotion recognition and digital gift adjustment

[1336] The server uses IBM Watson to analyze the user's emotions, analyzing facial expressions and text inputs while the user is using the system. For example, if the server detects that the user is sad, it will generate a personalized digital gift with a comforting message.

[1337] Sending digital gifts

[1338] The server sends the generated personalized digital gift to the user via email or messenger app using SMTP protocol or messenger API. The user can view the received digital gift and share it with friends.

[1339] Examples

[1340] As a concrete example, consider a case where a user uploads a photo of a friend and converts the friend's face into a sticker in a messenger app.

[1341] 1. The user opens the dedicated app on their smartphone and selects and uploads photos of their friends.

[1342] 2. The server receives the uploaded photo, pre-processes the image using OpenCV, and then uses TensorFlow's face recognition model to extract the friend's facial features.

[1343] 3. The server uses Adobe Photoshop API to generate stamps based on the extracted facial features, adjusting the facial contours and facial expressions to fit the stamp design.

[1344] 4. The server uses IBM Watson to analyze the user's emotions and, if the user is recognized as sad, generates a personalized digital gift with a comforting message.

[1345] 5. The server sends the generated personalized digital gift to the user through the Facebook Messenger API. The user can view the received digital gift and share it with their friends.

[1346] Examples of prompt statements

[1347] Upload a photo of your friend and turn their face into a sticker. Generate and send personalized digital gifts based on your user's emotions.

[1348] By inputting this prompt sentence into the generative AI model, the system will carry out the above process and provide the user with an original stamp.

[1349] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1350] Step 1:

[1351] A user uses a terminal to upload a friend's image to the system. The user opens the system's web interface or a dedicated app, selects an image file, and presses the upload button. The input is the image file selected by the user, and the output is the image data sent to the server.

[1352] Step 2:

[1353] The server receives the uploaded image. The received image data is first preprocessed using an image processing library (e.g., OpenCV). Preprocessing includes image resizing, noise removal, color correction, etc. The input is the image data sent by the user, and the output is the preprocessed image data.

[1354] Step 3:

[1355] The server inputs the preprocessed image data into a machine learning algorithm (e.g., TensorFlow's facial recognition model) to extract facial features. The feature extraction process identifies key features such as the facial contour, eyes, nose, and mouth. The input is the preprocessed image data, and the output is the extracted facial feature data.

[1356] Step 4:

[1357] The server generates a stamp using an image processing library (e.g., Adobe Photoshop API) based on the extracted facial feature data. Specifically, it adjusts the facial contours and facial expressions to fit the stamp design. The input is the facial feature data, and the output is the generated stamp image.

[1358] Step 5:

[1359] The server recognizes the user's emotions using an emotion engine (e.g., IBM Watson). It analyzes emotions from facial expressions and text inputs while the user is using the system. The input is the user's facial expression data and text data, and the output is the recognized emotion data.

[1360] Step 6:

[1361] The server tailors the personalized digital gift including the generated stamp based on the recognized emotion data, e.g., if the user is recognized as sad, it generates a digital gift including a comforting message, where the input is the emotion data and the stamp image, and the output is the tailored personalized digital gift.

[1362] Step 7:

[1363] The server sends the generated personalized digital gift to the user via email or a messenger app using the SMTP protocol or a messenger API (e.g., Facebook Messenger API). The input is the personalized digital gift and the output is the digital gift sent to the user.

[1364] (Application example 2)

[1365] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1366] In conventional messenger apps and video content, it was difficult for users to generate personalized characters and stamps based on their friends' photos. In addition, there was a lack of functionality to adjust content according to the user's emotions, so there was a need to improve the user experience. Furthermore, there were limited means to effectively provide the generated content.

[1367] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1368] In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting the face and dialogue of a character of a messenger app stamp or video content based on the photo, a means for providing the conversion result as a personalized digital gift, and a means for recognizing the emotion of the user and adjusting the personalized digital gift based on the emotion. This allows the user to easily generate a personalized character or stamp based on a friend's photo, and further allows the content to be adjusted according to the user's emotion. In addition, the generated content can be effectively provided.

[1369] "User" means an individual who utilizes the System to upload photos and receive personalized digital gifts.

[1370] A "photo" is image data uploaded by a user, which includes features such as a friend's face.

[1371] "Messenger app stamps" are digital stamps used within messenger apps and are generated based on photos uploaded by users.

[1372] "Video content characters" are people or characters that appear in videos, whose faces and dialogue are transformed based on photos uploaded by users.

[1373] "Dialogue" refers to conversations or statements made by characters in video content or messenger apps.

[1374] "Personalized Digital Gifts" are individualized digital content that is generated based on photos and emotions uploaded by users.

[1375] A "machine learning algorithm" is an algorithm for extracting features from photographs and transforming people's faces and dialogue based on those features.

[1376] "Means for recognizing emotions" refers to technology for analyzing a user's emotions and adjusting content based on those emotions.

[1377] The "means of provision" refers to the means of sending the generated personalized digital gift to the user via email or a messenger app.

[1378] The system for implementing the present invention receives a photo uploaded by a user, converts the face and dialogue of a character in a messenger app stamp or video content based on the photo, and provides a generated personalized digital gift. It also includes a function of recognizing the user's emotions and adjusting the personalized digital gift based on the emotions.

[1379] Hardware and Software Used

[1380] Hardware: Smartphone, head-mounted display

[1381] Software: OpenCV, Dlib, EmotionRecognizer (emotion recognition library), VideoEditor (video editing library)

[1382] System Configuration

[1383] 1. User terminal: A device that users use to upload photos. This includes smartphones and head-mounted displays.

[1384] 2. Server: Receives the uploaded photos, uses machine learning algorithms to extract facial features, and processes them to transform the faces and dialogue of characters in the video content.

[1385] 3. Emotion Recognition Engine: Analyzes user emotions and tailors personalized digital gifts based on those emotions.

[1386] 4. Delivery method: The generated personalized digital gift is sent to the user via email or messenger app.

[1387] Data processing and calculation

[1388] 1. Facial feature extraction: The server extracts facial features from the uploaded photo using OpenCV and Dlib, which obtains the facial landmark points in the photo.

[1389] 2. Emotion Recognition: The server recognizes the user's emotion using EmotionRecognizer, which analyzes the user's emotional state when he uploads a photo.

[1390] 3. Video Editing: The server uses VideoEditor to edit the video content based on the extracted facial features and the recognized emotions, so that the faces and dialogues of the characters in the video are transformed into the faces of the user's friends and the content is adjusted according to their emotions.

[1391] Examples

[1392] Consider a case where a user uploads a photo of a friend and converts the friend's face into a character in a video content. For example, if the user wants to "make the friend's face into an animated character and create a touching scene," the system converts the friend's face into an animated character and generates a video including the touching scene.

[1393] Examples of prompt statements

[1394] Upload a photo of your friend, transform their face into a cartoon character, and generate a video with some touching moments.

[1395] In this way, users can enjoy personalized video content.

[1396] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1397] Step 1:

[1398] A user uploads a photo of a friend.

[1399] Input: A friend's photo file selected by the user.

[1400] Output: The photo data uploaded to the server.

[1401] Specific operation: A user selects a photo of a friend using a smartphone or a head-mounted display and uploads it to the system. The server receives the photo data.

[1402] Step 2:

[1403] The server extracts facial features from the photo.

[1404] Input: The uploaded photo data.

[1405] Output: Extracted facial feature data (landmark points).

[1406] What it does: The server uses OpenCV and Dlib to detect faces in photos and extract facial landmark points, which gives us facial feature data.

[1407] Step 3:

[1408] The server recognizes the user's emotions.

[1409] Input: The uploaded photo data.

[1410] Output: The recognized emotion data.

[1411] What it does: The server uses EmotionRecognizer to analyze the photo and recognize the user's emotion, which analyzes the user's emotional state when uploading the photo.

[1412] Step 4:

[1413] The server edits the video content based on facial features and emotions.

[1414] Input: extracted facial feature data, recognized emotion data, original video content.

[1415] Output: The edited video content.

[1416] How it works: The server uses VideoEditor to convert the character's face in the video into the friend's face based on the extracted facial feature data, and adjusts the video content based on the recognized emotion data, thereby generating personalized video content.

[1417] Step 5:

[1418] The server provides the generated personalized digital gift.

[1419] Input: The edited video content.

[1420] Output: A personalized digital gift provided to the user.

[1421] How it works: The server sends the generated video content to the user via email or messenger app, so that the user can receive personalized video content.

[1422] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1423] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1424] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1425] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1426] [Fourth embodiment]

[1427] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1428] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1429] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1430] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1431] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1432] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1433] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1434] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1435] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1436] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1437] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1438] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1439] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1440] "Example 1"

[1441] The system of the present invention includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a scene in a cartoon or movie based on the photo, and a means for providing the conversion result as a personalized digital gift. Specifically, a user uploads his / her own photo through a website or application. Next, the system uses a machine learning algorithm to extract features from the photo and converts the face or dialogue of the person based on the features. Finally, the generated personalized digital gift is sent to the user via email or a messenger app.

[1442] "Example 2"

[1443] As a specific example, consider a case where a user uploads a photo of a friend and converts the friend's face into a stamp for a messenger app. First, the user uploads the friend's photo to the system. Next, the system uses a machine learning algorithm to extract the friend's facial features and converts the face of the person in the stamp into the friend's face based on the features. Finally, the system sends the generated personalized digital gift to the user via email or a messenger app. In this way, the user can get an original stamp with the friend's face drawn on it.

[1444] The process flow of each embodiment will be described below.

[1445] "Example 1"

[1446] Step 1: The user uploads a photo of themselves via a website or application.

[1447] Step 2: The system uses machine learning algorithms to extract features from the photo.

[1448] Step 3: The system transforms the person's face and dialogue based on the extracted features.

[1449] Step 4: The generated personalized digital gift is sent to the user via email or messenger app.

[1450] "Example 2"

[1451] Step 1: A user uploads a photo of their friend into the system.

[1452] Step 2: The system uses machine learning algorithms to extract your friend's facial features.

[1453] Step 3: The system converts the face of the person on the stamp into the face of a friend based on the extracted features. Step 4: The system sends the generated personalized digital gift to the user via email or messenger app.

[1454] Example 1

[1455] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1456] In the conventional digital gift generation system, when generating personalized digital gifts based on photos uploaded by users, the conversion of faces and dialogues is done manually, which is time-consuming and labor-intensive. In addition, there are also issues that the quality of the generated digital gifts is inconsistent and may not meet the user's expectations. Furthermore, there is a lack of a means to quickly and reliably provide the generated digital gifts to the users.

[1457] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1458] In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a stamp of a communication application or a face or dialogue of a person in a scene of a cartoon or a video work based on the image, a means for providing the conversion result as a personalized digital gift, a means for using a machine learning algorithm to extract features from the image, a means for using a generative AI model to generate a face or dialogue of a person based on the features, and a means for transmitting the generated personalized digital gift to the user via an electronic communication means. This makes it possible to quickly and efficiently generate a high-quality, consistent personalized digital gift based on an image uploaded by a user and provide it to the user.

[1459] "User" means an individual or entity that utilizes the System to upload an image and receive a personalized digital gift.

[1460] "Images" are visual data such as photographs and illustrations that users upload to the system.

[1461] A "communication application" is software for sending and receiving messages, and has the function of sending stamps and images.

[1462] A "stamp" is a small image or icon used within a communication application to express an emotion or message.

[1463] "Manga" is a series of pictures or illustrations intended to visually represent a story.

[1464] A "visual work" is any medium that contains a visually presented story or information, such as a film or television program.

[1465] A "machine learning algorithm" is a computational technique for learning patterns and features from data and making predictions and classifications.

[1466] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate new data.

[1467] A "personalized digital gift" is a digital gift that is customized for a particular user based on the user's image and information.

[1468] "Electronic communication means" refers to means for sending and receiving data via the Internet, email, messenger apps, etc.

[1469] The present invention is a system for generating and providing personalized digital gifts based on images uploaded by users. A specific embodiment of the system will be described below.

[1470] System Configuration

[1471] Hardware and Software

[1472] The server uses the following hardware and software:

[1473] Hardware: High-performance server machine (e.g. cloud server with CPU, GPU, and memory)

[1474] Software: Machine learning algorithms (e.g. TensorFlow), generative AI models (e.g. GPT-4, DALL-E), databases (e.g. Amazon S3), communications (e.g. SMTP servers, messenger APIs)

[1475] Processing flow

[1476] 1. Upload your photo

[1477] A user accesses a website or application using a smartphone or PC. For example, the user opens an application on the smartphone, takes a photo of his / her face using the camera function, and presses the upload button. This sends the photo data to the application.

[1478] 2. Receiving photos

[1479] The server receives the images uploaded by the user. Specifically, the server uses an Amazon S3 bucket to store the image data. The server receives the HTTP request, analyzes the image data, and stores it.

[1480] 3. Feature Extraction

[1481] The server uses machine learning algorithms to extract features from the images. It uses TensorFlow for this process. The server feeds the image data into a TensorFlow model to identify features such as the contours of the face and the location of the eyes, nose, and mouth. For example, it uses a facial recognition model to obtain the coordinates of each part of the face.

[1482] 4. Face and dialogue transformation

[1483] Based on the extracted features, the server converts the faces and dialogue of people in stamps for communication applications, comics, and scenes in video works. Here, generative AI models (GPT-4, DALL-E) are used. The server uses DALL-E to synthesize the user's face into a specific scene and uses GPT-4 to generate the dialogue content. For example, the server synthesizes the user's face into a scene from a famous science fiction movie, and changes the dialogue in that scene to the user's name or a specific phrase.

[1484] 5. Create personalized digital gifts

[1485] The server generates a personalized digital gift based on the conversion result. The server combines the generated images and text to create a digital gift for the user. For example, it creates a digital stamp with the user's face or a comic strip featuring the user.

[1486] 6. Sending Digital Gifts

[1487] The server transmits the generated personalized digital gift to the user via electronic communication means. The server transmits a download link for the digital gift using the user's email address or messenger app ID. For example, the server transmits an email containing a download link for the digital gift to the user's email address.

[1488] Examples and prompts

[1489] Examples

[1490] Users upload a photo of their face through the app and receive a digital gift that combines their face with a scene from the famous sci-fi movie "XXX" based on the photo.

[1491] Examples of prompt statements

[1492] "Based on the face photo uploaded by the user, please composite the face into a scene from the famous science fiction movie "XXX" and generate a digital gift that includes a specific dialogue."

[1493] In this way, users can easily create and enjoy their own special digital gifts.

[1494] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1495] Step 1:

[1496] A user accesses a website or application using a smartphone or PC. The user takes a photo of his or her face using the camera function and presses the upload button. This sends the photo data to the application.

[1497] Input: A photo of the user's face

[1498] Output: The photo data sent to the application.

[1499] Step 2:

[1500] The server receives the images uploaded by the user: the server receives the HTTP request, parses the image data, and stores it in an Amazon S3 bucket.

[1501] Input: Photo data sent from the application

[1502] Output: Image data stored in an Amazon S3 bucket

[1503] Step 3:

[1504] The server uses machine learning algorithms to extract facial features from the images, using TensorFlow. The server feeds the image data into a TensorFlow model to identify features such as the facial contours and the location of the eyes, nose, and mouth.

[1505] Input: Image data stored in an Amazon S3 bucket

[1506] Output: Feature data such as facial contours and the positions of eyes, nose, and mouth

[1507] Step 4:

[1508] Based on the extracted features, the server converts the faces and dialogue of people in stamps for communication applications, comics, and scenes in video works. Here, generative AI models (GPT-4, DALL-E) are used. The server uses DALL-E to synthesize the user's face into a specific scene and uses GPT-4 to generate the dialogue content.

[1509] Input: Facial feature data

[1510] Output: Synthesized face image and generated dialogue content

[1511] Step 5:

[1512] The server generates a personalized digital gift based on the conversion result. The server combines the generated images and text to create a digital gift exclusive to the user.

[1513] Input: Synthetic face image and generated dialogue

[1514] Output: personalized digital gift (images and text)

[1515] Step 6:

[1516] The server transmits the generated personalized digital gift to the user via electronic communication means. The server uses the user's email address or messenger app ID to send a download link for the digital gift.

[1517] Input: Personalized digital gift (image and / or text)

[1518] Output: Download link for the digital gift sent to the user

[1519] (Application example 1)

[1520] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1521] Conventional advertising systems have had the problem that it is difficult to provide users with personalized advertisements, and the effectiveness of advertisements is limited. In addition, there is a lack of a way for users to easily generate personalized advertisements based on their own photos, so it is not possible to improve the degree of personalization of advertisements.

[1522] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a cartoon or movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, a means for generating the conversion result as an advertising material, and a means for providing the generated advertising material to a user. This makes it possible to generate a personalized advertisement based on the user's photo.

[1523] "Means for receiving photos uploaded by users" refers to a function that enables users to send their own photos to a server via the Internet and receive the photo data.

[1524] "Means for changing the faces and dialogue of people appearing in messenger app stamps, comics, or movie scenes based on the photograph" refers to a function for changing the faces and dialogue of people appearing in messenger app stamps, comics, or movie scenes using features extracted from a photograph uploaded by a user.

[1525] The "means for providing the conversion result as a personalized digital gift" is a function for individually providing digital content including the converted face and dialogue to the user.

[1526] The "means for generating the conversion result as advertising material" is a function for creating advertising material such as advertising banners and video advertisements using the converted faces and dialogue.

[1527] The "means for providing the created advertising material to the user" is a function for providing the created advertising material to the user so that the user can use it.

[1528] A "machine learning algorithm" is an algorithm that enables a computer to learn patterns from data and then use the results of that learning to make predictions and classify new data.

[1529] "Feature extraction" is the process of extracting important information or patterns from an image or data.

[1530] A "personalized digital gift" is a digital gift that is customized based on a user's individual information and preferences.

[1531] "Advertising Materials" refers to content such as images, videos, and text used in an advertising campaign.

[1532] A system for implementing this invention includes means for receiving photos uploaded by a user, means for converting messenger app stamps or the faces and dialogue of people in cartoon or movie scenes based on the photos, means for providing the results of the conversion as a personalized digital gift, means for generating the results of the conversion as advertising material, and means for providing the generated advertising material to a user.

[1533] The server receives photos uploaded by users over the Internet. The received photos are pre-processed using image processing libraries such as OpenCV and PIL. Then, machine learning algorithms (e.g., face recognition model using Keras) are used to extract facial features from the photos. This feature extraction process provides important information about the user's face.

[1534] Based on the extracted features, a generative AI model is used to transform the faces and dialogue of people in messenger app stickers, comics, and movie scenes. The transformed results are then generated as personalized digital gifts. Furthermore, the same transformed results are used to generate advertising materials (e.g., advertising banners and video ads).

[1535] The generated personalized digital gifts and advertising materials are then sent to the user via email or messenger apps, allowing users to easily generate personalized ads based on their own photos and display them in a special way for friends and family.

[1536] As a concrete example, consider a case where a user uploads a photo of themselves and generates a personalized ad by superimposing their face onto an advertising banner for a specific product (e.g. a new smartphone), which can then be shared with the user's friends and family via social media or messaging apps.

[1537] An example of a prompt is as follows:

[1538] "Upload a photo of yourself and we'll generate personalized ads for you that will be shown specially to your friends and family."

[1539] In this way, it becomes possible to generate personalized advertisements based on the user's photographs, thereby improving the effectiveness of the advertisements.

[1540] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1541] Step 1:

[1542] A user uploads a photo. The user sends his / her photo to the server via the Internet. The input is the user's photo data, and the output is the photo data stored on the server.

[1543] Step 2:

[1544] The server receives the photos and performs pre-processing. The server uses image processing libraries such as OpenCV and PIL to convert the photos to the appropriate format and adjust the size. The input is the user's photo data and the output is the pre-processed photo data.

[1545] Step 3:

[1546] The server uses machine learning algorithms to extract facial features from photos. The server uses a Keras facial recognition model to extract facial features from the preprocessed photo data. The input is the preprocessed photo data, and the output is the facial feature data.

[1547] Step 4:

[1548] The server uses a generative AI model to convert the faces and dialogue of people in messenger app stamps, comics, and movie scenes based on facial feature data. The input is facial feature data, and the output is the converted face and dialogue data.

[1549] Step 5:

[1550] The server generates the transformed data as a personalized digital gift. The server uses the transformed facial and / or interaction data to create a digital gift. The input is the transformed facial and / or interaction data, and the output is a personalized digital gift.

[1551] Step 6:

[1552] The server generates the converted data as advertising material. The server uses the converted face and dialogue data to create advertising material such as advertising banners and video ads. The input is the converted face and dialogue data, and the output is advertising material.

[1553] Step 7:

[1554] The server provides the generated personalized digital gift and advertising material to the user. The server sends the generated digital gift and advertising material to the user via email or a messenger app. The input is the personalized digital gift and advertising material, and the output is the digital gift and advertising material sent to the user.

[1555] Example 2

[1556] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1557] Conventional messenger apps and digital gift delivery methods make it difficult for users to easily create individually customized content. In addition, converting friends' faces into stamps and other digital content requires specialized knowledge and skills, making it difficult for general users to use. Furthermore, there is a lack of a way to quickly and easily share the generated content.

[1558] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1559] In this invention, the server includes a means for receiving an image uploaded by a user, a means for extracting facial features based on the image using a machine learning algorithm, a means for generating a facial image for a stamp based on the features using a generative AI model, and a means for providing the generated stamp image to the user. This allows the user to convert the face of a friend into a stamp or other digital content and share it quickly and easily without requiring specialized knowledge or skills.

[1560] "User" means an individual or organization that utilizes the system to upload images and receive generated stamp images.

[1561] "Image" refers to visual data such as photographs or drawings uploaded by users.

[1562] A "server" is a computer system for processing images received from a user and providing a generated stamp image.

[1563] "Machine learning algorithms" are computational techniques used to extract facial features from images, and include face detection algorithms and facial landmark detection algorithms.

[1564] A "face detection algorithm" is a computational method for locating faces within an image.

[1565] A "facial landmark detection algorithm" is a computational method for identifying facial feature points (landmark points).

[1566] A "generative AI model" is an artificial intelligence model that generates facial images for stamps based on extracted facial features.

[1567] A "stamp image" is a facial image created by a generative AI model that can be used in messenger apps, etc.

[1568] The "means for providing" is a method for sending the generated stamp image to the user via email or a messenger app.

[1569] This invention is a system that generates face images for stamps using a machine learning algorithm and a generative AI model based on an image uploaded by a user, and provides the image to the user. A specific embodiment of this system will be described below.

[1570] First, the user uses a terminal (smartphone or PC) to access the system's web interface or a dedicated application. The user clicks the photo upload button and selects the friend's photo file. After selecting, the user presses the upload button to send the photo to the server.

[1571] The server receives the image sent by the user. It temporarily stores the received image data and then applies a face detection algorithm. Specifically, it uses OpenCV's Haar Cascade or Dlib's face detection function to detect faces in the image. Once face detection is complete, the server obtains the coordinate information of the face.

[1572] Next, the server uses Dlib's facial landmark detection feature to extract facial features. By extracting facial landmark points, detailed facial features can be captured. The extracted landmark points are used in the next processing step.

[1573] The server uses a generative AI model (e.g., GAN: Generative Adversarial Network) to generate a face image for the stamp based on the extracted facial features. The GAN model uses the extracted landmark points as input to generate a face image for the stamp. The generated face image is saved in a format that the user can use in messenger apps, etc.

[1574] Finally, the server provides the generated stamp image to the user via email or a messenger app (e.g., LINE or WhatsApp). The user can download the received stamp image and use it in the messenger app.

[1575] As a concrete example, consider the case where a user uploads a photo of a friend and converts the friend's face into a LINE stamp. The user selects the photo in the LINE app on their smartphone and uploads it to the system. The server uses OpenCV to detect faces in the received photo and Dlib to extract facial features. It then uses GAN to convert the friend's face into a stamp and sends the generated stamp to the user as a LINE message. The user can use the received stamp in LINE chats.

[1576] An example of a prompt is, "Upload a photo of a friend and turn their face into a LINE stamp."

[1577] The system allows users, without any specialist knowledge or skills, to convert their friends' faces into stamps and other digital content that can be shared quickly and easily.

[1578] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1579] Step 1:

[1580] A user uploads a photo of a friend.

[1581] Specifically, the user uses a terminal (smartphone or PC) to access the system's web interface or a dedicated application. The user clicks the photo upload button and selects a friend's photo file. After selecting, the user presses the upload button, and the photo is sent to the server.

[1582] Input: A photo file selected by the user

[1583] Output: Photo data sent to the server

[1584] Step 2:

[1585] The server receives the photo and performs face detection.

[1586] Specifically, the server receives the photo sent by the user and temporarily stores it. Then, it detects the face in the photo using OpenCV's Haar Cascade and Dlib's face detection function. Once face detection is complete, the server obtains the face coordinate information.

[1587] Input: Photo data sent by the user

[1588] Output: Face coordinate information

[1589] Step 3:

[1590] The server extracts facial features.

[1591] Specifically, the server uses Dlib's facial landmark detection feature to extract facial landmark points, which allow detailed facial features to be identified, and these points are used in the next processing step.

[1592] Input: Face coordinate information

[1593] Output: Facial landmark points

[1594] Step 4:

[1595] The server generates a face image for the stamp.

[1596] Specifically, the server uses a generative AI model (for example, a Generative Adversarial Network (GAN)) to generate a facial image for the stamp. The GAN model uses the extracted landmark points as input to generate a facial image for the stamp. The generated facial image is saved in a format that the user can use in messenger apps, etc.

[1597] Input: Facial landmark points

[1598] Output: Face image for stamp

[1599] Step 5:

[1600] The server provides the generated stamp image to the user.

[1601] Specifically, the server sends the generated stamp image to the user via email or a messenger app (e.g., LINE or WhatsApp). The user can download the received stamp image and use it in the messenger app.

[1602] Input: Generated stamp image

[1603] Output: Stamp image sent to user

[1604] (Application example 2)

[1605] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1606] In autonomous vehicles, there is a demand for providing passengers with personalized entertainment and information. However, with conventional systems, it is difficult to provide content that reflects the individual characteristics of each passenger, which makes it difficult to increase passenger satisfaction.

[1607] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1608] In this invention, the server includes a means for receiving an image uploaded by a user, a means for generating an avatar for the infotainment system of the autonomous vehicle based on the image, and a means for displaying the generated avatar on a display inside the vehicle, thereby generating a personalized avatar based on the passenger's face, and providing customized entertainment and information to the passenger.

[1609] "User" refers to an individual or organization that uses the System.

[1610] "Upload" refers to the act of a user sending data from their device to a server.

[1611] "Image" refers to visual information stored in digital format.

[1612] An "autonomous vehicle" refers to a vehicle that drives autonomously using artificial intelligence and sensor technology.

[1613] An "infotainment system" refers to an in-car system that integrates information and entertainment.

[1614] An "avatar" refers to a digital character that reflects the characteristics of a user.

[1615] "Generation" refers to the process by which a system creates new data based on input data.

[1616] "Display" refers to a device for displaying visual information.

[1617] A "machine learning algorithm" refers to a computational technique for learning patterns from data and making predictions or classifications.

[1618] "Features" refer to important information extracted from images or data.

[1619] "Entertainment" refers to content that provides enjoyment and entertainment to users.

[1620] "Information" refers to data and knowledge that is useful to the user.

[1621] "Customization" refers to adjustment to the needs and preferences of an individual user.

[1622] A system for implementing the invention includes means for receiving an image uploaded by a user, means for generating an avatar for an infotainment system of an autonomous vehicle based on the image, and means for displaying the generated avatar on an in-vehicle display.

[1623] System Programming

[1624] The system operates as follows.

[1625] 1. User uploads an image:

[1626] Users upload images from their devices to the system, which transmits them to a server and stores them for processing.

[1627] 2. The server processes the image:

[1628] The server uses machine learning algorithms to extract facial features from the received images: it uses OpenCV to detect faces and Keras to extract features using a pre-trained facial feature extraction model.

[1629] 3. Create an avatar:

[1630] Based on the extracted features, a generative AI model is used to generate an avatar, which is a digital character that reflects the user's facial features.

[1631] 4. Display on the display:

[1632] The generated avatar will be displayed on the autonomous vehicle's infotainment system display, allowing passengers to enjoy a personalized avatar based on their own face.

[1633] Hardware and Software Used

[1634] Hardware:

[1635] Cameras in self-driving vehicles: Used to capture passengers' faces.

[1636] Display: Used to display the generated avatar.

[1637] software:

[1638] OpenCV: Used for face detection.

[1639] Keras: Used to run the face feature extraction model.

[1640] PIL (Python Imaging Library): Used to process the generated avatar as an image.

[1641] Examples

[1642] For example, when a passenger enters the vehicle, a camera will automatically capture their face and the system will generate an avatar based on that image, which will then be displayed on the vehicle's display to provide customized entertainment and information to the passenger.

[1643] Examples of prompt statements

[1644] "Develop a system that takes a picture of a passenger's face and generates a personalized avatar based on that face. The avatar will be used by the infotainment system inside the car."

[1645] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1646] Step 1:

[1647] A user uploads an image.

[1648] A user selects an image from his / her device and uploads it to the system. The input is the image file stored on the user's device, and the output is the image data sent to the server. The server temporarily stores the received image data.

[1649] Step 2:

[1650] The server processes the images.

[1651] The server performs face detection on the received image data using OpenCV. The input is the uploaded image data, and the output is the area information of the detected face. Specifically, the image is converted to grayscale and a face detection algorithm is applied to identify the position of the face.

[1652] Step 3:

[1653] The server extracts facial features.

[1654] The server extracts features from the face detection results using a face feature extraction model pre-trained with Keras. The input is face region information, and the output is a face feature vector. Specifically, the detected face region is resized and input to the model to obtain a feature vector.

[1655] Step 4:

[1656] The server generates the avatar.

[1657] The server generates an avatar using a generative AI model based on the extracted facial feature vector. The input is the facial feature vector, and the output is the generated avatar image. Specifically, the feature vector is input into the generative AI model to generate the avatar image.

[1658] Step 5:

[1659] The server displays the avatar on the display.

[1660] The server transmits the generated avatar image to the display of the infotainment system of the autonomous vehicle. The input is the generated avatar image, and the output is the avatar displayed on the display. Specifically, the server converts the avatar image into a format suitable for the display and transmits a signal to display it on the display in the car.

[1661] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1662] "Example 1"

[1663] In one embodiment of the present invention, a system is provided that includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp, a cartoon or a movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, and an emotion engine for recognizing the emotion of the user. The emotion engine recognizes the emotion from the user's facial expression, tone of voice, text input, etc., and adjusts the generation of the personalized digital gift based on the emotion. For example, if it is recognized that the user is happy, the system generates a personalized digital gift that expresses joy.

[1664] "Example 2"

[1665] In another embodiment of the present invention, a user uploads a photo of a friend to the system, and the system uses a machine learning algorithm to extract the friend's facial features. The system then converts the person's face on the stamp into the friend's face based on the extracted features. Furthermore, the system recognizes the user's emotion with an emotion engine and adjusts the generated personalized digital gift based on the emotion. For example, if the user is recognized as sad, the system generates a personalized digital gift with a comforting message.

[1666] The process flow of each embodiment will be described below.

[1667] "Example 1"

[1668] Step 1: A user uploads a photo to the system.

[1669] Step 2: The system receives the uploaded photo and extracts the person's facial features from the photo.

[1670] Step 3: Based on the extracted features, the system converts the faces and dialogue of people in messenger app stickers, comic book and movie scenes.

[1671] Step 4: The system uses the emotion engine to recognize the user's emotion.

[1672] Step 5: The system tailors the generated personalized digital gift based on the recognized emotion and provides it to the user.

[1673] "Example 2"

[1674] Step 1: A user uploads a photo of their friend into the system.

[1675] Step 2: The system receives the uploaded photo and uses machine learning algorithms to extract your friend's facial features.

[1676] Step 3: The system converts the face of the person in the stamp into the face of a friend based on the extracted features. Step 4: The system uses an emotion engine to recognize the user's emotion.

[1677] Step 5: The system tailors the generated personalized digital gift based on the recognized emotion and provides it to the user.

[1678] Example 1

[1679] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1680] Conventional digital gift generation systems provide uniform content without considering the user's emotions, making it difficult to provide a personalized experience. In addition, conversion processing based on images uploaded by users was limited, making it difficult to convert into various formats such as messenger app stamps, comics, and video scenes.

[1681] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a face or dialogue of a person in a communication application stamp, a cartoon or a video scene based on the image, a means for providing the conversion result as a personalized digital gift, and an emotion engine for recognizing the user's emotion. This makes it possible to generate a personalized digital gift based on the user's emotion.

[1682] "User" refers to any person who utilizes the System to upload an image and receive a personalized digital gift.

[1683] "Image" refers to visual data such as photographs or drawings uploaded by users.

[1684] "Communication application" refers to software that enables users to send and receive messages, such as a messenger application or chat application.

[1685] "Stamps" refer to small images or icons used within communication applications.

[1686] "Manga" refers to a series of pictures or illustrations that visually depict a story.

[1687] A "film scene" refers to a portion of a visual medium such as a film or video.

[1688] "Conversion" refers to the process of changing original images or data into a different format or content.

[1689] "Personalized digital gifts" refers to individually customized digital content generated based on a user's image and emotions.

[1690] An "emotion engine" refers to software or algorithms that recognize emotions from a user's facial expressions, tone of voice, text input, etc.

[1691] A "machine learning algorithm" is a mathematical model that learns patterns from data and makes predictions or classifications.

[1692] The present invention is a system for generating and providing a personalized digital gift based on an image uploaded by a user. The system includes a means for a user to upload an image, a means for analyzing and transforming the image, a means for providing the transformation result as a personalized digital gift, and an emotion engine for recognizing the user's emotion.

[1693] Hardware and Software Used

[1694] Hardware

[1695] Server: Receives, analyzes, transforms images, recognizes emotions, and generates and delivers digital gifts.

[1696] Device: The device (computer, smartphone, tablet, etc.) through which a user uploads images.

[1697] software

[1698] Website or application: Provides an interface for users to upload images.

[1699] Machine learning algorithms used to extract features from images and perform transformations (e.g. OpenCV, Dlib, TensorFlow, PyTorch).

[1700] Emotion engine: Software for recognizing user emotions (e.g. Emotion API).

[1701] Generative AI models: Models for image transformation and dialogue generation (e.g., StyleGAN, GPT-3).

[1702] Mail Server and Messenger API: Used to send the generated digital gifts to users.

[1703] Examples

[1704] A user accesses a website and uploads an image of their own smiling face. The server receives the image and uses a machine learning algorithm to extract facial features. Next, the emotion engine recognizes the user's smile as "happiness." Based on this information, the server generates a stamp for a communication application that uses the user's face to express happiness. Finally, the generated stamp is sent to the user's email address.

[1705] Examples of prompt statements

[1706] "Generate stickers for a communication application that express joy based on images uploaded by users. The user's emotions are recognized from their smiles."

[1707] In this way, it is possible to generate and offer personalized digital gifts based on the user's emotions.

[1708] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1709] Step 1: User uploads an image

[1710] A user uploads his / her own image through a website or application by clicking an upload button and selecting an image file from a file selection dialog. The input is the user's image file, and the output is the image data sent to the server.

[1711] Step 2: The server receives the image

[1712] The server receives the image sent by the user and saves it in the database. Specifically, it receives the HTTP request, saves the image file in a temporary directory, and then saves it in the database. The input is the image data sent by the user, and the output is the path of the saved image file.

[1713] Step 3: The server analyzes the image.

[1714] The server uses machine learning algorithms to extract facial features from images. Specifically, it uses OpenCV and Dlib to perform face detection and identify the coordinates of facial landmarks (eyes, nose, mouth, etc.). The input is the path to the saved image file, and the output is the facial feature data.

[1715] Step 4: The server recognizes the emotion.

[1716] The server uses the emotion engine to recognize the user's emotion from the image. Specifically, it sends facial feature data to the Emotion API and classifies the emotion. The input is the facial feature data, and the output is the recognized emotion data.

[1717] Step 5: The server performs the conversion process.

[1718] The server performs conversion processing using a generative AI model based on the extracted facial features and recognized emotions. Specifically, it uses StyleGAN to convert the user's face into a stamp for a communication application, and uses GPT-3 to generate dialogue that matches the emotion. The input is facial feature data and emotion data, and the output is the converted stamp and dialogue data.

[1719] Step 6: The server generates the personalized digital gift.

[1720] The server generates a personalized digital gift based on the converted results. Specifically, it creates a digital card that includes a stamp with the user's face and a message that matches the user's emotion. The input is the converted stamp and conversation data, and the output is the generated digital gift.

[1721] Step 7: The server sends the digital gift.

[1722] The server sends the generated personalized digital gift to the user via email or a communication application. Specifically, the server uses the SMTP protocol or a messenger API to send the digital gift. The input is the generated digital gift and the output is the digital gift sent to the user.

[1723] (Application example 1)

[1724] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1725] In the conventional personalized digital gift generation system, since the conversion is performed uniformly without considering the user's emotions, it is not possible to provide an appropriate digital gift according to the user's emotions. In addition, if the generated digital gift does not match the user's emotions, there is a problem that the user's satisfaction decreases. Furthermore, since there is a lack of means to effectively deliver the generated digital gift, it is difficult to deliver it to the user quickly and reliably.

[1726] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1727] In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or a cartoon or movie scene based on the photo, a means for providing the conversion result as a personalized digital gift, a means including an emotion engine for recognizing the emotion of the user, a means for adjusting the generation of the personalized digital gift based on the emotion, and a means for sending the generated personalized digital gift to the user via email or a messenger app. This makes it possible to generate and quickly deliver a personalized digital gift appropriate to the emotion of the user.

[1728] "Means for receiving photos uploaded by users" refers to a function that enables users to send their own photos to the system via the Internet and for the server to receive those photos.

[1729] "Messenger app stamps" are images or icons used within messenger applications to allow users to express emotions or messages.

[1730] "Means of transforming the faces and dialogue of characters in manga or movie scenes" refers to technology that replaces the faces and dialogue of characters appearing in manga or movie scenes with the user's face and dialogue based on photos uploaded by the user.

[1731] A "personalized digital gift" is a digital gift that is customized based on a user's individual characteristics and emotions.

[1732] An "emotion engine" is software that recognizes emotions from a user's facial expressions, tone of voice, text input, etc., and adjusts the system's operation based on those emotions.

[1733] A "machine learning algorithm" is a computational technique for learning patterns based on large amounts of data and making predictions and classifications.

[1734] "Means of sending via email or messenger app" refers to a function for sending the generated personalized digital gift to a user via email or messenger application.

[1735] An embodiment of the present invention will be described. First, a user uses a terminal such as a smartphone or a personal computer to access a dedicated application or website. The user sends a photo through an interface for uploading the user's own photo. The photo is received by a server.

[1736] The server uses a machine learning algorithm to extract features from the received photos. The algorithm uses image processing libraries such as OpenCV to detect facial feature points and analyze the shape and expression of the face. Then, based on the extracted features, it converts the faces and dialogue of people in messenger app stamps, comics, and movie scenes. For the conversion, it uses technologies such as FaceSwap and DeepFake.

[1737] In addition, the server recognizes the user's emotions using an emotion engine. The emotion engine uses software such as EmotionRecognizer to analyze emotions from the user's facial expressions, tone of voice, text input, etc. Based on the recognized emotions, the server adjusts the generation of personalized digital gifts. For example, if the user is happy, the server generates a digital gift that expresses joy.

[1738] The generated personalized digital gift is then sent to the user from the server via email or a messenger app, using a communication library such as MessagingService.

[1739] For example, if a user uploads a photo of themselves feeling happy, the system will recognize that emotion and edit the character in the movie scene to make them look happy. The generated video will be sent to the user's email address.

[1740] An example of a prompt sentence might be:

[1741] "Create an application that converts a character's face in a movie scene into the user's face based on a photo uploaded by the user, and edits the scene based on the user's emotions. Please also include a function to send the generated video to the user."

[1742] In this way, personalized digital gifts appropriate to the user's emotions can be generated and quickly delivered.

[1743] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1744] Step 1:

[1745] The user uses the device to access a dedicated application or website and uploads their own photo.

[1746] Input: User photo

[1747] Output: The user's photo sent to the server.

[1748] How it works: A user selects a photo through the application interface and clicks the upload button, and the photo is sent to the server over the Internet.

[1749] Step 2:

[1750] Based on the photos received by the server, a machine learning algorithm is used to extract features from the photos.

[1751] Input: The user's photo sent to the server

[1752] Output: Extracted facial feature data

[1753] Specific operation: The server uses image processing libraries such as OpenCV to detect facial feature points from photos and analyze facial shape and facial expressions.

[1754] Step 3:

[1755] Based on the extracted features, the server converts the faces and dialogue of people in messenger app stamps, manga, and movie scenes.

[1756] Input: Extracted facial feature data

[1757] Output: Converted character faces and dialogue

[1758] How it works: The server uses techniques such as FaceSwap and DeepFake to swap the user's face with that of a character in a movie scene and adapts the dialogue to match the user's characteristics.

[1759] Step 4:

[1760] The server recognizes the user's emotions using an emotion engine.

[1761] Input: User photo

[1762] Output: Recognized emotion data

[1763] How it works: The server uses software such as EmotionRecognizer to analyze emotions from the user's facial expressions, tone of voice, text input, etc.

[1764] Step 5:

[1765] The server coordinates the generation of the personalized digital gift based on the recognized emotion.

[1766] Input: Recognized emotion data, transformed character faces and dialogue

[1767] Output: Personalized digital gifts tailored based on emotions

[1768] Specific operation: The server generates digital gifts that reflect emotions such as joy or sadness according to the recognized emotions.

[1769] Step 6:

[1770] The server then sends the generated personalized digital gift to the user via email or a messenger app.

[1771] Input: Personalized digital gifts tailored based on emotions

[1772] Output: A personalized digital gift sent to the user.

[1773] Specific operation: The server uses a communication library such as MessagingService to send the generated digital gift to the user's email address or messenger app.

[1774] Example 2

[1775] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1776] In conventional systems, it was difficult to provide personalized digital gifts that took into account the user's emotions when converting stamps in messenger apps and faces and dialogues in visual media based on photos uploaded by the user. In addition, there was a lack of means to appropriately send the generated digital gifts to the user. This limited the user experience and made it difficult to provide services that met individual needs.

[1777] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1778] In this invention, the server includes a means for receiving an image uploaded by a user, a means for converting a face or dialogue of a person in a messenger app stamp or visual media based on the image, a means for providing the conversion result as a personalized digital gift, a means for recognizing a user's emotion and adjusting the personalized digital gift based on the emotion, and a means for sending the personalized digital gift to the user, thereby enabling the generation and sending of a personalized digital gift according to the user's emotion.

[1779] "User" means any individual or entity that utilizes the System to upload images and receive personalized digital gifts.

[1780] An "image" is data containing visual information, such as a photograph or drawing, that a user uploads to the system.

[1781] A "messenger app" is a software application that enables users to exchange messages, including text, images, and audio, with other users.

[1782] "Stamps" are small images used within messenger apps that express specific facial expressions or characters.

[1783] "Visual media" refers to media content that includes visual elements, such as manga and movies.

[1784] A "machine learning algorithm" is a computational technique for learning patterns from data and making predictions or classifications.

[1785] A "feature" is an identifiable element in an image, such as a facial contour, eyes, nose, or mouth.

[1786] A "personalized digital gift" is a digital gift that is individually generated based on a user's input and emotions.

[1787] An "emotion engine" is software or algorithms for analyzing and recognizing a user's emotions.

[1788] An "electronic means of communication" is any means for sending or receiving information in digital form, such as email or a messaging app.

[1789] This invention is a system that converts the faces and dialogues of people in stamps and visual media of messenger apps based on images uploaded by users and provides them as personalized digital gifts. A specific embodiment of this system will be described below.

[1790] System Configuration

[1791] Hardware

[1792] Server: A computer system with powerful processors and large memory capacity that runs image processing and machine learning algorithms.

[1793] Terminal: A smartphone or PC used by a user to upload images and receive generated digital gifts.

[1794] software

[1795] Image processing library: Software for pre-processing and transforming images, such as OpenCV or Adobe Photoshop API.

[1796] Machine learning algorithms: Algorithms for extracting features from images and performing transformations, such as TensorFlow.

[1797] Emotion engine: Software for recognizing user emotions, such as IBM Watson or Microsoft Azure Emotion API.

[1798] Electronic communication means: Means for sending the generated digital gift to the user, such as SMTP protocol or messenger API (e.g. Facebook Messenger API).

[1799] System Operation

[1800] Uploading an image

[1801] Users use their devices to upload pictures of their friends to the system, either through the system's web interface or a dedicated app.

[1802] Image preprocessing and feature extraction

[1803] The server receives the uploaded image, first pre-processes the image using OpenCV, then uses TensorFlow's face recognition model to extract the facial features of the friend. In this feature extraction process, the main feature points such as the facial contour, eyes, nose, and mouth are identified.

[1804] Stamp Generation

[1805] The server uses Adobe Photoshop API to generate stamps based on the extracted facial features, adjusting the facial contours and facial expressions to fit the stamp design.

[1806] Emotion recognition and digital gift adjustment

[1807] The server uses IBM Watson to analyze the user's emotions, analyzing facial expressions and text inputs while the user is using the system. For example, if the server detects that the user is sad, it will generate a personalized digital gift with a comforting message.

[1808] Sending digital gifts

[1809] The server sends the generated personalized digital gift to the user via email or messenger app using SMTP protocol or messenger API. The user can view the received digital gift and share it with friends.

[1810] Examples

[1811] As a concrete example, consider a case where a user uploads a photo of a friend and converts the friend's face into a sticker in a messenger app.

[1812] 1. The user opens the dedicated app on their smartphone and selects and uploads photos of their friends.

[1813] 2. The server receives the uploaded photo, pre-processes the image using OpenCV, and then uses TensorFlow's face recognition model to extract the friend's facial features.

[1814] 3. The server uses Adobe Photoshop API to generate stamps based on the extracted facial features, adjusting the facial contours and facial expressions to fit the stamp design.

[1815] 4. The server uses IBM Watson to analyze the user's emotions and, if the user is recognized as sad, generates a personalized digital gift with a comforting message.

[1816] 5. The server sends the generated personalized digital gift to the user through the Facebook Messenger API. The user can view the received digital gift and share it with their friends.

[1817] Examples of prompt statements

[1818] Upload a photo of your friend and turn their face into a sticker. Generate and send personalized digital gifts based on your user's emotions.

[1819] By inputting this prompt sentence into the generative AI model, the system will carry out the above process and provide the user with an original stamp.

[1820] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1821] Step 1:

[1822] A user uses a terminal to upload a friend's image to the system. The user opens the system's web interface or a dedicated app, selects an image file, and presses the upload button. The input is the image file selected by the user, and the output is the image data sent to the server.

[1823] Step 2:

[1824] The server receives the uploaded image. The received image data is first preprocessed using an image processing library (e.g., OpenCV). Preprocessing includes image resizing, noise removal, color correction, etc. The input is the image data sent by the user, and the output is the preprocessed image data.

[1825] Step 3:

[1826] The server inputs the preprocessed image data into a machine learning algorithm (e.g., TensorFlow's facial recognition model) to extract facial features. The feature extraction process identifies key features such as the facial contour, eyes, nose, and mouth. The input is the preprocessed image data, and the output is the extracted facial feature data.

[1827] Step 4:

[1828] The server generates a stamp using an image processing library (e.g., Adobe Photoshop API) based on the extracted facial feature data. Specifically, it adjusts the facial contours and facial expressions to fit the stamp design. The input is the facial feature data, and the output is the generated stamp image.

[1829] Step 5:

[1830] The server recognizes the user's emotions using an emotion engine (e.g., IBM Watson). It analyzes emotions from facial expressions and text inputs while the user is using the system. The input is the user's facial expression data and text data, and the output is the recognized emotion data.

[1831] Step 6:

[1832] The server tailors the personalized digital gift including the generated stamp based on the recognized emotion data, e.g., if the user is recognized as sad, it generates a digital gift including a comforting message, where the input is the emotion data and the stamp image, and the output is the tailored personalized digital gift.

[1833] Step 7:

[1834] The server sends the generated personalized digital gift to the user via email or a messenger app using the SMTP protocol or a messenger API (e.g., Facebook Messenger API). The input is the personalized digital gift and the output is the digital gift sent to the user.

[1835] (Application example 2)

[1836] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1837] In conventional messenger apps and video content, it was difficult for users to generate personalized characters and stamps based on their friends' photos. In addition, there was a lack of functionality to adjust content according to the user's emotions, so there was a need to improve the user experience. Furthermore, there were limited means to effectively provide the generated content.

[1838] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1839] In this invention, the server includes a means for receiving a photo uploaded by a user, a means for converting the face and dialogue of a character of a messenger app stamp or video content based on the photo, a means for providing the conversion result as a personalized digital gift, and a means for recognizing the emotion of the user and adjusting the personalized digital gift based on the emotion. This allows the user to easily generate a personalized character or stamp based on a friend's photo, and further allows the content to be adjusted according to the user's emotion. In addition, the generated content can be effectively provided.

[1840] "User" means an individual who utilizes the System to upload photos and receive personalized digital gifts.

[1841] A "photo" is image data uploaded by a user, which includes features such as a friend's face.

[1842] "Messenger app stamps" are digital stamps used within messenger apps and are generated based on photos uploaded by users.

[1843] "Video content characters" are people or characters that appear in videos, whose faces and dialogue are transformed based on photos uploaded by users.

[1844] "Dialogue" refers to conversations or statements made by characters in video content or messenger apps.

[1845] "Personalized Digital Gifts" are individualized digital content that is generated based on photos and emotions uploaded by users.

[1846] A "machine learning algorithm" is an algorithm for extracting features from photographs and transforming people's faces and dialogue based on those features.

[1847] "Means for recognizing emotions" refers to technology for analyzing a user's emotions and adjusting content based on those emotions.

[1848] The "means of provision" refers to the means of sending the generated personalized digital gift to the user via email or a messenger app.

[1849] The system for implementing the present invention receives a photo uploaded by a user, converts the face and dialogue of a character in a messenger app stamp or video content based on the photo, and provides a generated personalized digital gift. It also includes a function of recognizing the user's emotions and adjusting the personalized digital gift based on the emotions.

[1850] Hardware and Software Used

[1851] Hardware: Smartphone, head-mounted display

[1852] Software: OpenCV, Dlib, EmotionRecognizer (emotion recognition library), VideoEditor (video editing library)

[1853] System Configuration

[1854] 1. User terminal: A device that users use to upload photos. This includes smartphones and head-mounted displays.

[1855] 2. Server: Receives the uploaded photos, uses machine learning algorithms to extract facial features, and processes them to transform the faces and dialogue of characters in the video content.

[1856] 3. Emotion Recognition Engine: Analyzes user emotions and tailors personalized digital gifts based on those emotions.

[1857] 4. Delivery method: The generated personalized digital gift is sent to the user via email or messenger app.

[1858] Data processing and calculation

[1859] 1. Facial feature extraction: The server extracts facial features from the uploaded photo using OpenCV and Dlib, which obtains the facial landmark points in the photo.

[1860] 2. Emotion Recognition: The server recognizes the user's emotion using EmotionRecognizer, which analyzes the user's emotional state when he uploads a photo.

[1861] 3. Video Editing: The server uses VideoEditor to edit the video content based on the extracted facial features and the recognized emotions, so that the faces and dialogues of the characters in the video are transformed into the faces of the user's friends and the content is adjusted according to their emotions.

[1862] E...

Claims

1. a means for receiving a facial photograph uploaded by a user; means for extracting facial features from said facial photograph using a machine learning algorithm; A means for generating a prompt sentence for instructing the generation of a digital gift in which a face of a messenger app stamp is converted into a face of the face photograph based on the extracted facial features, or a digital gift in which a face of a person in a scene of a designated video work is converted into a face of the face photograph; A means for generating a personalized digital gift based on the face of the received face photograph using the generated prompt sentence and the generative AI model; A providing means for providing the generated digital gift as a personalized digital gift; A system including:

2. The system of claim 1 , wherein the providing means sends the generated personalized digital gift to the designated user via email or a messenger app.

3. A means for recognizing an emotional state of a user from the user's facial expression, tone of voice, or input text information; means for adjusting the generated digital gift based on the estimated emotional state of the user; The system of claim 1 further comprising:

Images