system

Abstract

Provide a system. [Solution] A means for a user to upload a photo using a terminal; a means for the device to preprocess the photo; a server receiving the preprocessed photograph and analyzing the photograph using an AI model; A means for automatically generating stamps based on the analysis results; means for returning the generated stamp to the terminal; The system includes means for the terminal to register the returned stamp with the communication application.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Today, users want to create original LINE stickers using their own photos. However, traditional methods require specialized image editing software and skills to create stickers, which is time-consuming and makes it difficult to protect privacy. Furthermore, these sticker creation tools are not user-friendly, making them difficult for average users to use. Given this background, there is a need for a system that allows users to easily generate LINE stickers from photos, protects privacy, and is easy for anyone to use. [Means for solving the problem]

[0005] The present invention provides a system including a means for a user to upload a photo using a terminal, a means for the terminal to preprocess the photo, a means for a server to receive the preprocessed photo and analyze the photo using an AI model, a means for automatically generating stamps based on the analysis results, a means for returning the generated stamps to the terminal, and a means for the terminal to register the returned stamps in a communication application.

[0006] Specifically, the device resizes, crops, and filters the photo, and also masks the face to protect privacy. The server selects the content of the stamp based on the specified theme and automatically adds text. This system allows users to easily and quickly create and use original LINE stamps without specialized skills or significant effort. In addition, the privacy protection function allows users to use the service with peace of mind.

[0007] "Photo" refers to image data taken by a user, typically acquired using a digital camera or smartphone.

[0008] A "terminal" refers to an electronic device used by a user, such as a computer or smartphone, and is a hardware device on which applications run.

[0009] "Upload" refers to the operation of a user sending photo data from their own device to a server.

[0010] "Preprocessing" refers to a series of steps taken to prepare photo data for analysis and use, including operations such as resizing, cropping, and filtering.

[0011] A "server" refers to a computer system that communicates with terminals via a network and provides data processing and services.

[0012] "AI model" refers to a computational model that uses artificial intelligence technology to learn and recognize patterns, and is used for photo analysis and stamp generation.

[0013] "Analysis" refers to the process of extracting and identifying specific features and information from photo data.

[0014] "Stamp" refers to a graphic item for use within a communication application, including decorations and messages based on user-generated data.

[0015] "Return" refers to the operation of sending the stamp data generated by the server back to the original terminal.

[0016] A "communication application" refers to software that allows users to send and receive messages and use stamps, and a typical example is a messaging application.

[0017] "Registering" refers to the operation of adding a generated stamp to a stamp library within a communication application. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention is a system that allows users to easily create LINE stamps from their own photos, and is realized by sequentially processing the photos: uploading, preprocessing, analyzing, stamp generation, and stamp usage. The specific details of each processing step are described below.

[0040] User operation

[0041] The device used by the user is an electronic device such as a smartphone or tablet. The user first launches the LINE stamp generation application, follows the instructions of the application to select a photo they have on hand, and uploads it to the server.

[0042] Terminal side processing

[0043] The device receives the photos uploaded by the user and performs the following preprocessing:

[0044] 1. Resize: Change the resolution of the photo to a size suitable for analysis or use.

[0045] 2. Crop: Cut out unnecessary parts of the photo and leave only the parts suitable for stamping.

[0046] 3. Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[0047] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[0048] Server-side processing

[0049] The server receives the pre-processed photo data from the device and analyzes it using the AI ​​model:

[0050] 1. Feature extraction: The AI ​​model extracts features from the photo (e.g., smile, pose).

[0051] 2. Topic response: Identify elements related to a given topic (e.g., "Holidays").

[0052] 3. Stamp generation: Based on the extracted features and the theme, an appropriate template is selected and a stamp is generated.

[0053] 4. Add text: Automatically add text that matches the theme (e.g., "It's fun!") to the stamp.

[0054] The generated stamp is sent back to the terminal from the server.

[0055] Registering stamps on the device

[0056] The device registers the received stamp data in the LINE app:

[0057] 1. Data reception: The terminal receives the generated stamp data from the server.

[0058] 2. Registration process: Add stamps to the LINE application.

[0059] This allows users to easily use LINE stamps.

[0060] Specific examples

[0061] Consider a case where a user goes on a family picnic on the weekend and uploads photos of the picnic to a LINE sticker generation app. The user selects a picnic photo from the smartphone gallery, and the app resizes, crops, filters, and blurs faces before sending it to the server.

[0062] The server uses an AI model to analyze the photo, extracting features such as a "smile" or "picnic scene," and generates stamps that correspond to the theme of "holidays." The stamps are accompanied by text such as "That's fun!"

[0063] The generated stamp set is sent back to the user's smartphone from the server and automatically registered in the LINE app, allowing the user to immediately use the original stamps in LINE chats with family and friends.

[0064] In this way, the present invention provides a system that allows users to easily create and use original LINE stamps. With privacy protection features and automated analysis, the system can be easily used by many more users.

[0065] The processing flow will be explained below.

[0066] Processing step details

[0067] Step 1:

[0068] The user launches the LINE stamp generation application on their smartphone and selects a photo from their gallery, then taps the "Upload" button on the app.

[0069] Step 2:

[0070] The device receives the photo selected by the user and resizes the photo to a resolution suitable for analysis and use.

[0071] Step 3:

[0072] The device will trim the background of the photo to extract only the parts suitable for making a stamp, and adjust the brightness and contrast of the image using filtering if necessary.

[0073] Step 4:

[0074] Using the device's AI technology, it automatically detects faces in photos and applies a mosaic process to protect privacy, making it difficult to identify individuals.

[0075] Step 5:

[0076] The device sends the preprocessed photo data to the server, which receives the data and uses an AI model to analyze the photo's features.

[0077] Step 6:

[0078] The server uses the analysis results to identify elements related to the specified topic (e.g., "holidays") and selects an appropriate stamp template based on the features most relevant to the topic.

[0079] Step 7:

[0080] The server automatically generates stamps based on the characteristics and theme of the photo. The generated stamps reflect elements of the user's photo and include text appropriate to the theme (e.g., "That's fun!").

[0081] Step 8:

[0082] The server packages the generated stamp data and sends it back to the terminal, which receives the data.

[0083] Step 9:

[0084] The device registers the received stamp data in the LINE application. Once registration is complete, the new stamp set will be added to the user's LINE stamp library.

[0085] Step 10:

[0086] Users can open the LINE app and use the newly added stamps to send messages to friends and family in the chat screen. Users can use the generated stamps as they like.

[0087] Through this series of processes, users can easily convert their photos into LINE stickers and use them conveniently. The entire processing flow is automated, making it a user-friendly system that also protects privacy.

[0088] Example 1

[0089] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0090] Conventionally, it has been difficult for users to easily generate original stamps using their own photos and use them in messaging applications. Furthermore, due to a lack of privacy protection and image optimization processing when generating stamps, it has been difficult to easily create high-quality stamps. The present invention aims to solve these problems and provide a system that allows users to easily generate high-quality original stamps.

[0091] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0092] In this invention, the server includes means for receiving the preprocessed image and analyzing the image using a machine learning model, means for automatically generating an image based on the analysis result, and means for returning the generated image to the information device, thereby enabling users to easily generate high-quality original stamps and use them in messaging applications.

[0093] A "user" is an individual or legal entity that uses a particular application or system.

[0094] "Information devices" refers to a wide range of devices, including electronic terminals such as smartphones, tablets, and personal computers.

[0095] "Image" refers to visual data such as photographs, paintings, graphics, and illustrations.

[0096] "Preprocessing" refers to the process of converting images into a format suitable for analysis and use by performing operations such as resizing, cropping, filtering, and face masking.

[0097] "Computer system" refers to the entire system that processes and analyzes data, including servers and cloud-based computing resources.

[0098] A "machine learning model" refers to an algorithm that learns from large amounts of data and automatically recognizes specific patterns and features.

[0099] "Analysis" is the process of using machine learning models to identify and extract specific features and patterns.

[0100] "Generation" is the process of creating new images or information based on analyzed data.

[0101] A "messaging application" is an application that allows users to send and receive text, images, stamps, etc.

[0102] This invention relates to a system that allows users to easily create original stamps using their own images. The system employs a method in which users upload images, which are preprocessed on their devices, then analyzed and generated on a server, and the generated stamps are registered in a messaging application.

[0103] Hardware and software used

[0104] Users access the system using information devices such as smartphones and tablets, and use a LINE stamp generation application that runs on these devices. On the server side, a high-performance cloud infrastructure is used, and Tensorflow (registered trademark) and OpenCV are used for the machine learning models.

[0105] Specific processing

[0106] 1. Launch the app

[0107] The user launches the LINE stamp generation application on their smartphone or tablet and taps the app icon to launch the app.

[0108] 2. Select and upload photos

[0109] Users select the photo they want to use as a stamp from their device's gallery and tap the "Upload" button in the app to send the photo to the server.

[0110] 3. Photo preprocessing

[0111] The device pre-processes the photo before sending it to the server. Specifically, it does the following:

[0112] Resize: Reduce the image resolution to 800x800 pixels.

[0113] Crop: Cut out unnecessary background parts of the photo and center only the person.

[0114] Filtering: Increases image brightness by 10% and contrast by 15%.

[0115] Face Mask: Using AI technology, the face is automatically detected and a 10x10 pixel mosaic is applied.

[0116] 4. Data Transmission

[0117] The photos are pre-processed on the terminal and sent to a server via the Internet.

[0118] 5. Photo Analysis

[0119] The server analyzes the photo using machine learning models (e.g. TensorFlow, OpenCV). Specifically, it performs the following steps:

[0120] Feature extraction: Extracting photo features such as smiles and poses.

[0121] Topic-based: Identify elements related to a given theme (e.g., "Holidays").

[0122] 6. Stamp generation

[0123] A stamp is generated based on the extracted features and themes. For example, a stamp of a smiling person can be generated with the text "It's fun!"

[0124] 7. Data return

[0125] The generated stamp data is returned from the server to the terminal.

[0126] 8. Registering with the LINE app

[0127] The device will register the received stamps in the LINE application. Specifically, select "Stamp Management" from the LINE app's settings menu and then select "Add New Stamps."

[0128] Specific examples

[0129] For example, consider a scenario where a user wants to upload photos of a family picnic on the weekend to a LINE sticker generation app. The user selects a picnic photo from the smartphone gallery and taps the "Upload" button. The app resizes the photo (e.g., to 800x800 pixels), crops unnecessary background, increases brightness by 10%, increases contrast by 15%, and blurs the face (e.g., blurs the face by 10x10 pixels), before sending it to the server.

[0130] The server uses a generative AI model to analyze the photo, extracting features such as a "smile" or "picnic scene," and generates stamps that correspond to the theme "holiday." The stamps are accompanied by the text "It's fun!" The generated stamp set is then sent back from the server to the user's smartphone, which then registers the stamps in the LINE application. The user can then immediately use these original stamps in LINE chats with family and friends.

[0131] Examples of prompt statements

[0132] Prompt: "Generate a fun stamp related to a holiday picnic. The photo shows a smiling family."

[0133] Example output from the AI ​​model: "A stamp of a family having a picnic, with the text 'This is fun!'"

[0134] As described above, the present invention provides a system that allows users to easily create original stamps using their own images and use them in messaging applications.

[0135] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0136] Step 1: User launches app

[0137] The user launches the LINE stamp generation application on an information device such as a smartphone or tablet. The app launches when the user taps a specific icon. The input is the operation of the information device, and the output is the launch of the application.

[0138] Step 2: User selects and uploads photo

[0139] The user selects the image they want to use as a stamp from the photo gallery in the app. After selecting, they press the "Upload" button to send the image to the server. The input is the user's selected image and instructions to upload, and the output is the selected image data sent from the device to the server.

[0140] Step 3: Pre-processing photos on the device

[0141] The device receives the images uploaded by the user and performs pre-processing. The specific pre-processing operations include:

[0142] Resize: Reduce the resolution of an image to 800x800 pixels. Take the original high-resolution image as input and get the resized image as output.

[0143] Cropping: Cut out unnecessary background parts and leave important parts (such as people) in the center. The input is the entire image, and the output is the cropped image.

[0144] Filtering: Optimizes the brightness and contrast of an image. The input is the original image, and the output is the filtered image. Specifically, the brightness is increased by 10% and the contrast is increased by 15%.

[0145] Face mask: AI technology automatically detects the face and applies a 10x10 pixel mosaic process. The input is the original image, and the output is an image with the face mosaicked.

[0146] Step 4: Send data from the device to the server

[0147] The device sends the preprocessed image data to the server. The input is the preprocessed image, and the output is the image data sent to the server. The transmission occurs over a network connection.

[0148] Step 5: Photo analysis by the server

[0149] The server analyzes the preprocessed images using machine learning models (e.g., TensorFlow, OpenCV). Specific operations include:

[0150] Feature extraction: Extract image features such as smiles and poses. The input is a preprocessed image, and the output is feature data.

[0151] Topic matching: Identify elements related to a specified theme (e.g., "holidays"). The input is theme information and feature data, and the output is the corresponding elements.

[0152] Step 6: Server-generated stamps

[0153] The server generates stamps based on the extracted features and themes. Specific operations include:

[0154] Template Selection: Select an appropriate stamp template. The input is feature data and a topic, and the output is the selected template.

[0155] Add Text: Automatically add text that matches the theme (e.g., "It's fun!") to the stamp. The input is a template and text data, and the output is a completed stamp image.

[0156] Step 7: Sending data back from the server to the device

[0157] The server returns the generated stamp data to the user's terminal. There is the generated stamp data as input and the stamp data sent to the terminal as output.

[0158] Step 8: Register stamps via terminal

[0159] The device registers the received stamp with the messaging application. Specific actions include:

[0160] Data reception: Receives stamp data sent from the server. There is the sent stamp data as input and the received stamp data as output.

[0161] Registration process: Registers stamps in the LINE application. Specifically, select "Stamp Management" from the LINE app's settings menu and execute "Add New Stamp." The stamp data received is the input, and the registered stamp is displayed in the LINE app as the output.

[0162] This allows users to easily create original LINE stamps and use them in messaging applications.

[0163] (Application example 1)

[0164] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0165] To improve the user experience in virtual stores, a system is needed that allows users to easily create original virtual avatars that reflect their own characteristics and use them to enjoy shopping and interacting with other users. However, current systems require advanced expertise and a significant amount of time for users to create avatars from their own photos, making them difficult for average users to use. Another problem is that the environment for properly registering and using generated avatars in virtual stores is not yet in place.

[0166] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0167] In this invention, the server includes means for analyzing a photo using a generative AI model, means for automatically generating a virtual avatar based on the analysis results, and means for selecting content for the virtual avatar based on a specified prompt and automatically adding customization elements. This allows users to easily generate original virtual avatars from their own photos without specialized knowledge or skills and use the avatars in virtual stores.

[0168] "Device" means the electronic device used by a User to upload photos and receive / register generated virtual avatars.

[0169] "Photo preprocessing" is the process of preparing photos for analysis and avatar generation, including resizing, cropping, filtering, and face masking.

[0170] The "server" is a central processing unit that receives pre-processed photos, analyzes the photos using a generative AI model, and generates and returns a virtual avatar based on the analysis results.

[0171] A "generative AI model" is an artificial intelligence algorithm that analyzes information from a photograph, extracts features, and generates a virtual avatar.

[0172] A "virtual avatar" is a digital character generated by extracting features from a user's photograph, and is used by the user in a virtual store.

[0173] A "prompt" is an instruction used for analysis, a piece of text that tells the generative AI model how to analyze a photo and what specific features it has.

[0174] A "virtual store" is a virtual commercial space where users can shop and interact using virtual avatars they create.

[0175] "Customization elements" are decorative elements such as the appearance, belongings, and background of a virtual avatar, and are automatically added based on user specifications or prompts.

[0176] "Analysis results" are data obtained after a photo is analyzed by a generative AI model, and include information based on feature extraction and prompt text.

[0177] This invention is a system that allows users to easily create original virtual avatars from their own photographs and use those avatars in virtual stores. The specific system configuration and processing method will be described below.

[0178] Main configuration

[0179] 1. User Device

[0180] User devices can be smartphones, tablets, smart glasses, head-mounted displays, etc. User devices provide a means for users to take, select, and upload their own photos.

[0181] 2. Server

[0182] The server receives the uploaded photo, analyzes it using a generative AI model, generates a virtual avatar based on the analysis results, adds any necessary customization elements, and sends it back to the user's device.

[0183] 3. Generative AI Models

[0184] A generative AI model is an artificial intelligence algorithm that extracts features from a photo and generates a virtual avatar based on them, using machine learning frameworks such as TensorFlow and PyTorch.

[0185] Processing method

[0186] 1. Upload a photo

[0187] Users select photos from their device's camera or gallery and upload them to the server through the application, which then imports the user's photos into the system.

[0188] 2. Photo pre-processing

[0189] The device performs pre-processing on the uploaded photo, including resizing, cropping, filtering, and face masking, to prepare the photo for analysis and avatar generation.

[0190] 3. Server-side analysis

[0191] The server receives the pre-processed photos and performs feature analysis using a generative AI model, which extracts features such as smiles and poses from the photos.

[0192] 4. Virtual Avatar Generation

[0193] Based on the extracted features, the server automatically generates a virtual avatar, determines the avatar's content based on the specified prompt, and adds customization elements, such as background, clothing, and accessories.

[0194] 5. Return and register the generated virtual avatar

[0195] The generated virtual avatar is sent back from the server to the user's device, which then registers the avatar in a virtual store, allowing the user to use the avatar for shopping and interacting with other users.

[0196] Specific examples

[0197] For example, a user can upload a photo of themselves using a virtual shopping app. The app processes the photo, analyzes it with an AI model to extract features, and generates a virtual avatar that reflects the user's characteristics. This avatar can then be used in a virtual store, allowing the user to shop or participate in events.

[0198] An example of a specified prompt sentence is:

[0199] Extract features from a user's facial photo and provide the data needed to generate a virtual avatar. Prioritize analysis of smiles, movement direction, and facial features.

[0200] This allows users to easily create their own original virtual avatar and enjoy a rich user experience in a virtual store.

[0201] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0202] Step 1:

[0203] The user uploads a photo on the device. They select a photo from the device's camera or gallery and press the "Upload" button to import the photo into the application. The input is the photo file selected by the user, and the output is the photo file to be uploaded. With the user's operation, the photo is ready to be sent to the server.

[0204] Step 2:

[0205] The device preprocesses the photo. The input is the uploaded photo file. The device first resizes the photo. Next, it crops it to keep only the important elements. After that, it filters the photo to adjust the brightness and contrast, and blurs the face if necessary. The output is the preprocessed photo data.

[0206] Step 3:

[0207] The server receives the preprocessed photos. The input is the preprocessed photo data, and the server prepares the received data for analysis. The output is the photo data ready for analysis.

[0208] Step 4:

[0209] The server analyzes the photo using a generative AI model. The input is photo data that can be analyzed, and the generative AI model extracts features from the photo (e.g., smile, facial expression, pose). The output is the feature extraction results, i.e., prominent feature data obtained from the photo.

[0210] Step 5:

[0211] The server generates a virtual avatar based on the specified prompt. The input is the feature extraction results and the prompt. The content of the virtual avatar is determined based on the analysis results, and customization elements (e.g., clothing, background, accessories) are automatically added. The output is the generated virtual avatar. An example of a prompt is: "Please extract features from the user's facial photograph and provide the data necessary to generate a virtual avatar. Please prioritize analysis of smiles, movement direction, and facial features."

[0212] Step 6:

[0213] The server returns the generated virtual avatar to the user's device. The input is the generated virtual avatar, and the output is the avatar data returned to the user's device. The server ensures that the avatar is properly delivered to the user's device through the return process.

[0214] Step 7:

[0215] The user's device registers the returned virtual avatar in the virtual store. The input is the returned avatar data, and the output is the avatar information registered in the virtual store. The device registers the avatar using the virtual store's API, and the user is ready to use the avatar to act in the virtual store.

[0216] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0217] This invention is a system that allows users to easily generate LINE stamps from their own photos, recognizing and reflecting the user's emotions in the process. This is achieved by sequentially processing the photos: uploading, preprocessing, analysis, emotion recognition, stamp generation, and stamp usage. The specific details of each processing step are described below.

[0218] User operation

[0219] Users simply launch the LINE stamp generation application on their smartphone and follow the app's instructions to select a photo from their gallery and upload it to the server.

[0220] Terminal side processing

[0221] The device receives the photos uploaded by the user and performs the following preprocessing:

[0222] 1. Resize: Change the resolution of the photo to a size suitable for analysis or use.

[0223] 2. Crop: Cut out unnecessary parts of the photo and leave only the parts suitable for stamping.

[0224] 3. Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[0225] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[0226] Server-side processing

[0227] The server receives the uploaded photo and processes it as follows:

[0228] 1. Feature extraction: The server uses an AI model to extract features from the photo (e.g., smile, pose).

[0229] 2. Emotion Recognition: The emotion engine analyzes photos and recognizes the user's emotions. For example, it identifies emotions such as "happiness" or "surprise" based on facial expressions and posture.

[0230] 3. Theme matching: Identify elements related to the specified theme (e.g., "holidays"). The server selects an appropriate stamp template based on the features most relevant to the theme.

[0231] 4. Stamp generation: Stamps are automatically generated based on the extracted features and themes. The generated stamps reflect elements of the user's photo and include text appropriate to the theme (e.g., "It's fun!"). The design and text of the stamps are also adjusted based on the results of emotion recognition.

[0232] The generated stamp is sent back from the server to the user's terminal.

[0233] Registering stamps on the device

[0234] The device registers the received stamp data in the LINE app:

[0235] 1. Data reception: The terminal receives the generated stamp data from the server.

[0236] 2. Registration process: Add stamps to the LINE application.

[0237] This allows users to easily use LINE stamps.

[0238] Specific examples

[0239] Consider the case where a user uploads photos taken during a trip to a LINE stamp generation app. The user selects and uploads photos from their smartphone gallery. The device then resizes, crops, filters, and blurs faces before sending the photos to the server.

[0240] The server uses an AI model to analyze the photo and extract features such as a "smile" or "tourist spot pose." The emotion engine recognizes that the user in the photo is "feeling happy." The server generates a stamp corresponding to the theme "travel," and adds text such as "Great trip!" to the stamp. Emotion recognition allows the stamp design to have bright colors and a fun atmosphere.

[0241] The generated stamp set is sent back to the user's smartphone from the server and automatically registered in the LINE app, allowing the user to immediately use the original stamps in LINE chats with family and friends.

[0242] In this way, the present invention provides a system that allows users to easily create and use original LINE stamps. By using emotion recognition functionality to generate stamps that reflect the user's emotions, a more personalized experience is provided.

[0243] The processing flow will be explained below.

[0244] Processing step details

[0245] Step 1:

[0246] The user launches the LINE stamp generation application on their smartphone and selects a photo from their gallery, then taps the "Upload" button on the app.

[0247] Step 2:

[0248] After the device receives the selected photo from the user, it resizes the photo, changing the photo resolution to a size suitable for analysis and use. For example, a 5000x4000 pixel photo is resized to 1000x800 pixels.

[0249] Step 3:

[0250] Your device will trim away unwanted background from your photo, determine the area you want to use, and crop the photo so that only the subject remains, for example, cropping only around a person.

[0251] Step 4:

[0252] Your phone will apply filters to your photos, including adjusting brightness, contrast, and color tone, for example, brightening dark photos and enhancing contrast.

[0253] Step 5:

[0254] Using the device's AI technology, it automatically detects faces in photos and applies mosaic processing to protect privacy. For example, it applies a mosaic filter to all faces in the photo.

[0255] Step 6:

[0256] The device sends the pre-processed photo data to the server, which receives the data.

[0257] Step 7:

[0258] The server analyzes the photo using an AI model, extracting features (e.g., smile, pose) and identifying key elements of the photo.

[0259] Step 8:

[0260] The server's emotion engine further analyzes the photo to identify the user's emotion, for example, identifying "happiness" from the curvature of the smile and the degree of eye opening.

[0261] Step 9:

[0262] The server identifies elements related to the specified topic (e.g., "holidays") and selects appropriate stamp templates based on the features most relevant to the topic.

[0263] Step 10:

[0264] The server automatically generates stamps based on the user's features, theme, and recognized emotion. The generated stamps reflect elements of the user's photo and automatically add text according to the theme. The text and design are adjusted based on the results of emotion recognition.

[0265] Step 11:

[0266] The generated stamp data is packaged and sent back from the server to the terminal, which receives this data.

[0267] Step 12:

[0268] The stamp data received by the device is registered in the LINE application. The stamp is added to the LINE app and becomes available for use by users.

[0269] Step 13:

[0270] Users can open the LINE app and use the newly added stamps to send messages to friends and family in the chat screen. Users can use the generated stamps as they like.

[0271] Through this series of processes, users can easily create original LINE stamps and use personalized stamps when communicating with friends. The emotion recognition function ensures that the generated stamps are more in line with the user's emotions, giving them special meaning.

[0272] Example 2

[0273] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0274] Conventional image editing and sticker generation systems make it difficult for users to easily create LINE stickers from their own photos and create personalized stickers that reflect their emotions. Furthermore, to improve the user experience during the sticker generation process, advanced image processing techniques and complex operations are often required, making them difficult for average users to use. Therefore, there is a need for a system that can easily and efficiently generate stickers using users' images and automatically adjust the design to reflect the user's emotions or specific themes.

[0275] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for analyzing images using an artificial intelligence model, means for recognizing the user's emotion in the image using an emotion recognition engine, and means for selecting the content of stamps based on a specified theme. This makes it possible to automatically extract features from images uploaded by users, recognize specific emotions, and further generate optimal stamps based on the results. This allows users to generate and use high-quality personalized stamps with simple operations.

[0276] "User" means an individual or entity that uses the System to upload images and generate and use stamps.

[0277] "Terminal" refers to a portable information terminal such as a smartphone or tablet used by a user.

[0278] "Image" refers to visual information such as photographs and illustrations that users upload using their devices.

[0279] "Preprocessing" refers to a series of operations that the device performs on the image, including resizing, cropping, filtering, and face masking.

[0280] "Server" refers to a remote computer system that analyzes pre-processed images received from a terminal and generates stamps.

[0281] "Artificial intelligence model" refers to a trained algorithm or program used for image analysis and stamp generation.

[0282] An "emotion recognition engine" refers to software that analyzes a user's facial expressions and posture in an image and identifies their emotions.

[0283] "Theme" refers to the theme or concept specified when creating a stamp.

[0284] "Stamps" refer to user-generated visual content used in communication applications such as LINE.

[0285] "Communication application" refers to software for exchanging messages using user-generated stamps.

[0286] "Resizing" refers to the process of changing the resolution of image data.

[0287] "Trimming" refers to the process of cutting out unnecessary parts of an image.

[0288] "Filtering" refers to the process of optimizing the brightness, contrast, color, etc. of an image.

[0289] "Facial masking" refers to the process of detecting facial areas in an image and applying mosaic processing to protect privacy.

[0290] MODE FOR CARRYING OUT THE INVENTION

[0291] This invention relates to a system that allows users to easily create LINE stamps from their own images and reflect the user's emotions. This system is realized by sequentially processing the image upload, preprocessing, analysis, emotion recognition, stamp generation, and stamp usage. The specific details of each processing step are described below.

[0292] User operation

[0293] The user launches the LINE stamp generation application on their smartphone. Following the application's instructions, the user selects the image they want to turn into a stamp from the image gallery stored on their smartphone and taps the "Upload" button. This operation sends the selected image data to the server.

[0294] Terminal side processing

[0295] The device receives the image uploaded by the user and performs the following preprocessing:

[0296] 1. Resizing: Changing the image resolution to a size suitable for analysis or use. Specifically, the device uses an image processing library (e.g., OpenCV) to change the number of pixels in the image.

[0297] 2. Cropping: Cut out unnecessary parts of the image and leave only the parts suitable for stamping. Specifically, the device applies an image processing algorithm to automatically crop the center of the image.

[0298] 3. Filtering: Optimizes the brightness, contrast, and color of the image. Specifically, the device performs filtering to adjust the overall color tone and brightness.

[0299] 4. Face Masking: To protect privacy, AI technology is used to automatically detect and blur faces. Specifically, the device uses a face recognition API (e.g., Google® Vision API) to apply a blur effect to the face.

[0300] The processed image data is sent to the server via an HTTP request.

[0301] Server-side processing

[0302] The server receives the pre-processed image and performs the following operations:

[0303] 1. Feature extraction: The server uses an artificial intelligence model (e.g., TensorFlow or PyTorch) to analyze image features (e.g., smile, pose).

[0304] 2. Emotion Recognition: Use an emotion recognition engine (e.g., EmotionAPI) to identify emotions (e.g., joy, surprise) from the user's facial expressions in the image.

[0305] 3. Theme response: Elements related to the specified theme (e.g., "travel") are identified, and the server refers to its internal database to select stamp templates related to the theme.

[0306] 4. Sticker generation: The server automatically generates stickers based on the extracted features and themes. Specifically, it uses a generative AI model (e.g., GANs model) to generate stickers that correspond to the image features and themes. Based on the results of emotion recognition, it adjusts the design and text (e.g., "That's fun!").

[0307] The generated stamp data is sent to the user's terminal as an HTTP response.

[0308] Registering stamps on the device

[0309] The device processes the received stamp data and registers it in the LINE application. The specific steps are as follows:

[0310] 1. Data reception: The terminal receives the stamp data returned from the server and stores it in a temporary file or memory.

[0311] 2. Registering to the LINE app: The device uses the LINE API to register the new stamp to the app.

[0312] Specific examples

[0313] Consider the case where a user uploads photos taken during a trip to a LINE sticker generation app. The user selects travel photos from the smartphone gallery and taps the "Upload" button. The device then resizes, crops, and filters the image data, blurring the faces, before sending it to the server. The server analyzes features from the images, such as smiles and poses at tourist spots, and uses an emotion engine to recognize "feelings of joy." Based on the theme "travel," the server generates stickers containing text such as "Wonderful trip!" The generated sticker set is then sent back from the server to the user's device and automatically registered in the LINE app. The user can then immediately use these original stickers in LINE chats with family and friends.

[0314] An example of a prompt to be input to the generative AI model is as follows:

[0315] 1. "Use photos from your trip to create LINE stamps that show you having fun."

[0316] 2. "Create a stamp with the theme of 'surprise' based on a photo of a surprised expression."

[0317] 3. "Create a stamp set that represents 'family togetherness' based on a family photo."

[0318] In this way, the system of the present invention allows users to create original LINE stamps with simple operations and use stamps that reflect their emotions.

[0319] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0320] Step 1:

[0321] The user launches the LINE stamp generation application on their smartphone, selects the image they want to use as a stamp from their device's gallery, and taps the "Upload" button.

[0322] Specific behavior: The user taps the app icon to launch it, follows the guide to select an image, and presses the "Upload" button.

[0323] Input: An image selected from your phone's gallery.

[0324] Output: Image data is sent to the server.

[0325] Step 2:

[0326] The device receives the uploaded image and performs the following preprocessing:

[0327] 1. Resize: Change the image resolution to a size suitable for analysis or use.

[0328] What happens: The device uses an image processing library (e.g., OpenCV) to scale the image to the appropriate resolution.

[0329] Input: Image data uploaded by the user.

[0330] Output: The resized image data.

[0331] 2. Trimming: Cut out unnecessary parts of the image and leave only the parts suitable for stamping.

[0332] What it does: Your device applies an image processing algorithm to automatically crop the center of the image.

[0333] Input: The resized image data.

[0334] Output: Cropped image data.

[0335] 3. Filtering: Optimizing the brightness, contrast, hue, etc. of the image.

[0336] What it does: Your device applies filters to adjust the overall color and brightness.

[0337] Input: Cropped image data.

[0338] Output: The filtered image data.

[0339] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[0340] What it does: The device uses a facial recognition API (e.g., Google Vision API) to apply a mosaic effect to the face.

[0341] Input: The filtered image data.

[0342] Output: Face mask processed image data.

[0343] Step 3:

[0344] The terminal sends the preprocessed image data to the server using an HTTP request.

[0345] Specific operation: The device generates an HTTP request and sends a payload containing image data to the server address.

[0346] Input: Face mask processed image data.

[0347] Output: The preprocessed image is sent to the server.

[0348] Step 4:

[0349] The server receives the pre-processed image and performs the following operations:

[0350] 1. Feature extraction: The server uses an artificial intelligence model (e.g., TensorFlow or PyTorch) to analyze image features (e.g., smile, pose).

[0351] How it works: The server inputs image data into a pre-trained AI model to extract key features.

[0352] Input: Preprocessed image data.

[0353] Output: Extracted feature data.

[0354] 2. Emotion Recognition: Use an emotion recognition engine (e.g., EmotionAPI) to identify emotions (e.g., joy, surprise) from the user's facial expressions in the image.

[0355] Specific operation: The server calls the emotion recognition API and analyzes emotions from the feature data.

[0356] Input: Extracted feature data.

[0357] Output: Recognized emotion data.

[0358] 3. Theme response: Elements related to the specified theme are identified, and the server refers to its internal database to select a stamp template related to the theme.

[0359] Specific operation: The server retrieves information relevant to the topic from the database and selects the relevant template.

[0360] Input: Recognized emotion data and a given topic.

[0361] Output: The selected stamp template.

[0362] 4. Sticker generation: The server automatically generates stickers based on the image features and themes. Specifically, the server uses a generative AI model (e.g., a GANs model) to generate stickers that correspond to the image features and themes. Based on the results of emotion recognition, the server adjusts the design and text (e.g., "That's fun!").

[0363] Specific operation: The server inputs image features and topic information into the GANs model to generate stamps. The emotion recognition results are reflected in the design and text.

[0364] Input: Selected stamp template, extracted features, and emotion data.

[0365] Output: The generated stamp data.

[0366] Step 5:

[0367] The server returns the generated stamp data to the user's terminal.

[0368] Specific operation: The server generates an HTTP response and sends a payload containing the stamp data to the user terminal.

[0369] Input: The generated stamp data.

[0370] Output: The stamp data is sent to the user's device.

[0371] Step 6:

[0372] The device processes the received stamp data and registers it in the LINE application:

[0373] 1. Data reception: The terminal receives the stamp data sent from the server and stores it in a temporary file or memory.

[0374] Specific operation: The device analyzes the HTTP response, obtains the stamp data, and saves it.

[0375] Input: Stamp data sent from the server.

[0376] Output: Temporary file or data held in memory.

[0377] 2. Registering to the LINE app: The device uses the LINE API to register the new stamp to the app.

[0378] Specific operation: The device calls the LINE API and adds the stamp to the application.

[0379] Input: Stamp data held in temporary file or memory.

[0380] Output: Stamps registered in the LINE app.

[0381] This allows users to immediately use the original LINE stamps they have created.

[0382] (Application example 2)

[0383] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0384] By using personalized stamps that reflect the emotions of the users who create LINE stamps as advertisements, it is possible to effectively deliver advertisements to target users. However, conventional systems lack a means to recognize users' emotions and generate personalized advertisements, making it difficult to implement effective advertising campaigns.

[0385] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for uploading photos, a means for pre-processing the photos, and a means for recognizing a user's emotions using an emotion recognition engine and adding text for personalized advertising to generate advertising stamps. This makes it possible to generate stamp designs that reflect the user's emotional state and create personalized advertising campaigns.

[0386] "User" is the end user who uploads photos and utilizes the stamp generation process.

[0387] A "terminal" is an electronic device such as a smartphone or tablet that is operated by a user.

[0388] The "means for uploading photos" is a function that allows a user to use a terminal to send photos that they have to a server.

[0389] "Preprocessing" refers to the process of resizing, cropping, filtering, etc., of uploaded photos to prepare them for subsequent analysis.

[0390] The "server" is a computer system that receives uploaded photos and performs various analytical processes and stamp generation.

[0391] An "AI model" is a mathematical model that uses artificial intelligence technology to analyze photos and extract features and recognize emotions.

[0392] An "emotion recognition engine" is a software component for analyzing and recognizing a user's emotions in a photo.

[0393] A "stamp" is an image generated for use within a communication application that reflects a user's photo and emotion recognition results.

[0394] "Means for automatically generating stamps" is a function that automatically creates stamps based on analysis results and emotion recognition results.

[0395] "Advertising stamps" are stamps used in promotional activities such as advertising campaigns, and include personalized messages or designs.

[0396] "Text for personalized advertisement" is text for an advertisement message that is generated by reflecting the result of user emotion recognition.

[0397] The "means for registering" is a function for adding the generated stamp to a communication application so that the user can immediately use it.

[0398] This invention is a system that allows users to easily create LINE stamps for advertising purposes from their own photos, and recognizes and reflects the user's emotions in the process. Details of its implementation are described below.

[0399] Hardware and software used

[0400] Hardware:

[0401] Device: Electronic device such as a smartphone or tablet

[0402] Server: A computer system for analysis and stamp generation

[0403] software:

[0404] TensorFlow: A library for building and using emotion recognition models

[0405] Pillow (PIL): An image processing library for pre-processing photos.

[0406] Examples of data processing and data calculation

[0407] 1. Upload and pre-process photos:

[0408] Users upload photos for advertising purposes using their devices, which undergo the following pre-processing:

[0409] Resize: Change the resolution of your photos to the optimal size.

[0410] Crop: Cut out unwanted parts of a photo, leaving only the center.

[0411] Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[0412] 2. Server analysis and emotion recognition:

[0413] The server receives the preprocessed photos and analyzes them using an AI model. An emotion recognition engine recognizes the user's emotions in the photos. Specifically, the AI ​​model uses TensorFlow to analyze emotions and identify emotions such as "joy" or "surprise."

[0414] 3. Automatic stamp generation:

[0415] Based on the results of the analysis and emotion recognition, advertising stamps are automatically generated. Specifically, advertising text is added according to the features in the photo and the recognized emotion. For example, if the recognized emotion is "joy," the text "Happy moment!" is added.

[0416] 4. Return and register your stamps:

[0417] The server returns the generated stamp to the device, which then automatically registers the received stamp in a communication application (e.g., LINE). This allows the user to immediately use the generated advertising stamp.

[0418] Examples of concrete examples and prompts

[0419] Examples:

[0420] Suppose an advertising agency uploads a photo of a smiling user to promote the launch of a new product. The device resizes, crops, and filters the photo, then sends it to a server. The server uses TensorFlow to analyze the photo and recognize the emotion "joy." The resulting sticker contains the text "Happy Moment!" and is used as a promotional sticker.

[0421] Example prompt sentence:

[0422] "Upload photos taken during the promotional event and generate LINE stickers that reflect the emotion of joy. Add text such as 'Fun Moments!' to the generated stickers."

[0423] This invention makes it easy to create personalized advertisements that reflect the user's emotions, and enables effective advertisement delivery to target users.

[0424] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0425] Step 1:

[0426] A user uploads a photo using a terminal. To do this, the user launches the terminal application, selects a photo for advertising from the gallery, and presses the "Upload" button. The input is the photo file selected by the user, and the output is the photo data sent to the server.

[0427] Step 2:

[0428] The device preprocesses the uploaded photos, specifically:

[0429] Resize: Change the photo resolution to 128x128 pixels.

[0430] Crop: Cut out the center to narrow the focus.

[0431] Filtering: Optimize brightness and contrast.

[0432] The input is the original uploaded photo file, and the output is preprocessed photo data that has been resized, cropped, and filtered.

[0433] Step 3:

[0434] The server receives the preprocessed photos and analyzes them using an AI model. Specifically, it extracts the user's facial expressions and facial features in the photos. The input is the preprocessed photo data, and the output is the analysis results of facial features and emotions.

[0435] Step 4:

[0436] The emotion recognition engine recognizes the user's emotions based on the analysis results. Specifically, an emotion recognition model using TensorFlow identifies emotions such as "happiness" and "surprise." The input is the analysis results by the AI ​​model, and the output is data on the recognized emotional state.

[0437] Step 5:

[0438] The server automatically generates advertising stamps based on the recognized emotions. Here, text (e.g., "A fun moment!") that matches the recognized emotion (e.g., "Joy") is added. The input is the emotion recognition result and photo data, and the output is the generated stamp data.

[0439] Step 6:

[0440] The generated stamp is returned to the terminal. The server sends the generated stamp data to the terminal. The input is the stamp data generated by the server, and the output is the stamp data sent to the terminal.

[0441] Step 7:

[0442] The device registers the returned stamps in a communication application (e.g., LINE). The user can use the generated stamps as is without performing any special operations. The input is the stamp data returned from the server, and the output is the stamps registered in the LINE app.

[0443] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0444] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0445] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0446] [Second embodiment]

[0447] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0448] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0449] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0450] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0451] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0452] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0453] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0454] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0455] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0456] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0457] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0458] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0459] The present invention is a system that allows users to easily create LINE stamps from their own photos, and is realized by sequentially processing the photos: uploading, preprocessing, analyzing, stamp generation, and stamp usage. The specific details of each processing step are described below.

[0460] User operation

[0461] The device used by the user is an electronic device such as a smartphone or tablet. The user first launches the LINE stamp generation application, follows the instructions of the application to select a photo they have on hand, and uploads it to the server.

[0462] Terminal side processing

[0463] The device receives the photos uploaded by the user and performs the following preprocessing:

[0464] 1. Resize: Change the resolution of the photo to a size suitable for analysis or use.

[0465] 2. Crop: Cut out unnecessary parts of the photo and leave only the parts suitable for stamping.

[0466] 3. Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[0467] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[0468] Server-side processing

[0469] The server receives the pre-processed photo data from the device and analyzes it using the AI ​​model:

[0470] 1. Feature extraction: The AI ​​model extracts features from the photo (e.g., smile, pose).

[0471] 2. Topic response: Identify elements related to a given topic (e.g., "Holidays").

[0472] 3. Stamp generation: Based on the extracted features and the theme, an appropriate template is selected and a stamp is generated.

[0473] 4. Add text: Automatically add text that matches the theme (e.g., "It's fun!") to the stamp.

[0474] The generated stamp is sent back to the terminal from the server.

[0475] Registering stamps on the device

[0476] The device registers the received stamp data in the LINE app:

[0477] 1. Data reception: The terminal receives the generated stamp data from the server.

[0478] 2. Registration process: Add stamps to the LINE application.

[0479] This allows users to easily use LINE stamps.

[0480] Specific examples

[0481] Consider a case where a user goes on a family picnic on the weekend and uploads photos of the picnic to a LINE sticker generation app. The user selects a picnic photo from the smartphone gallery, and the app resizes, crops, filters, and blurs faces before sending it to the server.

[0482] The server uses an AI model to analyze the photo, extracting features such as a "smile" or "picnic scene," and generates stamps that correspond to the theme of "holidays." The stamps are accompanied by text such as "That's fun!"

[0483] The generated stamp set is sent back to the user's smartphone from the server and automatically registered in the LINE app, allowing the user to immediately use the original stamps in LINE chats with family and friends.

[0484] In this way, the present invention provides a system that allows users to easily create and use original LINE stamps. With privacy protection features and automated analysis, the system can be easily used by many more users.

[0485] The processing flow will be explained below.

[0486] Processing step details

[0487] Step 1:

[0488] The user launches the LINE stamp generation application on their smartphone and selects a photo from their gallery, then taps the "Upload" button on the app.

[0489] Step 2:

[0490] The device receives the photo selected by the user and resizes the photo to a resolution suitable for analysis and use.

[0491] Step 3:

[0492] The device will trim the background of the photo to extract only the parts suitable for making a stamp, and adjust the brightness and contrast of the image using filtering if necessary.

[0493] Step 4:

[0494] Using the device's AI technology, it automatically detects faces in photos and applies a mosaic process to protect privacy, making it difficult to identify individuals.

[0495] Step 5:

[0496] The device sends the preprocessed photo data to the server, which receives the data and uses an AI model to analyze the photo's features.

[0497] Step 6:

[0498] The server uses the analysis results to identify elements related to the specified topic (e.g., "holidays") and selects an appropriate stamp template based on the features most relevant to the topic.

[0499] Step 7:

[0500] The server automatically generates stamps based on the characteristics and theme of the photo. The generated stamps reflect elements of the user's photo and include text appropriate to the theme (e.g., "That's fun!").

[0501] Step 8:

[0502] The server packages the generated stamp data and sends it back to the terminal, which receives the data.

[0503] Step 9:

[0504] The device registers the received stamp data in the LINE application. Once registration is complete, the new stamp set will be added to the user's LINE stamp library.

[0505] Step 10:

[0506] Users can open the LINE app and use the newly added stamps to send messages to friends and family in the chat screen. Users can use the generated stamps as they like.

[0507] Through this series of processes, users can easily convert their photos into LINE stickers and use them conveniently. The entire processing flow is automated, making it a user-friendly system that also protects privacy.

[0508] Example 1

[0509] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0510] Conventionally, it has been difficult for users to easily generate original stamps using their own photos and use them in messaging applications. Furthermore, due to a lack of privacy protection and image optimization processing when generating stamps, it has been difficult to easily create high-quality stamps. The present invention aims to solve these problems and provide a system that allows users to easily generate high-quality original stamps.

[0511] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0512] In this invention, the server includes means for receiving the preprocessed image and analyzing the image using a machine learning model, means for automatically generating an image based on the analysis result, and means for returning the generated image to the information device, thereby enabling users to easily generate high-quality original stamps and use them in messaging applications.

[0513] A "user" is an individual or legal entity that uses a particular application or system.

[0514] "Information devices" refers to a wide range of devices, including electronic terminals such as smartphones, tablets, and personal computers.

[0515] "Image" refers to visual data such as photographs, paintings, graphics, and illustrations.

[0516] "Preprocessing" refers to the process of converting images into a format suitable for analysis and use by performing operations such as resizing, cropping, filtering, and face masking.

[0517] "Computer system" refers to the entire system that processes and analyzes data, including servers and cloud-based computing resources.

[0518] A "machine learning model" refers to an algorithm that learns from large amounts of data and automatically recognizes specific patterns and features.

[0519] "Analysis" is the process of using machine learning models to identify and extract specific features and patterns.

[0520] "Generation" is the process of creating new images or information based on analyzed data.

[0521] A "messaging application" is an application that allows users to send and receive text, images, stamps, etc.

[0522] This invention relates to a system that allows users to easily create original stamps using their own images. The system employs a method in which users upload images, which are preprocessed on their devices, then analyzed and generated on a server, and the generated stamps are registered in a messaging application.

[0523] Hardware and software used

[0524] Users access the system using devices such as smartphones and tablets, and use a LINE stamp generation application that runs on these devices. On the server side, a high-performance cloud infrastructure is used, and TensorFlow and OpenCV are used for machine learning models.

[0525] Specific processing

[0526] 1. Launch the app

[0527] The user launches the LINE stamp generation application on their smartphone or tablet and taps the app icon to launch the app.

[0528] 2. Select and upload photos

[0529] Users select the photo they want to use as a stamp from their device's gallery and tap the "Upload" button in the app to send the photo to the server.

[0530] 3. Photo preprocessing

[0531] The device pre-processes the photo before sending it to the server. Specifically, it does the following:

[0532] Resize: Reduce the image resolution to 800x800 pixels.

[0533] Crop: Cut out unnecessary background parts of the photo and center only the person.

[0534] Filtering: Increases image brightness by 10% and contrast by 15%.

[0535] Face Mask: Using AI technology, the face is automatically detected and a 10x10 pixel mosaic is applied.

[0536] 4. Data Transmission

[0537] The photos are pre-processed on the terminal and sent to a server via the Internet.

[0538] 5. Photo Analysis

[0539] The server analyzes the photo using machine learning models (e.g. TensorFlow, OpenCV). Specifically, it performs the following steps:

[0540] Feature extraction: Extracting photo features such as smiles and poses.

[0541] Topic-based: Identify elements related to a given theme (e.g., "Holidays").

[0542] 6. Stamp generation

[0543] A stamp is generated based on the extracted features and themes. For example, a stamp of a smiling person can be generated with the text "It's fun!"

[0544] 7. Data return

[0545] The generated stamp data is returned from the server to the terminal.

[0546] 8. Registering with the LINE app

[0547] The device will register the received stamps in the LINE application. Specifically, select "Stamp Management" from the LINE app's settings menu and then select "Add New Stamps."

[0548] Specific examples

[0549] For example, consider a scenario where a user wants to upload photos of a family picnic on the weekend to a LINE sticker generation app. The user selects a picnic photo from the smartphone gallery and taps the "Upload" button. The app resizes the photo (e.g., to 800x800 pixels), crops unnecessary background, increases brightness by 10%, increases contrast by 15%, and blurs the face (e.g., blurs the face by 10x10 pixels), before sending it to the server.

[0550] The server uses a generative AI model to analyze the photo, extracting features such as a "smile" or "picnic scene," and generates stamps that correspond to the theme "holiday." The stamps are accompanied by the text "It's fun!" The generated stamp set is then sent back from the server to the user's smartphone, which then registers the stamps in the LINE application. The user can then immediately use these original stamps in LINE chats with family and friends.

[0551] Examples of prompt statements

[0552] Prompt: "Generate a fun stamp related to a holiday picnic. The photo shows a smiling family."

[0553] Example output from the AI ​​model: "A stamp of a family having a picnic, with the text 'This is fun!'"

[0554] As described above, the present invention provides a system that allows users to easily create original stamps using their own images and use them in messaging applications.

[0555] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0556] Step 1: User launches app

[0557] The user launches the LINE stamp generation application on an information device such as a smartphone or tablet. The app launches when the user taps a specific icon. The input is the operation of the information device, and the output is the launch of the application.

[0558] Step 2: User selects and uploads photo

[0559] The user selects the image they want to use as a stamp from the photo gallery in the app. After selecting, they press the "Upload" button to send the image to the server. The input is the user's selected image and instructions to upload, and the output is the selected image data sent from the device to the server.

[0560] Step 3: Pre-processing photos on the device

[0561] The device receives the images uploaded by the user and performs pre-processing. The specific pre-processing operations include:

[0562] Resize: Reduce the resolution of an image to 800x800 pixels. Take the original high-resolution image as input and get the resized image as output.

[0563] Cropping: Cut out unnecessary background parts and leave important parts (such as people) in the center. The input is the entire image, and the output is the cropped image.

[0564] Filtering: Optimizes the brightness and contrast of an image. The input is the original image, and the output is the filtered image. Specifically, the brightness is increased by 10% and the contrast is increased by 15%.

[0565] Face mask: AI technology automatically detects the face and applies a 10x10 pixel mosaic process. The input is the original image, and the output is an image with the face mosaicked.

[0566] Step 4: Send data from the device to the server

[0567] The device sends the preprocessed image data to the server. The input is the preprocessed image, and the output is the image data sent to the server. The transmission occurs over a network connection.

[0568] Step 5: Photo analysis by the server

[0569] The server analyzes the preprocessed images using machine learning models (e.g., TensorFlow, OpenCV). Specific operations include:

[0570] Feature extraction: Extract image features such as smiles and poses. The input is a preprocessed image, and the output is feature data.

[0571] Topic matching: Identify elements related to a specified theme (e.g., "holidays"). The input is theme information and feature data, and the output is the corresponding elements.

[0572] Step 6: Server-generated stamps

[0573] The server generates stamps based on the extracted features and themes. Specific operations include:

[0574] Template Selection: Select an appropriate stamp template. The input is feature data and a topic, and the output is the selected template.

[0575] Add Text: Automatically add text that matches the theme (e.g., "It's fun!") to the stamp. The input is a template and text data, and the output is a completed stamp image.

[0576] Step 7: Sending data back from the server to the device

[0577] The server returns the generated stamp data to the user's terminal. There is the generated stamp data as input and the stamp data sent to the terminal as output.

[0578] Step 8: Register stamps via terminal

[0579] The device registers the received stamp with the messaging application. Specific actions include:

[0580] Data reception: Receives stamp data sent from the server. There is the sent stamp data as input and the received stamp data as output.

[0581] Registration process: Registers stamps in the LINE application. Specifically, select "Stamp Management" from the LINE app's settings menu and execute "Add New Stamp." The stamp data received is the input, and the registered stamp is displayed in the LINE app as the output.

[0582] This allows users to easily create original LINE stamps and use them in messaging applications.

[0583] (Application example 1)

[0584] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0585] To improve the user experience in virtual stores, a system is needed that allows users to easily create original virtual avatars that reflect their own characteristics and use them to enjoy shopping and interacting with other users. However, current systems require advanced expertise and a significant amount of time for users to create avatars from their own photos, making them difficult for average users to use. Another problem is that the environment for properly registering and using generated avatars in virtual stores is not yet in place.

[0586] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0587] In this invention, the server includes means for analyzing a photo using a generative AI model, means for automatically generating a virtual avatar based on the analysis results, and means for selecting content for the virtual avatar based on a specified prompt and automatically adding customization elements. This allows users to easily generate original virtual avatars from their own photos without specialized knowledge or skills and use the avatars in virtual stores.

[0588] "Device" means the electronic device used by a User to upload photos and receive / register generated virtual avatars.

[0589] "Photo preprocessing" is the process of preparing photos for analysis and avatar generation, including resizing, cropping, filtering, and face masking.

[0590] The "server" is a central processing unit that receives pre-processed photos, analyzes the photos using a generative AI model, and generates and returns a virtual avatar based on the analysis results.

[0591] A "generative AI model" is an artificial intelligence algorithm that analyzes information from a photograph, extracts features, and generates a virtual avatar.

[0592] A "virtual avatar" is a digital character generated by extracting features from a user's photograph, and is used by the user in a virtual store.

[0593] A "prompt" is an instruction used for analysis, a piece of text that tells the generative AI model how to analyze a photo and what specific features it has.

[0594] A "virtual store" is a virtual commercial space where users can shop and interact using virtual avatars they create.

[0595] "Customization elements" are decorative elements such as the appearance, belongings, and background of a virtual avatar, and are automatically added based on user specifications or prompts.

[0596] "Analysis results" are data obtained after a photo is analyzed by a generative AI model, and include information based on feature extraction and prompt text.

[0597] This invention is a system that allows users to easily create original virtual avatars from their own photographs and use those avatars in virtual stores. The specific system configuration and processing method will be described below.

[0598] Main configuration

[0599] 1. User Device

[0600] User devices can be smartphones, tablets, smart glasses, head-mounted displays, etc. User devices provide a means for users to take, select, and upload their own photos.

[0601] 2. Server

[0602] The server receives the uploaded photo, analyzes it using a generative AI model, generates a virtual avatar based on the analysis results, adds any necessary customization elements, and sends it back to the user's device.

[0603] 3. Generative AI Models

[0604] A generative AI model is an artificial intelligence algorithm that extracts features from a photo and generates a virtual avatar based on them, using machine learning frameworks such as TensorFlow and PyTorch.

[0605] Processing method

[0606] 1. Upload a photo

[0607] Users select photos from their device's camera or gallery and upload them to the server through the application, which then imports the user's photos into the system.

[0608] 2. Photo pre-processing

[0609] The device performs pre-processing on the uploaded photo, including resizing, cropping, filtering, and face masking, to prepare the photo for analysis and avatar generation.

[0610] 3. Server-side analysis

[0611] The server receives the pre-processed photos and performs feature analysis using a generative AI model, which extracts features such as smiles and poses from the photos.

[0612] 4. Virtual Avatar Generation

[0613] Based on the extracted features, the server automatically generates a virtual avatar, determines the avatar's content based on the specified prompt, and adds customization elements, such as background, clothing, and accessories.

[0614] 5. Return and register the generated virtual avatar

[0615] The generated virtual avatar is sent back from the server to the user's device, which then registers the avatar in a virtual store, allowing the user to use the avatar for shopping and interacting with other users.

[0616] Specific examples

[0617] For example, a user can upload a photo of themselves using a virtual shopping app. The app processes the photo, analyzes it with an AI model to extract features, and generates a virtual avatar that reflects the user's characteristics. This avatar can then be used in a virtual store, allowing the user to shop or participate in events.

[0618] An example of a specified prompt sentence is:

[0619] Extract features from a user's facial photo and provide the data needed to generate a virtual avatar. Prioritize analysis of smiles, movement direction, and facial features.

[0620] This allows users to easily create their own original virtual avatar and enjoy a rich user experience in a virtual store.

[0621] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0622] Step 1:

[0623] The user uploads a photo on the device. They select a photo from the device's camera or gallery and press the "Upload" button to import the photo into the application. The input is the photo file selected by the user, and the output is the photo file to be uploaded. With the user's operation, the photo is ready to be sent to the server.

[0624] Step 2:

[0625] The device preprocesses the photo. The input is the uploaded photo file. The device first resizes the photo. Next, it crops it to keep only the important elements. After that, it filters the photo to adjust the brightness and contrast, and blurs the face if necessary. The output is the preprocessed photo data.

[0626] Step 3:

[0627] The server receives the preprocessed photos. The input is the preprocessed photo data, and the server prepares the received data for analysis. The output is the photo data ready for analysis.

[0628] Step 4:

[0629] The server analyzes the photo using a generative AI model. The input is photo data that can be analyzed, and the generative AI model extracts features from the photo (e.g., smile, facial expression, pose). The output is the feature extraction results, i.e., prominent feature data obtained from the photo.

[0630] Step 5:

[0631] The server generates a virtual avatar based on the specified prompt. The input is the feature extraction results and the prompt. The content of the virtual avatar is determined based on the analysis results, and customization elements (e.g., clothing, background, accessories) are automatically added. The output is the generated virtual avatar. An example of a prompt is: "Please extract features from the user's facial photograph and provide the data necessary to generate a virtual avatar. Please prioritize analysis of smiles, movement direction, and facial features."

[0632] Step 6:

[0633] The server returns the generated virtual avatar to the user's device. The input is the generated virtual avatar, and the output is the avatar data returned to the user's device. The server ensures that the avatar is properly delivered to the user's device through the return process.

[0634] Step 7:

[0635] The user's device registers the returned virtual avatar in the virtual store. The input is the returned avatar data, and the output is the avatar information registered in the virtual store. The device registers the avatar using the virtual store's API, and the user is ready to use the avatar to act in the virtual store.

[0636] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0637] This invention is a system that allows users to easily generate LINE stamps from their own photos, recognizing and reflecting the user's emotions in the process. This is achieved by sequentially processing the photos: uploading, preprocessing, analysis, emotion recognition, stamp generation, and stamp usage. The specific details of each processing step are described below.

[0638] User operation

[0639] Users simply launch the LINE stamp generation application on their smartphone and follow the app's instructions to select a photo from their gallery and upload it to the server.

[0640] Terminal side processing

[0641] The device receives the photos uploaded by the user and performs the following preprocessing:

[0642] 1. Resize: Change the resolution of the photo to a size suitable for analysis or use.

[0643] 2. Crop: Cut out unnecessary parts of the photo and leave only the parts suitable for stamping.

[0644] 3. Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[0645] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[0646] Server-side processing

[0647] The server receives the uploaded photo and processes it as follows:

[0648] 1. Feature extraction: The server uses an AI model to extract features from the photo (e.g., smile, pose).

[0649] 2. Emotion Recognition: The emotion engine analyzes photos and recognizes the user's emotions. For example, it identifies emotions such as "happiness" or "surprise" based on facial expressions and posture.

[0650] 3. Theme matching: Identify elements related to the specified theme (e.g., "holidays"). The server selects an appropriate stamp template based on the features most relevant to the theme.

[0651] 4. Stamp generation: Stamps are automatically generated based on the extracted features and themes. The generated stamps reflect elements of the user's photo and include text appropriate to the theme (e.g., "It's fun!"). The design and text of the stamps are also adjusted based on the results of emotion recognition.

[0652] The generated stamp is sent back from the server to the user's terminal.

[0653] Registering stamps on the device

[0654] The device registers the received stamp data in the LINE app:

[0655] 1. Data reception: The terminal receives the generated stamp data from the server.

[0656] 2. Registration process: Add stamps to the LINE application.

[0657] This allows users to easily use LINE stamps.

[0658] Specific examples

[0659] Consider the case where a user uploads photos taken during a trip to a LINE stamp generation app. The user selects and uploads photos from their smartphone gallery. The device then resizes, crops, filters, and blurs faces before sending the photos to the server.

[0660] The server uses an AI model to analyze the photo and extract features such as a "smile" or "tourist spot pose." The emotion engine recognizes that the user in the photo is "feeling happy." The server generates a stamp corresponding to the theme "travel," and adds text such as "Great trip!" to the stamp. Emotion recognition allows the stamp design to have bright colors and a fun atmosphere.

[0661] The generated stamp set is sent back to the user's smartphone from the server and automatically registered in the LINE app, allowing the user to immediately use the original stamps in LINE chats with family and friends.

[0662] In this way, the present invention provides a system that allows users to easily create and use original LINE stamps. By using emotion recognition functionality to generate stamps that reflect the user's emotions, a more personalized experience is provided.

[0663] The processing flow will be explained below.

[0664] Processing step details

[0665] Step 1:

[0666] The user launches the LINE stamp generation application on their smartphone and selects a photo from their gallery, then taps the "Upload" button on the app.

[0667] Step 2:

[0668] After the device receives the selected photo from the user, it resizes the photo, changing the photo resolution to a size suitable for analysis and use. For example, a 5000x4000 pixel photo is resized to 1000x800 pixels.

[0669] Step 3:

[0670] Your device will trim away unwanted background from your photo, determine the area you want to use, and crop the photo so that only the subject remains, for example, cropping only around a person.

[0671] Step 4:

[0672] Your phone will apply filters to your photos, including adjusting brightness, contrast, and color tone, for example, brightening dark photos and enhancing contrast.

[0673] Step 5:

[0674] Using the device's AI technology, it automatically detects faces in photos and applies mosaic processing to protect privacy. For example, it applies a mosaic filter to all faces in the photo.

[0675] Step 6:

[0676] The device sends the pre-processed photo data to the server, which receives the data.

[0677] Step 7:

[0678] The server analyzes the photo using an AI model, extracting features (e.g., smile, pose) and identifying key elements of the photo.

[0679] Step 8:

[0680] The server's emotion engine further analyzes the photo to identify the user's emotion, for example, identifying "happiness" from the curvature of the smile and the degree of eye opening.

[0681] Step 9:

[0682] The server identifies elements related to the specified topic (e.g., "holidays") and selects appropriate stamp templates based on the features most relevant to the topic.

[0683] Step 10:

[0684] The server automatically generates stamps based on the user's features, theme, and recognized emotion. The generated stamps reflect elements of the user's photo and automatically add text according to the theme. The text and design are adjusted based on the results of emotion recognition.

[0685] Step 11:

[0686] The generated stamp data is packaged and sent back from the server to the terminal, which receives this data.

[0687] Step 12:

[0688] The stamp data received by the device is registered in the LINE application. The stamp is added to the LINE app and becomes available for use by users.

[0689] Step 13:

[0690] Users can open the LINE app and use the newly added stamps to send messages to friends and family in the chat screen. Users can use the generated stamps as they like.

[0691] Through this series of processes, users can easily create original LINE stamps and use personalized stamps when communicating with friends. The emotion recognition function ensures that the generated stamps are more in line with the user's emotions, giving them special meaning.

[0692] Example 2

[0693] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0694] Conventional image editing and sticker generation systems make it difficult for users to easily create LINE stickers from their own photos and create personalized stickers that reflect their emotions. Furthermore, to improve the user experience during the sticker generation process, advanced image processing techniques and complex operations are often required, making them difficult for average users to use. Therefore, there is a need for a system that can easily and efficiently generate stickers using users' images and automatically adjust the design to reflect the user's emotions or specific themes.

[0695] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for analyzing images using an artificial intelligence model, means for recognizing the user's emotion in the image using an emotion recognition engine, and means for selecting the content of stamps based on a specified theme. This makes it possible to automatically extract features from images uploaded by users, recognize specific emotions, and further generate optimal stamps based on the results. This allows users to generate and use high-quality personalized stamps with simple operations.

[0696] "User" means an individual or entity that uses the System to upload images and generate and use stamps.

[0697] "Terminal" refers to a portable information terminal such as a smartphone or tablet used by a user.

[0698] "Image" refers to visual information such as photographs and illustrations that users upload using their devices.

[0699] "Preprocessing" refers to a series of operations that the device performs on the image, including resizing, cropping, filtering, and face masking.

[0700] "Server" refers to a remote computer system that analyzes pre-processed images received from a terminal and generates stamps.

[0701] "Artificial intelligence model" refers to a trained algorithm or program used for image analysis and stamp generation.

[0702] An "emotion recognition engine" refers to software that analyzes a user's facial expressions and posture in an image and identifies their emotions.

[0703] "Theme" refers to the theme or concept specified when creating a stamp.

[0704] "Stamps" refer to user-generated visual content used in communication applications such as LINE.

[0705] "Communication application" refers to software for exchanging messages using user-generated stamps.

[0706] "Resizing" refers to the process of changing the resolution of image data.

[0707] "Trimming" refers to the process of cutting out unnecessary parts of an image.

[0708] "Filtering" refers to the process of optimizing the brightness, contrast, color, etc. of an image.

[0709] "Facial masking" refers to the process of detecting facial areas in an image and applying mosaic processing to protect privacy.

[0710] MODE FOR CARRYING OUT THE INVENTION

[0711] This invention relates to a system that allows users to easily create LINE stamps from their own images and reflect the user's emotions. This system is realized by sequentially processing the image upload, preprocessing, analysis, emotion recognition, stamp generation, and stamp usage. The specific details of each processing step are described below.

[0712] User operation

[0713] The user launches the LINE stamp generation application on their smartphone. Following the application's instructions, the user selects the image they want to turn into a stamp from the image gallery stored on their smartphone and taps the "Upload" button. This operation sends the selected image data to the server.

[0714] Terminal side processing

[0715] The device receives the image uploaded by the user and performs the following preprocessing:

[0716] 1. Resizing: Changing the image resolution to a size suitable for analysis or use. Specifically, the device uses an image processing library (e.g., OpenCV) to change the number of pixels in the image.

[0717] 2. Cropping: Cut out unnecessary parts of the image and leave only the parts suitable for stamping. Specifically, the device applies an image processing algorithm to automatically crop the center of the image.

[0718] 3. Filtering: Optimizes the brightness, contrast, and color of the image. Specifically, the device performs filtering to adjust the overall color tone and brightness.

[0719] 4. Face Masking: To protect privacy, AI technology is used to automatically detect and blur faces. Specifically, the device uses a face recognition API (e.g., Google Vision API) to apply a blur effect to the face.

[0720] The processed image data is sent to the server via an HTTP request.

[0721] Server-side processing

[0722] The server receives the pre-processed image and performs the following operations:

[0723] 1. Feature extraction: The server uses an artificial intelligence model (e.g., TensorFlow or PyTorch) to analyze image features (e.g., smile, pose).

[0724] 2. Emotion Recognition: Use an emotion recognition engine (e.g., EmotionAPI) to identify emotions (e.g., joy, surprise) from the user's facial expressions in the image.

[0725] 3. Theme response: Elements related to the specified theme (e.g., "travel") are identified, and the server refers to its internal database to select stamp templates related to the theme.

[0726] 4. Sticker generation: The server automatically generates stickers based on the extracted features and themes. Specifically, it uses a generative AI model (e.g., GANs model) to generate stickers that correspond to the image features and themes. Based on the results of emotion recognition, it adjusts the design and text (e.g., "That's fun!").

[0727] The generated stamp data is sent to the user's terminal as an HTTP response.

[0728] Registering stamps on the device

[0729] The device processes the received stamp data and registers it in the LINE application. The specific steps are as follows:

[0730] 1. Data reception: The terminal receives the stamp data returned from the server and stores it in a temporary file or memory.

[0731] 2. Registering to the LINE app: The device uses the LINE API to register the new stamp to the app.

[0732] Specific examples

[0733] Consider the case where a user uploads photos taken during a trip to a LINE sticker generation app. The user selects travel photos from the smartphone gallery and taps the "Upload" button. The device then resizes, crops, and filters the image data, blurring the faces, before sending it to the server. The server analyzes features from the images, such as smiles and poses at tourist spots, and uses an emotion engine to recognize "feelings of joy." Based on the theme "travel," the server generates stickers containing text such as "Wonderful trip!" The generated sticker set is then sent back from the server to the user's device and automatically registered in the LINE app. The user can then immediately use these original stickers in LINE chats with family and friends.

[0734] An example of a prompt to be input to the generative AI model is as follows:

[0735] 1. "Use photos from your trip to create LINE stamps that show you having fun."

[0736] 2. "Create a stamp with the theme of 'surprise' based on a photo of a surprised expression."

[0737] 3. "Create a stamp set that represents 'family togetherness' based on a family photo."

[0738] In this way, the system of the present invention allows users to create original LINE stamps with simple operations and use stamps that reflect their emotions.

[0739] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0740] Step 1:

[0741] The user launches the LINE stamp generation application on their smartphone, selects the image they want to use as a stamp from their device's gallery, and taps the "Upload" button.

[0742] Specific behavior: The user taps the app icon to launch it, follows the guide to select an image, and presses the "Upload" button.

[0743] Input: An image selected from your phone's gallery.

[0744] Output: Image data is sent to the server.

[0745] Step 2:

[0746] The device receives the uploaded image and performs the following preprocessing:

[0747] 1. Resize: Change the image resolution to a size suitable for analysis or use.

[0748] What happens: The device uses an image processing library (e.g., OpenCV) to scale the image to the appropriate resolution.

[0749] Input: Image data uploaded by the user.

[0750] Output: The resized image data.

[0751] 2. Trimming: Cut out unnecessary parts of the image and leave only the parts suitable for stamping.

[0752] What it does: Your device applies an image processing algorithm to automatically crop the center of the image.

[0753] Input: The resized image data.

[0754] Output: Cropped image data.

[0755] 3. Filtering: Optimizing the brightness, contrast, hue, etc. of the image.

[0756] What it does: Your device applies filters to adjust the overall color and brightness.

[0757] Input: Cropped image data.

[0758] Output: The filtered image data.

[0759] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[0760] What it does: The device uses a facial recognition API (e.g., Google Vision API) to apply a mosaic effect to the face.

[0761] Input: The filtered image data.

[0762] Output: Face mask processed image data.

[0763] Step 3:

[0764] The terminal sends the preprocessed image data to the server using an HTTP request.

[0765] Specific operation: The device generates an HTTP request and sends a payload containing image data to the server address.

[0766] Input: Face mask processed image data.

[0767] Output: The preprocessed image is sent to the server.

[0768] Step 4:

[0769] The server receives the pre-processed image and performs the following operations:

[0770] 1. Feature extraction: The server uses an artificial intelligence model (e.g., TensorFlow or PyTorch) to analyze image features (e.g., smile, pose).

[0771] How it works: The server inputs image data into a pre-trained AI model to extract key features.

[0772] Input: Preprocessed image data.

[0773] Output: Extracted feature data.

[0774] 2. Emotion Recognition: Use an emotion recognition engine (e.g., EmotionAPI) to identify emotions (e.g., joy, surprise) from the user's facial expressions in the image.

[0775] Specific operation: The server calls the emotion recognition API and analyzes emotions from the feature data.

[0776] Input: Extracted feature data.

[0777] Output: Recognized emotion data.

[0778] 3. Theme response: Elements related to the specified theme are identified, and the server refers to its internal database to select a stamp template related to the theme.

[0779] Specific operation: The server retrieves information relevant to the topic from the database and selects the relevant template.

[0780] Input: Recognized emotion data and a given topic.

[0781] Output: The selected stamp template.

[0782] 4. Sticker generation: The server automatically generates stickers based on the image features and themes. Specifically, the server uses a generative AI model (e.g., a GANs model) to generate stickers that correspond to the image features and themes. Based on the results of emotion recognition, the server adjusts the design and text (e.g., "That's fun!").

[0783] Specific operation: The server inputs image features and topic information into the GANs model to generate stamps. The emotion recognition results are reflected in the design and text.

[0784] Input: Selected stamp template, extracted features, and emotion data.

[0785] Output: The generated stamp data.

[0786] Step 5:

[0787] The server returns the generated stamp data to the user's terminal.

[0788] Specific operation: The server generates an HTTP response and sends a payload containing the stamp data to the user terminal.

[0789] Input: The generated stamp data.

[0790] Output: The stamp data is sent to the user's device.

[0791] Step 6:

[0792] The device processes the received stamp data and registers it in the LINE application:

[0793] 1. Data reception: The terminal receives the stamp data sent from the server and stores it in a temporary file or memory.

[0794] Specific operation: The device analyzes the HTTP response, obtains the stamp data, and saves it.

[0795] Input: Stamp data sent from the server.

[0796] Output: Temporary file or data held in memory.

[0797] 2. Registering to the LINE app: The device uses the LINE API to register the new stamp to the app.

[0798] Specific operation: The device calls the LINE API and adds the stamp to the application.

[0799] Input: Stamp data held in temporary file or memory.

[0800] Output: Stamps registered in the LINE app.

[0801] This allows users to immediately use the original LINE stamps they have created.

[0802] (Application example 2)

[0803] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0804] By using personalized stamps that reflect the emotions of the users who create LINE stamps as advertisements, it is possible to effectively deliver advertisements to target users. However, conventional systems lack a means to recognize users' emotions and generate personalized advertisements, making it difficult to implement effective advertising campaigns.

[0805] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for uploading photos, a means for pre-processing the photos, and a means for recognizing a user's emotions using an emotion recognition engine and adding text for personalized advertising to generate advertising stamps. This makes it possible to generate stamp designs that reflect the user's emotional state and create personalized advertising campaigns.

[0806] "User" is the end user who uploads photos and utilizes the stamp generation process.

[0807] A "terminal" is an electronic device such as a smartphone or tablet that is operated by a user.

[0808] The "means for uploading photos" is a function that allows a user to use a terminal to send photos that they have to a server.

[0809] "Preprocessing" refers to the process of resizing, cropping, filtering, etc., of uploaded photos to prepare them for subsequent analysis.

[0810] The "server" is a computer system that receives uploaded photos and performs various analytical processes and stamp generation.

[0811] An "AI model" is a mathematical model that uses artificial intelligence technology to analyze photos and extract features and recognize emotions.

[0812] An "emotion recognition engine" is a software component for analyzing and recognizing a user's emotions in a photo.

[0813] A "stamp" is an image generated for use within a communication application that reflects a user's photo and emotion recognition results.

[0814] "Means for automatically generating stamps" is a function that automatically creates stamps based on analysis results and emotion recognition results.

[0815] "Advertising stamps" are stamps used in promotional activities such as advertising campaigns, and include personalized messages or designs.

[0816] "Text for personalized advertisement" is text for an advertisement message that is generated by reflecting the result of user emotion recognition.

[0817] The "means for registering" is a function for adding the generated stamp to a communication application so that the user can immediately use it.

[0818] This invention is a system that allows users to easily create LINE stamps for advertising purposes from their own photos, and recognizes and reflects the user's emotions in the process. Details of its implementation are described below.

[0819] Hardware and software used

[0820] Hardware:

[0821] Device: Electronic device such as a smartphone or tablet

[0822] Server: A computer system for analysis and stamp generation

[0823] software:

[0824] TensorFlow: A library for building and using emotion recognition models

[0825] Pillow (PIL): An image processing library for pre-processing photos.

[0826] Examples of data processing and data calculation

[0827] 1. Upload and pre-process photos:

[0828] Users upload photos for advertising purposes using their devices, which undergo the following pre-processing:

[0829] Resize: Change the resolution of your photos to the optimal size.

[0830] Crop: Cut out unwanted parts of a photo, leaving only the center.

[0831] Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[0832] 2. Server analysis and emotion recognition:

[0833] The server receives the preprocessed photos and analyzes them using an AI model. An emotion recognition engine recognizes the user's emotions in the photos. Specifically, the AI ​​model uses TensorFlow to analyze emotions and identify emotions such as "joy" or "surprise."

[0834] 3. Automatic stamp generation:

[0835] Based on the results of the analysis and emotion recognition, advertising stamps are automatically generated. Specifically, advertising text is added according to the features in the photo and the recognized emotion. For example, if the recognized emotion is "joy," the text "Happy moment!" is added.

[0836] 4. Return and register your stamps:

[0837] The server returns the generated stamp to the device, which then automatically registers the received stamp in a communication application (e.g., LINE). This allows the user to immediately use the generated advertising stamp.

[0838] Examples of concrete examples and prompts

[0839] Examples:

[0840] Suppose an advertising agency uploads a photo of a smiling user to promote the launch of a new product. The device resizes, crops, and filters the photo, then sends it to a server. The server uses TensorFlow to analyze the photo and recognize the emotion "joy." The resulting sticker contains the text "Happy Moment!" and is used as a promotional sticker.

[0841] Example prompt sentence:

[0842] "Upload photos taken during the promotional event and generate LINE stickers that reflect the emotion of joy. Add text such as 'Fun Moments!' to the generated stickers."

[0843] This invention makes it easy to create personalized advertisements that reflect the user's emotions, and enables effective advertisement delivery to target users.

[0844] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0845] Step 1:

[0846] A user uploads a photo using a terminal. To do this, the user launches the terminal application, selects a photo for advertising from the gallery, and presses the "Upload" button. The input is the photo file selected by the user, and the output is the photo data sent to the server.

[0847] Step 2:

[0848] The device preprocesses the uploaded photos, specifically:

[0849] Resize: Change the photo resolution to 128x128 pixels.

[0850] Crop: Cut out the center to narrow the focus.

[0851] Filtering: Optimize brightness and contrast.

[0852] The input is the original uploaded photo file, and the output is preprocessed photo data that has been resized, cropped, and filtered.

[0853] Step 3:

[0854] The server receives the preprocessed photos and analyzes them using an AI model. Specifically, it extracts the user's facial expressions and facial features in the photos. The input is the preprocessed photo data, and the output is the analysis results of facial features and emotions.

[0855] Step 4:

[0856] The emotion recognition engine recognizes the user's emotions based on the analysis results. Specifically, an emotion recognition model using TensorFlow identifies emotions such as "happiness" and "surprise." The input is the analysis results by the AI ​​model, and the output is data on the recognized emotional state.

[0857] Step 5:

[0858] The server automatically generates advertising stamps based on the recognized emotions. Here, text (e.g., "A fun moment!") that matches the recognized emotion (e.g., "Joy") is added. The input is the emotion recognition result and photo data, and the output is the generated stamp data.

[0859] Step 6:

[0860] The generated stamp is returned to the terminal. The server sends the generated stamp data to the terminal. The input is the stamp data generated by the server, and the output is the stamp data sent to the terminal.

[0861] Step 7:

[0862] The device registers the returned stamps in a communication application (e.g., LINE). The user can use the generated stamps as is without performing any special operations. The input is the stamp data returned from the server, and the output is the stamps registered in the LINE app.

[0863] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0864] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0865] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0866] [Third embodiment]

[0867] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0868] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0869] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0870] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0871] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0872] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0873] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0874] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0875] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0876] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0877] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0878] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0879] The present invention is a system that allows users to easily create LINE stamps from their own photos, and is realized by sequentially processing the photos: uploading, preprocessing, analyzing, stamp generation, and stamp usage. The specific details of each processing step are described below.

[0880] User operation

[0881] The device used by the user is an electronic device such as a smartphone or tablet. The user first launches the LINE stamp generation application, follows the instructions of the application to select a photo they have on hand, and uploads it to the server.

[0882] Terminal side processing

[0883] The device receives the photos uploaded by the user and performs the following preprocessing:

[0884] 1. Resize: Change the resolution of the photo to a size suitable for analysis or use.

[0885] 2. Crop: Cut out unnecessary parts of the photo and leave only the parts suitable for stamping.

[0886] 3. Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[0887] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[0888] Server-side processing

[0889] The server receives the pre-processed photo data from the device and analyzes it using the AI ​​model:

[0890] 1. Feature extraction: The AI ​​model extracts features from the photo (e.g., smile, pose).

[0891] 2. Topic response: Identify elements related to a given topic (e.g., "Holidays").

[0892] 3. Stamp generation: Based on the extracted features and the theme, an appropriate template is selected and a stamp is generated.

[0893] 4. Add text: Automatically add text that matches the theme (e.g., "It's fun!") to the stamp.

[0894] The generated stamp is sent back to the terminal from the server.

[0895] Registering stamps on the device

[0896] The device registers the received stamp data in the LINE app:

[0897] 1. Data reception: The terminal receives the generated stamp data from the server.

[0898] 2. Registration process: Add stamps to the LINE application.

[0899] This allows users to easily use LINE stamps.

[0900] Specific examples

[0901] Consider a case where a user goes on a family picnic on the weekend and uploads photos of the picnic to a LINE sticker generation app. The user selects a picnic photo from the smartphone gallery, and the app resizes, crops, filters, and blurs faces before sending it to the server.

[0902] The server uses an AI model to analyze the photo, extracting features such as a "smile" or "picnic scene," and generates stamps that correspond to the theme of "holidays." The stamps are accompanied by text such as "That's fun!"

[0903] The generated stamp set is sent back to the user's smartphone from the server and automatically registered in the LINE app, allowing the user to immediately use the original stamps in LINE chats with family and friends.

[0904] In this way, the present invention provides a system that allows users to easily create and use original LINE stamps. With privacy protection features and automated analysis, the system can be easily used by many more users.

[0905] The processing flow will be explained below.

[0906] Processing step details

[0907] Step 1:

[0908] The user launches the LINE stamp generation application on their smartphone and selects a photo from their gallery, then taps the "Upload" button on the app.

[0909] Step 2:

[0910] The device receives the photo selected by the user and resizes the photo to a resolution suitable for analysis and use.

[0911] Step 3:

[0912] The device will trim the background of the photo to extract only the parts suitable for making a stamp, and adjust the brightness and contrast of the image using filtering if necessary.

[0913] Step 4:

[0914] Using the device's AI technology, it automatically detects faces in photos and applies a mosaic process to protect privacy, making it difficult to identify individuals.

[0915] Step 5:

[0916] The device sends the preprocessed photo data to the server, which receives the data and uses an AI model to analyze the photo's features.

[0917] Step 6:

[0918] The server uses the analysis results to identify elements related to the specified topic (e.g., "holidays") and selects an appropriate stamp template based on the features most relevant to the topic.

[0919] Step 7:

[0920] The server automatically generates stamps based on the characteristics and theme of the photo. The generated stamps reflect elements of the user's photo and include text appropriate to the theme (e.g., "That's fun!").

[0921] Step 8:

[0922] The server packages the generated stamp data and sends it back to the terminal, which receives the data.

[0923] Step 9:

[0924] The device registers the received stamp data in the LINE application. Once registration is complete, the new stamp set will be added to the user's LINE stamp library.

[0925] Step 10:

[0926] Users can open the LINE app and use the newly added stamps to send messages to friends and family in the chat screen. Users can use the generated stamps as they like.

[0927] Through this series of processes, users can easily convert their photos into LINE stickers and use them conveniently. The entire processing flow is automated, making it a user-friendly system that also protects privacy.

[0928] Example 1

[0929] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0930] Conventionally, it has been difficult for users to easily generate original stamps using their own photos and use them in messaging applications. Furthermore, due to a lack of privacy protection and image optimization processing when generating stamps, it has been difficult to easily create high-quality stamps. The present invention aims to solve these problems and provide a system that allows users to easily generate high-quality original stamps.

[0931] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0932] In this invention, the server includes means for receiving the preprocessed image and analyzing the image using a machine learning model, means for automatically generating an image based on the analysis result, and means for returning the generated image to the information device, thereby enabling users to easily generate high-quality original stamps and use them in messaging applications.

[0933] A "user" is an individual or legal entity that uses a particular application or system.

[0934] "Information devices" refers to a wide range of devices, including electronic terminals such as smartphones, tablets, and personal computers.

[0935] "Image" refers to visual data such as photographs, paintings, graphics, and illustrations.

[0936] "Preprocessing" refers to the process of converting images into a format suitable for analysis and use by performing operations such as resizing, cropping, filtering, and face masking.

[0937] "Computer system" refers to the entire system that processes and analyzes data, including servers and cloud-based computing resources.

[0938] A "machine learning model" refers to an algorithm that learns from large amounts of data and automatically recognizes specific patterns and features.

[0939] "Analysis" is the process of using machine learning models to identify and extract specific features and patterns.

[0940] "Generation" is the process of creating new images or information based on analyzed data.

[0941] A "messaging application" is an application that allows users to send and receive text, images, stamps, etc.

[0942] This invention relates to a system that allows users to easily create original stamps using their own images. The system employs a method in which users upload images, which are preprocessed on their devices, then analyzed and generated on a server, and the generated stamps are registered in a messaging application.

[0943] Hardware and software used

[0944] Users access the system using devices such as smartphones and tablets, and use a LINE stamp generation application that runs on these devices. On the server side, a high-performance cloud infrastructure is used, and TensorFlow and OpenCV are used for machine learning models.

[0945] Specific processing

[0946] 1. Launch the app

[0947] The user launches the LINE stamp generation application on their smartphone or tablet and taps the app icon to launch the app.

[0948] 2. Select and upload photos

[0949] Users select the photo they want to use as a stamp from their device's gallery and tap the "Upload" button in the app to send the photo to the server.

[0950] 3. Photo preprocessing

[0951] The device pre-processes the photo before sending it to the server. Specifically, it does the following:

[0952] Resize: Reduce the image resolution to 800x800 pixels.

[0953] Crop: Cut out unnecessary background parts of the photo and center only the person.

[0954] Filtering: Increases image brightness by 10% and contrast by 15%.

[0955] Face Mask: Using AI technology, the face is automatically detected and a 10x10 pixel mosaic is applied.

[0956] 4. Data Transmission

[0957] The photos are pre-processed on the terminal and sent to a server via the Internet.

[0958] 5. Photo Analysis

[0959] The server analyzes the photo using machine learning models (e.g. TensorFlow, OpenCV). Specifically, it performs the following steps:

[0960] Feature extraction: Extracting photo features such as smiles and poses.

[0961] Topic-based: Identify elements related to a given theme (e.g., "Holidays").

[0962] 6. Stamp generation

[0963] A stamp is generated based on the extracted features and themes. For example, a stamp of a smiling person can be generated with the text "It's fun!"

[0964] 7. Data return

[0965] The generated stamp data is returned from the server to the terminal.

[0966] 8. Registering with the LINE app

[0967] The device will register the received stamps in the LINE application. Specifically, select "Stamp Management" from the LINE app's settings menu and then select "Add New Stamps."

[0968] Specific examples

[0969] For example, consider a scenario where a user wants to upload photos of a family picnic on the weekend to a LINE sticker generation app. The user selects a picnic photo from the smartphone gallery and taps the "Upload" button. The app resizes the photo (e.g., to 800x800 pixels), crops unnecessary background, increases brightness by 10%, increases contrast by 15%, and blurs the face (e.g., blurs the face by 10x10 pixels), before sending it to the server.

[0970] The server uses a generative AI model to analyze the photo, extracting features such as a "smile" or "picnic scene," and generates stamps that correspond to the theme "holiday." The stamps are accompanied by the text "It's fun!" The generated stamp set is then sent back from the server to the user's smartphone, which then registers the stamps in the LINE application. The user can then immediately use these original stamps in LINE chats with family and friends.

[0971] Examples of prompt statements

[0972] Prompt: "Generate a fun stamp related to a holiday picnic. The photo shows a smiling family."

[0973] Example output from the AI ​​model: "A stamp of a family having a picnic, with the text 'This is fun!'"

[0974] As described above, the present invention provides a system that allows users to easily create original stamps using their own images and use them in messaging applications.

[0975] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0976] Step 1: User launches app

[0977] The user launches the LINE stamp generation application on an information device such as a smartphone or tablet. The app launches when the user taps a specific icon. The input is the operation of the information device, and the output is the launch of the application.

[0978] Step 2: User selects and uploads photo

[0979] The user selects the image they want to use as a stamp from the photo gallery in the app. After selecting, they press the "Upload" button to send the image to the server. The input is the user's selected image and instructions to upload, and the output is the selected image data sent from the device to the server.

[0980] Step 3: Pre-processing photos on the device

[0981] The device receives the images uploaded by the user and performs pre-processing. The specific pre-processing operations include:

[0982] Resize: Reduce the resolution of an image to 800x800 pixels. Take the original high-resolution image as input and get the resized image as output.

[0983] Cropping: Cut out unnecessary background parts and leave important parts (such as people) in the center. The input is the entire image, and the output is the cropped image.

[0984] Filtering: Optimizes the brightness and contrast of an image. The input is the original image, and the output is the filtered image. Specifically, the brightness is increased by 10% and the contrast is increased by 15%.

[0985] Face mask: AI technology automatically detects the face and applies a 10x10 pixel mosaic process. The input is the original image, and the output is an image with the face mosaicked.

[0986] Step 4: Send data from the device to the server

[0987] The device sends the preprocessed image data to the server. The input is the preprocessed image, and the output is the image data sent to the server. The transmission occurs over a network connection.

[0988] Step 5: Photo analysis by the server

[0989] The server analyzes the preprocessed images using machine learning models (e.g., TensorFlow, OpenCV). Specific operations include:

[0990] Feature extraction: Extract image features such as smiles and poses. The input is a preprocessed image, and the output is feature data.

[0991] Topic matching: Identify elements related to a specified theme (e.g., "holidays"). The input is theme information and feature data, and the output is the corresponding elements.

[0992] Step 6: Server-generated stamps

[0993] The server generates stamps based on the extracted features and themes. Specific operations include:

[0994] Template Selection: Select an appropriate stamp template. The input is feature data and a topic, and the output is the selected template.

[0995] Add Text: Automatically add text that matches the theme (e.g., "It's fun!") to the stamp. The input is a template and text data, and the output is a completed stamp image.

[0996] Step 7: Sending data back from the server to the device

[0997] The server returns the generated stamp data to the user's terminal. There is the generated stamp data as input and the stamp data sent to the terminal as output.

[0998] Step 8: Register stamps via terminal

[0999] The device registers the received stamp with the messaging application. Specific actions include:

[1000] Data reception: Receives stamp data sent from the server. There is the sent stamp data as input and the received stamp data as output.

[1001] Registration process: Registers stamps in the LINE application. Specifically, select "Stamp Management" from the LINE app's settings menu and execute "Add New Stamp." The stamp data received is the input, and the registered stamp is displayed in the LINE app as the output.

[1002] This allows users to easily create original LINE stamps and use them in messaging applications.

[1003] (Application example 1)

[1004] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1005] To improve the user experience in virtual stores, a system is needed that allows users to easily create original virtual avatars that reflect their own characteristics and use them to enjoy shopping and interacting with other users. However, current systems require advanced expertise and a significant amount of time for users to create avatars from their own photos, making them difficult for average users to use. Another problem is that the environment for properly registering and using generated avatars in virtual stores is not yet in place.

[1006] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1007] In this invention, the server includes means for analyzing a photo using a generative AI model, means for automatically generating a virtual avatar based on the analysis results, and means for selecting content for the virtual avatar based on a specified prompt and automatically adding customization elements. This allows users to easily generate original virtual avatars from their own photos without specialized knowledge or skills and use the avatars in virtual stores.

[1008] "Device" means the electronic device used by a User to upload photos and receive / register generated virtual avatars.

[1009] "Photo preprocessing" is the process of preparing photos for analysis and avatar generation, including resizing, cropping, filtering, and face masking.

[1010] The "server" is a central processing unit that receives pre-processed photos, analyzes the photos using a generative AI model, and generates and returns a virtual avatar based on the analysis results.

[1011] A "generative AI model" is an artificial intelligence algorithm that analyzes information from a photograph, extracts features, and generates a virtual avatar.

[1012] A "virtual avatar" is a digital character generated by extracting features from a user's photograph, and is used by the user in a virtual store.

[1013] A "prompt" is an instruction used for analysis, a piece of text that tells the generative AI model how to analyze a photo and what specific features it has.

[1014] A "virtual store" is a virtual commercial space where users can shop and interact using virtual avatars they create.

[1015] "Customization elements" are decorative elements such as the appearance, belongings, and background of a virtual avatar, and are automatically added based on user specifications or prompts.

[1016] "Analysis results" are data obtained after a photo is analyzed by a generative AI model, and include information based on feature extraction and prompt text.

[1017] This invention is a system that allows users to easily create original virtual avatars from their own photographs and use those avatars in virtual stores. The specific system configuration and processing method will be described below.

[1018] Main configuration

[1019] 1. User Device

[1020] User devices can be smartphones, tablets, smart glasses, head-mounted displays, etc. User devices provide a means for users to take, select, and upload their own photos.

[1021] 2. Server

[1022] The server receives the uploaded photo, analyzes it using a generative AI model, generates a virtual avatar based on the analysis results, adds any necessary customization elements, and sends it back to the user's device.

[1023] 3. Generative AI Models

[1024] A generative AI model is an artificial intelligence algorithm that extracts features from a photo and generates a virtual avatar based on them, using machine learning frameworks such as TensorFlow and PyTorch.

[1025] Processing method

[1026] 1. Upload a photo

[1027] Users select photos from their device's camera or gallery and upload them to the server through the application, which then imports the user's photos into the system.

[1028] 2. Photo pre-processing

[1029] The device performs pre-processing on the uploaded photo, including resizing, cropping, filtering, and face masking, to prepare the photo for analysis and avatar generation.

[1030] 3. Server-side analysis

[1031] The server receives the pre-processed photos and performs feature analysis using a generative AI model, which extracts features such as smiles and poses from the photos.

[1032] 4. Virtual Avatar Generation

[1033] Based on the extracted features, the server automatically generates a virtual avatar, determines the avatar's content based on the specified prompt, and adds customization elements, such as background, clothing, and accessories.

[1034] 5. Return and register the generated virtual avatar

[1035] The generated virtual avatar is sent back from the server to the user's device, which then registers the avatar in a virtual store, allowing the user to use the avatar for shopping and interacting with other users.

[1036] Specific examples

[1037] For example, a user can upload a photo of themselves using a virtual shopping app. The app processes the photo, analyzes it with an AI model to extract features, and generates a virtual avatar that reflects the user's characteristics. This avatar can then be used in a virtual store, allowing the user to shop or participate in events.

[1038] An example of a specified prompt sentence is:

[1039] Extract features from a user's facial photo and provide the data needed to generate a virtual avatar. Prioritize analysis of smiles, movement direction, and facial features.

[1040] This allows users to easily create their own original virtual avatar and enjoy a rich user experience in a virtual store.

[1041] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1042] Step 1:

[1043] The user uploads a photo on the device. They select a photo from the device's camera or gallery and press the "Upload" button to import the photo into the application. The input is the photo file selected by the user, and the output is the photo file to be uploaded. With the user's operation, the photo is ready to be sent to the server.

[1044] Step 2:

[1045] The device preprocesses the photo. The input is the uploaded photo file. The device first resizes the photo. Next, it crops it to keep only the important elements. After that, it filters the photo to adjust the brightness and contrast, and blurs the face if necessary. The output is the preprocessed photo data.

[1046] Step 3:

[1047] The server receives the preprocessed photos. The input is the preprocessed photo data, and the server prepares the received data for analysis. The output is the photo data ready for analysis.

[1048] Step 4:

[1049] The server analyzes the photo using a generative AI model. The input is photo data that can be analyzed, and the generative AI model extracts features from the photo (e.g., smile, facial expression, pose). The output is the feature extraction results, i.e., prominent feature data obtained from the photo.

[1050] Step 5:

[1051] The server generates a virtual avatar based on the specified prompt. The input is the feature extraction results and the prompt. The content of the virtual avatar is determined based on the analysis results, and customization elements (e.g., clothing, background, accessories) are automatically added. The output is the generated virtual avatar. An example of a prompt is: "Please extract features from the user's facial photograph and provide the data necessary to generate a virtual avatar. Please prioritize analysis of smiles, movement direction, and facial features."

[1052] Step 6:

[1053] The server returns the generated virtual avatar to the user's device. The input is the generated virtual avatar, and the output is the avatar data returned to the user's device. The server ensures that the avatar is properly delivered to the user's device through the return process.

[1054] Step 7:

[1055] The user's device registers the returned virtual avatar in the virtual store. The input is the returned avatar data, and the output is the avatar information registered in the virtual store. The device registers the avatar using the virtual store's API, and the user is ready to use the avatar to act in the virtual store.

[1056] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1057] This invention is a system that allows users to easily generate LINE stamps from their own photos, recognizing and reflecting the user's emotions in the process. This is achieved by sequentially processing the photos: uploading, preprocessing, analysis, emotion recognition, stamp generation, and stamp usage. The specific details of each processing step are described below.

[1058] User operation

[1059] Users simply launch the LINE stamp generation application on their smartphone and follow the app's instructions to select a photo from their gallery and upload it to the server.

[1060] Terminal side processing

[1061] The device receives the photos uploaded by the user and performs the following preprocessing:

[1062] 1. Resize: Change the resolution of the photo to a size suitable for analysis or use.

[1063] 2. Crop: Cut out unnecessary parts of the photo and leave only the parts suitable for stamping.

[1064] 3. Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[1065] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[1066] Server-side processing

[1067] The server receives the uploaded photo and processes it as follows:

[1068] 1. Feature extraction: The server uses an AI model to extract features from the photo (e.g., smile, pose).

[1069] 2. Emotion Recognition: The emotion engine analyzes photos and recognizes the user's emotions. For example, it identifies emotions such as "happiness" or "surprise" based on facial expressions and posture.

[1070] 3. Theme matching: Identify elements related to the specified theme (e.g., "holidays"). The server selects an appropriate stamp template based on the features most relevant to the theme.

[1071] 4. Stamp generation: Stamps are automatically generated based on the extracted features and themes. The generated stamps reflect elements of the user's photo and include text appropriate to the theme (e.g., "It's fun!"). The design and text of the stamps are also adjusted based on the results of emotion recognition.

[1072] The generated stamp is sent back from the server to the user's terminal.

[1073] Registering stamps on the device

[1074] The device registers the received stamp data in the LINE app:

[1075] 1. Data reception: The terminal receives the generated stamp data from the server.

[1076] 2. Registration process: Add stamps to the LINE application.

[1077] This allows users to easily use LINE stamps.

[1078] Specific examples

[1079] Consider the case where a user uploads photos taken during a trip to a LINE stamp generation app. The user selects and uploads photos from their smartphone gallery. The device then resizes, crops, filters, and blurs faces before sending the photos to the server.

[1080] The server uses an AI model to analyze the photo and extract features such as a "smile" or "tourist spot pose." The emotion engine recognizes that the user in the photo is "feeling happy." The server generates a stamp corresponding to the theme "travel," and adds text such as "Great trip!" to the stamp. Emotion recognition allows the stamp design to have bright colors and a fun atmosphere.

[1081] The generated stamp set is sent back to the user's smartphone from the server and automatically registered in the LINE app, allowing the user to immediately use the original stamps in LINE chats with family and friends.

[1082] In this way, the present invention provides a system that allows users to easily create and use original LINE stamps. By using emotion recognition functionality to generate stamps that reflect the user's emotions, a more personalized experience is provided.

[1083] The processing flow will be explained below.

[1084] Processing step details

[1085] Step 1:

[1086] The user launches the LINE stamp generation application on their smartphone and selects a photo from their gallery, then taps the "Upload" button on the app.

[1087] Step 2:

[1088] After the device receives the selected photo from the user, it resizes the photo, changing the photo resolution to a size suitable for analysis and use. For example, a 5000x4000 pixel photo is resized to 1000x800 pixels.

[1089] Step 3:

[1090] Your device will trim away unwanted background from your photo, determine the area you want to use, and crop the photo so that only the subject remains, for example, cropping only around a person.

[1091] Step 4:

[1092] Your phone will apply filters to your photos, including adjusting brightness, contrast, and color tone, for example, brightening dark photos and enhancing contrast.

[1093] Step 5:

[1094] Using the device's AI technology, it automatically detects faces in photos and applies mosaic processing to protect privacy. For example, it applies a mosaic filter to all faces in the photo.

[1095] Step 6:

[1096] The device sends the pre-processed photo data to the server, which receives the data.

[1097] Step 7:

[1098] The server analyzes the photo using an AI model, extracting features (e.g., smile, pose) and identifying key elements of the photo.

[1099] Step 8:

[1100] The server's emotion engine further analyzes the photo to identify the user's emotion, for example, identifying "happiness" from the curvature of the smile and the degree of eye opening.

[1101] Step 9:

[1102] The server identifies elements related to the specified topic (e.g., "holidays") and selects appropriate stamp templates based on the features most relevant to the topic.

[1103] Step 10:

[1104] The server automatically generates stamps based on the user's features, theme, and recognized emotion. The generated stamps reflect elements of the user's photo and automatically add text according to the theme. The text and design are adjusted based on the results of emotion recognition.

[1105] Step 11:

[1106] The generated stamp data is packaged and sent back from the server to the terminal, which receives this data.

[1107] Step 12:

[1108] The stamp data received by the device is registered in the LINE application. The stamp is added to the LINE app and becomes available for use by users.

[1109] Step 13:

[1110] Users can open the LINE app and use the newly added stamps to send messages to friends and family in the chat screen. Users can use the generated stamps as they like.

[1111] Through this series of processes, users can easily create original LINE stamps and use personalized stamps when communicating with friends. The emotion recognition function ensures that the generated stamps are more in line with the user's emotions, giving them special meaning.

[1112] Example 2

[1113] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1114] Conventional image editing and sticker generation systems make it difficult for users to easily create LINE stickers from their own photos and create personalized stickers that reflect their emotions. Furthermore, to improve the user experience during the sticker generation process, advanced image processing techniques and complex operations are often required, making them difficult for average users to use. Therefore, there is a need for a system that can easily and efficiently generate stickers using users' images and automatically adjust the design to reflect the user's emotions or specific themes.

[1115] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for analyzing images using an artificial intelligence model, means for recognizing the user's emotion in the image using an emotion recognition engine, and means for selecting the content of stamps based on a specified theme. This makes it possible to automatically extract features from images uploaded by users, recognize specific emotions, and further generate optimal stamps based on the results. This allows users to generate and use high-quality personalized stamps with simple operations.

[1116] "User" means an individual or entity that uses the System to upload images and generate and use stamps.

[1117] "Terminal" refers to a portable information terminal such as a smartphone or tablet used by a user.

[1118] "Image" refers to visual information such as photographs and illustrations that users upload using their devices.

[1119] "Preprocessing" refers to a series of operations that the device performs on the image, including resizing, cropping, filtering, and face masking.

[1120] "Server" refers to a remote computer system that analyzes pre-processed images received from a terminal and generates stamps.

[1121] "Artificial intelligence model" refers to a trained algorithm or program used for image analysis and stamp generation.

[1122] An "emotion recognition engine" refers to software that analyzes a user's facial expressions and posture in an image and identifies their emotions.

[1123] "Theme" refers to the theme or concept specified when creating a stamp.

[1124] "Stamps" refer to user-generated visual content used in communication applications such as LINE.

[1125] "Communication application" refers to software for exchanging messages using user-generated stamps.

[1126] "Resizing" refers to the process of changing the resolution of image data.

[1127] "Trimming" refers to the process of cutting out unnecessary parts of an image.

[1128] "Filtering" refers to the process of optimizing the brightness, contrast, color, etc. of an image.

[1129] "Facial masking" refers to the process of detecting facial areas in an image and applying mosaic processing to protect privacy.

[1130] MODE FOR CARRYING OUT THE INVENTION

[1131] This invention relates to a system that allows users to easily create LINE stamps from their own images and reflect the user's emotions. This system is realized by sequentially processing the image upload, preprocessing, analysis, emotion recognition, stamp generation, and stamp usage. The specific details of each processing step are described below.

[1132] User operation

[1133] The user launches the LINE stamp generation application on their smartphone. Following the application's instructions, the user selects the image they want to turn into a stamp from the image gallery stored on their smartphone and taps the "Upload" button. This operation sends the selected image data to the server.

[1134] Terminal side processing

[1135] The device receives the image uploaded by the user and performs the following preprocessing:

[1136] 1. Resizing: Changing the image resolution to a size suitable for analysis or use. Specifically, the device uses an image processing library (e.g., OpenCV) to change the number of pixels in the image.

[1137] 2. Cropping: Cut out unnecessary parts of the image and leave only the parts suitable for stamping. Specifically, the device applies an image processing algorithm to automatically crop the center of the image.

[1138] 3. Filtering: Optimizes the brightness, contrast, and color of the image. Specifically, the device performs filtering to adjust the overall color tone and brightness.

[1139] 4. Face Masking: To protect privacy, AI technology is used to automatically detect and blur faces. Specifically, the device uses a face recognition API (e.g., Google Vision API) to apply a blur effect to the face.

[1140] The processed image data is sent to the server via an HTTP request.

[1141] Server-side processing

[1142] The server receives the pre-processed image and performs the following operations:

[1143] 1. Feature extraction: The server uses an artificial intelligence model (e.g., TensorFlow or PyTorch) to analyze image features (e.g., smile, pose).

[1144] 2. Emotion Recognition: Use an emotion recognition engine (e.g., EmotionAPI) to identify emotions (e.g., joy, surprise) from the user's facial expressions in the image.

[1145] 3. Theme response: Elements related to the specified theme (e.g., "travel") are identified, and the server refers to its internal database to select stamp templates related to the theme.

[1146] 4. Sticker generation: The server automatically generates stickers based on the extracted features and themes. Specifically, it uses a generative AI model (e.g., GANs model) to generate stickers that correspond to the image features and themes. Based on the results of emotion recognition, it adjusts the design and text (e.g., "That's fun!").

[1147] The generated stamp data is sent to the user's terminal as an HTTP response.

[1148] Registering stamps on the device

[1149] The device processes the received stamp data and registers it in the LINE application. The specific steps are as follows:

[1150] 1. Data reception: The terminal receives the stamp data returned from the server and stores it in a temporary file or memory.

[1151] 2. Registering to the LINE app: The device uses the LINE API to register the new stamp to the app.

[1152] Specific examples

[1153] Consider the case where a user uploads photos taken during a trip to a LINE sticker generation app. The user selects travel photos from the smartphone gallery and taps the "Upload" button. The device then resizes, crops, and filters the image data, blurring the faces, before sending it to the server. The server analyzes features from the images, such as smiles and poses at tourist spots, and uses an emotion engine to recognize "feelings of joy." Based on the theme "travel," the server generates stickers containing text such as "Wonderful trip!" The generated sticker set is then sent back from the server to the user's device and automatically registered in the LINE app. The user can then immediately use these original stickers in LINE chats with family and friends.

[1154] An example of a prompt to be input to the generative AI model is as follows:

[1155] 1. "Use photos from your trip to create LINE stamps that show you having fun."

[1156] 2. "Create a stamp with the theme of 'surprise' based on a photo of a surprised expression."

[1157] 3. "Create a stamp set that represents 'family togetherness' based on a family photo."

[1158] In this way, the system of the present invention allows users to create original LINE stamps with simple operations and use stamps that reflect their emotions.

[1159] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1160] Step 1:

[1161] The user launches the LINE stamp generation application on their smartphone, selects the image they want to use as a stamp from their device's gallery, and taps the "Upload" button.

[1162] Specific behavior: The user taps the app icon to launch it, follows the guide to select an image, and presses the "Upload" button.

[1163] Input: An image selected from your phone's gallery.

[1164] Output: Image data is sent to the server.

[1165] Step 2:

[1166] The device receives the uploaded image and performs the following preprocessing:

[1167] 1. Resize: Change the image resolution to a size suitable for analysis or use.

[1168] What happens: The device uses an image processing library (e.g., OpenCV) to scale the image to the appropriate resolution.

[1169] Input: Image data uploaded by the user.

[1170] Output: The resized image data.

[1171] 2. Trimming: Cut out unnecessary parts of the image and leave only the parts suitable for stamping.

[1172] What it does: Your device applies an image processing algorithm to automatically crop the center of the image.

[1173] Input: The resized image data.

[1174] Output: Cropped image data.

[1175] 3. Filtering: Optimizing the brightness, contrast, hue, etc. of the image.

[1176] What it does: Your device applies filters to adjust the overall color and brightness.

[1177] Input: Cropped image data.

[1178] Output: The filtered image data.

[1179] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[1180] What it does: The device uses a facial recognition API (e.g., Google Vision API) to apply a mosaic effect to the face.

[1181] Input: The filtered image data.

[1182] Output: Face mask processed image data.

[1183] Step 3:

[1184] The terminal sends the preprocessed image data to the server using an HTTP request.

[1185] Specific operation: The device generates an HTTP request and sends a payload containing image data to the server address.

[1186] Input: Face mask processed image data.

[1187] Output: The preprocessed image is sent to the server.

[1188] Step 4:

[1189] The server receives the pre-processed image and performs the following operations:

[1190] 1. Feature extraction: The server uses an artificial intelligence model (e.g., TensorFlow or PyTorch) to analyze image features (e.g., smile, pose).

[1191] How it works: The server inputs image data into a pre-trained AI model to extract key features.

[1192] Input: Preprocessed image data.

[1193] Output: Extracted feature data.

[1194] 2. Emotion Recognition: Use an emotion recognition engine (e.g., EmotionAPI) to identify emotions (e.g., joy, surprise) from the user's facial expressions in the image.

[1195] Specific operation: The server calls the emotion recognition API and analyzes emotions from the feature data.

[1196] Input: Extracted feature data.

[1197] Output: Recognized emotion data.

[1198] 3. Theme response: Elements related to the specified theme are identified, and the server refers to its internal database to select a stamp template related to the theme.

[1199] Specific operation: The server retrieves information relevant to the topic from the database and selects the relevant template.

[1200] Input: Recognized emotion data and a given topic.

[1201] Output: The selected stamp template.

[1202] 4. Sticker generation: The server automatically generates stickers based on the image features and themes. Specifically, the server uses a generative AI model (e.g., a GANs model) to generate stickers that correspond to the image features and themes. Based on the results of emotion recognition, the server adjusts the design and text (e.g., "That's fun!").

[1203] Specific operation: The server inputs image features and topic information into the GANs model to generate stamps. The emotion recognition results are reflected in the design and text.

[1204] Input: Selected stamp template, extracted features, and emotion data.

[1205] Output: The generated stamp data.

[1206] Step 5:

[1207] The server returns the generated stamp data to the user's terminal.

[1208] Specific operation: The server generates an HTTP response and sends a payload containing the stamp data to the user terminal.

[1209] Input: The generated stamp data.

[1210] Output: The stamp data is sent to the user's device.

[1211] Step 6:

[1212] The device processes the received stamp data and registers it in the LINE application:

[1213] 1. Data reception: The terminal receives the stamp data sent from the server and stores it in a temporary file or memory.

[1214] Specific operation: The device analyzes the HTTP response, obtains the stamp data, and saves it.

[1215] Input: Stamp data sent from the server.

[1216] Output: Temporary file or data held in memory.

[1217] 2. Registering to the LINE app: The device uses the LINE API to register the new stamp to the app.

[1218] Specific operation: The device calls the LINE API and adds the stamp to the application.

[1219] Input: Stamp data held in temporary file or memory.

[1220] Output: Stamps registered in the LINE app.

[1221] This allows users to immediately use the original LINE stamps they have created.

[1222] (Application example 2)

[1223] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1224] By using personalized stamps that reflect the emotions of the users who create LINE stamps as advertisements, it is possible to effectively deliver advertisements to target users. However, conventional systems lack a means to recognize users' emotions and generate personalized advertisements, making it difficult to implement effective advertising campaigns.

[1225] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for uploading photos, a means for pre-processing the photos, and a means for recognizing a user's emotions using an emotion recognition engine and adding text for personalized advertising to generate advertising stamps. This makes it possible to generate stamp designs that reflect the user's emotional state and create personalized advertising campaigns.

[1226] "User" is the end user who uploads photos and utilizes the stamp generation process.

[1227] A "terminal" is an electronic device such as a smartphone or tablet that is operated by a user.

[1228] The "means for uploading photos" is a function that allows a user to use a terminal to send photos that they have to a server.

[1229] "Preprocessing" refers to the process of resizing, cropping, filtering, etc., of uploaded photos to prepare them for subsequent analysis.

[1230] The "server" is a computer system that receives uploaded photos and performs various analytical processes and stamp generation.

[1231] An "AI model" is a mathematical model that uses artificial intelligence technology to analyze photos and extract features and recognize emotions.

[1232] An "emotion recognition engine" is a software component for analyzing and recognizing a user's emotions in a photo.

[1233] A "stamp" is an image generated for use within a communication application that reflects a user's photo and emotion recognition results.

[1234] "Means for automatically generating stamps" is a function that automatically creates stamps based on analysis results and emotion recognition results.

[1235] "Advertising stamps" are stamps used in promotional activities such as advertising campaigns, and include personalized messages or designs.

[1236] "Text for personalized advertisement" is text for an advertisement message that is generated by reflecting the result of user emotion recognition.

[1237] The "means for registering" is a function for adding the generated stamp to a communication application so that the user can immediately use it.

[1238] This invention is a system that allows users to easily create LINE stamps for advertising purposes from their own photos, and recognizes and reflects the user's emotions in the process. Details of its implementation are described below.

[1239] Hardware and software used

[1240] Hardware:

[1241] Device: Electronic device such as a smartphone or tablet

[1242] Server: A computer system for analysis and stamp generation

[1243] software:

[1244] TensorFlow: A library for building and using emotion recognition models

[1245] Pillow (PIL): An image processing library for pre-processing photos.

[1246] Examples of data processing and data calculation

[1247] 1. Upload and pre-process photos:

[1248] Users upload photos for advertising purposes using their devices, which undergo the following pre-processing:

[1249] Resize: Change the resolution of your photos to the optimal size.

[1250] Crop: Cut out unwanted parts of a photo, leaving only the center.

[1251] Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[1252] 2. Server analysis and emotion recognition:

[1253] The server receives the preprocessed photos and analyzes them using an AI model. An emotion recognition engine recognizes the user's emotions in the photos. Specifically, the AI ​​model uses TensorFlow to analyze emotions and identify emotions such as "joy" or "surprise."

[1254] 3. Automatic stamp generation:

[1255] Based on the results of the analysis and emotion recognition, advertising stamps are automatically generated. Specifically, advertising text is added according to the features in the photo and the recognized emotion. For example, if the recognized emotion is "joy," the text "Happy moment!" is added.

[1256] 4. Return and register your stamps:

[1257] The server returns the generated stamp to the device, which then automatically registers the received stamp in a communication application (e.g., LINE). This allows the user to immediately use the generated advertising stamp.

[1258] Examples of concrete examples and prompts

[1259] Examples:

[1260] Suppose an advertising agency uploads a photo of a smiling user to promote the launch of a new product. The device resizes, crops, and filters the photo, then sends it to a server. The server uses TensorFlow to analyze the photo and recognize the emotion "joy." The resulting sticker contains the text "Happy Moment!" and is used as a promotional sticker.

[1261] Example prompt sentence:

[1262] "Upload photos taken during the promotional event and generate LINE stickers that reflect the emotion of joy. Add text such as 'Fun Moments!' to the generated stickers."

[1263] This invention makes it easy to create personalized advertisements that reflect the user's emotions, and enables effective advertisement delivery to target users.

[1264] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1265] Step 1:

[1266] A user uploads a photo using a terminal. To do this, the user launches the terminal application, selects a photo for advertising from the gallery, and presses the "Upload" button. The input is the photo file selected by the user, and the output is the photo data sent to the server.

[1267] Step 2:

[1268] The device preprocesses the uploaded photos, specifically:

[1269] Resize: Change the photo resolution to 128x128 pixels.

[1270] Crop: Cut out the center to narrow the focus.

[1271] Filtering: Optimize brightness and contrast.

[1272] The input is the original uploaded photo file, and the output is preprocessed photo data that has been resized, cropped, and filtered.

[1273] Step 3:

[1274] The server receives the preprocessed photos and analyzes them using an AI model. Specifically, it extracts the user's facial expressions and facial features in the photos. The input is the preprocessed photo data, and the output is the analysis results of facial features and emotions.

[1275] Step 4:

[1276] The emotion recognition engine recognizes the user's emotions based on the analysis results. Specifically, an emotion recognition model using TensorFlow identifies emotions such as "happiness" and "surprise." The input is the analysis results by the AI ​​model, and the output is data on the recognized emotional state.

[1277] Step 5:

[1278] The server automatically generates advertising stamps based on the recognized emotions. Here, text (e.g., "A fun moment!") that matches the recognized emotion (e.g., "Joy") is added. The input is the emotion recognition result and photo data, and the output is the generated stamp data.

[1279] Step 6:

[1280] The generated stamp is returned to the terminal. The server sends the generated stamp data to the terminal. The input is the stamp data generated by the server, and the output is the stamp data sent to the terminal.

[1281] Step 7:

[1282] The device registers the returned stamps in a communication application (e.g., LINE). The user can use the generated stamps as is without performing any special operations. The input is the stamp data returned from the server, and the output is the stamps registered in the LINE app.

[1283] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1284] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1285] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1286] [Fourth embodiment]

[1287] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1288] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1289] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1290] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1291] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1292] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1293] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1294] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1295] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1296] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1297] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1298] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1299] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1300] The present invention is a system that allows users to easily create LINE stamps from their own photos, and is realized by sequentially processing the photos: uploading, preprocessing, analyzing, stamp generation, and stamp usage. The specific details of each processing step are described below.

[1301] User operation

[1302] The device used by the user is an electronic device such as a smartphone or tablet. The user first launches the LINE stamp generation application, follows the instructions of the application to select a photo they have on hand, and uploads it to the server.

[1303] Terminal side processing

[1304] The device receives the photos uploaded by the user and performs the following preprocessing:

[1305] 1. Resize: Change the resolution of the photo to a size suitable for analysis or use.

[1306] 2. Crop: Cut out unnecessary parts of the photo and leave only the parts suitable for stamping.

[1307] 3. Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[1308] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[1309] Server-side processing

[1310] The server receives the pre-processed photo data from the device and analyzes it using the AI ​​model:

[1311] 1. Feature extraction: The AI ​​model extracts features from the photo (e.g., smile, pose).

[1312] 2. Topic response: Identify elements related to a given topic (e.g., "Holidays").

[1313] 3. Stamp generation: Based on the extracted features and the theme, an appropriate template is selected and a stamp is generated.

[1314] 4. Add text: Automatically add text that matches the theme (e.g., "It's fun!") to the stamp.

[1315] The generated stamp is sent back to the terminal from the server.

[1316] Registering stamps on the device

[1317] The device registers the received stamp data in the LINE app:

[1318] 1. Data reception: The terminal receives the generated stamp data from the server.

[1319] 2. Registration process: Add stamps to the LINE application.

[1320] This allows users to easily use LINE stamps.

[1321] Specific examples

[1322] Consider a case where a user goes on a family picnic on the weekend and uploads photos of the picnic to a LINE sticker generation app. The user selects a picnic photo from the smartphone gallery, and the app resizes, crops, filters, and blurs faces before sending it to the server.

[1323] The server uses an AI model to analyze the photo, extracting features such as a "smile" or "picnic scene," and generates stamps that correspond to the theme of "holidays." The stamps are accompanied by text such as "That's fun!"

[1324] The generated stamp set is sent back to the user's smartphone from the server and automatically registered in the LINE app, allowing the user to immediately use the original stamps in LINE chats with family and friends.

[1325] In this way, the present invention provides a system that allows users to easily create and use original LINE stamps. With privacy protection features and automated analysis, the system can be easily used by many more users.

[1326] The processing flow will be explained below.

[1327] Processing step details

[1328] Step 1:

[1329] The user launches the LINE stamp generation application on their smartphone and selects a photo from their gallery, then taps the "Upload" button on the app.

[1330] Step 2:

[1331] The device receives the photo selected by the user and resizes the photo to a resolution suitable for analysis and use.

[1332] Step 3:

[1333] The device will trim the background of the photo to extract only the parts suitable for making a stamp, and adjust the brightness and contrast of the image using filtering if necessary.

[1334] Step 4:

[1335] Using the device's AI technology, it automatically detects faces in photos and applies a mosaic process to protect privacy, making it difficult to identify individuals.

[1336] Step 5:

[1337] The device sends the preprocessed photo data to the server, which receives the data and uses an AI model to analyze the photo's features.

[1338] Step 6:

[1339] The server uses the analysis results to identify elements related to the specified topic (e.g., "holidays") and selects an appropriate stamp template based on the features most relevant to the topic.

[1340] Step 7:

[1341] The server automatically generates stamps based on the characteristics and theme of the photo. The generated stamps reflect elements of the user's photo and include text appropriate to the theme (e.g., "That's fun!").

[1342] Step 8:

[1343] The server packages the generated stamp data and sends it back to the terminal, which receives the data.

[1344] Step 9:

[1345] The device registers the received stamp data in the LINE application. Once registration is complete, the new stamp set will be added to the user's LINE stamp library.

[1346] Step 10:

[1347] Users can open the LINE app and use the newly added stamps to send messages to friends and family in the chat screen. Users can use the generated stamps as they like.

[1348] Through this series of processes, users can easily convert their photos into LINE stickers and use them conveniently. The entire processing flow is automated, making it a user-friendly system that also protects privacy.

[1349] Example 1

[1350] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1351] Conventionally, it has been difficult for users to easily generate original stamps using their own photos and use them in messaging applications. Furthermore, due to a lack of privacy protection and image optimization processing when generating stamps, it has been difficult to easily create high-quality stamps. The present invention aims to solve these problems and provide a system that allows users to easily generate high-quality original stamps.

[1352] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1353] In this invention, the server includes means for receiving the preprocessed image and analyzing the image using a machine learning model, means for automatically generating an image based on the analysis result, and means for returning the generated image to the information device, thereby enabling users to easily generate high-quality original stamps and use them in messaging applications.

[1354] A "user" is an individual or legal entity that uses a particular application or system.

[1355] "Information devices" refers to a wide range of devices, including electronic terminals such as smartphones, tablets, and personal computers.

[1356] "Image" refers to visual data such as photographs, paintings, graphics, and illustrations.

[1357] "Preprocessing" refers to the process of converting images into a format suitable for analysis and use by performing operations such as resizing, cropping, filtering, and face masking.

[1358] "Computer system" refers to the entire system that processes and analyzes data, including servers and cloud-based computing resources.

[1359] A "machine learning model" refers to an algorithm that learns from large amounts of data and automatically recognizes specific patterns and features.

[1360] "Analysis" is the process of using machine learning models to identify and extract specific features and patterns.

[1361] "Generation" is the process of creating new images or information based on analyzed data.

[1362] A "messaging application" is an application that allows users to send and receive text, images, stamps, etc.

[1363] This invention relates to a system that allows users to easily create original stamps using their own images. The system employs a method in which users upload images, which are preprocessed on their devices, then analyzed and generated on a server, and the generated stamps are registered in a messaging application.

[1364] Hardware and software used

[1365] Users access the system using devices such as smartphones and tablets, and use a LINE stamp generation application that runs on these devices. On the server side, a high-performance cloud infrastructure is used, and TensorFlow and OpenCV are used for machine learning models.

[1366] Specific processing

[1367] 1. Launch the app

[1368] The user launches the LINE stamp generation application on their smartphone or tablet and taps the app icon to launch the app.

[1369] 2. Select and upload photos

[1370] Users select the photo they want to use as a stamp from their device's gallery and tap the "Upload" button in the app to send the photo to the server.

[1371] 3. Photo preprocessing

[1372] The device pre-processes the photo before sending it to the server. Specifically, it does the following:

[1373] Resize: Reduce the image resolution to 800x800 pixels.

[1374] Crop: Cut out unnecessary background parts of the photo and center only the person.

[1375] Filtering: Increases image brightness by 10% and contrast by 15%.

[1376] Face Mask: Using AI technology, the face is automatically detected and a 10x10 pixel mosaic is applied.

[1377] 4. Data Transmission

[1378] The photos are pre-processed on the terminal and sent to a server via the Internet.

[1379] 5. Photo Analysis

[1380] The server analyzes the photo using machine learning models (e.g. TensorFlow, OpenCV). Specifically, it performs the following steps:

[1381] Feature extraction: Extracting photo features such as smiles and poses.

[1382] Topic-based: Identify elements related to a given theme (e.g., "Holidays").

[1383] 6. Stamp generation

[1384] A stamp is generated based on the extracted features and themes. For example, a stamp of a smiling person can be generated with the text "It's fun!"

[1385] 7. Data return

[1386] The generated stamp data is returned from the server to the terminal.

[1387] 8. Registering with the LINE app

[1388] The device will register the received stamps in the LINE application. Specifically, select "Stamp Management" from the LINE app's settings menu and then select "Add New Stamps."

[1389] Specific examples

[1390] For example, consider a scenario where a user wants to upload photos of a family picnic on the weekend to a LINE sticker generation app. The user selects a picnic photo from the smartphone gallery and taps the "Upload" button. The app resizes the photo (e.g., to 800x800 pixels), crops unnecessary background, increases brightness by 10%, increases contrast by 15%, and blurs the face (e.g., blurs the face by 10x10 pixels), before sending it to the server.

[1391] The server uses a generative AI model to analyze the photo, extracting features such as a "smile" or "picnic scene," and generates stamps that correspond to the theme "holiday." The stamps are accompanied by the text "It's fun!" The generated stamp set is then sent back from the server to the user's smartphone, which then registers the stamps in the LINE application. The user can then immediately use these original stamps in LINE chats with family and friends.

[1392] Examples of prompt statements

[1393] Prompt: "Generate a fun stamp related to a holiday picnic. The photo shows a smiling family."

[1394] Example output from the AI ​​model: "A stamp of a family having a picnic, with the text 'This is fun!'"

[1395] As described above, the present invention provides a system that allows users to easily create original stamps using their own images and use them in messaging applications.

[1396] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1397] Step 1: User launches app

[1398] The user launches the LINE stamp generation application on an information device such as a smartphone or tablet. The app launches when the user taps a specific icon. The input is the operation of the information device, and the output is the launch of the application.

[1399] Step 2: User selects and uploads photo

[1400] The user selects the image they want to use as a stamp from the photo gallery in the app. After selecting, they press the "Upload" button to send the image to the server. The input is the user's selected image and instructions to upload, and the output is the selected image data sent from the device to the server.

[1401] Step 3: Pre-processing photos on the device

[1402] The device receives the images uploaded by the user and performs pre-processing. The specific pre-processing operations include:

[1403] Resize: Reduce the resolution of an image to 800x800 pixels. Take the original high-resolution image as input and get the resized image as output.

[1404] Cropping: Cut out unnecessary background parts and leave important parts (such as people) in the center. The input is the entire image, and the output is the cropped image.

[1405] Filtering: Optimizes the brightness and contrast of an image. The input is the original image, and the output is the filtered image. Specifically, the brightness is increased by 10% and the contrast is increased by 15%.

[1406] Face mask: AI technology automatically detects the face and applies a 10x10 pixel mosaic process. The input is the original image, and the output is an image with the face mosaicked.

[1407] Step 4: Send data from the device to the server

[1408] The device sends the preprocessed image data to the server. The input is the preprocessed image, and the output is the image data sent to the server. The transmission occurs over a network connection.

[1409] Step 5: Photo analysis by the server

[1410] The server analyzes the preprocessed images using machine learning models (e.g., TensorFlow, OpenCV). Specific operations include:

[1411] Feature extraction: Extract image features such as smiles and poses. The input is a preprocessed image, and the output is feature data.

[1412] Topic matching: Identify elements related to a specified theme (e.g., "holidays"). The input is theme information and feature data, and the output is the corresponding elements.

[1413] Step 6: Server-generated stamps

[1414] The server generates stamps based on the extracted features and themes. Specific operations include:

[1415] Template Selection: Select an appropriate stamp template. The input is feature data and a topic, and the output is the selected template.

[1416] Add Text: Automatically add text that matches the theme (e.g., "It's fun!") to the stamp. The input is a template and text data, and the output is a completed stamp image.

[1417] Step 7: Sending data back from the server to the device

[1418] The server returns the generated stamp data to the user's terminal. There is the generated stamp data as input and the stamp data sent to the terminal as output.

[1419] Step 8: Register stamps via terminal

[1420] The device registers the received stamp with the messaging application. Specific actions include:

[1421] Data reception: Receives stamp data sent from the server. There is the sent stamp data as input and the received stamp data as output.

[1422] Registration process: Registers stamps in the LINE application. Specifically, select "Stamp Management" from the LINE app's settings menu and execute "Add New Stamp." The stamp data received is the input, and the registered stamp is displayed in the LINE app as the output.

[1423] This allows users to easily create original LINE stamps and use them in messaging applications.

[1424] (Application example 1)

[1425] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1426] To improve the user experience in virtual stores, a system is needed that allows users to easily create original virtual avatars that reflect their own characteristics and use them to enjoy shopping and interacting with other users. However, current systems require advanced expertise and a significant amount of time for users to create avatars from their own photos, making them difficult for average users to use. Another problem is that the environment for properly registering and using generated avatars in virtual stores is not yet in place.

[1427] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1428] In this invention, the server includes means for analyzing a photo using a generative AI model, means for automatically generating a virtual avatar based on the analysis results, and means for selecting content for the virtual avatar based on a specified prompt and automatically adding customization elements. This allows users to easily generate original virtual avatars from their own photos without specialized knowledge or skills and use the avatars in virtual stores.

[1429] "Device" means the electronic device used by a User to upload photos and receive / register generated virtual avatars.

[1430] "Photo preprocessing" is the process of preparing photos for analysis and avatar generation, including resizing, cropping, filtering, and face masking.

[1431] The "server" is a central processing unit that receives pre-processed photos, analyzes the photos using a generative AI model, and generates and returns a virtual avatar based on the analysis results.

[1432] A "generative AI model" is an artificial intelligence algorithm that analyzes information from a photograph, extracts features, and generates a virtual avatar.

[1433] A "virtual avatar" is a digital character generated by extracting features from a user's photograph, and is used by the user in a virtual store.

[1434] A "prompt" is an instruction used for analysis, a piece of text that tells the generative AI model how to analyze a photo and what specific features it has.

[1435] A "virtual store" is a virtual commercial space where users can shop and interact using virtual avatars they create.

[1436] "Customization elements" are decorative elements such as the appearance, belongings, and background of a virtual avatar, and are automatically added based on user specifications or prompts.

[1437] "Analysis results" are data obtained after a photo is analyzed by a generative AI model, and include information based on feature extraction and prompt text.

[1438] This invention is a system that allows users to easily create original virtual avatars from their own photographs and use those avatars in virtual stores. The specific system configuration and processing method will be described below.

[1439] Main configuration

[1440] 1. User Device

[1441] User devices can be smartphones, tablets, smart glasses, head-mounted displays, etc. User devices provide a means for users to take, select, and upload their own photos.

[1442] 2. Server

[1443] The server receives the uploaded photo, analyzes it using a generative AI model, generates a virtual avatar based on the analysis results, adds any necessary customization elements, and sends it back to the user's device.

[1444] 3. Generative AI Models

[1445] A generative AI model is an artificial intelligence algorithm that extracts features from a photo and generates a virtual avatar based on them, using machine learning frameworks such as TensorFlow and PyTorch.

[1446] Processing method

[1447] 1. Upload a photo

[1448] Users select photos from their device's camera or gallery and upload them to the server through the application, which then imports the user's photos into the system.

[1449] 2. Photo pre-processing

[1450] The device performs pre-processing on the uploaded photo, including resizing, cropping, filtering, and face masking, to prepare the photo for analysis and avatar generation.

[1451] 3. Server-side analysis

[1452] The server receives the pre-processed photos and performs feature analysis using a generative AI model, which extracts features such as smiles and poses from the photos.

[1453] 4. Virtual Avatar Generation

[1454] Based on the extracted features, the server automatically generates a virtual avatar, determines the avatar's content based on the specified prompt, and adds customization elements, such as background, clothing, and accessories.

[1455] 5. Return and register the generated virtual avatar

[1456] The generated virtual avatar is sent back from the server to the user's device, which then registers the avatar in a virtual store, allowing the user to use the avatar for shopping and interacting with other users.

[1457] Specific examples

[1458] For example, a user can upload a photo of themselves using a virtual shopping app. The app processes the photo, analyzes it with an AI model to extract features, and generates a virtual avatar that reflects the user's characteristics. This avatar can then be used in a virtual store, allowing the user to shop or participate in events.

[1459] An example of a specified prompt sentence is:

[1460] Extract features from a user's facial photo and provide the data needed to generate a virtual avatar. Prioritize analysis of smiles, movement direction, and facial features.

[1461] This allows users to easily create their own original virtual avatar and enjoy a rich user experience in a virtual store.

[1462] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1463] Step 1:

[1464] The user uploads a photo on the device. They select a photo from the device's camera or gallery and press the "Upload" button to import the photo into the application. The input is the photo file selected by the user, and the output is the photo file to be uploaded. With the user's operation, the photo is ready to be sent to the server.

[1465] Step 2:

[1466] The device preprocesses the photo. The input is the uploaded photo file. The device first resizes the photo. Next, it crops it to keep only the important elements. After that, it filters the photo to adjust the brightness and contrast, and blurs the face if necessary. The output is the preprocessed photo data.

[1467] Step 3:

[1468] The server receives the preprocessed photos. The input is the preprocessed photo data, and the server prepares the received data for analysis. The output is the photo data ready for analysis.

[1469] Step 4:

[1470] The server analyzes the photo using a generative AI model. The input is photo data that can be analyzed, and the generative AI model extracts features from the photo (e.g., smile, facial expression, pose). The output is the feature extraction results, i.e., prominent feature data obtained from the photo.

[1471] Step 5:

[1472] The server generates a virtual avatar based on the specified prompt. The input is the feature extraction results and the prompt. The content of the virtual avatar is determined based on the analysis results, and customization elements (e.g., clothing, background, accessories) are automatically added. The output is the generated virtual avatar. An example of a prompt is: "Please extract features from the user's facial photograph and provide the data necessary to generate a virtual avatar. Please prioritize analysis of smiles, movement direction, and facial features."

[1473] Step 6:

[1474] The server returns the generated virtual avatar to the user's device. The input is the generated virtual avatar, and the output is the avatar data returned to the user's device. The server ensures that the avatar is properly delivered to the user's device through the return process.

[1475] Step 7:

[1476] The user's device registers the returned virtual avatar in the virtual store. The input is the returned avatar data, and the output is the avatar information registered in the virtual store. The device registers the avatar using the virtual store's API, and the user is ready to use the avatar to act in the virtual store.

[1477] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1478] This invention is a system that allows users to easily generate LINE stamps from their own photos, recognizing and reflecting the user's emotions in the process. This is achieved by sequentially processing the photos: uploading, preprocessing, analysis, emotion recognition, stamp generation, and stamp usage. The specific details of each processing step are described below.

[1479] User operation

[1480] Users simply launch the LINE stamp generation application on their smartphone and follow the app's instructions to select a photo from their gallery and upload it to the server.

[1481] Terminal side processing

[1482] The device receives the photos uploaded by the user and performs the following preprocessing:

[1483] 1. Resize: Change the resolution of the photo to a size suitable for analysis or use.

[1484] 2. Crop: Cut out unnecessary parts of the photo and leave only the parts suitable for stamping.

[1485] 3. Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[1486] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[1487] Server-side processing

[1488] The server receives the uploaded photo and processes it as follows:

[1489] 1. Feature extraction: The server uses an AI model to extract features from the photo (e.g., smile, pose).

[1490] 2. Emotion Recognition: The emotion engine analyzes photos and recognizes the user's emotions. For example, it identifies emotions such as "happiness" or "surprise" based on facial expressions and posture.

[1491] 3. Theme matching: Identify elements related to the specified theme (e.g., "holidays"). The server selects an appropriate stamp template based on the features most relevant to the theme.

[1492] 4. Stamp generation: Stamps are automatically generated based on the extracted features and themes. The generated stamps reflect elements of the user's photo and include text appropriate to the theme (e.g., "It's fun!"). The design and text of the stamps are also adjusted based on the results of emotion recognition.

[1493] The generated stamp is sent back from the server to the user's terminal.

[1494] Registering stamps on the device

[1495] The device registers the received stamp data in the LINE app:

[1496] 1. Data reception: The terminal receives the generated stamp data from the server.

[1497] 2. Registration process: Add stamps to the LINE application.

[1498] This allows users to easily use LINE stamps.

[1499] Specific examples

[1500] Consider the case where a user uploads photos taken during a trip to a LINE stamp generation app. The user selects and uploads photos from their smartphone gallery. The device then resizes, crops, filters, and blurs faces before sending the photos to the server.

[1501] The server uses an AI model to analyze the photo and extract features such as a "smile" or "tourist spot pose." The emotion engine recognizes that the user in the photo is "feeling happy." The server generates a stamp corresponding to the theme "travel," and adds text such as "Great trip!" to the stamp. Emotion recognition allows the stamp design to have bright colors and a fun atmosphere.

[1502] The generated stamp set is sent back to the user's smartphone from the server and automatically registered in the LINE app, allowing the user to immediately use the original stamps in LINE chats with family and friends.

[1503] In this way, the present invention provides a system that allows users to easily create and use original LINE stamps. By using emotion recognition functionality to generate stamps that reflect the user's emotions, a more personalized experience is provided.

[1504] The processing flow will be explained below.

[1505] Processing step details

[1506] Step 1:

[1507] The user launches the LINE stamp generation application on their smartphone and selects a photo from their gallery, then taps the "Upload" button on the app.

[1508] Step 2:

[1509] After the device receives the selected photo from the user, it resizes the photo, changing the photo resolution to a size suitable for analysis and use. For example, a 5000x4000 pixel photo is resized to 1000x800 pixels.

[1510] Step 3:

[1511] Your device will trim away unwanted background from your photo, determine the area you want to use, and crop the photo so that only the subject remains, for example, cropping only around a person.

[1512] Step 4:

[1513] Your phone will apply filters to your photos, including adjusting brightness, contrast, and color tone, for example, brightening dark photos and enhancing contrast.

[1514] Step 5:

[1515] Using the device's AI technology, it automatically detects faces in photos and applies mosaic processing to protect privacy. For example, it applies a mosaic filter to all faces in the photo.

[1516] Step 6:

[1517] The device sends the pre-processed photo data to the server, which receives the data.

[1518] Step 7:

[1519] The server analyzes the photo using an AI model, extracting features (e.g., smile, pose) and identifying key elements of the photo.

[1520] Step 8:

[1521] The server's emotion engine further analyzes the photo to identify the user's emotion, for example, identifying "happiness" from the curvature of the smile and the degree of eye opening.

[1522] Step 9:

[1523] The server identifies elements related to the specified topic (e.g., "holidays") and selects appropriate stamp templates based on the features most relevant to the topic.

[1524] Step 10:

[1525] The server automatically generates stamps based on the user's features, theme, and recognized emotion. The generated stamps reflect elements of the user's photo and automatically add text according to the theme. The text and design are adjusted based on the results of emotion recognition.

[1526] Step 11:

[1527] The generated stamp data is packaged and sent back from the server to the terminal, which receives this data.

[1528] Step 12:

[1529] The stamp data received by the device is registered in the LINE application. The stamp is added to the LINE app and becomes available for use by users.

[1530] Step 13:

[1531] Users can open the LINE app and use the newly added stamps to send messages to friends and family in the chat screen. Users can use the generated stamps as they like.

[1532] Through this series of processes, users can easily create original LINE stamps and use personalized stamps when communicating with friends. The emotion recognition function ensures that the generated stamps are more in line with the user's emotions, giving them special meaning.

[1533] Example 2

[1534] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1535] Conventional image editing and sticker generation systems make it difficult for users to easily create LINE stickers from their own photos and create personalized stickers that reflect their emotions. Furthermore, to improve the user experience during the sticker generation process, advanced image processing techniques and complex operations are often required, making them difficult for average users to use. Therefore, there is a need for a system that can easily and efficiently generate stickers using users' images and automatically adjust the design to reflect the user's emotions or specific themes.

[1536] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for analyzing images using an artificial intelligence model, means for recognizing the user's emotion in the image using an emotion recognition engine, and means for selecting the content of stamps based on a specified theme. This makes it possible to automatically extract features from images uploaded by users, recognize specific emotions, and further generate optimal stamps based on the results. This allows users to generate and use high-quality personalized stamps with simple operations.

[1537] "User" means an individual or entity that uses the System to upload images and generate and use stamps.

[1538] "Terminal" refers to a portable information terminal such as a smartphone or tablet used by a user.

[1539] "Image" refers to visual information such as photographs and illustrations that users upload using their devices.

[1540] "Preprocessing" refers to a series of operations that the device performs on the image, including resizing, cropping, filtering, and face masking.

[1541] "Server" refers to a remote computer system that analyzes pre-processed images received from a terminal and generates stamps.

[1542] "Artificial intelligence model" refers to a trained algorithm or program used for image analysis and stamp generation.

[1543] An "emotion recognition engine" refers to software that analyzes a user's facial expressions and posture in an image and identifies their emotions.

[1544] "Theme" refers to the theme or concept specified when creating a stamp.

[1545] "Stamps" refer to user-generated visual content used in communication applications such as LINE.

[1546] "Communication application" refers to software for exchanging messages using user-generated stamps.

[1547] "Resizing" refers to the process of changing the resolution of image data.

[1548] "Trimming" refers to the process of cutting out unnecessary parts of an image.

[1549] "Filtering" refers to the process of optimizing the brightness, contrast, color, etc. of an image.

[1550] "Facial masking" refers to the process of detecting facial areas in an image and applying mosaic processing to protect privacy.

[1551] MODE FOR CARRYING OUT THE INVENTION

[1552] This invention relates to a system that allows users to easily create LINE stamps from their own images and reflect the user's emotions. This system is realized by sequentially processing the image upload, preprocessing, analysis, emotion recognition, stamp generation, and stamp usage. The specific details of each processing step are described below.

[1553] User operation

[1554] The user launches the LINE stamp generation application on their smartphone. Following the application's instructions, the user selects the image they want to turn into a stamp from the image gallery stored on their smartphone and taps the "Upload" button. This operation sends the selected image data to the server.

[1555] Terminal side processing

[1556] The device receives the image uploaded by the user and performs the following preprocessing:

[1557] 1. Resizing: Changing the image resolution to a size suitable for analysis or use. Specifically, the device uses an image processing library (e.g., OpenCV) to change the number of pixels in the image.

[1558] 2. Cropping: Cut out unnecessary parts of the image and leave only the parts suitable for stamping. Specifically, the device applies an image processing algorithm to automatically crop the center of the image.

[1559] 3. Filtering: Optimizes the brightness, contrast, and color of the image. Specifically, the device performs filtering to adjust the overall color tone and brightness.

[1560] 4. Face Masking: To protect privacy, AI technology is used to automatically detect and blur faces. Specifically, the device uses a face recognition API (e.g., Google Vision API) to apply a blur effect to the face.

[1561] The processed image data is sent to the server via an HTTP request.

[1562] Server-side processing

[1563] The server receives the pre-processed image and performs the following operations:

[1564] 1. Feature extraction: The server uses an artificial intelligence model (e.g., TensorFlow or PyTorch) to analyze image features (e.g., smile, pose).

[1565] 2. Emotion Recognition: Use an emotion recognition engine (e.g., EmotionAPI) to identify emotions (e.g., joy, surprise) from the user's facial expressions in the image.

[1566] 3. Theme response: Elements related to the specified theme (e.g., "travel") are identified, and the server refers to its internal database to select stamp templates related to the theme.

[1567] 4. Sticker generation: The server automatically generates stickers based on the extracted features and themes. Specifically, it uses a generative AI model (e.g., GANs model) to generate stickers that correspond to the image features and themes. Based on the results of emotion recognition, it adjusts the design and text (e.g., "That's fun!").

[1568] The generated stamp data is sent to the user's terminal as an HTTP response.

[1569] Registering stamps on the device

[1570] The device processes the received stamp data and registers it in the LINE application. The specific steps are as follows:

[1571] 1. Data reception: The terminal receives the stamp data returned from the server and stores it in a temporary file or memory.

[1572] 2. Registering to the LINE app: The device uses the LINE API to register the new stamp to the app.

[1573] Specific examples

[1574] Consider the case where a user uploads photos taken during a trip to a LINE sticker generation app. The user selects travel photos from the smartphone gallery and taps the "Upload" button. The device then resizes, crops, and filters the image data, blurring the faces, before sending it to the server. The server analyzes features from the images, such as smiles and poses at tourist spots, and uses an emotion engine to recognize "feelings of joy." Based on the theme "travel," the server generates stickers containing text such as "Wonderful trip!" The generated sticker set is then sent back from the server to the user's device and automatically registered in the LINE app. The user can then immediately use these original stickers in LINE chats with family and friends.

[1575] An example of a prompt to be input to the generative AI model is as follows:

[1576] 1. "Use photos from your trip to create LINE stamps that show you having fun."

[1577] 2. "Create a stamp with the theme of 'surprise' based on a photo of a surprised expression."

[1578] 3. "Create a stamp set that represents 'family togetherness' based on a family photo."

[1579] In this way, the system of the present invention allows users to create original LINE stamps with simple operations and use stamps that reflect their emotions.

[1580] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1581] Step 1:

[1582] The user launches the LINE stamp generation application on their smartphone, selects the image they want to use as a stamp from their device's gallery, and taps the "Upload" button.

[1583] Specific behavior: The user taps the app icon to launch it, follows the guide to select an image, and presses the "Upload" button.

[1584] Input: An image selected from your phone's gallery.

[1585] Output: Image data is sent to the server.

[1586] Step 2:

[1587] The device receives the uploaded image and performs the following preprocessing:

[1588] 1. Resize: Change the image resolution to a size suitable for analysis or use.

[1589] What happens: The device uses an image processing library (e.g., OpenCV) to scale the image to the appropriate resolution.

[1590] Input: Image data uploaded by the user.

[1591] Output: The resized image data.

[1592] 2. Trimming: Cut out unnecessary parts of the image and leave only the parts suitable for stamping.

[1593] What it does: Your device applies an image processing algorithm to automatically crop the center of the image.

[1594] Input: The resized image data.

[1595] Output: Cropped image data.

[1596] 3. Filtering: Optimizing the brightness, contrast, hue, etc. of the image.

[1597] What it does: Your device applies filters to adjust the overall color and brightness.

[1598] Input: Cropped image data.

[1599] Output: The filtered image data.

[1600] 4. Face mask: To protect privacy, AI technology is used to automatically detect and blur facial areas.

[1601] What it does: The device uses a facial recognition API (e.g., Google Vision API) to apply a mosaic effect to the face.

[1602] Input: The filtered image data.

[1603] Output: Face mask processed image data.

[1604] Step 3:

[1605] The terminal sends the preprocessed image data to the server using an HTTP request.

[1606] Specific operation: The device generates an HTTP request and sends a payload containing image data to the server address.

[1607] Input: Face mask processed image data.

[1608] Output: The preprocessed image is sent to the server.

[1609] Step 4:

[1610] The server receives the pre-processed image and performs the following operations:

[1611] 1. Feature extraction: The server uses an artificial intelligence model (e.g., TensorFlow or PyTorch) to analyze image features (e.g., smile, pose).

[1612] How it works: The server inputs image data into a pre-trained AI model to extract key features.

[1613] Input: Preprocessed image data.

[1614] Output: Extracted feature data.

[1615] 2. Emotion Recognition: Use an emotion recognition engine (e.g., EmotionAPI) to identify emotions (e.g., joy, surprise) from the user's facial expressions in the image.

[1616] Specific operation: The server calls the emotion recognition API and analyzes emotions from the feature data.

[1617] Input: Extracted feature data.

[1618] Output: Recognized emotion data.

[1619] 3. Theme response: Elements related to the specified theme are identified, and the server refers to its internal database to select a stamp template related to the theme.

[1620] Specific operation: The server retrieves information relevant to the topic from the database and selects the relevant template.

[1621] Input: Recognized emotion data and a given topic.

[1622] Output: The selected stamp template.

[1623] 4. Sticker generation: The server automatically generates stickers based on the image features and themes. Specifically, the server uses a generative AI model (e.g., a GANs model) to generate stickers that correspond to the image features and themes. Based on the results of emotion recognition, the server adjusts the design and text (e.g., "That's fun!").

[1624] Specific operation: The server inputs image features and topic information into the GANs model to generate stamps. The emotion recognition results are reflected in the design and text.

[1625] Input: Selected stamp template, extracted features, and emotion data.

[1626] Output: The generated stamp data.

[1627] Step 5:

[1628] The server returns the generated stamp data to the user's terminal.

[1629] Specific operation: The server generates an HTTP response and sends a payload containing the stamp data to the user terminal.

[1630] Input: The generated stamp data.

[1631] Output: The stamp data is sent to the user's device.

[1632] Step 6:

[1633] The device processes the received stamp data and registers it in the LINE application:

[1634] 1. Data reception: The terminal receives the stamp data sent from the server and stores it in a temporary file or memory.

[1635] Specific operation: The device analyzes the HTTP response, obtains the stamp data, and saves it.

[1636] Input: Stamp data sent from the server.

[1637] Output: Temporary file or data held in memory.

[1638] 2. Registering to the LINE app: The device uses the LINE API to register the new stamp to the app.

[1639] Specific operation: The device calls the LINE API and adds the stamp to the application.

[1640] Input: Stamp data held in temporary file or memory.

[1641] Output: Stamps registered in the LINE app.

[1642] This allows users to immediately use the original LINE stamps they have created.

[1643] (Application example 2)

[1644] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1645] By using personalized stamps that reflect the emotions of the users who create LINE stamps as advertisements, it is possible to effectively deliver advertisements to target users. However, conventional systems lack a means to recognize users' emotions and generate personalized advertisements, making it difficult to implement effective advertising campaigns.

[1646] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for uploading photos, a means for pre-processing the photos, and a means for recognizing a user's emotions using an emotion recognition engine and adding text for personalized advertising to generate advertising stamps. This makes it possible to generate stamp designs that reflect the user's emotional state and create personalized advertising campaigns.

[1647] "User" is the end user who uploads photos and utilizes the stamp generation process.

[1648] A "terminal" is an electronic device such as a smartphone or tablet that is operated by a user.

[1649] The "means for uploading photos" is a function that allows a user to use a terminal to send photos that they have to a server.

[1650] "Preprocessing" refers to the process of resizing, cropping, filtering, etc., of uploaded photos to prepare them for subsequent analysis.

[1651] The "server" is a computer system that receives uploaded photos and performs various analytical processes and stamp generation.

[1652] An "AI model" is a mathematical model that uses artificial intelligence technology to analyze photos and extract features and recognize emotions.

[1653] An "emotion recognition engine" is a software component for analyzing and recognizing a user's emotions in a photo.

[1654] A "stamp" is an image generated for use within a communication application that reflects a user's photo and emotion recognition results.

[1655] "Means for automatically generating stamps" is a function that automatically creates stamps based on analysis results and emotion recognition results.

[1656] "Advertising stamps" are stamps used in promotional activities such as advertising campaigns, and include personalized messages or designs.

[1657] "Text for personalized advertisement" is text for an advertisement message that is generated by reflecting the result of user emotion recognition.

[1658] The "means for registering" is a function for adding the generated stamp to a communication application so that the user can immediately use it.

[1659] This invention is a system that allows users to easily create LINE stamps for advertising purposes from their own photos, and recognizes and reflects the user's emotions in the process. Details of its implementation are described below.

[1660] Hardware and software used

[1661] Hardware:

[1662] Device: Electronic device such as a smartphone or tablet

[1663] Server: A computer system for analysis and stamp generation

[1664] software:

[1665] TensorFlow: A library for building and using emotion recognition models

[1666] Pillow (PIL): An image processing library for pre-processing photos.

[1667] Examples of data processing and data calculation

[1668] 1. Upload and pre-process photos:

[1669] Users upload photos for advertising purposes using their devices, which undergo the following pre-processing:

[1670] Resize: Change the resolution of your photos to the optimal size.

[1671] Crop: Cut out unwanted parts of a photo, leaving only the center.

[1672] Filtering: Optimize the brightness, contrast, color, etc. of your photos.

[1673] 2. Server analysis and emotion recognition:

[1674] The server receives the preprocessed photos and analyzes them using an AI model. An emotion recognition engine recognizes the user's emotions in the photos. Specifically, the AI ​​model uses TensorFlow to analyze emotions and identify emotions such as "joy" or "surprise."

[1675] 3. Automatic stamp generation:

[1676] Based on the results of the analysis and emotion recognition, advertising stamps are automatically generated. Specifically, advertising text is added according to the features in the photo and the recognized emotion. For example, if the recognized emotion is "joy," the text "Happy moment!" is added.

[1677] 4. Return and register your stamps:

[1678] The server returns the generated stamp to the device, which then automatically registers the received stamp in a communication application (e.g., LINE). This allows the user to immediately use the generated advertising stamp.

[1679] Examples of concrete examples and prompts

[1680] Examples:

[1681] Suppose an advertising agency uploads a photo of a smiling user to promote the launch of a new product. The device resizes, crops, and filters the photo, then sends it to a server. The server uses TensorFlow to analyze the photo and recognize the emotion "joy." The resulting sticker contains the text "Happy Moment!" and is used as a promotional sticker.

[1682] Example prompt sentence:

[1683] "Upload photos taken during the promotional event and generate LINE stickers that reflect the emotion of joy. Add text such as 'Fun Moments!' to the generated stickers."

[1684] This invention makes it easy to create personalized advertisements that reflect the user's emotions, and enables effective advertisement delivery to target users.

[1685] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1686] Step 1:

[1687] A user uploads a photo using a terminal. To do this, the user launches the terminal application, selects a photo for advertising from the gallery, and presses the "Upload" button. The input is the photo file selected by the user, and the output is the photo data sent to the server.

[1688] Step 2:

[1689] The device preprocesses the uploaded photos, specifically:

[1690] Resize: Change the photo resolution to 128x128 pixels.

[1691] Crop: Cut out the center to narrow the focus.

[1692] Filtering: Optimize brightness and contrast.

[1693] The input is the original uploaded photo file, and the output is preprocessed photo data that has been resized, cropped, and filtered.

[1694] Step 3:

[1695] The server receives the preprocessed photos and analyzes them using an AI model. Specifically, it extracts the user's facial expressions and facial features in the photos. The input is the preprocessed photo data, and the output is the analysis results of facial features and emotions.

[1696] Step 4:

[1697] The emotion recognition engine recognizes the user's emotions based on the analysis results. Specifically, an emotion recognition model using TensorFlow identifies emotions such as "happiness" and "surprise." The input is the analysis results by the AI ​​model, and the output is data on the recognized emotional state.

[1698] Step 5:

[1699] The server automatically generates advertising stamps based on the recognized emotions. Here, text (e.g., "A fun moment!") that matches the recognized emotion (e.g., "Joy") is added. The input is the emotion recognition result and photo data, and the output is the generated stamp data.

[1700] Step 6:

[1701] The generated stamp is returned to the terminal. The server sends the generated stamp data to the terminal. The input is the stamp data generated by the server, and the output is the stamp data sent to the terminal.

[1702] Step 7:

[1703] The device registers the returned stamps in a communication application (e.g., LINE). The user can use the generated stamps as is without performing any special operations. The input is the stamp data returned from the server, and the output is the stamps registered in the LINE app.

[1704] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1705] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1706] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1707] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1708] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1709] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1710] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1711] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1712] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1713] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1714] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1715] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1716] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1717] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1718] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1719] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1720] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1721] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1722] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1723] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1724] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1725] The following is further disclosed regarding the above embodiment.

[1726] (Claim 1)

[1727] A means for a user to upload a photo using a terminal;

[1728] a means for the device to preprocess the photo;

[1729] a server receiving the preprocessed photograph and analyzing the photograph using an AI model;

[1730] A means for automatically generating stamps based on the analysis results;

[1731] means for returning the generated stamp to the terminal;

[1732] The system includes means for the terminal to register the returned stamp with the communication application.

[1733] (Claim 2)

[1734] 10. The system of claim 1, wherein the terminal resizes, crops, and filters the photo.

[1735] (Claim 3)

[1736] 2. The system according to claim 1, wherein the server selects the content of the stamp based on a specified theme and automatically adds text.

[1737] "Example 1"

[1738] (Claim 1)

[1739] A means for a user to upload an image using an information device;

[1740] means for the information device to preprocess the image;

[1741] means for a computer system to receive the preprocessed image and analyze the image using a machine learning model;

[1742] means for automatically generating an image based on the analysis results;

[1743] means for returning the generated image to the information device;

[1744] The system includes means for the information appliance to register the returned image with a messaging application.

[1745] (Claim 2)

[1746] 10. The system of claim 1, wherein the information appliance resizes, crops, and filters the image.

[1747] (Claim 3)

[1748] 10. The system of claim 1, wherein the computer system selects image content based on a specified theme and automatically adds text.

[1749] "Application Example 1"

[1750] (Claim 1)

[1751] A means for a user to upload a photo using a terminal;

[1752] a means for the device to preprocess the photo;

[1753] a server receiving the preprocessed photograph and analyzing the photograph using the generative AI model;

[1754] A means for automatically generating a virtual avatar based on the analysis results;

[1755] A means for returning the generated virtual avatar to the terminal;

[1756] The system includes a means for registering the returned virtual avatar in a virtual store.

[1757] (Claim 2)

[1758] 10. The system of claim 1, wherein the terminal resizes, crops, and filters the photo.

[1759] (Claim 3)

[1760] 2. The system of claim 1, wherein the server selects content for the virtual avatar based on a specified prompt sentence and automatically adds customization elements.

[1761] "Example 2: Combining Emotion Engines"

[1762] (Claim 1)

[1763] A means for a user to upload an image using a terminal;

[1764] means for the terminal to preprocess the image;

[1765] means for the server to receive the preprocessed image and analyze the image using an artificial intelligence model;

[1766] means for recognizing a user's emotion in an image using an emotion recognition engine;

[1767] A means for selecting the content of the stamp based on a specified theme;

[1768] means for automatically generating stamps based on the results of the analysis and emotion recognition;

[1769] means for returning the generated stamp to the terminal;

[1770] The system includes means for the terminal to register the returned stamp with the communication application.

[1771] (Claim 2)

[1772] 10. The system of claim 1, wherein the terminal performs image resizing, cropping, filtering, and face masking.

[1773] (Claim 3)

[1774] The system of claim 1, wherein the server detects elements related to the specified theme, selects the content of the stamp, and further adjusts the design and text based on the results of the analysis and emotion recognition.

[1775] "Application example 2 when combining emotion engines"

[1776] (Claim 1)

[1777] A means for a user to upload a photo using a terminal;

[1778] a means for the device to preprocess the photo;

[1779] a server receiving the preprocessed photograph and analyzing the photograph using an AI model;

[1780] A means for automatically generating stamps based on the analysis results;

[1781] means for returning the generated stamp to the terminal;

[1782] A means for the terminal to register the returned stamp in a communication application;

[1783] means for recognizing a user's emotion using an emotion recognition engine and adding text for personalized advertising to generate stamps for advertising;

[1784] A system including:

[1785] (Claim 2)

[1786] 10. The system of claim 1, wherein the terminal resizes, crops, and filters the photo.

[1787] (Claim 3)

[1788] 2. The system according to claim 1, wherein the server selects the content of the stamp based on a specified theme and automatically adds text.

[1789] (Claim 4)

[1790] 10. The system of claim 1, wherein the system generates stamp designs that reflect the emotional state of a target user to generate stamps for advertising, thereby creating personalized advertising campaigns. [Explanation of symbols]

[1791] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for a user to upload a photo using a terminal; a means for the device to preprocess the photo; a server receiving the preprocessed photograph and analyzing the photograph using an AI model; A means for automatically generating stamps based on the analysis results; means for returning the generated stamp to the terminal; The system includes means for the terminal to register the returned stamp with the communication application.

2. The system of claim 1, wherein the terminal resizes, crops, and filters the photo.

3. 2. The system according to claim 1, wherein the server selects the content of the stamp based on a specified theme and automatically adds text.

Images