System

Abstract

A system is provided.SOLUTION: An image upload means for a user to register his / her own face photograph, a means for a server to receive the face photograph and perform image analysis, an image generation means for the server to generate images of the user wearing various costumes based on the analyzed face photograph, a means for the server to store the generated images in a database and transmit the images to a user terminal, a means for the user to browse the transmitted catalog and select a favorite costume, a reservation information transmission means for the user to make a try-on reservation and transmit the information to the server, and a means for the server to store the reservation information received from the user in a database, means for notifying a sales / rental operator.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention solves the problem of the time and effort required to select an outfit, such as a furisode or wedding dress. Conventional methods require users to expend a great deal of time and effort to select the most suitable outfit from a large number of options, making efficient selection difficult. Furthermore, multiple visits to the store are required to try on the outfit, which increases fatigue due to the physical travel and repeated fitting. The objective of this invention is to solve this problem and provide a method that allows users to efficiently and effectively select the most suitable outfit. [Means for solving the problem]

[0005] The present invention provides a system that includes an image uploading means for users to register their own facial photos, a server receiving the facial photos and performing image analysis, an image generation means for the server generating images of the user wearing various outfits based on the analyzed facial photos, a server storing the generated images in a database and transmitting them to the user's terminal, a means for the user to browse the transmitted catalog and select an outfit they like, a reservation information transmission means for the user to make a fitting reservation and transmit that information to the server, and a server storing the reservation information received from the user in a database and notifying the sales / rental business. This allows users to browse the outfit catalog featuring themselves as models based on their own facial photos and efficiently select outfits from the comfort of their own homes. Furthermore, fitting reservations can be easily made through the app, reducing the number of physical store visits and improving the efficiency of outfit selection.

[0006] The "image upload means" is a function that allows a user to upload a photo of their face to the application.

[0007] The "image analysis means" is a function that processes the facial photograph received by the server from the user, extracts facial features, and converts them into a format suitable for image generation.

[0008] "Image generation means" refers to the AI ​​technology or algorithm that the server uses to generate new images by combining the user's facial photo with various outfits based on the results of image analysis.

[0009] The "database storage means" is a function for temporarily or permanently storing generated images and associated metadata on a server or cloud storage.

[0010] "Transmission means to user terminal" refers to the communication function and protocol that the server uses to transmit the generated costume catalog to the user's terminal.

[0011] "Means for browsing the catalog" refers to the interface or function that allows the user to use the application to browse the costume catalog sent from the server.

[0012] "Try-on reservation means" refers to the online form or in-app reservation system that allows users to make a reservation to try on outfits selected from the catalog.

[0013] "Reservation information transmission means" refers to the communication function and protocol for transmitting the fitting reservation information entered by the user to the server.

[0014] "Means for notifying sales / rental businesses" refers to a system or protocol for notifying sales or rental businesses of the corresponding costumes of the fitting reservation information received by the server.

[0015] "Contract reward means" refers to a function or algorithm for calculating and recording a contract reward when a contract is concluded by the server.

[0016] "Means for extracting facial features" refers to the technology that allows the server to identify facial features from a user's facial photograph and extract the information necessary for image generation.

[0017] "Preprocessing means for standardizing facial images" refers to processing as part of image analysis to convert facial photographs into a format suitable for image generation, such as adjusting the resolution of the photograph and cropping out the facial area. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. The system includes an application that runs on the user's terminal, and image generation and data processing on the server side.

[0040] Specific operation of the system

[0041] 1. Registering a user's photo

[0042] Users launch the application and log in. After logging in, they can either take a photo of themselves using the camera or select and upload a photo from their existing photo library.

[0043] Example: If a user wants to choose a furisode kimono for their Coming of Age Ceremony, they can take a photo of their face directly into the app and upload it.

[0044] 2. Processing facial photos

[0045] The server receives the face photo uploaded by the user and uses image analysis algorithms to extract facial features, adjusting the resolution and cropping the face to standardize the face image.

[0046] Example: The server uses a library such as OpenCV to perform face recognition and convert the data into a standardized format.

[0047] 3. Clothing catalog generation using image generation AI

[0048] The server sends a request to the image generation AI based on the preprocessed facial image to generate images of the user wearing various costumes, which are then stored in a database.

[0049] Example: The server uses algorithms such as GAN to generate images of the user wearing a gorgeous furisode or an elegant wedding dress.

[0050] 4. Provision of catalogues

[0051] The server then sends the generated costume catalog to the user's device, where the user can browse the catalog featuring themselves as models and select their favorite outfit from the multiple options.

[0052] Example: A user uses an app to browse images of various furisode kimonos and choose their favorite design.

[0053] 5. Booking a try-on appointment

[0054] To try on an outfit selected from the catalog, the user fills out a reservation form within the app and makes a reservation. The reservation information is sent to the server.

[0055] Example: To try on the furisode kimono of a user's choice, the user enters the desired date and time and store information to confirm the reservation.

[0056] 6. Manage fitting reservations

[0057] The server stores the received reservation information in a database, notifies the costume sales / rental business, and, upon receiving confirmation from the business, notifies the user of the reservation details.

[0058] Example: The server notifies the business of the user's fitting reservation by email, and after confirmation from the business, notifies the user of the fitting date.

[0059] 7. Calculating and recording commissions

[0060] When a transaction is confirmed, the server records the information and calculates the commission for the transaction. This commission is provided by the sales / rental business and is used to maintain and operate the system.

[0061] Example: When a user actually rents a furisode, that information is recorded and the commission from the business is calculated.

[0062] In this way, users can browse a catalog based on their own face photo and make a fitting reservation to efficiently select the most suitable outfit. Sales / rental businesses also benefit from being able to efficiently approach customers through the system. By combining server-side image analysis, generation, and management functions, it is possible to significantly improve the user experience.

[0063] The processing flow will be explained below.

[0064] Step 1:

[0065] The user launches the application and logs in. They enter their user ID and password and access their account through the authentication screen.

[0066] Step 2:

[0067] Users can take a photo of themselves using the camera function or select and upload a photo from their existing photo library, then press the "Upload" button to send the photo to the server.

[0068] Step 3:

[0069] The server receives the face photo uploaded by the user, processes the HTTP request, and temporarily stores the image data.

[0070] Step 4:

[0071] The server analyzes the received facial photos and uses a facial recognition algorithm to crop the facial area, which involves using the OpenCV library to identify the face location.

[0072] Step 5:

[0073] The server extracts the facial area and standardizes the resolution, size, etc. This preprocessing allows the image generation AI to generate stable, high-quality images.

[0074] Step 6:

[0075] The server sends the preprocessed facial image to the image generation AI and requests it to generate images of the user wearing various outfits. The image generation AI then uses techniques such as GAN to generate multiple outfit patterns.

[0076] Step 7:

[0077] The server receives the generated images, adds corresponding metadata (e.g., type of costume, color, design) to each, and stores them in a database.

[0078] Step 8:

[0079] The server sends the generated clothing catalog to the user's device, sending the constructed catalog data as an HTTP response, allowing the user to view it in real time.

[0080] Step 9:

[0081] Users can browse the costume catalog sent to them within the application and select the costume they like. They can also check detailed information about the costume and other users' ratings by touching the screen.

[0082] Step 10:

[0083] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[0084] Step 11:

[0085] The device sends the fitting reservation information entered by the user to the server, generating an HTTP POST request and sending the reservation data to the server.

[0086] Step 12:

[0087] The server stores the received reservation information in a database, generates a reservation ID, and records the reservation details.

[0088] Step 13:

[0089] The server sends a fitting reservation notification to the corresponding costume sales / rental business via API or email.

[0090] Step 14:

[0091] The server receives the reservation confirmation from the business and notifies the user of the confirmation information. It also generates a notification message to inform the user of the fitting details.

[0092] Step 15:

[0093] When a transaction is confirmed, the server records the information in a database, saving data such as the transaction ID, transaction date and time, user ID, and business ID.

[0094] Step 16:

[0095] The server calculates the reward for the successful transaction and records the information. It calculates the reward amount based on the contract reward from the business operator and saves the corresponding record in the database.

[0096] The above is the specific flow of the program processing of this system, which allows users to efficiently and effectively select the most suitable outfit and easily make a fitting reservation.

[0097] Example 1

[0098] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] Today, users want to be able to try on various outfits and view the results in an efficient and reliable way using their own facial photos. However, current systems require cumbersome facial photo registration and analysis, which does not improve the user experience. Furthermore, efficient management is required for providing the generated images and accepting try-on reservations. It is necessary to solve these issues and provide a system that is convenient for both users and sales / rental businesses.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0101] In this invention, the server includes an image uploading means for users to register their own facial photos, a means for the server to receive the facial photos and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photos, a means for the server to save the generated images in a database and send them to a user terminal, a means for the user to browse the sent catalog and select a favorite outfit, a reservation information sending means for the user to make a try-on reservation and send that information to the server, a means for the server to save the reservation information received from the user in a database and notify the sales / rental company, an automatic adjustment means for the user terminal to automatically adjust the facial photos to an appropriate format, and a means for the server to execute an algorithm to suggest outfit combinations based on the facial images. This allows users to efficiently try on and select outfits based on their own facial photos.

[0102] "User" refers to an individual who uses this system to upload a photo of their face and generate images of them trying on clothes.

[0103] "Image upload means" refers to the function that allows users to send photos of their faces to the system.

[0104] "Server" refers to a computer system that analyzes facial photos received from users, generates costumed images using image generation AI, stores them in a database, and sends them to the user's device.

[0105] "Image analysis means" refers to the function of analyzing facial photographs received by the server and extracting facial features.

[0106] "Image generation means" refers to the function of the server to generate images of users wearing various costumes using a generative AI model based on analyzed facial photos.

[0107] "Database" refers to an information management system for storing images generated by the server and user fitting reservation information.

[0108] "User terminal" refers to a device such as a smartphone, tablet, or PC that a user uses to operate an application.

[0109] A "catalog" refers to a collection of images of users wearing costumes generated by the server, which can be viewed and selected by users.

[0110] The "reservation information transmission means" refers to a function for transmitting reservation information for trying on the costume selected by the user to the server.

[0111] "Automatic adjustment means" refers to a function that allows a user's device to automatically adjust uploaded facial photos into an appropriate format.

[0112] "Contract reward" refers to the reward calculated by the system based on transaction information when a user actually rents or purchases an outfit.

[0113] A "generative AI model" refers to an image generation algorithm used by the server, which generates costume images based on a user's facial photo using a generative artificial network (GAN) or similar.

[0114] A "prompt sentence" refers to an input sentence that specifies the expected result for a generative AI model.

[0115] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. The system includes an application that runs on the user's terminal, and image generation and data processing on the server side.

[0116] User Device

[0117] The user terminal includes devices such as smartphones, tablets, and PCs. The user uses the terminal to launch the application and take a photo of their face or select and upload it from a photo library. The terminal is equipped with an automatic adjustment means, which automatically adjusts the uploaded face photo to an appropriate format.

[0118] Example: When a user is choosing a furisode kimono for their Coming of Age Ceremony, they can use the app to take and upload a photo of their face.

[0119] server

[0120] The server is the central system that analyzes the received facial photos and generates images of the user in various costumes. The server uses the following specific software:

[0121] OpenCV: Used for analyzing and preprocessing face images.

[0122] TENSORFLOW®: Used for image generation using generative AI models (e.g., GANs).

[0123] The server has a pre-processing means for extracting the user's facial features using face recognition technology and standardizing the facial image, and also has a function for storing the generated costume image in a database and transmitting it to the user's terminal.

[0124] Specific behavior:

[0125] The server uses the OpenCV library to analyze facial features and inputs the standardized facial images into a generative AI model based on TensorFlow, which generates images of the user wearing various outfits. The images are stored in a database and then sent to the user's device.

[0126] user

[0127] Users can browse the sent catalog on the app and select their favorite outfits. They can then make a reservation to try on the outfits they selected within the app and send that information to the server. The server stores this information and notifies the sales / rental company.

[0128] Example: A user browses a catalog of furisode kimonos on the app, selects their favorite design, and makes a reservation to try them on.

[0129] Calculation of commission

[0130] The server records the information of the costumes that have been sold and calculates the commission for the sale. This commission is provided by the sales / rental business and is used for the maintenance and operation of the system.

[0131] Prompt Sentence Examples

[0132] "I want to choose a furisode for my Coming-of-Age Ceremony. Please upload a photo of your face and generate images of you trying on various furisode."

[0133] "Try on wedding dresses based on your own photo and create a catalog."

[0134] This allows users to efficiently try on and select outfits based on their own face photos, and sales / rental businesses can also use the system to effectively approach customers.

[0135] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0136] Step 1:

[0137] The user launches the application and logs in, which gives them access to the system, and then uses the camera function to take a photo of their face or upload one from their existing photo library.

[0138] Input: User authentication information (ID, password), face photo

[0139] Output: Face photo sent to server

[0140] Specific actions: Press the "Take Photo" button in the app to take a new photo or press the "Choose from Library" button to select an existing photo and upload a face photo. Press the "Upload" button to send the face photo to the server.

[0141] Step 2:

[0142] The server uses the OpenCV library to run a facial recognition algorithm on the face photo received from the user, extracting facial features, and then standardizes the face image (adjusting resolution and cropping the face) to complete preprocessing.

[0143] Input: Uploaded face photo

[0144] Output: Preprocessed face image

[0145] Specific operation: The server detects faces in the received photos using cv2.CascadeClassifier in the OpenCV library, crops the face area, and resizes the image to a resolution of 256x256 pixels. This completes the standardization process.

[0146] Step 3:

[0147] The server uses the preprocessed facial image to send a request to a generative AI model (likely a GAN) using the TensorFlow library to generate images of the user wearing various costumes, which are then stored in a database.

[0148] Input: Preprocessed face image

[0149] Output: An image of the user wearing the generated outfit

[0150] How it works: The preprocessed face image is input into a GAN model in the TensorFlow library, which then generates images of the user wearing various costumes and stores them in a database.

[0151] Step 4:

[0152] The server retrieves the generated outfit catalog (outfit images based on the user's face photo) from the database and sends it to the user's device. The user can browse this catalog using the app and select the outfit they like.

[0153] Input: Generated images stored in a database

[0154] Output: Outfit catalog sent to user's device

[0155] Specific operation: When a user presses the "View Catalog" button, the server retrieves the user's generated images from the database and sends them to the user's device. The user then views them on the app and presses the "Favorite" button to select an outfit.

[0156] Step 5:

[0157] The user fills out the fitting reservation form for the outfit they have chosen within the app and makes a reservation to try on the outfit. The reservation information is sent to the server.

[0158] Input: Selected outfit, desired fitting date and time, and other reservation information

[0159] Output: Reservation information sent to the server

[0160] Specific operation: The user presses the "Book a try-on" button, enters the desired date and time and store information in the reservation form, and presses the "Confirm reservation" button to send the reservation information to the server.

[0161] Step 6:

[0162] The server stores the received reservation information in a database and notifies the corresponding sales / rental business. After confirmation, the business notifies the user of the try-on details.

[0163] Input: Booking information received from the user

[0164] Output: Notification to sales / rental businesses, notification of try-on details to users

[0165] Specific operation: The server saves the reservation information in a database, notifies the business via email or API, and once confirmation is received from the business, notifies the user of the fitting reservation details.

[0166] Step 7:

[0167] When a transaction is confirmed, the server records the information and calculates the commission for the transaction. This commission is paid by the sales / rental business and is used to maintain and operate the system.

[0168] Input: Deal information

[0169] Output: Contract reward

[0170] Specific operation: When a user tries on an outfit and actually purchases it, the server records the transaction information (sale or rental details) in the database. The server calculates the reward based on this information and charges the business operator. This reward is used to operate the system.

[0171] (Application example 1)

[0172] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0173] Conventional outfit selection systems make it difficult for users to efficiently try on a large number of outfits, requiring them to visit a store in person. Furthermore, the generation of outfit try-on images based on the user's facial photograph often lacks accuracy and variety. Furthermore, the quality and real-time nature of the generated outfit try-on images cannot be guaranteed, resulting in a poor user experience and a cumbersome outfit selection process.

[0174] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0175] In this invention, the server includes an image uploading means for a user to register a facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to store the generated images in a database and send them to a user terminal, a means for the user to browse the transmitted catalog and select a favorite outfit, a reservation information sending means for the user to make a fitting reservation and send that information to the server, a means for the server to store the reservation information received from the user in a database and notify the sales / rental business, a means for generating a prompt message to send to an image generation AI based on the user's characteristics, and a means for providing an image generated based on the prompt message. This allows a user to efficiently generate and select images of trying on outfits through a virtual fitting room.

[0176] The "image uploading means" is a function that allows a user to transfer a photo of their face from their terminal to the server.

[0177] The "image analysis means" is a function that extracts and analyzes facial features from the facial photograph received by the server.

[0178] The "image generation means" is a function that automatically generates images of the user wearing various outfits based on the analyzed facial photograph.

[0179] The "database storage means" is a function for storing the generated try-on images in a database.

[0180] The "user terminal transmission means" is a function for transmitting the try-on images generated by the server to the user's terminal.

[0181] The "catalog viewing means" is a function that allows the user to view the transmitted costume catalog on the terminal.

[0182] The "try-on reservation information sending means" is a function for sending reservation information for trying on the outfit selected by the user to the server.

[0183] The "reservation information storage means" is a function for storing reservation information received from a user in a database.

[0184] The "Sales / Rental Company Notification Method" is a function for notifying sales or rental companies of the user's reservation information.

[0185] The "prompt generation means" is a function for generating instructions to be sent to the image generation AI based on the user's characteristics.

[0186] The "image providing means" is a function for providing the generated image to the user.

[0187] As an embodiment of the present invention, a system for running a virtual fitting room application on a smartphone is constructed. The specific operation of this system will be described below.

[0188] First, the user launches the smartphone application and logs in. After logging in, the user can either take a photo of their face using the application's camera function or select and upload a photo from their existing photo library. This process is carried out by the image uploading means.

[0189] The server then processes the received facial photo using image analysis tools to extract facial features, using libraries such as OpenCV, and standardizes the analyzed facial photo, preparing it for the next step.

[0190] The server generates a prompt to be sent to the image generation AI based on the analysis results of the facial photo. For example, a prompt such as "Please generate multiple photos of a woman in her twenties wearing a casual dress" is created using the prompt generation means.

[0191] Based on the prompt, the image generation means uses algorithms such as GAN (generative adversarial network) using TensorFlow or PyTorch to generate images of the user wearing various costumes. The generated images are stored in a database and then managed by the database storage means.

[0192] The generated try-on images are sent to the smartphone app by the user terminal transmission means. The user can then use the catalog browsing means to view the sent try-on images on the app and select the outfit they like.

[0193] When a user makes a reservation to try on clothes, they fill out the reservation form in the app and send the information to the server. This is done by the try-on reservation information sending means. The server saves the received reservation information in the database saving means and simultaneously notifies the sales / rental business. The sales / rental business notifying means is used to notify the relevant parties of the user's reservation information, ensuring a smooth try-on experience.

[0194] This system not only allows users to easily make fitting reservations and select outfits, but also enables efficient operation for sales / rental businesses. This allows users to experience virtual fittings at home or anywhere and easily select the appropriate outfit.

[0195] For example, if a user wants to choose a furisode for their coming-of-age ceremony, they take a photo of their face directly into the app and upload it, and a prompt such as "Please generate photos of a woman in her 20s wearing multiple casual dresses" is sent to the generative AI model. This model then generates an image of the user wearing the furisode, and the user then selects a furisode based on that image and makes a reservation to try it on.

[0196] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0197] Step 1:

[0198] A user launches a smartphone application and logs in. They can then use the app's camera to take a photo of their face or select and upload a photo from their existing photo library.

[0199] Input: User's face photo

[0200] Output: Face photo data to the server

[0201] Specific operation: Send the facial photo data to the server using the image upload method.

[0202] Step 2:

[0203] The server processes the received facial photos using image analysis tools to extract facial features, performs facial recognition using libraries such as OpenCV, and converts them into a standardized format.

[0204] Input: User's face photo data

[0205] Output: Extracted facial feature data

[0206] Specific operation: The received facial photo is analyzed using OpenCV, and facial features are extracted and standardized.

[0207] Step 3:

[0208] The server generates a prompt sentence to be sent to the image generation AI based on the analysis results of the facial photo. Using the prompt sentence generation means, it creates an instruction sentence based on specific conditions.

[0209] Input: Extracted facial feature data

[0210] Output: prompt statement

[0211] Specific operation: Based on the analysis results, generate a prompt such as "Please generate multiple photos of a woman in her 20s wearing a casual dress."

[0212] Step 4:

[0213] The server sends prompts to an image generation AI model (e.g., GAN), which generates images of the user trying on various outfits.

[0214] Input: prompt statement

[0215] Output: Try-on image data

[0216] Specific operation: The image generation AI model is run based on the prompt sentence, and the generated image is obtained.

[0217] Step 5:

[0218] The generated try-on images are stored in a database by a database storage means.

[0219] Input: Try-on image data

[0220] Output: Try-on images stored in a database

[0221] Specific operation: Save the generated try-on images in an image database.

[0222] Step 6:

[0223] The server transmits the generated try-on images to a smartphone app via a user terminal transmission means, allowing the user to view them.

[0224] Input: Try-on images stored in the database

[0225] Output: Try-on image sent to user's device

[0226] Specific operation: The generated try-on image is sent to the user's device.

[0227] Step 7:

[0228] The user browses the sent catalog, selects a favorite outfit, and checks the try-on images using the catalog browsing means.

[0229] Input: Try-on image sent to user's device

[0230] Output: User selected outfit information

[0231] What it does: The user browses through try-on images within the app and selects the outfit that best suits them.

[0232] Step 8:

[0233] The user makes a reservation for trying on the selected outfit, and transmits the reservation information to the server using the try-on reservation information transmitting means.

[0234] Input: Information about the outfit selected by the user, fitting reservation information

[0235] Output: Reservation information sent to the server

[0236] Specific operation: Fill out the reservation form within the app and send the fitting reservation information to the server.

[0237] Step 9:

[0238] The server stores the received reservation information in a database using a database storage means, and notifies the sales / rental business of the reservation information.

[0239] Input: Reservation information sent to the server

[0240] Output: Reservation information stored in the database, notified businesses

[0241] What it does: Stores the reservation information and notifies the sales / rental business via email or notification system.

[0242] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0243] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. Furthermore, the system incorporates an emotion engine that recognizes the user's emotions, adding a function that suggests optimal outfits based on the user's emotional state.

[0244] Specific operation of the system

[0245] 1. Registering a user's photo

[0246] The user launches the application and logs in. After logging in, they take a photo of themselves or select one from their existing photo library and upload it. They then press the "Upload" button to send the photo to the server.

[0247] Example: If a user wants to choose a furisode kimono for their Coming of Age Ceremony, they can take a photo of their face directly into the app and upload it.

[0248] 2. Processing facial photos

[0249] The server receives the face photo uploaded by the user, uses image analysis algorithms to extract facial features, crops the extracted face area, and standardizes the resolution and size.

[0250] Example: The server uses a library such as OpenCV to perform face recognition and convert the data into a standardized format.

[0251] 3. Clothing catalog generation using image generation AI

[0252] The server sends a request to the image generation AI based on the preprocessed facial image to generate images of the user wearing various costumes, which are then stored in a database.

[0253] Example: The server uses algorithms such as GAN to generate images of the user wearing a gorgeous furisode or an elegant wedding dress.

[0254] 4. Provision of catalogues

[0255] The server sends the generated costume catalog to the user's terminal, and the catalog data can be viewed by the user in real time.

[0256] Example: A user uses the app to view images of various furisode kimonos and compare multiple designs.

[0257] 5. Emotion Recognition by Emotion Engine

[0258] The server-based emotion engine analyzes the user's facial expressions and estimates their emotional state. This emotion data is collected in real time as the user browses through the clothing catalog.

[0259] Example: The server uses the webcam to capture and analyze facial expressions in real time while the user is viewing the image.

[0260] 6. Recommendations for the best costume

[0261] The server then suggests the most suitable outfits for the user based on the estimated emotional state, including selecting outfits that match the user's preferences and filtering colors and designs based on the emotional state.

[0262] Example: If a user shows a happy expression when seeing a particular furisode, the system will prioritize suggesting that outfit.

[0263] 7. Dynamic catalog updates

[0264] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the contents of the catalog in response to the changes.

[0265] Example: Prioritize a list of outfits that the user is interested in, and add related outfit recommendations to the next page they view.

[0266] 8. Reservations for fitting

[0267] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[0268] Example: To try on the furisode kimono of a user's choice, the user enters the desired date and time and store information to confirm the reservation.

[0269] 9. Submitting and Managing Reservation Information

[0270] The terminal sends the fitting reservation information entered by the user to the server, which stores the received reservation information in a database and notifies the corresponding sales / rental business.

[0271] Example: The server sends reservation information via the business's API or email, and after receiving confirmation from the business, notifies the user of the fitting date.

[0272] 10. Calculating and recording commissions

[0273] When the server confirms the successful transaction, it records the information in a database and calculates the reward according to the successful transaction.

[0274] Example: When a user rents a furisode after trying it on, the server records the data and calculates the fee based on the contract fee from the business operator.

[0275] In this way, the system of the present invention allows users to simulate trying on various outfits based on their own facial photographs, and by receiving optimal outfit suggestions based on their emotional state, they can efficiently and satisfactorily select outfits. The system also has the advantage for sales / rental businesses, as it allows them to efficiently approach customers by utilizing the user's emotions.

[0276] The processing flow will be explained below.

[0277] Step 1:

[0278] The user starts the application and logs in. They enter their user ID and password for authentication.

[0279] Step 2:

[0280] Users can take a photo of themselves using the camera function or select and upload a photo from their existing photo library, then press the "Upload" button to send the photo to the server.

[0281] Step 3:

[0282] The server receives the face photo uploaded by the user, processes the HTTP request, and temporarily stores the image data.

[0283] Step 4:

[0284] The server analyzes the received facial photos and uses a facial recognition algorithm to crop the facial area, including locating the face using the OpenCV library.

[0285] Step 5:

[0286] The server extracts the facial area and standardizes the resolution, size, etc. This preprocessing allows the image generation AI to generate stable, high-quality images.

[0287] Step 6:

[0288] The server sends the preprocessed facial image to the image generation AI and requests it to generate images of the user wearing various outfits. The image generation AI then uses techniques such as GAN to generate multiple outfit patterns.

[0289] Step 7:

[0290] The server receives the generated images, adds corresponding metadata (e.g., type of costume, color, design) to each, and stores them in a database.

[0291] Step 8:

[0292] The server sends the generated clothing catalog to the user's device, sending the constructed catalog data as an HTTP response, allowing the user to view it in real time.

[0293] Step 9:

[0294] Users can browse the costume catalog sent to them within the application and select the costume they like. They can also check detailed information about the costume and other users' ratings by touching the screen.

[0295] Step 10:

[0296] The emotion engine built into the server analyzes the user's facial expressions and estimates their emotional state. The emotion engine uses a webcam to capture and analyze facial expressions in real time while the user is viewing the image.

[0297] Step 11:

[0298] The server then suggests the most suitable outfits for the user based on the estimated emotional state, including selecting outfits that match the user's preferences and filtering colors and designs based on the emotional state.

[0299] Step 12:

[0300] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the catalog contents accordingly. It prioritizes the display of outfits that the user is interested in, and also adds related outfit recommendations to the next page viewed.

[0301] Step 13:

[0302] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[0303] Step 14:

[0304] The device sends the fitting reservation information entered by the user to the server, generating an HTTP POST request and sending the reservation data to the server.

[0305] Step 15:

[0306] The server stores the received reservation information in a database, generates a reservation ID, and records the reservation details.

[0307] Step 16:

[0308] The server sends a fitting reservation notification to the corresponding costume sales / rental business via API or email.

[0309] Step 17:

[0310] The server receives the reservation confirmation from the business and notifies the user of the confirmation information. It also generates a notification message to inform the user of the fitting details.

[0311] Step 18:

[0312] When a transaction is confirmed, the server records the information in a database, saving data such as the transaction ID, transaction date and time, user ID, and business ID.

[0313] Step 19:

[0314] The server calculates the reward for the successful transaction and records the information. It calculates the reward amount based on the contract reward from the business operator and saves the corresponding record in the database.

[0315] This is the specific flow of the program processing of this system. By combining it with an emotion engine, users can receive optimal outfit suggestions based on their emotional state, realizing an efficient and satisfying outfit selection process.

[0316] Example 2

[0317] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0318] In conventional online shopping systems, it was difficult for users to check how clothes would look when tried on using their own photos. Furthermore, there was no system that could provide optimal suggestions based on the user's emotional state, which prevented users from increasing satisfaction. Furthermore, the means by which users could make reservations for try-ons was inefficient, and it was time-consuming for businesses to handle the process.

[0319] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an image uploading means for a user to register their own facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to store the generated images in a database and send them to a user terminal, a means for the user to browse the sent catalog and select a favorite outfit, an emotion recognition means for analyzing the user's facial expression and estimating their emotional state using an emotion engine built into the server, a means for the server to suggest optimal outfits based on the estimated emotional state, a means for the server to dynamically update the catalog content, a reservation information sending means for the user to make a try-on reservation and send the information to the server, and a means for the server to save the reservation information received from the user in a database and notify the sales / rental company. This allows the user to simulate trying on various outfits based on their own facial photo and further receive optimal outfit suggestions based on their emotional state, enabling efficient and satisfying outfit selection.

[0320] "User" refers to anyone who uses this system.

[0321] "Image upload means" refers to the means by which a user can register a photo of their face in the system.

[0322] A "server" refers to a computing device that centrally handles the processing of the entire system.

[0323] "Image analysis means" refers to a means for analyzing facial photographs received by the server and extracting facial features.

[0324] "Image generation means" refers to the means by which the server generates images of the user wearing various costumes based on the analyzed facial photograph.

[0325] A "database" refers to a collection of data that a system uses to efficiently store and manage data.

[0326] "User device" refers to the electronic device (e.g., smartphone, personal computer) used by the user for access.

[0327] "Catalogue" refers to the collection of images of various outfits that are generated and provided by the System and that can be viewed by Users.

[0328] "Emotion engine" refers to the component within the system that analyzes the user's facial expressions and infers their emotional state.

[0329] "Emotion recognition means" refers to a means for analyzing a user's facial expressions using an emotion engine and estimating their emotional state.

[0330] The "means for suggesting optimal outfits" refers to a means for suggesting optimal outfits to a user based on an estimated emotional state.

[0331] "Means for dynamic updating" refers to a means by which the server updates the catalog contents in real time.

[0332] The "reservation information transmission means" refers to a means for transmitting information to the server that allows a user to make a reservation for trying on clothes.

[0333] "Sales / Rental Business" refers to a business that offers costumes for sale or rental.

[0334] This invention relates to a system that generates images based on a user's facial photograph, allowing the user to try on various outfits and efficiently select outfits. This system also incorporates an emotion engine that recognizes the user's emotions, and includes a function to suggest the most suitable outfit based on the user's emotional state.

[0335] First, users launch the application and log in to register their own face photo. Then, they can take a photo of their face or select one from their existing photo library and upload it. The uploaded face photo is sent to the server and stored in a database. For this purpose, users take and upload photos using devices such as smartphones or computers.

[0336] The server then inputs the received facial photo into an image analysis algorithm to extract facial features. This analysis uses libraries such as OpenCV. The extracted facial features are cropped and standardized in resolution and size. The standardized image is then sent to the next processing step.

[0337] Next, the server inputs the preprocessed facial image into the image generation AI, and provides the following prompt to the generation AI model:

[0338] Use a photo of the user's face to generate an image of them wearing a gorgeous furisode or an elegant wedding dress.

[0339] The server uses algorithms such as GAN (Generative Adversarial Network) to generate images of the user's face dressed in various outfits. The generated images are stored in a database, along with associated metadata. This generated outfit catalog is then sent from the server to the user's device, where it can be viewed in real time.

[0340] While the user is browsing the clothing catalog, the emotion engine built into the server captures the user's facial expressions with a webcam and analyzes their emotions using a facial expression analysis algorithm. The emotion data is collected and the user's emotional state is estimated.

[0341] The server then suggests the most suitable outfits based on this emotional state. The suggestion algorithm selects the most suitable outfits by taking into account the user's emotional data and past preference data. The server also dynamically updates the catalog contents according to changes in emotions and sends them to the user's device in real time. This dynamic update allows the user to continue browsing outfits that interest them and related designs.

[0342] When a user finds an outfit they like, they can make a reservation to try it on. They fill out a reservation form within the app and submit information such as the desired date and time and the store. The device sends this information to the server, which stores the received reservation information in a database. The server then notifies the sales / rental company, and after receiving confirmation from the company, notifies the user of the try-on date and time.

[0343] Finally, if the user tries on the clothes and makes a purchase, the server records the information in a database, calculates the commission for the purchase, and notifies the business operator. This process allows users to efficiently choose clothes that satisfy them.

[0344] In this way, the system of the present invention enables a simulation of trying on clothes based on the user's facial photograph, and suggests optimal outfits based on the user's emotional state. It also efficiently handles everything from trying-on reservations to calculating and managing commissions, providing convenience to both users and businesses.

[0345] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0346] Step 1:

[0347] Registering a user's photo

[0348] Users launch the application, log in, and then click the "Upload Face Photo" button to either take a photo or select one from their existing photo library and upload it.

[0349] Input: User's face photo (either a newly taken photo or a photo selected from the library)

[0350] Output: The uploaded face photo is sent to the server and stored in the database.

[0351] Specific operation: The user uses a smartphone or computer to take or select a photo of their face and send it to the server via the app.

[0352] Step 2:

[0353] Face photo processing

[0354] The server inputs the facial photo received from the user into an image analysis algorithm to extract facial features. Facial recognition is performed using the OpenCV library.

[0355] Input: A photo of the face sent by the user

[0356] Output: extracted facial features and standardized face image

[0357] Specific operation: The server uses OpenCV to perform facial recognition and converts the facial image to a standard size of 500x500 pixels.

[0358] Step 3:

[0359] Clothing catalog generation using image generation AI

[0360] The server inputs the preprocessed facial image into the image generation AI, and gives the following prompt to the generation AI model:

[0361] Use a photo of the user's face to generate an image of them wearing a gorgeous furisode or an elegant wedding dress.

[0362] Input: Standardized face image and prompt text

[0363] Output: Images of the user wearing various generated outfits

[0364] How it works: The server uses a Generative Adversarial Network (GAN) to generate images of the user wearing gorgeous furisode or elegant dresses based on a photo of their face.

[0365] Step 4:

[0366] Providing a catalog

[0367] The server then sends the generated costume catalog to the user's device, where it can be viewed in real time.

[0368] Input: Generated outfit catalog

[0369] Output: Catalog display on user terminal

[0370] Specific operation: The server sends the generated image data to the user's terminal, allowing the user to view the images in real time on their smartphone or computer.

[0371] Step 5:

[0372] Emotion recognition by emotion engine

[0373] The emotion engine built into the server captures the user's facial expressions using a webcam and analyzes their emotions using an expression analysis algorithm.

[0374] Input: User's facial expressions (images and video captured in real time)

[0375] Output: Estimated emotional state data

[0376] Specific operation: While the user is browsing the catalog, the server captures the user's facial expressions via a webcam and performs emotion recognition.

[0377] Step 6:

[0378] Optimal costume suggestions

[0379] The server then suggests the best outfit for the user based on the estimated emotional state. The suggestion algorithm takes into account emotional data and past preference data.

[0380] Input: Estimated emotional state data and historical preference data

[0381] Output: Optimal costume suggestion data

[0382] Specific operation: The server selects the most suitable outfit based on the user's emotional state and past preference data and suggests it to the user.

[0383] Step 7:

[0384] Dynamic catalog updates

[0385] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the contents of the catalog in response to the changes.

[0386] Input: Real-time changing emotional state data

[0387] Output: Dynamically updated catalog

[0388] Specific operation: The server monitors changes in the user's emotional state in real time, and automatically updates the contents of the catalog and sends them to the user whenever the emotion changes.

[0389] Step 8:

[0390] Accepting reservations for try-on

[0391] To make a reservation to try on the selected outfit, the user fills out a reservation form within the app, inputting information such as the desired date and time and the store, and submitting it.

[0392] Input: Trial reservation information entered by the user

[0393] Output: Reservation information sent to the server

[0394] Specific operation: The user uses the app to enter the desired date and time and store information, and presses the send button to send the reservation information to the server.

[0395] Step 9:

[0396] Submit and manage your reservations

[0397] The server stores the received reservation information in a database and notifies the corresponding sales / rental business.

[0398] Input: Booking information received from the user

[0399] Output: Reservation information stored in the database and notifications to businesses

[0400] Specific operation: The server records the reservation information in a database, notifies the business via API or email, and once confirmation is received, notifies the user of the fitting date.

[0401] Step 10:

[0402] Calculating and recording commissions

[0403] When the server confirms that a contract has been concluded, it records the information in a database, calculates the contract fee, and notifies the business operator.

[0404] Input: Data related to the deal

[0405] Output: Calculated commission and notification

[0406] Specific operation: The server records the contract information in a database, calculates the contract fee, and notifies the business operator.

[0407] (Application example 2)

[0408] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0409] When choosing an outfit, users have to try on many outfits in person, which takes time and effort. Furthermore, when choosing outfits online, there is a risk of making the wrong choice because there are limited ways to check whether the outfit suits the user. Furthermore, outfits suggested without taking into account the user's emotional state may result in low satisfaction. A new system is needed to solve these problems.

[0410] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an image upload means for the user to register their own facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to save the generated images in a database and send them to the user terminal, a means for the user to browse the sent catalog and select a favorite outfit, a reservation information sending means for the user to make a try-on reservation and send that information to the server, a means for recognizing the user's emotional state in real time using an emotion engine and suggesting the most suitable outfit based on the emotional state, and a means for the server to save the reservation information received from the user in a database and notify the sales / rental company. This allows the user to receive the most suitable outfit suggestions in real time based on their facial photo, realizing an efficient and satisfying outfit selection process.

[0411] A "user" is a person who uses this system to register a photo of their face, select an outfit, and make a fitting reservation.

[0412] The "image uploading means" refers to a method and device that allows a user to register a photograph of his or her face.

[0413] The "server" is a central system that analyzes facial photos received from users and performs image generation and emotion recognition.

[0414] "Means for performing image analysis" refers to a technology in which the server processes a facial photo sent by the user and extracts facial features.

[0415] The "image generation means" is a technology that generates virtual images of users wearing various costumes based on facial photographs analyzed by the server.

[0416] The "database" is an information management system for storing generated images and reservation information.

[0417] "User Terminal" means a device through which a User accesses and operates this System.

[0418] A "catalog" is a list of costumes generated in a format that can be viewed by users.

[0419] The "means for making a fitting reservation" refers to a method and device for making a reservation to actually try on the outfit selected by the user.

[0420] The "reservation information transmission means" is a technology that transmits the try-on reservation information entered by the user to the server.

[0421] The "Emotion Engine" is an artificial intelligence technology that analyzes a user's facial expressions and recognizes their emotional state in real time.

[0422] The "means for suggesting optimal outfits" is a technology that dynamically selects and presents optimal outfits to users based on emotional data recognized by the emotion engine.

[0423] The "sales / rental company" is a service provider that actually provides the costumes based on the generated images and fitting reservation information.

[0424] System Overview

[0425] The embodiment of this invention is a virtual store system where users can try on various outfits based on their facial photographs. This system is composed of a user terminal, a server, an image generation means, an emotion engine, a database, and a sales / rental agent.

[0426] Hardware used

[0427] User devices: smartphones, tablets, etc.

[0428] Server: High-performance computer (e.g., Amazon EC2)

[0429] Data storage: Cloud storage (e.g. Amazon S3)

[0430] Software used

[0431] Face Recognition Library: OpenCV

[0432] Emotion recognition engines: Affectiva and RealEyes

[0433] Image generation AI: GAN (Generative Adversarial Network)

[0434] Backend: Python (Flask / Django)

[0435] Data management: MySQL (registered trademark)

[0436] Mobile App Development: React Native

[0437] Specific operation of the system

[0438] 1. User photo registration:

[0439] Users launch the smartphone app and log in. After logging in, they can take a photo of themselves or select one from their existing photo library and upload it. The uploaded image is sent to the server.

[0440] 2. Facial Photo Analysis:

[0441] The server analyzes the facial photo received from the user using OpenCV, extracts facial features, and converts them into a standardized format.

[0442] 3. Image generation:

[0443] The server uses GAN to generate images of the user wearing various costumes based on the analyzed facial images, and these generated images are stored in a database.

[0444] 4. Catalog provided by:

[0445] The server sends the generated clothing catalog to the user's device, where the user can browse the catalog and compare multiple clothing items.

[0446] 5. Emotion recognition:

[0447] The emotion engine analyzes users' facial expressions to recognize their emotional state in real time, thereby collecting emotional data as users browse the catalog.

[0448] 6. Best outfit suggestions:

[0449] The server then suggests the most suitable outfit for the user based on the emotional data obtained from the emotion engine. This suggestion is made in real time, increasing user satisfaction.

[0450] 7. Try-on appointment:

[0451] Users can make reservations to try on selected outfits. The reservation information is sent to the server and stored in a database. The server then notifies the sales / rental company of this information.

[0452] Specific examples

[0453] For example, consider a case where a user using a smartphone app is choosing a furisode for their coming-of-age ceremony. The user uploads a photo of their face into the app, which analyzes it and generates an image of them wearing the furisode. As the user browses the catalog, the emotion engine analyzes their facial expressions, and if they show a happy expression, that furisode will be suggested as a priority. If the user wants to try on the furisode, they can make a reservation through the app, and the sales / rental company will be notified.

[0454] Example of input prompt for generative AI model

[0455] "A 20-year-old woman is choosing a furisode for her Coming-of-Age Ceremony. Based on a photo of her face, if she is smiling, she will try on a red furisode; if she is relaxed, she will try on a blue furisode; if she is deep in thought, she will try on a white furisode."

[0456] In this way, the present invention provides a specific system operation method and technology that allows users to select outfits efficiently and satisfactorily.

[0457] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0458] Step 1:

[0459] User device: The user launches the application and logs in. The user takes a photo of their face with their smartphone camera or selects a photo from their existing photo library and sends it to the server using an image upload method.

[0460] Input: User's face photo

[0461] Output: Sending photo data to the server

[0462] Specific operation: The user takes a photo of their face with their smartphone and taps the upload button.

[0463] Step 2:

[0464] Server: The server receives the face photo sent by the user and analyzes the face image using OpenCV, extracts facial features, and standardizes the face image.

[0465] Input: Face photo sent to the server

[0466] Output: Standardized face image

[0467] What it does: The server applies a facial recognition algorithm to identify the location of facial contours, eyes, nose, etc. Based on the identified facial features, it resizes the image to a standard size (e.g., 224x224 pixels).

[0468] Step 3:

[0469] Server: The server uses GAN to generate images of the user wearing various costumes based on the analyzed standardized facial image. The generated images are stored in a database.

[0470] Input: Standardized face image

[0471] Output: Images of the user in various outfits

[0472] Specific operation: The server inputs a standardized face image into the GAN model, generates images of the person trying on multiple outfits, and stores each image in a database.

[0473] Step 4:

[0474] Server: The server sends the generated costume catalog to the user's device. The catalog contains images of the user wearing various costumes generated based on their facial photograph.

[0475] Input: Generated garment image

[0476] Output: Sending catalog data to the user's device

[0477] Specific operation: The server compiles the generated costume images into a catalog format and sends it to the user's device. The user can then view the catalog on their smartphone.

[0478] Step 5:

[0479] User device: The emotion engine monitors the user's facial expressions in real time and estimates their emotional state. Meanwhile, the user can browse the catalog and select their favorite outfit.

[0480] Input: User's facial expression data

[0481] Output: Estimated user emotional state

[0482] How it works: The smartphone camera is used to capture the user's facial expressions, and the emotion recognition engine analyzes the data to infer their emotional state. For example, if the user is smiling, the emotion engine will recognize it as "joy."

[0483] Step 6:

[0484] Server: Based on the emotional state estimated by the emotion engine, the server prioritizes and suggests the most suitable outfits for the user.

[0485] Input: Estimated user emotional state

[0486] Output: Recommendation of optimal outfits to the user

[0487] How it works: The server receives data from the emotion engine, re-ranks the outfits in the catalog based on the estimated emotional state, and displays them to the user. For example, if the emotion of "joy" is recognized, bright-colored outfits will be displayed preferentially.

[0488] Step 7:

[0489] User terminal: The user selects a favorite outfit and makes a reservation to try it on. The try-on reservation information is sent to the server using the reservation information sending means.

[0490] Input: User's fitting reservation information (desired date and time, store, etc.)

[0491] Output: Send reservation information to the server

[0492] Specific operation: If a user wants to try on an outfit selected from the catalog, they fill out the reservation form and tap the submit button.

[0493] Step 8:

[0494] Server: The server stores the reservation information received from the user in a database and notifies the sales / rental agent.

[0495] Input: Try-on reservation information

[0496] Output: Save to database and notify vendor

[0497] Specific operation: The server records the reservation information in a database and notifies the sales / rental agent of the information via API or email.

[0498] example

[0499] Example of an input prompt for a generative AI model:

[0500] "A 20-year-old woman is choosing a furisode for her Coming-of-Age Ceremony. Based on a photo of her face, if she is smiling, she will try on a red furisode; if she is relaxed, she will try on a blue furisode; if she is deep in thought, she will try on a white furisode."

[0501] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0502] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0503] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0504] [Second embodiment]

[0505] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0506] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0507] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0508] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0509] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0510] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0511] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0512] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0513] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0514] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0515] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0516] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0517] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. The system includes an application that runs on the user's terminal, and image generation and data processing on the server side.

[0518] Specific operation of the system

[0519] 1. Registering a user's photo

[0520] Users launch the application and log in. After logging in, they can either take a photo of themselves using the camera or select and upload a photo from their existing photo library.

[0521] Example: If a user wants to choose a furisode kimono for their Coming of Age Ceremony, they can take a photo of their face directly into the app and upload it.

[0522] 2. Processing facial photos

[0523] The server receives the face photo uploaded by the user and uses image analysis algorithms to extract facial features, adjusting the resolution and cropping the face to standardize the face image.

[0524] Example: The server uses a library such as OpenCV to perform face recognition and convert the data into a standardized format.

[0525] 3. Clothing catalog generation using image generation AI

[0526] The server sends a request to the image generation AI based on the preprocessed facial image to generate images of the user wearing various costumes, which are then stored in a database.

[0527] Example: The server uses algorithms such as GAN to generate images of the user wearing a gorgeous furisode or an elegant wedding dress.

[0528] 4. Provision of catalogues

[0529] The server then sends the generated costume catalog to the user's device, where the user can browse the catalog featuring themselves as models and select their favorite outfit from the multiple options.

[0530] Example: A user uses an app to browse images of various furisode kimonos and choose their favorite design.

[0531] 5. Booking a try-on appointment

[0532] To try on an outfit selected from the catalog, the user fills out a reservation form within the app and makes a reservation. The reservation information is sent to the server.

[0533] Example: To try on the furisode kimono of a user's choice, the user enters the desired date and time and store information to confirm the reservation.

[0534] 6. Manage fitting reservations

[0535] The server stores the received reservation information in a database, notifies the costume sales / rental business, and, upon receiving confirmation from the business, notifies the user of the reservation details.

[0536] Example: The server notifies the business of the user's fitting reservation by email, and after confirmation from the business, notifies the user of the fitting date.

[0537] 7. Calculating and recording commissions

[0538] When a transaction is confirmed, the server records the information and calculates the commission for the transaction. This commission is provided by the sales / rental business and is used to maintain and operate the system.

[0539] Example: When a user actually rents a furisode, that information is recorded and the commission from the business is calculated.

[0540] In this way, users can browse a catalog based on their own face photo and make a fitting reservation to efficiently select the most suitable outfit. Sales / rental businesses also benefit from being able to efficiently approach customers through the system. By combining server-side image analysis, generation, and management functions, it is possible to significantly improve the user experience.

[0541] The processing flow will be explained below.

[0542] Step 1:

[0543] The user launches the application and logs in. They enter their user ID and password and access their account through the authentication screen.

[0544] Step 2:

[0545] Users can take a photo of themselves using the camera function or select and upload a photo from their existing photo library, then press the "Upload" button to send the photo to the server.

[0546] Step 3:

[0547] The server receives the face photo uploaded by the user, processes the HTTP request, and temporarily stores the image data.

[0548] Step 4:

[0549] The server analyzes the received facial photos and uses a facial recognition algorithm to crop the facial area, which involves using the OpenCV library to identify the face location.

[0550] Step 5:

[0551] The server extracts the facial area and standardizes the resolution, size, etc. This preprocessing allows the image generation AI to generate stable, high-quality images.

[0552] Step 6:

[0553] The server sends the preprocessed facial image to the image generation AI and requests it to generate images of the user wearing various outfits. The image generation AI then uses techniques such as GAN to generate multiple outfit patterns.

[0554] Step 7:

[0555] The server receives the generated images, adds corresponding metadata (e.g., type of costume, color, design) to each, and stores them in a database.

[0556] Step 8:

[0557] The server sends the generated clothing catalog to the user's device, sending the constructed catalog data as an HTTP response, allowing the user to view it in real time.

[0558] Step 9:

[0559] Users can browse the costume catalog sent to them within the application and select the costume they like. They can also check detailed information about the costume and other users' ratings by touching the screen.

[0560] Step 10:

[0561] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[0562] Step 11:

[0563] The device sends the fitting reservation information entered by the user to the server, generating an HTTP POST request and sending the reservation data to the server.

[0564] Step 12:

[0565] The server stores the received reservation information in a database, generates a reservation ID, and records the reservation details.

[0566] Step 13:

[0567] The server sends a fitting reservation notification to the corresponding costume sales / rental business via API or email.

[0568] Step 14:

[0569] The server receives the reservation confirmation from the business and notifies the user of the confirmation information. It also generates a notification message to inform the user of the fitting details.

[0570] Step 15:

[0571] When a transaction is confirmed, the server records the information in a database, saving data such as the transaction ID, transaction date and time, user ID, and business ID.

[0572] Step 16:

[0573] The server calculates the reward for the successful transaction and records the information. It calculates the reward amount based on the contract reward from the business operator and saves the corresponding record in the database.

[0574] The above is the specific flow of the program processing of this system, which allows users to efficiently and effectively select the most suitable outfit and easily make a fitting reservation.

[0575] Example 1

[0576] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0577] Today, users want to be able to try on various outfits and view the results in an efficient and reliable way using their own facial photos. However, current systems require cumbersome facial photo registration and analysis, which does not improve the user experience. Furthermore, efficient management is required for providing the generated images and accepting try-on reservations. It is necessary to solve these issues and provide a system that is convenient for both users and sales / rental businesses.

[0578] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0579] In this invention, the server includes an image uploading means for users to register their own facial photos, a means for the server to receive the facial photos and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photos, a means for the server to save the generated images in a database and send them to a user terminal, a means for the user to browse the sent catalog and select a favorite outfit, a reservation information sending means for the user to make a try-on reservation and send that information to the server, a means for the server to save the reservation information received from the user in a database and notify the sales / rental company, an automatic adjustment means for the user terminal to automatically adjust the facial photos to an appropriate format, and a means for the server to execute an algorithm to suggest outfit combinations based on the facial images. This allows users to efficiently try on and select outfits based on their own facial photos.

[0580] "User" refers to an individual who uses this system to upload a photo of their face and generate images of them trying on clothes.

[0581] "Image upload means" refers to the function that allows users to send photos of their faces to the system.

[0582] "Server" refers to a computer system that analyzes facial photos received from users, generates costumed images using image generation AI, stores them in a database, and sends them to the user's device.

[0583] "Image analysis means" refers to the function of analyzing facial photographs received by the server and extracting facial features.

[0584] "Image generation means" refers to the function of the server to generate images of users wearing various costumes using a generative AI model based on analyzed facial photos.

[0585] "Database" refers to an information management system for storing images generated by the server and user fitting reservation information.

[0586] "User terminal" refers to a device such as a smartphone, tablet, or PC that a user uses to operate an application.

[0587] A "catalog" refers to a collection of images of users wearing costumes generated by the server, which can be viewed and selected by users.

[0588] The "reservation information transmission means" refers to a function for transmitting reservation information for trying on the costume selected by the user to the server.

[0589] "Automatic adjustment means" refers to a function that allows a user's device to automatically adjust uploaded facial photos into an appropriate format.

[0590] "Contract reward" refers to the reward calculated by the system based on transaction information when a user actually rents or purchases an outfit.

[0591] A "generative AI model" refers to an image generation algorithm used by the server, which generates costume images based on a user's facial photo using a generative artificial network (GAN) or similar.

[0592] A "prompt sentence" refers to an input sentence that specifies the expected result for a generative AI model.

[0593] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. The system includes an application that runs on the user's terminal, and image generation and data processing on the server side.

[0594] User Device

[0595] The user terminal includes devices such as smartphones, tablets, and PCs. The user uses the terminal to launch the application and take a photo of their face or select and upload it from a photo library. The terminal is equipped with an automatic adjustment means, which automatically adjusts the uploaded face photo to an appropriate format.

[0596] Example: When a user is choosing a furisode kimono for their Coming of Age Ceremony, they can use the app to take and upload a photo of their face.

[0597] server

[0598] The server is the central system that analyzes the received facial photos and generates images of the user in various costumes. The server uses the following specific software:

[0599] OpenCV: Used for analyzing and preprocessing face images.

[0600] TensorFlow: Used for image generation using generative AI models (e.g., GANs).

[0601] The server has a pre-processing means for extracting the user's facial features using face recognition technology and standardizing the facial image, and also has a function for storing the generated costume image in a database and transmitting it to the user's terminal.

[0602] Specific behavior:

[0603] The server uses the OpenCV library to analyze facial features and inputs the standardized facial images into a generative AI model based on TensorFlow, which generates images of the user wearing various outfits. The images are stored in a database and then sent to the user's device.

[0604] user

[0605] Users can browse the sent catalog on the app and select their favorite outfits. They can then make a reservation to try on the outfits they selected within the app and send that information to the server. The server stores this information and notifies the sales / rental company.

[0606] Example: A user browses a catalog of furisode kimonos on the app, selects their favorite design, and makes a reservation to try them on.

[0607] Calculation of commission

[0608] The server records the information of the costumes that have been sold and calculates the commission for the sale. This commission is provided by the sales / rental business and is used for the maintenance and operation of the system.

[0609] Prompt Sentence Examples

[0610] "I want to choose a furisode for my Coming-of-Age Ceremony. Please upload a photo of your face and generate images of you trying on various furisode."

[0611] "Try on wedding dresses based on your own photo and create a catalog."

[0612] This allows users to efficiently try on and select outfits based on their own face photos, and sales / rental businesses can also use the system to effectively approach customers.

[0613] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0614] Step 1:

[0615] The user launches the application and logs in, which gives them access to the system, and then uses the camera function to take a photo of their face or upload one from their existing photo library.

[0616] Input: User authentication information (ID, password), face photo

[0617] Output: Face photo sent to server

[0618] Specific actions: Press the "Take Photo" button in the app to take a new photo or press the "Choose from Library" button to select an existing photo and upload a face photo. Press the "Upload" button to send the face photo to the server.

[0619] Step 2:

[0620] The server uses the OpenCV library to run a facial recognition algorithm on the face photo received from the user, extracting facial features, and then standardizes the face image (adjusting resolution and cropping the face) to complete preprocessing.

[0621] Input: Uploaded face photo

[0622] Output: Preprocessed face image

[0623] Specific operation: The server detects faces in the received photos using cv2.CascadeClassifier in the OpenCV library, crops the face area, and resizes the image to a resolution of 256x256 pixels. This completes the standardization process.

[0624] Step 3:

[0625] The server uses the preprocessed facial image to send a request to a generative AI model (likely a GAN) using the TensorFlow library to generate images of the user wearing various costumes, which are then stored in a database.

[0626] Input: Preprocessed face image

[0627] Output: An image of the user wearing the generated outfit

[0628] How it works: The preprocessed face image is input into a GAN model in the TensorFlow library, which then generates images of the user wearing various costumes and stores them in a database.

[0629] Step 4:

[0630] The server retrieves the generated outfit catalog (outfit images based on the user's face photo) from the database and sends it to the user's device. The user can browse this catalog using the app and select the outfit they like.

[0631] Input: Generated images stored in a database

[0632] Output: Outfit catalog sent to user's device

[0633] Specific operation: When a user presses the "View Catalog" button, the server retrieves the user's generated images from the database and sends them to the user's device. The user then views them on the app and presses the "Favorite" button to select an outfit.

[0634] Step 5:

[0635] The user fills out the fitting reservation form for the outfit they have chosen within the app and makes a reservation to try on the outfit. The reservation information is sent to the server.

[0636] Input: Selected outfit, desired fitting date and time, and other reservation information

[0637] Output: Reservation information sent to the server

[0638] Specific operation: The user presses the "Book a try-on" button, enters the desired date and time and store information in the reservation form, and presses the "Confirm reservation" button to send the reservation information to the server.

[0639] Step 6:

[0640] The server stores the received reservation information in a database and notifies the corresponding sales / rental business. After confirmation, the business notifies the user of the try-on details.

[0641] Input: Booking information received from the user

[0642] Output: Notification to sales / rental businesses, notification of try-on details to users

[0643] Specific operation: The server saves the reservation information in a database, notifies the business via email or API, and once confirmation is received from the business, notifies the user of the fitting reservation details.

[0644] Step 7:

[0645] When a transaction is confirmed, the server records the information and calculates the commission for the transaction. This commission is paid by the sales / rental business and is used to maintain and operate the system.

[0646] Input: Deal information

[0647] Output: Contract reward

[0648] Specific operation: When a user tries on an outfit and actually purchases it, the server records the transaction information (sale or rental details) in the database. The server calculates the reward based on this information and charges the business operator. This reward is used to operate the system.

[0649] (Application example 1)

[0650] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0651] Conventional outfit selection systems make it difficult for users to efficiently try on a large number of outfits, requiring them to visit a store in person. Furthermore, the generation of outfit try-on images based on the user's facial photograph often lacks accuracy and variety. Furthermore, the quality and real-time nature of the generated outfit try-on images cannot be guaranteed, resulting in a poor user experience and a cumbersome outfit selection process.

[0652] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0653] In this invention, the server includes an image uploading means for a user to register a facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to store the generated images in a database and send them to a user terminal, a means for the user to browse the transmitted catalog and select a favorite outfit, a reservation information sending means for the user to make a fitting reservation and send that information to the server, a means for the server to store the reservation information received from the user in a database and notify the sales / rental business, a means for generating a prompt message to send to an image generation AI based on the user's characteristics, and a means for providing an image generated based on the prompt message. This allows a user to efficiently generate and select images of trying on outfits through a virtual fitting room.

[0654] The "image uploading means" is a function that allows a user to transfer a photo of their face from their terminal to the server.

[0655] The "image analysis means" is a function that extracts and analyzes facial features from the facial photograph received by the server.

[0656] The "image generation means" is a function that automatically generates images of the user wearing various outfits based on the analyzed facial photograph.

[0657] The "database storage means" is a function for storing the generated try-on images in a database.

[0658] The "user terminal transmission means" is a function for transmitting the try-on images generated by the server to the user's terminal.

[0659] The "catalog viewing means" is a function that allows the user to view the transmitted costume catalog on the terminal.

[0660] The "try-on reservation information sending means" is a function for sending reservation information for trying on the outfit selected by the user to the server.

[0661] The "reservation information storage means" is a function for storing reservation information received from a user in a database.

[0662] The "Sales / Rental Company Notification Method" is a function for notifying sales or rental companies of the user's reservation information.

[0663] The "prompt generation means" is a function for generating instructions to be sent to the image generation AI based on the user's characteristics.

[0664] The "image providing means" is a function for providing the generated image to the user.

[0665] As an embodiment of the present invention, a system for running a virtual fitting room application on a smartphone is constructed. The specific operation of this system will be described below.

[0666] First, the user launches the smartphone application and logs in. After logging in, the user can either take a photo of their face using the application's camera function or select and upload a photo from their existing photo library. This process is carried out by the image uploading means.

[0667] The server then processes the received facial photo using image analysis tools to extract facial features, using libraries such as OpenCV, and standardizes the analyzed facial photo, preparing it for the next step.

[0668] The server generates a prompt to be sent to the image generation AI based on the analysis results of the facial photo. For example, a prompt such as "Please generate multiple photos of a woman in her twenties wearing a casual dress" is created using the prompt generation means.

[0669] Based on the prompt, the image generation means uses algorithms such as GAN (generative adversarial network) using TensorFlow or PyTorch to generate images of the user wearing various costumes. The generated images are stored in a database and then managed by the database storage means.

[0670] The generated try-on images are sent to the smartphone app by the user terminal transmission means. The user can then use the catalog browsing means to view the sent try-on images on the app and select the outfit they like.

[0671] When a user makes a reservation to try on clothes, they fill out the reservation form in the app and send the information to the server. This is done by the try-on reservation information sending means. The server saves the received reservation information in the database saving means and simultaneously notifies the sales / rental business. The sales / rental business notifying means is used to notify the relevant parties of the user's reservation information, ensuring a smooth try-on experience.

[0672] This system not only allows users to easily make fitting reservations and select outfits, but also enables efficient operation for sales / rental businesses. This allows users to experience virtual fittings at home or anywhere and easily select the appropriate outfit.

[0673] For example, if a user wants to choose a furisode for their coming-of-age ceremony, they take a photo of their face directly into the app and upload it, and a prompt such as "Please generate photos of a woman in her 20s wearing multiple casual dresses" is sent to the generative AI model. This model then generates an image of the user wearing the furisode, and the user then selects a furisode based on that image and makes a reservation to try it on.

[0674] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0675] Step 1:

[0676] A user launches a smartphone application and logs in. They can then use the app's camera to take a photo of their face or select and upload a photo from their existing photo library.

[0677] Input: User's face photo

[0678] Output: Face photo data to the server

[0679] Specific operation: Send the facial photo data to the server using the image upload method.

[0680] Step 2:

[0681] The server processes the received facial photos using image analysis tools to extract facial features, performs facial recognition using libraries such as OpenCV, and converts them into a standardized format.

[0682] Input: User's face photo data

[0683] Output: Extracted facial feature data

[0684] Specific operation: The received facial photo is analyzed using OpenCV, and facial features are extracted and standardized.

[0685] Step 3:

[0686] The server generates a prompt sentence to be sent to the image generation AI based on the analysis results of the facial photo. Using the prompt sentence generation means, it creates an instruction sentence based on specific conditions.

[0687] Input: Extracted facial feature data

[0688] Output: prompt statement

[0689] Specific operation: Based on the analysis results, generate a prompt such as "Please generate multiple photos of a woman in her 20s wearing a casual dress."

[0690] Step 4:

[0691] The server sends prompts to an image generation AI model (e.g., GAN), which generates images of the user trying on various outfits.

[0692] Input: prompt statement

[0693] Output: Try-on image data

[0694] Specific operation: The image generation AI model is run based on the prompt sentence, and the generated image is obtained.

[0695] Step 5:

[0696] The generated try-on images are stored in a database by a database storage means.

[0697] Input: Try-on image data

[0698] Output: Try-on images stored in a database

[0699] Specific operation: Save the generated try-on images in an image database.

[0700] Step 6:

[0701] The server transmits the generated try-on images to a smartphone app via a user terminal transmission means, allowing the user to view them.

[0702] Input: Try-on images stored in the database

[0703] Output: Try-on image sent to user's device

[0704] Specific operation: The generated try-on image is sent to the user's device.

[0705] Step 7:

[0706] The user browses the sent catalog, selects a favorite outfit, and checks the try-on images using the catalog browsing means.

[0707] Input: Try-on image sent to user's device

[0708] Output: User selected outfit information

[0709] What it does: The user browses through try-on images within the app and selects the outfit that best suits them.

[0710] Step 8:

[0711] The user makes a reservation for trying on the selected outfit, and transmits the reservation information to the server using the try-on reservation information transmitting means.

[0712] Input: Information about the outfit selected by the user, fitting reservation information

[0713] Output: Reservation information sent to the server

[0714] Specific operation: Fill out the reservation form within the app and send the fitting reservation information to the server.

[0715] Step 9:

[0716] The server stores the received reservation information in a database using a database storage means, and notifies the sales / rental business of the reservation information.

[0717] Input: Reservation information sent to the server

[0718] Output: Reservation information stored in the database, notified businesses

[0719] What it does: Stores the reservation information and notifies the sales / rental business via email or notification system.

[0720] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0721] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. Furthermore, the system incorporates an emotion engine that recognizes the user's emotions, adding a function that suggests optimal outfits based on the user's emotional state.

[0722] Specific operation of the system

[0723] 1. Registering a user's photo

[0724] The user launches the application and logs in. After logging in, they take a photo of themselves or select one from their existing photo library and upload it. They then press the "Upload" button to send the photo to the server.

[0725] Example: If a user wants to choose a furisode kimono for their Coming of Age Ceremony, they can take a photo of their face directly into the app and upload it.

[0726] 2. Processing facial photos

[0727] The server receives the face photo uploaded by the user, uses image analysis algorithms to extract facial features, crops the extracted face area, and standardizes the resolution and size.

[0728] Example: The server uses a library such as OpenCV to perform face recognition and convert the data into a standardized format.

[0729] 3. Clothing catalog generation using image generation AI

[0730] The server sends a request to the image generation AI based on the preprocessed facial image to generate images of the user wearing various costumes, which are then stored in a database.

[0731] Example: The server uses algorithms such as GAN to generate images of the user wearing a gorgeous furisode or an elegant wedding dress.

[0732] 4. Provision of catalogues

[0733] The server sends the generated costume catalog to the user's terminal, and the catalog data can be viewed by the user in real time.

[0734] Example: A user uses the app to view images of various furisode kimonos and compare multiple designs.

[0735] 5. Emotion Recognition by Emotion Engine

[0736] The server-based emotion engine analyzes the user's facial expressions and estimates their emotional state. This emotion data is collected in real time as the user browses through the clothing catalog.

[0737] Example: The server uses the webcam to capture and analyze facial expressions in real time while the user is viewing the image.

[0738] 6. Recommendations for the best costume

[0739] The server then suggests the most suitable outfits for the user based on the estimated emotional state, including selecting outfits that match the user's preferences and filtering colors and designs based on the emotional state.

[0740] Example: If a user shows a happy expression when seeing a particular furisode, the system will prioritize suggesting that outfit.

[0741] 7. Dynamic catalog updates

[0742] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the contents of the catalog in response to the changes.

[0743] Example: Prioritize a list of outfits that the user is interested in, and add related outfit recommendations to the next page they view.

[0744] 8. Reservations for fitting

[0745] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[0746] Example: To try on the furisode kimono of a user's choice, the user enters the desired date and time and store information to confirm the reservation.

[0747] 9. Submitting and Managing Reservation Information

[0748] The terminal sends the fitting reservation information entered by the user to the server, which stores the received reservation information in a database and notifies the corresponding sales / rental business.

[0749] Example: The server sends reservation information via the business's API or email, and after receiving confirmation from the business, notifies the user of the fitting date.

[0750] 10. Calculating and recording commissions

[0751] When the server confirms the successful transaction, it records the information in a database and calculates the reward according to the successful transaction.

[0752] Example: When a user rents a furisode after trying it on, the server records the data and calculates the fee based on the contract fee from the business operator.

[0753] In this way, the system of the present invention allows users to simulate trying on various outfits based on their own facial photographs, and by receiving optimal outfit suggestions based on their emotional state, they can efficiently and satisfactorily select outfits. The system also has the advantage for sales / rental businesses, as it allows them to efficiently approach customers by utilizing the user's emotions.

[0754] The processing flow will be explained below.

[0755] Step 1:

[0756] The user starts the application and logs in. They enter their user ID and password for authentication.

[0757] Step 2:

[0758] Users can take a photo of themselves using the camera function or select and upload a photo from their existing photo library, then press the "Upload" button to send the photo to the server.

[0759] Step 3:

[0760] The server receives the face photo uploaded by the user, processes the HTTP request, and temporarily stores the image data.

[0761] Step 4:

[0762] The server analyzes the received facial photos and uses a facial recognition algorithm to crop the facial area, including locating the face using the OpenCV library.

[0763] Step 5:

[0764] The server extracts the facial area and standardizes the resolution, size, etc. This preprocessing allows the image generation AI to generate stable, high-quality images.

[0765] Step 6:

[0766] The server sends the preprocessed facial image to the image generation AI and requests it to generate images of the user wearing various outfits. The image generation AI then uses techniques such as GAN to generate multiple outfit patterns.

[0767] Step 7:

[0768] The server receives the generated images, adds corresponding metadata (e.g., type of costume, color, design) to each, and stores them in a database.

[0769] Step 8:

[0770] The server sends the generated clothing catalog to the user's device, sending the constructed catalog data as an HTTP response, allowing the user to view it in real time.

[0771] Step 9:

[0772] Users can browse the costume catalog sent to them within the application and select the costume they like. They can also check detailed information about the costume and other users' ratings by touching the screen.

[0773] Step 10:

[0774] The emotion engine built into the server analyzes the user's facial expressions and estimates their emotional state. The emotion engine uses a webcam to capture and analyze facial expressions in real time while the user is viewing the image.

[0775] Step 11:

[0776] The server then suggests the most suitable outfits for the user based on the estimated emotional state, including selecting outfits that match the user's preferences and filtering colors and designs based on the emotional state.

[0777] Step 12:

[0778] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the catalog contents accordingly. It prioritizes the display of outfits that the user is interested in, and also adds related outfit recommendations to the next page viewed.

[0779] Step 13:

[0780] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[0781] Step 14:

[0782] The device sends the fitting reservation information entered by the user to the server, generating an HTTP POST request and sending the reservation data to the server.

[0783] Step 15:

[0784] The server stores the received reservation information in a database, generates a reservation ID, and records the reservation details.

[0785] Step 16:

[0786] The server sends a fitting reservation notification to the corresponding costume sales / rental business via API or email.

[0787] Step 17:

[0788] The server receives the reservation confirmation from the business and notifies the user of the confirmation information. It also generates a notification message to inform the user of the fitting details.

[0789] Step 18:

[0790] When a transaction is confirmed, the server records the information in a database, saving data such as the transaction ID, transaction date and time, user ID, and business ID.

[0791] Step 19:

[0792] The server calculates the reward for the successful transaction and records the information. It calculates the reward amount based on the contract reward from the business operator and saves the corresponding record in the database.

[0793] This is the specific flow of the program processing of this system. By combining it with an emotion engine, users can receive optimal outfit suggestions based on their emotional state, realizing an efficient and satisfying outfit selection process.

[0794] Example 2

[0795] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0796] In conventional online shopping systems, it was difficult for users to check how clothes would look when tried on using their own photos. Furthermore, there was no system that could provide optimal suggestions based on the user's emotional state, which prevented users from increasing satisfaction. Furthermore, the means by which users could make reservations for try-ons was inefficient, and it was time-consuming for businesses to handle the process.

[0797] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an image uploading means for a user to register their own facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to store the generated images in a database and send them to a user terminal, a means for the user to browse the sent catalog and select a favorite outfit, an emotion recognition means for analyzing the user's facial expression and estimating their emotional state using an emotion engine built into the server, a means for the server to suggest optimal outfits based on the estimated emotional state, a means for the server to dynamically update the catalog content, a reservation information sending means for the user to make a try-on reservation and send the information to the server, and a means for the server to save the reservation information received from the user in a database and notify the sales / rental company. This allows the user to simulate trying on various outfits based on their own facial photo and further receive optimal outfit suggestions based on their emotional state, enabling efficient and satisfying outfit selection.

[0798] "User" refers to anyone who uses this system.

[0799] "Image upload means" refers to the means by which a user can register a photo of their face in the system.

[0800] A "server" refers to a computing device that centrally handles the processing of the entire system.

[0801] "Image analysis means" refers to a means for analyzing facial photographs received by the server and extracting facial features.

[0802] "Image generation means" refers to the means by which the server generates images of the user wearing various costumes based on the analyzed facial photograph.

[0803] A "database" refers to a collection of data that a system uses to efficiently store and manage data.

[0804] "User device" refers to the electronic device (e.g., smartphone, personal computer) used by the user for access.

[0805] "Catalogue" refers to the collection of images of various outfits that are generated and provided by the System and that can be viewed by Users.

[0806] "Emotion engine" refers to the component within the system that analyzes the user's facial expressions and infers their emotional state.

[0807] "Emotion recognition means" refers to a means for analyzing a user's facial expressions using an emotion engine and estimating their emotional state.

[0808] The "means for suggesting optimal outfits" refers to a means for suggesting optimal outfits to a user based on an estimated emotional state.

[0809] "Means for dynamic updating" refers to a means by which the server updates the catalog contents in real time.

[0810] The "reservation information transmission means" refers to a means for transmitting information to the server that allows a user to make a reservation for trying on clothes.

[0811] "Sales / Rental Business" refers to a business that offers costumes for sale or rental.

[0812] This invention relates to a system that generates images based on a user's facial photograph, allowing the user to try on various outfits and efficiently select outfits. This system also incorporates an emotion engine that recognizes the user's emotions, and includes a function to suggest the most suitable outfit based on the user's emotional state.

[0813] First, users launch the application and log in to register their own face photo. Then, they can take a photo of their face or select one from their existing photo library and upload it. The uploaded face photo is sent to the server and stored in a database. For this purpose, users take and upload photos using devices such as smartphones or computers.

[0814] The server then inputs the received facial photo into an image analysis algorithm to extract facial features. This analysis uses libraries such as OpenCV. The extracted facial features are cropped and standardized in resolution and size. The standardized image is then sent to the next processing step.

[0815] Next, the server inputs the preprocessed facial image into the image generation AI, and provides the following prompt to the generation AI model:

[0816] Use a photo of the user's face to generate an image of them wearing a gorgeous furisode or an elegant wedding dress.

[0817] The server uses algorithms such as GAN (Generative Adversarial Network) to generate images of the user's face dressed in various outfits. The generated images are stored in a database, along with associated metadata. This generated outfit catalog is then sent from the server to the user's device, where it can be viewed in real time.

[0818] While the user is browsing the clothing catalog, the emotion engine built into the server captures the user's facial expressions with a webcam and analyzes their emotions using a facial expression analysis algorithm. The emotion data is collected and the user's emotional state is estimated.

[0819] The server then suggests the most suitable outfits based on this emotional state. The suggestion algorithm selects the most suitable outfits by taking into account the user's emotional data and past preference data. The server also dynamically updates the catalog contents according to changes in emotions and sends them to the user's device in real time. This dynamic update allows the user to continue browsing outfits that interest them and related designs.

[0820] When a user finds an outfit they like, they can make a reservation to try it on. They fill out a reservation form within the app and submit information such as the desired date and time and the store. The device sends this information to the server, which stores the received reservation information in a database. The server then notifies the sales / rental company, and after receiving confirmation from the company, notifies the user of the try-on date and time.

[0821] Finally, if the user tries on the clothes and makes a purchase, the server records the information in a database, calculates the commission for the purchase, and notifies the business operator. This process allows users to efficiently choose clothes that satisfy them.

[0822] In this way, the system of the present invention enables a simulation of trying on clothes based on the user's facial photograph, and suggests optimal outfits based on the user's emotional state. It also efficiently handles everything from trying-on reservations to calculating and managing commissions, providing convenience to both users and businesses.

[0823] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0824] Step 1:

[0825] Registering a user's photo

[0826] Users launch the application, log in, and then click the "Upload Face Photo" button to either take a photo or select one from their existing photo library and upload it.

[0827] Input: User's face photo (either a newly taken photo or a photo selected from the library)

[0828] Output: The uploaded face photo is sent to the server and stored in the database.

[0829] Specific operation: The user uses a smartphone or computer to take or select a photo of their face and send it to the server via the app.

[0830] Step 2:

[0831] Face photo processing

[0832] The server inputs the facial photo received from the user into an image analysis algorithm to extract facial features. Facial recognition is performed using the OpenCV library.

[0833] Input: A photo of the face sent by the user

[0834] Output: extracted facial features and standardized face image

[0835] Specific operation: The server uses OpenCV to perform facial recognition and converts the facial image to a standard size of 500x500 pixels.

[0836] Step 3:

[0837] Clothing catalog generation using image generation AI

[0838] The server inputs the preprocessed facial image into the image generation AI, and gives the following prompt to the generation AI model:

[0839] Use a photo of the user's face to generate an image of them wearing a gorgeous furisode or an elegant wedding dress.

[0840] Input: Standardized face image and prompt text

[0841] Output: Images of the user wearing various generated outfits

[0842] How it works: The server uses a Generative Adversarial Network (GAN) to generate images of the user wearing gorgeous furisode or elegant dresses based on a photo of their face.

[0843] Step 4:

[0844] Providing a catalog

[0845] The server then sends the generated costume catalog to the user's device, where it can be viewed in real time.

[0846] Input: Generated outfit catalog

[0847] Output: Catalog display on user terminal

[0848] Specific operation: The server sends the generated image data to the user's terminal, allowing the user to view the images in real time on their smartphone or computer.

[0849] Step 5:

[0850] Emotion recognition by emotion engine

[0851] The emotion engine built into the server captures the user's facial expressions using a webcam and analyzes their emotions using an expression analysis algorithm.

[0852] Input: User's facial expressions (images and video captured in real time)

[0853] Output: Estimated emotional state data

[0854] Specific operation: While the user is browsing the catalog, the server captures the user's facial expressions via a webcam and performs emotion recognition.

[0855] Step 6:

[0856] Optimal costume suggestions

[0857] The server then suggests the best outfit for the user based on the estimated emotional state. The suggestion algorithm takes into account emotional data and past preference data.

[0858] Input: Estimated emotional state data and historical preference data

[0859] Output: Optimal costume suggestion data

[0860] Specific operation: The server selects the most suitable outfit based on the user's emotional state and past preference data and suggests it to the user.

[0861] Step 7:

[0862] Dynamic catalog updates

[0863] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the contents of the catalog in response to the changes.

[0864] Input: Real-time changing emotional state data

[0865] Output: Dynamically updated catalog

[0866] Specific operation: The server monitors changes in the user's emotional state in real time, and automatically updates the contents of the catalog and sends them to the user whenever the emotion changes.

[0867] Step 8:

[0868] Accepting reservations for try-on

[0869] To make a reservation to try on the selected outfit, the user fills out a reservation form within the app, inputting information such as the desired date and time and the store, and submitting it.

[0870] Input: Trial reservation information entered by the user

[0871] Output: Reservation information sent to the server

[0872] Specific operation: The user uses the app to enter the desired date and time and store information, and presses the send button to send the reservation information to the server.

[0873] Step 9:

[0874] Submit and manage your reservations

[0875] The server stores the received reservation information in a database and notifies the corresponding sales / rental business.

[0876] Input: Booking information received from the user

[0877] Output: Reservation information stored in the database and notifications to businesses

[0878] Specific operation: The server records the reservation information in a database, notifies the business via API or email, and once confirmation is received, notifies the user of the fitting date.

[0879] Step 10:

[0880] Calculating and recording commissions

[0881] When the server confirms that a contract has been concluded, it records the information in a database, calculates the contract fee, and notifies the business operator.

[0882] Input: Data related to the deal

[0883] Output: Calculated commission and notification

[0884] Specific operation: The server records the contract information in a database, calculates the contract fee, and notifies the business operator.

[0885] (Application example 2)

[0886] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0887] When choosing an outfit, users have to try on many outfits in person, which takes time and effort. Furthermore, when choosing outfits online, there is a risk of making the wrong choice because there are limited ways to check whether the outfit suits the user. Furthermore, outfits suggested without taking into account the user's emotional state may result in low satisfaction. A new system is needed to solve these problems.

[0888] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an image upload means for the user to register their own facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to save the generated images in a database and send them to the user terminal, a means for the user to browse the sent catalog and select a favorite outfit, a reservation information sending means for the user to make a try-on reservation and send that information to the server, a means for recognizing the user's emotional state in real time using an emotion engine and suggesting the most suitable outfit based on the emotional state, and a means for the server to save the reservation information received from the user in a database and notify the sales / rental company. This allows the user to receive the most suitable outfit suggestions in real time based on their facial photo, realizing an efficient and satisfying outfit selection process.

[0889] A "user" is a person who uses this system to register a photo of their face, select an outfit, and make a fitting reservation.

[0890] The "image uploading means" refers to a method and device that allows a user to register a photograph of his or her face.

[0891] The "server" is a central system that analyzes facial photos received from users and performs image generation and emotion recognition.

[0892] "Means for performing image analysis" refers to a technology in which the server processes a facial photo sent by the user and extracts facial features.

[0893] The "image generation means" is a technology that generates virtual images of users wearing various costumes based on facial photographs analyzed by the server.

[0894] The "database" is an information management system for storing generated images and reservation information.

[0895] "User Terminal" means a device through which a User accesses and operates this System.

[0896] A "catalog" is a list of costumes generated in a format that can be viewed by users.

[0897] The "means for making a fitting reservation" refers to a method and device for making a reservation to actually try on the outfit selected by the user.

[0898] The "reservation information transmission means" is a technology that transmits the try-on reservation information entered by the user to the server.

[0899] The "Emotion Engine" is an artificial intelligence technology that analyzes a user's facial expressions and recognizes their emotional state in real time.

[0900] The "means for suggesting optimal outfits" is a technology that dynamically selects and presents optimal outfits to users based on emotional data recognized by the emotion engine.

[0901] The "sales / rental company" is a service provider that actually provides the costumes based on the generated images and fitting reservation information.

[0902] System Overview

[0903] The embodiment of this invention is a virtual store system where users can try on various outfits based on their facial photographs. This system is composed of a user terminal, a server, an image generation means, an emotion engine, a database, and a sales / rental agent.

[0904] Hardware used

[0905] User devices: smartphones, tablets, etc.

[0906] Server: High-performance computer (e.g., Amazon EC2)

[0907] Data storage: Cloud storage (e.g. Amazon S3)

[0908] Software used

[0909] Face Recognition Library: OpenCV

[0910] Emotion recognition engines: Affectiva and RealEyes

[0911] Image generation AI: GAN (Generative Adversarial Network)

[0912] Backend: Python (Flask / Django)

[0913] Data management: MySQL

[0914] Mobile App Development: React Native

[0915] Specific operation of the system

[0916] 1. User photo registration:

[0917] Users launch the smartphone app and log in. After logging in, they can take a photo of themselves or select one from their existing photo library and upload it. The uploaded image is sent to the server.

[0918] 2. Facial Photo Analysis:

[0919] The server analyzes the facial photo received from the user using OpenCV, extracts facial features, and converts them into a standardized format.

[0920] 3. Image generation:

[0921] The server uses GAN to generate images of the user wearing various costumes based on the analyzed facial images, and these generated images are stored in a database.

[0922] 4. Catalog provided by:

[0923] The server sends the generated clothing catalog to the user's device, where the user can browse the catalog and compare multiple clothing items.

[0924] 5. Emotion recognition:

[0925] The emotion engine analyzes users' facial expressions to recognize their emotional state in real time, thereby collecting emotional data as users browse the catalog.

[0926] 6. Best outfit suggestions:

[0927] The server then suggests the most suitable outfit for the user based on the emotional data obtained from the emotion engine. This suggestion is made in real time, increasing user satisfaction.

[0928] 7. Try-on appointment:

[0929] Users can make reservations to try on selected outfits. The reservation information is sent to the server and stored in a database. The server then notifies the sales / rental company of this information.

[0930] Specific examples

[0931] For example, consider a case where a user using a smartphone app is choosing a furisode for their coming-of-age ceremony. The user uploads a photo of their face into the app, which analyzes it and generates an image of them wearing the furisode. As the user browses the catalog, the emotion engine analyzes their facial expressions, and if they show a happy expression, that furisode will be suggested as a priority. If the user wants to try on the furisode, they can make a reservation through the app, and the sales / rental company will be notified.

[0932] Example of input prompt for generative AI model

[0933] "A 20-year-old woman is choosing a furisode for her Coming-of-Age Ceremony. Based on a photo of her face, if she is smiling, she will try on a red furisode; if she is relaxed, she will try on a blue furisode; if she is deep in thought, she will try on a white furisode."

[0934] In this way, the present invention provides a specific system operation method and technology that allows users to select outfits efficiently and satisfactorily.

[0935] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0936] Step 1:

[0937] User device: The user launches the application and logs in. The user takes a photo of their face with their smartphone camera or selects a photo from their existing photo library and sends it to the server using an image upload method.

[0938] Input: User's face photo

[0939] Output: Sending photo data to the server

[0940] Specific operation: The user takes a photo of their face with their smartphone and taps the upload button.

[0941] Step 2:

[0942] Server: The server receives the face photo sent by the user and analyzes the face image using OpenCV, extracts facial features, and standardizes the face image.

[0943] Input: Face photo sent to the server

[0944] Output: Standardized face image

[0945] What it does: The server applies a facial recognition algorithm to identify the location of facial contours, eyes, nose, etc. Based on the identified facial features, it resizes the image to a standard size (e.g., 224x224 pixels).

[0946] Step 3:

[0947] Server: The server uses GAN to generate images of the user wearing various costumes based on the analyzed standardized facial image. The generated images are stored in a database.

[0948] Input: Standardized face image

[0949] Output: Images of the user in various outfits

[0950] Specific operation: The server inputs a standardized face image into the GAN model, generates images of the person trying on multiple outfits, and stores each image in a database.

[0951] Step 4:

[0952] Server: The server sends the generated costume catalog to the user's device. The catalog contains images of the user wearing various costumes generated based on their facial photograph.

[0953] Input: Generated garment image

[0954] Output: Sending catalog data to the user's device

[0955] Specific operation: The server compiles the generated costume images into a catalog format and sends it to the user's device. The user can then view the catalog on their smartphone.

[0956] Step 5:

[0957] User device: The emotion engine monitors the user's facial expressions in real time and estimates their emotional state. Meanwhile, the user can browse the catalog and select their favorite outfit.

[0958] Input: User's facial expression data

[0959] Output: Estimated user emotional state

[0960] How it works: The smartphone camera is used to capture the user's facial expressions, and the emotion recognition engine analyzes the data to infer their emotional state. For example, if the user is smiling, the emotion engine will recognize it as "joy."

[0961] Step 6:

[0962] Server: Based on the emotional state estimated by the emotion engine, the server prioritizes and suggests the most suitable outfits for the user.

[0963] Input: Estimated user emotional state

[0964] Output: Recommendation of optimal outfits to the user

[0965] How it works: The server receives data from the emotion engine, re-ranks the outfits in the catalog based on the estimated emotional state, and displays them to the user. For example, if the emotion of "joy" is recognized, bright-colored outfits will be displayed preferentially.

[0966] Step 7:

[0967] User terminal: The user selects a favorite outfit and makes a reservation to try it on. The try-on reservation information is sent to the server using the reservation information sending means.

[0968] Input: User's fitting reservation information (desired date and time, store, etc.)

[0969] Output: Send reservation information to the server

[0970] Specific operation: If a user wants to try on an outfit selected from the catalog, they fill out the reservation form and tap the submit button.

[0971] Step 8:

[0972] Server: The server stores the reservation information received from the user in a database and notifies the sales / rental agent.

[0973] Input: Try-on reservation information

[0974] Output: Save to database and notify vendor

[0975] Specific operation: The server records the reservation information in a database and notifies the sales / rental agent of the information via API or email.

[0976] example

[0977] Example of an input prompt for a generative AI model:

[0978] "A 20-year-old woman is choosing a furisode for her Coming-of-Age Ceremony. Based on a photo of her face, if she is smiling, she will try on a red furisode; if she is relaxed, she will try on a blue furisode; if she is deep in thought, she will try on a white furisode."

[0979] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0980] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0981] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0982] [Third embodiment]

[0983] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0984] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0985] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0986] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0987] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0988] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0989] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0990] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0991] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0992] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0993] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0994] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0995] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. The system includes an application that runs on the user's terminal, and image generation and data processing on the server side.

[0996] Specific operation of the system

[0997] 1. Registering a user's photo

[0998] Users launch the application and log in. After logging in, they can either take a photo of themselves using the camera or select and upload a photo from their existing photo library.

[0999] Example: If a user wants to choose a furisode kimono for their Coming of Age Ceremony, they can take a photo of their face directly into the app and upload it.

[1000] 2. Processing facial photos

[1001] The server receives the face photo uploaded by the user and uses image analysis algorithms to extract facial features, adjusting the resolution and cropping the face to standardize the face image.

[1002] Example: The server uses a library such as OpenCV to perform face recognition and convert the data into a standardized format.

[1003] 3. Clothing catalog generation using image generation AI

[1004] The server sends a request to the image generation AI based on the preprocessed facial image to generate images of the user wearing various costumes, which are then stored in a database.

[1005] Example: The server uses algorithms such as GAN to generate images of the user wearing a gorgeous furisode or an elegant wedding dress.

[1006] 4. Provision of catalogues

[1007] The server then sends the generated costume catalog to the user's device, where the user can browse the catalog featuring themselves as models and select their favorite outfit from the multiple options.

[1008] Example: A user uses an app to browse images of various furisode kimonos and choose their favorite design.

[1009] 5. Booking a try-on appointment

[1010] To try on an outfit selected from the catalog, the user fills out a reservation form within the app and makes a reservation. The reservation information is sent to the server.

[1011] Example: To try on the furisode kimono of a user's choice, the user enters the desired date and time and store information to confirm the reservation.

[1012] 6. Manage fitting reservations

[1013] The server stores the received reservation information in a database, notifies the costume sales / rental business, and, upon receiving confirmation from the business, notifies the user of the reservation details.

[1014] Example: The server notifies the business of the user's fitting reservation by email, and after confirmation from the business, notifies the user of the fitting date.

[1015] 7. Calculating and recording commissions

[1016] When a transaction is confirmed, the server records the information and calculates the commission for the transaction. This commission is provided by the sales / rental business and is used to maintain and operate the system.

[1017] Example: When a user actually rents a furisode, that information is recorded and the commission from the business is calculated.

[1018] In this way, users can browse a catalog based on their own face photo and make a fitting reservation to efficiently select the most suitable outfit. Sales / rental businesses also benefit from being able to efficiently approach customers through the system. By combining server-side image analysis, generation, and management functions, it is possible to significantly improve the user experience.

[1019] The processing flow will be explained below.

[1020] Step 1:

[1021] The user launches the application and logs in. They enter their user ID and password and access their account through the authentication screen.

[1022] Step 2:

[1023] Users can take a photo of themselves using the camera function or select and upload a photo from their existing photo library, then press the "Upload" button to send the photo to the server.

[1024] Step 3:

[1025] The server receives the face photo uploaded by the user, processes the HTTP request, and temporarily stores the image data.

[1026] Step 4:

[1027] The server analyzes the received facial photos and uses a facial recognition algorithm to crop the facial area, which involves using the OpenCV library to identify the face location.

[1028] Step 5:

[1029] The server extracts the facial area and standardizes the resolution, size, etc. This preprocessing allows the image generation AI to generate stable, high-quality images.

[1030] Step 6:

[1031] The server sends the preprocessed facial image to the image generation AI and requests it to generate images of the user wearing various outfits. The image generation AI then uses techniques such as GAN to generate multiple outfit patterns.

[1032] Step 7:

[1033] The server receives the generated images, adds corresponding metadata (e.g., type of costume, color, design) to each, and stores them in a database.

[1034] Step 8:

[1035] The server sends the generated clothing catalog to the user's device, sending the constructed catalog data as an HTTP response, allowing the user to view it in real time.

[1036] Step 9:

[1037] Users can browse the costume catalog sent to them within the application and select the costume they like. They can also check detailed information about the costume and other users' ratings by touching the screen.

[1038] Step 10:

[1039] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[1040] Step 11:

[1041] The device sends the fitting reservation information entered by the user to the server, generating an HTTP POST request and sending the reservation data to the server.

[1042] Step 12:

[1043] The server stores the received reservation information in a database, generates a reservation ID, and records the reservation details.

[1044] Step 13:

[1045] The server sends a fitting reservation notification to the corresponding costume sales / rental business via API or email.

[1046] Step 14:

[1047] The server receives the reservation confirmation from the business and notifies the user of the confirmation information. It also generates a notification message to inform the user of the fitting details.

[1048] Step 15:

[1049] When a transaction is confirmed, the server records the information in a database, saving data such as the transaction ID, transaction date and time, user ID, and business ID.

[1050] Step 16:

[1051] The server calculates the reward for the successful transaction and records the information. It calculates the reward amount based on the contract reward from the business operator and saves the corresponding record in the database.

[1052] The above is the specific flow of the program processing of this system, which allows users to efficiently and effectively select the most suitable outfit and easily make a fitting reservation.

[1053] Example 1

[1054] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1055] Today, users want to be able to try on various outfits and view the results in an efficient and reliable way using their own facial photos. However, current systems require cumbersome facial photo registration and analysis, which does not improve the user experience. Furthermore, efficient management is required for providing the generated images and accepting try-on reservations. It is necessary to solve these issues and provide a system that is convenient for both users and sales / rental businesses.

[1056] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1057] In this invention, the server includes an image uploading means for users to register their own facial photos, a means for the server to receive the facial photos and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photos, a means for the server to save the generated images in a database and send them to a user terminal, a means for the user to browse the sent catalog and select a favorite outfit, a reservation information sending means for the user to make a try-on reservation and send that information to the server, a means for the server to save the reservation information received from the user in a database and notify the sales / rental company, an automatic adjustment means for the user terminal to automatically adjust the facial photos to an appropriate format, and a means for the server to execute an algorithm to suggest outfit combinations based on the facial images. This allows users to efficiently try on and select outfits based on their own facial photos.

[1058] "User" refers to an individual who uses this system to upload a photo of their face and generate images of them trying on clothes.

[1059] "Image upload means" refers to the function that allows users to send photos of their faces to the system.

[1060] "Server" refers to a computer system that analyzes facial photos received from users, generates costumed images using image generation AI, stores them in a database, and sends them to the user's device.

[1061] "Image analysis means" refers to the function of analyzing facial photographs received by the server and extracting facial features.

[1062] "Image generation means" refers to the function of the server to generate images of users wearing various costumes using a generative AI model based on analyzed facial photos.

[1063] "Database" refers to an information management system for storing images generated by the server and user fitting reservation information.

[1064] "User terminal" refers to a device such as a smartphone, tablet, or PC that a user uses to operate an application.

[1065] A "catalog" refers to a collection of images of users wearing costumes generated by the server, which can be viewed and selected by users.

[1066] The "reservation information transmission means" refers to a function for transmitting reservation information for trying on the costume selected by the user to the server.

[1067] "Automatic adjustment means" refers to a function that allows a user's device to automatically adjust uploaded facial photos into an appropriate format.

[1068] "Contract reward" refers to the reward calculated by the system based on transaction information when a user actually rents or purchases an outfit.

[1069] A "generative AI model" refers to an image generation algorithm used by the server, which generates costume images based on a user's facial photo using a generative artificial network (GAN) or similar.

[1070] A "prompt sentence" refers to an input sentence that specifies the expected result for a generative AI model.

[1071] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. The system includes an application that runs on the user's terminal, and image generation and data processing on the server side.

[1072] User Device

[1073] The user terminal includes devices such as smartphones, tablets, and PCs. The user uses the terminal to launch the application and take a photo of their face or select and upload it from a photo library. The terminal is equipped with an automatic adjustment means, which automatically adjusts the uploaded face photo to an appropriate format.

[1074] Example: When a user is choosing a furisode kimono for their Coming of Age Ceremony, they can use the app to take and upload a photo of their face.

[1075] server

[1076] The server is the central system that analyzes the received facial photos and generates images of the user in various costumes. The server uses the following specific software:

[1077] OpenCV: Used for analyzing and preprocessing face images.

[1078] TensorFlow: Used for image generation using generative AI models (e.g., GANs).

[1079] The server has a pre-processing means for extracting the user's facial features using face recognition technology and standardizing the facial image, and also has a function for storing the generated costume image in a database and transmitting it to the user's terminal.

[1080] Specific behavior:

[1081] The server uses the OpenCV library to analyze facial features and inputs the standardized facial images into a generative AI model based on TensorFlow, which generates images of the user wearing various outfits. The images are stored in a database and then sent to the user's device.

[1082] user

[1083] Users can browse the sent catalog on the app and select their favorite outfits. They can then make a reservation to try on the outfits they selected within the app and send that information to the server. The server stores this information and notifies the sales / rental company.

[1084] Example: A user browses a catalog of furisode kimonos on the app, selects their favorite design, and makes a reservation to try them on.

[1085] Calculation of commission

[1086] The server records the information of the costumes that have been sold and calculates the commission for the sale. This commission is provided by the sales / rental business and is used for the maintenance and operation of the system.

[1087] Prompt Sentence Examples

[1088] "I want to choose a furisode for my Coming-of-Age Ceremony. Please upload a photo of your face and generate images of you trying on various furisode."

[1089] "Try on wedding dresses based on your own photo and create a catalog."

[1090] This allows users to efficiently try on and select outfits based on their own face photos, and sales / rental businesses can also use the system to effectively approach customers.

[1091] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1092] Step 1:

[1093] The user launches the application and logs in, which gives them access to the system, and then uses the camera function to take a photo of their face or upload one from their existing photo library.

[1094] Input: User authentication information (ID, password), face photo

[1095] Output: Face photo sent to server

[1096] Specific actions: Press the "Take Photo" button in the app to take a new photo or press the "Choose from Library" button to select an existing photo and upload a face photo. Press the "Upload" button to send the face photo to the server.

[1097] Step 2:

[1098] The server uses the OpenCV library to run a facial recognition algorithm on the face photo received from the user, extracting facial features, and then standardizes the face image (adjusting resolution and cropping the face) to complete preprocessing.

[1099] Input: Uploaded face photo

[1100] Output: Preprocessed face image

[1101] Specific operation: The server detects faces in the received photos using cv2.CascadeClassifier in the OpenCV library, crops the face area, and resizes the image to a resolution of 256x256 pixels. This completes the standardization process.

[1102] Step 3:

[1103] The server uses the preprocessed facial image to send a request to a generative AI model (likely a GAN) using the TensorFlow library to generate images of the user wearing various costumes, which are then stored in a database.

[1104] Input: Preprocessed face image

[1105] Output: An image of the user wearing the generated outfit

[1106] How it works: The preprocessed face image is input into a GAN model in the TensorFlow library, which then generates images of the user wearing various costumes and stores them in a database.

[1107] Step 4:

[1108] The server retrieves the generated outfit catalog (outfit images based on the user's face photo) from the database and sends it to the user's device. The user can browse this catalog using the app and select the outfit they like.

[1109] Input: Generated images stored in a database

[1110] Output: Outfit catalog sent to user's device

[1111] Specific operation: When a user presses the "View Catalog" button, the server retrieves the user's generated images from the database and sends them to the user's device. The user then views them on the app and presses the "Favorite" button to select an outfit.

[1112] Step 5:

[1113] The user fills out the fitting reservation form for the outfit they have chosen within the app and makes a reservation to try on the outfit. The reservation information is sent to the server.

[1114] Input: Selected outfit, desired fitting date and time, and other reservation information

[1115] Output: Reservation information sent to the server

[1116] Specific operation: The user presses the "Book a try-on" button, enters the desired date and time and store information in the reservation form, and presses the "Confirm reservation" button to send the reservation information to the server.

[1117] Step 6:

[1118] The server stores the received reservation information in a database and notifies the corresponding sales / rental business. After confirmation, the business notifies the user of the try-on details.

[1119] Input: Booking information received from the user

[1120] Output: Notification to sales / rental businesses, notification of try-on details to users

[1121] Specific operation: The server saves the reservation information in a database, notifies the business via email or API, and once confirmation is received from the business, notifies the user of the fitting reservation details.

[1122] Step 7:

[1123] When a transaction is confirmed, the server records the information and calculates the commission for the transaction. This commission is paid by the sales / rental business and is used to maintain and operate the system.

[1124] Input: Deal information

[1125] Output: Contract reward

[1126] Specific operation: When a user tries on an outfit and actually purchases it, the server records the transaction information (sale or rental details) in the database. The server calculates the reward based on this information and charges the business operator. This reward is used to operate the system.

[1127] (Application example 1)

[1128] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1129] Conventional outfit selection systems make it difficult for users to efficiently try on a large number of outfits, requiring them to visit a store in person. Furthermore, the generation of outfit try-on images based on the user's facial photograph often lacks accuracy and variety. Furthermore, the quality and real-time nature of the generated outfit try-on images cannot be guaranteed, resulting in a poor user experience and a cumbersome outfit selection process.

[1130] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1131] In this invention, the server includes an image uploading means for a user to register a facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to store the generated images in a database and send them to a user terminal, a means for the user to browse the transmitted catalog and select a favorite outfit, a reservation information sending means for the user to make a fitting reservation and send that information to the server, a means for the server to store the reservation information received from the user in a database and notify the sales / rental business, a means for generating a prompt message to send to an image generation AI based on the user's characteristics, and a means for providing an image generated based on the prompt message. This allows a user to efficiently generate and select images of trying on outfits through a virtual fitting room.

[1132] The "image uploading means" is a function that allows a user to transfer a photo of their face from their terminal to the server.

[1133] The "image analysis means" is a function that extracts and analyzes facial features from the facial photograph received by the server.

[1134] The "image generation means" is a function that automatically generates images of the user wearing various outfits based on the analyzed facial photograph.

[1135] The "database storage means" is a function for storing the generated try-on images in a database.

[1136] The "user terminal transmission means" is a function for transmitting the try-on images generated by the server to the user's terminal.

[1137] The "catalog viewing means" is a function that allows the user to view the transmitted costume catalog on the terminal.

[1138] The "try-on reservation information sending means" is a function for sending reservation information for trying on the outfit selected by the user to the server.

[1139] The "reservation information storage means" is a function for storing reservation information received from a user in a database.

[1140] The "Sales / Rental Company Notification Method" is a function for notifying sales or rental companies of the user's reservation information.

[1141] The "prompt generation means" is a function for generating instructions to be sent to the image generation AI based on the user's characteristics.

[1142] The "image providing means" is a function for providing the generated image to the user.

[1143] As an embodiment of the present invention, a system for running a virtual fitting room application on a smartphone is constructed. The specific operation of this system will be described below.

[1144] First, the user launches the smartphone application and logs in. After logging in, the user can either take a photo of their face using the application's camera function or select and upload a photo from their existing photo library. This process is carried out by the image uploading means.

[1145] The server then processes the received facial photo using image analysis tools to extract facial features, using libraries such as OpenCV, and standardizes the analyzed facial photo, preparing it for the next step.

[1146] The server generates a prompt to be sent to the image generation AI based on the analysis results of the facial photo. For example, a prompt such as "Please generate multiple photos of a woman in her twenties wearing a casual dress" is created using the prompt generation means.

[1147] Based on the prompt, the image generation means uses algorithms such as GAN (generative adversarial network) using TensorFlow or PyTorch to generate images of the user wearing various costumes. The generated images are stored in a database and then managed by the database storage means.

[1148] The generated try-on images are sent to the smartphone app by the user terminal transmission means. The user can then use the catalog browsing means to view the sent try-on images on the app and select the outfit they like.

[1149] When a user makes a reservation to try on clothes, they fill out the reservation form in the app and send the information to the server. This is done by the try-on reservation information sending means. The server saves the received reservation information in the database saving means and simultaneously notifies the sales / rental business. The sales / rental business notifying means is used to notify the relevant parties of the user's reservation information, ensuring a smooth try-on experience.

[1150] This system not only allows users to easily make fitting reservations and select outfits, but also enables efficient operation for sales / rental businesses. This allows users to experience virtual fittings at home or anywhere and easily select the appropriate outfit.

[1151] For example, if a user wants to choose a furisode for their coming-of-age ceremony, they take a photo of their face directly into the app and upload it, and a prompt such as "Please generate photos of a woman in her 20s wearing multiple casual dresses" is sent to the generative AI model. This model then generates an image of the user wearing the furisode, and the user then selects a furisode based on that image and makes a reservation to try it on.

[1152] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1153] Step 1:

[1154] A user launches a smartphone application and logs in. They can then use the app's camera to take a photo of their face or select and upload a photo from their existing photo library.

[1155] Input: User's face photo

[1156] Output: Face photo data to the server

[1157] Specific operation: Send the facial photo data to the server using the image upload method.

[1158] Step 2:

[1159] The server processes the received facial photos using image analysis tools to extract facial features, performs facial recognition using libraries such as OpenCV, and converts them into a standardized format.

[1160] Input: User's face photo data

[1161] Output: Extracted facial feature data

[1162] Specific operation: The received facial photo is analyzed using OpenCV, and facial features are extracted and standardized.

[1163] Step 3:

[1164] The server generates a prompt sentence to be sent to the image generation AI based on the analysis results of the facial photo. Using the prompt sentence generation means, it creates an instruction sentence based on specific conditions.

[1165] Input: Extracted facial feature data

[1166] Output: prompt statement

[1167] Specific operation: Based on the analysis results, generate a prompt such as "Please generate multiple photos of a woman in her 20s wearing a casual dress."

[1168] Step 4:

[1169] The server sends prompts to an image generation AI model (e.g., GAN), which generates images of the user trying on various outfits.

[1170] Input: prompt statement

[1171] Output: Try-on image data

[1172] Specific operation: The image generation AI model is run based on the prompt sentence, and the generated image is obtained.

[1173] Step 5:

[1174] The generated try-on images are stored in a database by a database storage means.

[1175] Input: Try-on image data

[1176] Output: Try-on images stored in a database

[1177] Specific operation: Save the generated try-on images in an image database.

[1178] Step 6:

[1179] The server transmits the generated try-on images to a smartphone app via a user terminal transmission means, allowing the user to view them.

[1180] Input: Try-on images stored in the database

[1181] Output: Try-on image sent to user's device

[1182] Specific operation: The generated try-on image is sent to the user's device.

[1183] Step 7:

[1184] The user browses the sent catalog, selects a favorite outfit, and checks the try-on images using the catalog browsing means.

[1185] Input: Try-on image sent to user's device

[1186] Output: User selected outfit information

[1187] What it does: The user browses through try-on images within the app and selects the outfit that best suits them.

[1188] Step 8:

[1189] The user makes a reservation for trying on the selected outfit, and transmits the reservation information to the server using the try-on reservation information transmitting means.

[1190] Input: Information about the outfit selected by the user, fitting reservation information

[1191] Output: Reservation information sent to the server

[1192] Specific operation: Fill out the reservation form within the app and send the fitting reservation information to the server.

[1193] Step 9:

[1194] The server stores the received reservation information in a database using a database storage means, and notifies the sales / rental business of the reservation information.

[1195] Input: Reservation information sent to the server

[1196] Output: Reservation information stored in the database, notified businesses

[1197] What it does: Stores the reservation information and notifies the sales / rental business via email or notification system.

[1198] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1199] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. Furthermore, the system incorporates an emotion engine that recognizes the user's emotions, adding a function that suggests optimal outfits based on the user's emotional state.

[1200] Specific operation of the system

[1201] 1. Registering a user's photo

[1202] The user launches the application and logs in. After logging in, they take a photo of themselves or select one from their existing photo library and upload it. They then press the "Upload" button to send the photo to the server.

[1203] Example: If a user wants to choose a furisode kimono for their Coming of Age Ceremony, they can take a photo of their face directly into the app and upload it.

[1204] 2. Processing facial photos

[1205] The server receives the face photo uploaded by the user, uses image analysis algorithms to extract facial features, crops the extracted face area, and standardizes the resolution and size.

[1206] Example: The server uses a library such as OpenCV to perform face recognition and convert the data into a standardized format.

[1207] 3. Clothing catalog generation using image generation AI

[1208] The server sends a request to the image generation AI based on the preprocessed facial image to generate images of the user wearing various costumes, which are then stored in a database.

[1209] Example: The server uses algorithms such as GAN to generate images of the user wearing a gorgeous furisode or an elegant wedding dress.

[1210] 4. Provision of catalogues

[1211] The server sends the generated costume catalog to the user's terminal, and the catalog data can be viewed by the user in real time.

[1212] Example: A user uses the app to view images of various furisode kimonos and compare multiple designs.

[1213] 5. Emotion Recognition by Emotion Engine

[1214] The server-based emotion engine analyzes the user's facial expressions and estimates their emotional state. This emotion data is collected in real time as the user browses through the clothing catalog.

[1215] Example: The server uses the webcam to capture and analyze facial expressions in real time while the user is viewing the image.

[1216] 6. Recommendations for the best costume

[1217] The server then suggests the most suitable outfits for the user based on the estimated emotional state, including selecting outfits that match the user's preferences and filtering colors and designs based on the emotional state.

[1218] Example: If a user shows a happy expression when seeing a particular furisode, the system will prioritize suggesting that outfit.

[1219] 7. Dynamic catalog updates

[1220] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the contents of the catalog in response to the changes.

[1221] Example: Prioritize a list of outfits that the user is interested in, and add related outfit recommendations to the next page they view.

[1222] 8. Reservations for fitting

[1223] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[1224] Example: To try on the furisode kimono of a user's choice, the user enters the desired date and time and store information to confirm the reservation.

[1225] 9. Submitting and Managing Reservation Information

[1226] The terminal sends the fitting reservation information entered by the user to the server, which stores the received reservation information in a database and notifies the corresponding sales / rental business.

[1227] Example: The server sends reservation information via the business's API or email, and after receiving confirmation from the business, notifies the user of the fitting date.

[1228] 10. Calculating and recording commissions

[1229] When the server confirms the successful transaction, it records the information in a database and calculates the reward according to the successful transaction.

[1230] Example: When a user rents a furisode after trying it on, the server records the data and calculates the fee based on the contract fee from the business operator.

[1231] In this way, the system of the present invention allows users to simulate trying on various outfits based on their own facial photographs, and by receiving optimal outfit suggestions based on their emotional state, they can efficiently and satisfactorily select outfits. The system also has the advantage for sales / rental businesses, as it allows them to efficiently approach customers by utilizing the user's emotions.

[1232] The processing flow will be explained below.

[1233] Step 1:

[1234] The user starts the application and logs in. They enter their user ID and password for authentication.

[1235] Step 2:

[1236] Users can take a photo of themselves using the camera function or select and upload a photo from their existing photo library, then press the "Upload" button to send the photo to the server.

[1237] Step 3:

[1238] The server receives the face photo uploaded by the user, processes the HTTP request, and temporarily stores the image data.

[1239] Step 4:

[1240] The server analyzes the received facial photos and uses a facial recognition algorithm to crop the facial area, including locating the face using the OpenCV library.

[1241] Step 5:

[1242] The server extracts the facial area and standardizes the resolution, size, etc. This preprocessing allows the image generation AI to generate stable, high-quality images.

[1243] Step 6:

[1244] The server sends the preprocessed facial image to the image generation AI and requests it to generate images of the user wearing various outfits. The image generation AI then uses techniques such as GAN to generate multiple outfit patterns.

[1245] Step 7:

[1246] The server receives the generated images, adds corresponding metadata (e.g., type of costume, color, design) to each, and stores them in a database.

[1247] Step 8:

[1248] The server sends the generated clothing catalog to the user's device, sending the constructed catalog data as an HTTP response, allowing the user to view it in real time.

[1249] Step 9:

[1250] Users can browse the costume catalog sent to them within the application and select the costume they like. They can also check detailed information about the costume and other users' ratings by touching the screen.

[1251] Step 10:

[1252] The emotion engine built into the server analyzes the user's facial expressions and estimates their emotional state. The emotion engine uses a webcam to capture and analyze facial expressions in real time while the user is viewing the image.

[1253] Step 11:

[1254] The server then suggests the most suitable outfits for the user based on the estimated emotional state, including selecting outfits that match the user's preferences and filtering colors and designs based on the emotional state.

[1255] Step 12:

[1256] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the catalog contents accordingly. It prioritizes the display of outfits that the user is interested in, and also adds related outfit recommendations to the next page viewed.

[1257] Step 13:

[1258] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[1259] Step 14:

[1260] The device sends the fitting reservation information entered by the user to the server, generating an HTTP POST request and sending the reservation data to the server.

[1261] Step 15:

[1262] The server stores the received reservation information in a database, generates a reservation ID, and records the reservation details.

[1263] Step 16:

[1264] The server sends a fitting reservation notification to the corresponding costume sales / rental business via API or email.

[1265] Step 17:

[1266] The server receives the reservation confirmation from the business and notifies the user of the confirmation information. It also generates a notification message to inform the user of the fitting details.

[1267] Step 18:

[1268] When a transaction is confirmed, the server records the information in a database, saving data such as the transaction ID, transaction date and time, user ID, and business ID.

[1269] Step 19:

[1270] The server calculates the reward for the successful transaction and records the information. It calculates the reward amount based on the contract reward from the business operator and saves the corresponding record in the database.

[1271] This is the specific flow of the program processing of this system. By combining it with an emotion engine, users can receive optimal outfit suggestions based on their emotional state, realizing an efficient and satisfying outfit selection process.

[1272] Example 2

[1273] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1274] In conventional online shopping systems, it was difficult for users to check how clothes would look when tried on using their own photos. Furthermore, there was no system that could provide optimal suggestions based on the user's emotional state, which prevented users from increasing satisfaction. Furthermore, the means by which users could make reservations for try-ons was inefficient, and it was time-consuming for businesses to handle the process.

[1275] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an image uploading means for a user to register their own facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to store the generated images in a database and send them to a user terminal, a means for the user to browse the sent catalog and select a favorite outfit, an emotion recognition means for analyzing the user's facial expression and estimating their emotional state using an emotion engine built into the server, a means for the server to suggest optimal outfits based on the estimated emotional state, a means for the server to dynamically update the catalog content, a reservation information sending means for the user to make a try-on reservation and send the information to the server, and a means for the server to save the reservation information received from the user in a database and notify the sales / rental company. This allows the user to simulate trying on various outfits based on their own facial photo and further receive optimal outfit suggestions based on their emotional state, enabling efficient and satisfying outfit selection.

[1276] "User" refers to anyone who uses this system.

[1277] "Image upload means" refers to the means by which a user can register a photo of their face in the system.

[1278] A "server" refers to a computing device that centrally handles the processing of the entire system.

[1279] "Image analysis means" refers to a means for analyzing facial photographs received by the server and extracting facial features.

[1280] "Image generation means" refers to the means by which the server generates images of the user wearing various costumes based on the analyzed facial photograph.

[1281] A "database" refers to a collection of data that a system uses to efficiently store and manage data.

[1282] "User device" refers to the electronic device (e.g., smartphone, personal computer) used by the user for access.

[1283] "Catalogue" refers to the collection of images of various outfits that are generated and provided by the System and that can be viewed by Users.

[1284] "Emotion engine" refers to the component within the system that analyzes the user's facial expressions and infers their emotional state.

[1285] "Emotion recognition means" refers to a means for analyzing a user's facial expressions using an emotion engine and estimating their emotional state.

[1286] The "means for suggesting optimal outfits" refers to a means for suggesting optimal outfits to a user based on an estimated emotional state.

[1287] "Means for dynamic updating" refers to a means by which the server updates the catalog contents in real time.

[1288] The "reservation information transmission means" refers to a means for transmitting information to the server that allows a user to make a reservation for trying on clothes.

[1289] "Sales / Rental Business" refers to a business that offers costumes for sale or rental.

[1290] This invention relates to a system that generates images based on a user's facial photograph, allowing the user to try on various outfits and efficiently select outfits. This system also incorporates an emotion engine that recognizes the user's emotions, and includes a function to suggest the most suitable outfit based on the user's emotional state.

[1291] First, users launch the application and log in to register their own face photo. Then, they can take a photo of their face or select one from their existing photo library and upload it. The uploaded face photo is sent to the server and stored in a database. For this purpose, users take and upload photos using devices such as smartphones or computers.

[1292] The server then inputs the received facial photo into an image analysis algorithm to extract facial features. This analysis uses libraries such as OpenCV. The extracted facial features are cropped and standardized in resolution and size. The standardized image is then sent to the next processing step.

[1293] Next, the server inputs the preprocessed facial image into the image generation AI, and provides the following prompt to the generation AI model:

[1294] Use a photo of the user's face to generate an image of them wearing a gorgeous furisode or an elegant wedding dress.

[1295] The server uses algorithms such as GAN (Generative Adversarial Network) to generate images of the user's face dressed in various outfits. The generated images are stored in a database, along with associated metadata. This generated outfit catalog is then sent from the server to the user's device, where it can be viewed in real time.

[1296] While the user is browsing the clothing catalog, the emotion engine built into the server captures the user's facial expressions with a webcam and analyzes their emotions using a facial expression analysis algorithm. The emotion data is collected and the user's emotional state is estimated.

[1297] The server then suggests the most suitable outfits based on this emotional state. The suggestion algorithm selects the most suitable outfits by taking into account the user's emotional data and past preference data. The server also dynamically updates the catalog contents according to changes in emotions and sends them to the user's device in real time. This dynamic update allows the user to continue browsing outfits that interest them and related designs.

[1298] When a user finds an outfit they like, they can make a reservation to try it on. They fill out a reservation form within the app and submit information such as the desired date and time and the store. The device sends this information to the server, which stores the received reservation information in a database. The server then notifies the sales / rental company, and after receiving confirmation from the company, notifies the user of the try-on date and time.

[1299] Finally, if the user tries on the clothes and makes a purchase, the server records the information in a database, calculates the commission for the purchase, and notifies the business operator. This process allows users to efficiently choose clothes that satisfy them.

[1300] In this way, the system of the present invention enables a simulation of trying on clothes based on the user's facial photograph, and suggests optimal outfits based on the user's emotional state. It also efficiently handles everything from trying-on reservations to calculating and managing commissions, providing convenience to both users and businesses.

[1301] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1302] Step 1:

[1303] Registering a user's photo

[1304] Users launch the application, log in, and then click the "Upload Face Photo" button to either take a photo or select one from their existing photo library and upload it.

[1305] Input: User's face photo (either a newly taken photo or a photo selected from the library)

[1306] Output: The uploaded face photo is sent to the server and stored in the database.

[1307] Specific operation: The user uses a smartphone or computer to take or select a photo of their face and send it to the server via the app.

[1308] Step 2:

[1309] Face photo processing

[1310] The server inputs the facial photo received from the user into an image analysis algorithm to extract facial features. Facial recognition is performed using the OpenCV library.

[1311] Input: A photo of the face sent by the user

[1312] Output: extracted facial features and standardized face image

[1313] Specific operation: The server uses OpenCV to perform facial recognition and converts the facial image to a standard size of 500x500 pixels.

[1314] Step 3:

[1315] Clothing catalog generation using image generation AI

[1316] The server inputs the preprocessed facial image into the image generation AI, and gives the following prompt to the generation AI model:

[1317] Use a photo of the user's face to generate an image of them wearing a gorgeous furisode or an elegant wedding dress.

[1318] Input: Standardized face image and prompt text

[1319] Output: Images of the user wearing various generated outfits

[1320] How it works: The server uses a Generative Adversarial Network (GAN) to generate images of the user wearing gorgeous furisode or elegant dresses based on a photo of their face.

[1321] Step 4:

[1322] Providing a catalog

[1323] The server then sends the generated costume catalog to the user's device, where it can be viewed in real time.

[1324] Input: Generated outfit catalog

[1325] Output: Catalog display on user terminal

[1326] Specific operation: The server sends the generated image data to the user's terminal, allowing the user to view the images in real time on their smartphone or computer.

[1327] Step 5:

[1328] Emotion recognition by emotion engine

[1329] The emotion engine built into the server captures the user's facial expressions using a webcam and analyzes their emotions using an expression analysis algorithm.

[1330] Input: User's facial expressions (images and video captured in real time)

[1331] Output: Estimated emotional state data

[1332] Specific operation: While the user is browsing the catalog, the server captures the user's facial expressions via a webcam and performs emotion recognition.

[1333] Step 6:

[1334] Optimal costume suggestions

[1335] The server then suggests the best outfit for the user based on the estimated emotional state. The suggestion algorithm takes into account emotional data and past preference data.

[1336] Input: Estimated emotional state data and historical preference data

[1337] Output: Optimal costume suggestion data

[1338] Specific operation: The server selects the most suitable outfit based on the user's emotional state and past preference data and suggests it to the user.

[1339] Step 7:

[1340] Dynamic catalog updates

[1341] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the contents of the catalog in response to the changes.

[1342] Input: Real-time changing emotional state data

[1343] Output: Dynamically updated catalog

[1344] Specific operation: The server monitors changes in the user's emotional state in real time, and automatically updates the contents of the catalog and sends them to the user whenever the emotion changes.

[1345] Step 8:

[1346] Accepting reservations for try-on

[1347] To make a reservation to try on the selected outfit, the user fills out a reservation form within the app, inputting information such as the desired date and time and the store, and submitting it.

[1348] Input: Trial reservation information entered by the user

[1349] Output: Reservation information sent to the server

[1350] Specific operation: The user uses the app to enter the desired date and time and store information, and presses the send button to send the reservation information to the server.

[1351] Step 9:

[1352] Submit and manage your reservations

[1353] The server stores the received reservation information in a database and notifies the corresponding sales / rental business.

[1354] Input: Booking information received from the user

[1355] Output: Reservation information stored in the database and notifications to businesses

[1356] Specific operation: The server records the reservation information in a database, notifies the business via API or email, and once confirmation is received, notifies the user of the fitting date.

[1357] Step 10:

[1358] Calculating and recording commissions

[1359] When the server confirms that a contract has been concluded, it records the information in a database, calculates the contract fee, and notifies the business operator.

[1360] Input: Data related to the deal

[1361] Output: Calculated commission and notification

[1362] Specific operation: The server records the contract information in a database, calculates the contract fee, and notifies the business operator.

[1363] (Application example 2)

[1364] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1365] When choosing an outfit, users have to try on many outfits in person, which takes time and effort. Furthermore, when choosing outfits online, there is a risk of making the wrong choice because there are limited ways to check whether the outfit suits the user. Furthermore, outfits suggested without taking into account the user's emotional state may result in low satisfaction. A new system is needed to solve these problems.

[1366] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an image upload means for the user to register their own facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to save the generated images in a database and send them to the user terminal, a means for the user to browse the sent catalog and select a favorite outfit, a reservation information sending means for the user to make a try-on reservation and send that information to the server, a means for recognizing the user's emotional state in real time using an emotion engine and suggesting the most suitable outfit based on the emotional state, and a means for the server to save the reservation information received from the user in a database and notify the sales / rental company. This allows the user to receive the most suitable outfit suggestions in real time based on their facial photo, realizing an efficient and satisfying outfit selection process.

[1367] A "user" is a person who uses this system to register a photo of their face, select an outfit, and make a fitting reservation.

[1368] The "image uploading means" refers to a method and device that allows a user to register a photograph of his or her face.

[1369] The "server" is a central system that analyzes facial photos received from users and performs image generation and emotion recognition.

[1370] "Means for performing image analysis" refers to a technology in which the server processes a facial photo sent by the user and extracts facial features.

[1371] The "image generation means" is a technology that generates virtual images of users wearing various costumes based on facial photographs analyzed by the server.

[1372] The "database" is an information management system for storing generated images and reservation information.

[1373] "User Terminal" means a device through which a User accesses and operates this System.

[1374] A "catalog" is a list of costumes generated in a format that can be viewed by users.

[1375] The "means for making a fitting reservation" refers to a method and device for making a reservation to actually try on the outfit selected by the user.

[1376] The "reservation information transmission means" is a technology that transmits the try-on reservation information entered by the user to the server.

[1377] The "Emotion Engine" is an artificial intelligence technology that analyzes a user's facial expressions and recognizes their emotional state in real time.

[1378] The "means for suggesting optimal outfits" is a technology that dynamically selects and presents optimal outfits to users based on emotional data recognized by the emotion engine.

[1379] The "sales / rental company" is a service provider that actually provides the costumes based on the generated images and fitting reservation information.

[1380] System Overview

[1381] The embodiment of this invention is a virtual store system where users can try on various outfits based on their facial photographs. This system is composed of a user terminal, a server, an image generation means, an emotion engine, a database, and a sales / rental agent.

[1382] Hardware used

[1383] User devices: smartphones, tablets, etc.

[1384] Server: High-performance computer (e.g., Amazon EC2)

[1385] Data storage: Cloud storage (e.g. Amazon S3)

[1386] Software used

[1387] Face Recognition Library: OpenCV

[1388] Emotion recognition engines: Affectiva and RealEyes

[1389] Image generation AI: GAN (Generative Adversarial Network)

[1390] Backend: Python (Flask / Django)

[1391] Data management: MySQL

[1392] Mobile App Development: React Native

[1393] Specific operation of the system

[1394] 1. User photo registration:

[1395] Users launch the smartphone app and log in. After logging in, they can take a photo of themselves or select one from their existing photo library and upload it. The uploaded image is sent to the server.

[1396] 2. Facial Photo Analysis:

[1397] The server analyzes the facial photo received from the user using OpenCV, extracts facial features, and converts them into a standardized format.

[1398] 3. Image generation:

[1399] The server uses GAN to generate images of the user wearing various costumes based on the analyzed facial images, and these generated images are stored in a database.

[1400] 4. Catalog provided by:

[1401] The server sends the generated clothing catalog to the user's device, where the user can browse the catalog and compare multiple clothing items.

[1402] 5. Emotion recognition:

[1403] The emotion engine analyzes users' facial expressions to recognize their emotional state in real time, thereby collecting emotional data as users browse the catalog.

[1404] 6. Best outfit suggestions:

[1405] The server then suggests the most suitable outfit for the user based on the emotional data obtained from the emotion engine. This suggestion is made in real time, increasing user satisfaction.

[1406] 7. Try-on appointment:

[1407] Users can make reservations to try on selected outfits. The reservation information is sent to the server and stored in a database. The server then notifies the sales / rental company of this information.

[1408] Specific examples

[1409] For example, consider a case where a user using a smartphone app is choosing a furisode for their coming-of-age ceremony. The user uploads a photo of their face into the app, which analyzes it and generates an image of them wearing the furisode. As the user browses the catalog, the emotion engine analyzes their facial expressions, and if they show a happy expression, that furisode will be suggested as a priority. If the user wants to try on the furisode, they can make a reservation through the app, and the sales / rental company will be notified.

[1410] Example of input prompt for generative AI model

[1411] "A 20-year-old woman is choosing a furisode for her Coming-of-Age Ceremony. Based on a photo of her face, if she is smiling, she will try on a red furisode; if she is relaxed, she will try on a blue furisode; if she is deep in thought, she will try on a white furisode."

[1412] In this way, the present invention provides a specific system operation method and technology that allows users to select outfits efficiently and satisfactorily.

[1413] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1414] Step 1:

[1415] User device: The user launches the application and logs in. The user takes a photo of their face with their smartphone camera or selects a photo from their existing photo library and sends it to the server using an image upload method.

[1416] Input: User's face photo

[1417] Output: Sending photo data to the server

[1418] Specific operation: The user takes a photo of their face with their smartphone and taps the upload button.

[1419] Step 2:

[1420] Server: The server receives the face photo sent by the user and analyzes the face image using OpenCV, extracts facial features, and standardizes the face image.

[1421] Input: Face photo sent to the server

[1422] Output: Standardized face image

[1423] What it does: The server applies a facial recognition algorithm to identify the location of facial contours, eyes, nose, etc. Based on the identified facial features, it resizes the image to a standard size (e.g., 224x224 pixels).

[1424] Step 3:

[1425] Server: The server uses GAN to generate images of the user wearing various costumes based on the analyzed standardized facial image. The generated images are stored in a database.

[1426] Input: Standardized face image

[1427] Output: Images of the user in various outfits

[1428] Specific operation: The server inputs a standardized face image into the GAN model, generates images of the person trying on multiple outfits, and stores each image in a database.

[1429] Step 4:

[1430] Server: The server sends the generated costume catalog to the user's device. The catalog contains images of the user wearing various costumes generated based on their facial photograph.

[1431] Input: Generated garment image

[1432] Output: Sending catalog data to the user's device

[1433] Specific operation: The server compiles the generated costume images into a catalog format and sends it to the user's device. The user can then view the catalog on their smartphone.

[1434] Step 5:

[1435] User device: The emotion engine monitors the user's facial expressions in real time and estimates their emotional state. Meanwhile, the user can browse the catalog and select their favorite outfit.

[1436] Input: User's facial expression data

[1437] Output: Estimated user emotional state

[1438] How it works: The smartphone camera is used to capture the user's facial expressions, and the emotion recognition engine analyzes the data to infer their emotional state. For example, if the user is smiling, the emotion engine will recognize it as "joy."

[1439] Step 6:

[1440] Server: Based on the emotional state estimated by the emotion engine, the server prioritizes and suggests the most suitable outfits for the user.

[1441] Input: Estimated user emotional state

[1442] Output: Recommendation of optimal outfits to the user

[1443] How it works: The server receives data from the emotion engine, re-ranks the outfits in the catalog based on the estimated emotional state, and displays them to the user. For example, if the emotion of "joy" is recognized, bright-colored outfits will be displayed preferentially.

[1444] Step 7:

[1445] User terminal: The user selects a favorite outfit and makes a reservation to try it on. The try-on reservation information is sent to the server using the reservation information sending means.

[1446] Input: User's fitting reservation information (desired date and time, store, etc.)

[1447] Output: Send reservation information to the server

[1448] Specific operation: If a user wants to try on an outfit selected from the catalog, they fill out the reservation form and tap the submit button.

[1449] Step 8:

[1450] Server: The server stores the reservation information received from the user in a database and notifies the sales / rental agent.

[1451] Input: Try-on reservation information

[1452] Output: Save to database and notify vendor

[1453] Specific operation: The server records the reservation information in a database and notifies the sales / rental agent of the information via API or email.

[1454] example

[1455] Example of an input prompt for a generative AI model:

[1456] "A 20-year-old woman is choosing a furisode for her Coming-of-Age Ceremony. Based on a photo of her face, if she is smiling, she will try on a red furisode; if she is relaxed, she will try on a blue furisode; if she is deep in thought, she will try on a white furisode."

[1457] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1458] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1459] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1460] [Fourth embodiment]

[1461] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1462] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1463] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1464] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1465] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1466] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1467] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1468] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1469] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1470] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1471] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1472] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1473] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1474] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. The system includes an application that runs on the user's terminal, and image generation and data processing on the server side.

[1475] Specific operation of the system

[1476] 1. Registering a user's photo

[1477] Users launch the application and log in. After logging in, they can either take a photo of themselves using the camera or select and upload a photo from their existing photo library.

[1478] Example: If a user wants to choose a furisode kimono for their Coming of Age Ceremony, they can take a photo of their face directly into the app and upload it.

[1479] 2. Processing facial photos

[1480] The server receives the face photo uploaded by the user and uses image analysis algorithms to extract facial features, adjusting the resolution and cropping the face to standardize the face image.

[1481] Example: The server uses a library such as OpenCV to perform face recognition and convert the data into a standardized format.

[1482] 3. Clothing catalog generation using image generation AI

[1483] The server sends a request to the image generation AI based on the preprocessed facial image to generate images of the user wearing various costumes, which are then stored in a database.

[1484] Example: The server uses algorithms such as GAN to generate images of the user wearing a gorgeous furisode or an elegant wedding dress.

[1485] 4. Provision of catalogues

[1486] The server then sends the generated costume catalog to the user's device, where the user can browse the catalog featuring themselves as models and select their favorite outfit from the multiple options.

[1487] Example: A user uses an app to browse images of various furisode kimonos and choose their favorite design.

[1488] 5. Booking a try-on appointment

[1489] To try on an outfit selected from the catalog, the user fills out a reservation form within the app and makes a reservation. The reservation information is sent to the server.

[1490] Example: To try on the furisode kimono of a user's choice, the user enters the desired date and time and store information to confirm the reservation.

[1491] 6. Manage fitting reservations

[1492] The server stores the received reservation information in a database, notifies the costume sales / rental business, and, upon receiving confirmation from the business, notifies the user of the reservation details.

[1493] Example: The server notifies the business of the user's fitting reservation by email, and after confirmation from the business, notifies the user of the fitting date.

[1494] 7. Calculating and recording commissions

[1495] When a transaction is confirmed, the server records the information and calculates the commission for the transaction. This commission is provided by the sales / rental business and is used to maintain and operate the system.

[1496] Example: When a user actually rents a furisode, that information is recorded and the commission from the business is calculated.

[1497] In this way, users can browse a catalog based on their own face photo and make a fitting reservation to efficiently select the most suitable outfit. Sales / rental businesses also benefit from being able to efficiently approach customers through the system. By combining server-side image analysis, generation, and management functions, it is possible to significantly improve the user experience.

[1498] The processing flow will be explained below.

[1499] Step 1:

[1500] The user launches the application and logs in. They enter their user ID and password and access their account through the authentication screen.

[1501] Step 2:

[1502] Users can take a photo of themselves using the camera function or select and upload a photo from their existing photo library, then press the "Upload" button to send the photo to the server.

[1503] Step 3:

[1504] The server receives the face photo uploaded by the user, processes the HTTP request, and temporarily stores the image data.

[1505] Step 4:

[1506] The server analyzes the received facial photos and uses a facial recognition algorithm to crop the facial area, which involves using the OpenCV library to identify the face location.

[1507] Step 5:

[1508] The server extracts the facial area and standardizes the resolution, size, etc. This preprocessing allows the image generation AI to generate stable, high-quality images.

[1509] Step 6:

[1510] The server sends the preprocessed facial image to the image generation AI and requests it to generate images of the user wearing various outfits. The image generation AI then uses techniques such as GAN to generate multiple outfit patterns.

[1511] Step 7:

[1512] The server receives the generated images, adds corresponding metadata (e.g., type of costume, color, design) to each, and stores them in a database.

[1513] Step 8:

[1514] The server sends the generated clothing catalog to the user's device, sending the constructed catalog data as an HTTP response, allowing the user to view it in real time.

[1515] Step 9:

[1516] Users can browse the costume catalog sent to them within the application and select the costume they like. They can also check detailed information about the costume and other users' ratings by touching the screen.

[1517] Step 10:

[1518] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[1519] Step 11:

[1520] The device sends the fitting reservation information entered by the user to the server, generating an HTTP POST request and sending the reservation data to the server.

[1521] Step 12:

[1522] The server stores the received reservation information in a database, generates a reservation ID, and records the reservation details.

[1523] Step 13:

[1524] The server sends a fitting reservation notification to the corresponding costume sales / rental business via API or email.

[1525] Step 14:

[1526] The server receives the reservation confirmation from the business and notifies the user of the confirmation information. It also generates a notification message to inform the user of the fitting details.

[1527] Step 15:

[1528] When a transaction is confirmed, the server records the information in a database, saving data such as the transaction ID, transaction date and time, user ID, and business ID.

[1529] Step 16:

[1530] The server calculates the reward for the successful transaction and records the information. It calculates the reward amount based on the contract reward from the business operator and saves the corresponding record in the database.

[1531] The above is the specific flow of the program processing of this system, which allows users to efficiently and effectively select the most suitable outfit and easily make a fitting reservation.

[1532] Example 1

[1533] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1534] Today, users want to be able to try on various outfits and view the results in an efficient and reliable way using their own facial photos. However, current systems require cumbersome facial photo registration and analysis, which does not improve the user experience. Furthermore, efficient management is required for providing the generated images and accepting try-on reservations. It is necessary to solve these issues and provide a system that is convenient for both users and sales / rental businesses.

[1535] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1536] In this invention, the server includes an image uploading means for users to register their own facial photos, a means for the server to receive the facial photos and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photos, a means for the server to save the generated images in a database and send them to a user terminal, a means for the user to browse the sent catalog and select a favorite outfit, a reservation information sending means for the user to make a try-on reservation and send that information to the server, a means for the server to save the reservation information received from the user in a database and notify the sales / rental company, an automatic adjustment means for the user terminal to automatically adjust the facial photos to an appropriate format, and a means for the server to execute an algorithm to suggest outfit combinations based on the facial images. This allows users to efficiently try on and select outfits based on their own facial photos.

[1537] "User" refers to an individual who uses this system to upload a photo of their face and generate images of them trying on clothes.

[1538] "Image upload means" refers to the function that allows users to send photos of their faces to the system.

[1539] "Server" refers to a computer system that analyzes facial photos received from users, generates costumed images using image generation AI, stores them in a database, and sends them to the user's device.

[1540] "Image analysis means" refers to the function of analyzing facial photographs received by the server and extracting facial features.

[1541] "Image generation means" refers to the function of the server to generate images of users wearing various costumes using a generative AI model based on analyzed facial photos.

[1542] "Database" refers to an information management system for storing images generated by the server and user fitting reservation information.

[1543] "User terminal" refers to a device such as a smartphone, tablet, or PC that a user uses to operate an application.

[1544] A "catalog" refers to a collection of images of users wearing costumes generated by the server, which can be viewed and selected by users.

[1545] The "reservation information transmission means" refers to a function for transmitting reservation information for trying on the costume selected by the user to the server.

[1546] "Automatic adjustment means" refers to a function that allows a user's device to automatically adjust uploaded facial photos into an appropriate format.

[1547] "Contract reward" refers to the reward calculated by the system based on transaction information when a user actually rents or purchases an outfit.

[1548] A "generative AI model" refers to an image generation algorithm used by the server, which generates costume images based on a user's facial photo using a generative artificial network (GAN) or similar.

[1549] A "prompt sentence" refers to an input sentence that specifies the expected result for a generative AI model.

[1550] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. The system includes an application that runs on the user's terminal, and image generation and data processing on the server side.

[1551] User Device

[1552] The user terminal includes devices such as smartphones, tablets, and PCs. The user uses the terminal to launch the application and take a photo of their face or select and upload it from a photo library. The terminal is equipped with an automatic adjustment means, which automatically adjusts the uploaded face photo to an appropriate format.

[1553] Example: When a user is choosing a furisode kimono for their Coming of Age Ceremony, they can use the app to take and upload a photo of their face.

[1554] server

[1555] The server is the central system that analyzes the received facial photos and generates images of the user in various costumes. The server uses the following specific software:

[1556] OpenCV: Used for analyzing and preprocessing face images.

[1557] TensorFlow: Used for image generation using generative AI models (e.g., GANs).

[1558] The server has a pre-processing means for extracting the user's facial features using face recognition technology and standardizing the facial image, and also has a function for storing the generated costume image in a database and transmitting it to the user's terminal.

[1559] Specific behavior:

[1560] The server uses the OpenCV library to analyze facial features and inputs the standardized facial images into a generative AI model based on TensorFlow, which generates images of the user wearing various outfits. The images are stored in a database and then sent to the user's device.

[1561] user

[1562] Users can browse the sent catalog on the app and select their favorite outfits. They can then make a reservation to try on the outfits they selected within the app and send that information to the server. The server stores this information and notifies the sales / rental company.

[1563] Example: A user browses a catalog of furisode kimonos on the app, selects their favorite design, and makes a reservation to try them on.

[1564] Calculation of commission

[1565] The server records the information of the costumes that have been sold and calculates the commission for the sale. This commission is provided by the sales / rental business and is used for the maintenance and operation of the system.

[1566] Prompt Sentence Examples

[1567] "I want to choose a furisode for my Coming-of-Age Ceremony. Please upload a photo of your face and generate images of you trying on various furisode."

[1568] "Try on wedding dresses based on your own photo and create a catalog."

[1569] This allows users to efficiently try on and select outfits based on their own face photos, and sales / rental businesses can also use the system to effectively approach customers.

[1570] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1571] Step 1:

[1572] The user launches the application and logs in, which gives them access to the system, and then uses the camera function to take a photo of their face or upload one from their existing photo library.

[1573] Input: User authentication information (ID, password), face photo

[1574] Output: Face photo sent to server

[1575] Specific actions: Press the "Take Photo" button in the app to take a new photo or press the "Choose from Library" button to select an existing photo and upload a face photo. Press the "Upload" button to send the face photo to the server.

[1576] Step 2:

[1577] The server uses the OpenCV library to run a facial recognition algorithm on the face photo received from the user, extracting facial features, and then standardizes the face image (adjusting resolution and cropping the face) to complete preprocessing.

[1578] Input: Uploaded face photo

[1579] Output: Preprocessed face image

[1580] Specific operation: The server detects faces in the received photos using cv2.CascadeClassifier in the OpenCV library, crops the face area, and resizes the image to a resolution of 256x256 pixels. This completes the standardization process.

[1581] Step 3:

[1582] The server uses the preprocessed facial image to send a request to a generative AI model (likely a GAN) using the TensorFlow library to generate images of the user wearing various costumes, which are then stored in a database.

[1583] Input: Preprocessed face image

[1584] Output: An image of the user wearing the generated outfit

[1585] How it works: The preprocessed face image is input into a GAN model in the TensorFlow library, which then generates images of the user wearing various costumes and stores them in a database.

[1586] Step 4:

[1587] The server retrieves the generated outfit catalog (outfit images based on the user's face photo) from the database and sends it to the user's device. The user can browse this catalog using the app and select the outfit they like.

[1588] Input: Generated images stored in a database

[1589] Output: Outfit catalog sent to user's device

[1590] Specific operation: When a user presses the "View Catalog" button, the server retrieves the user's generated images from the database and sends them to the user's device. The user then views them on the app and presses the "Favorite" button to select an outfit.

[1591] Step 5:

[1592] The user fills out the fitting reservation form for the outfit they have chosen within the app and makes a reservation to try on the outfit. The reservation information is sent to the server.

[1593] Input: Selected outfit, desired fitting date and time, and other reservation information

[1594] Output: Reservation information sent to the server

[1595] Specific operation: The user presses the "Book a try-on" button, enters the desired date and time and store information in the reservation form, and presses the "Confirm reservation" button to send the reservation information to the server.

[1596] Step 6:

[1597] The server stores the received reservation information in a database and notifies the corresponding sales / rental business. After confirmation, the business notifies the user of the try-on details.

[1598] Input: Booking information received from the user

[1599] Output: Notification to sales / rental businesses, notification of try-on details to users

[1600] Specific operation: The server saves the reservation information in a database, notifies the business via email or API, and once confirmation is received from the business, notifies the user of the fitting reservation details.

[1601] Step 7:

[1602] When a transaction is confirmed, the server records the information and calculates the commission for the transaction. This commission is paid by the sales / rental business and is used to maintain and operate the system.

[1603] Input: Deal information

[1604] Output: Contract reward

[1605] Specific operation: When a user tries on an outfit and actually purchases it, the server records the transaction information (sale or rental details) in the database. The server calculates the reward based on this information and charges the business operator. This reward is used to operate the system.

[1606] (Application example 1)

[1607] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1608] Conventional outfit selection systems make it difficult for users to efficiently try on a large number of outfits, requiring them to visit a store in person. Furthermore, the generation of outfit try-on images based on the user's facial photograph often lacks accuracy and variety. Furthermore, the quality and real-time nature of the generated outfit try-on images cannot be guaranteed, resulting in a poor user experience and a cumbersome outfit selection process.

[1609] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1610] In this invention, the server includes an image uploading means for a user to register a facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to store the generated images in a database and send them to a user terminal, a means for the user to browse the transmitted catalog and select a favorite outfit, a reservation information sending means for the user to make a fitting reservation and send that information to the server, a means for the server to store the reservation information received from the user in a database and notify the sales / rental business, a means for generating a prompt message to send to an image generation AI based on the user's characteristics, and a means for providing an image generated based on the prompt message. This allows a user to efficiently generate and select images of trying on outfits through a virtual fitting room.

[1611] The "image uploading means" is a function that allows a user to transfer a photo of their face from their terminal to the server.

[1612] The "image analysis means" is a function that extracts and analyzes facial features from the facial photograph received by the server.

[1613] The "image generation means" is a function that automatically generates images of the user wearing various outfits based on the analyzed facial photograph.

[1614] The "database storage means" is a function for storing the generated try-on images in a database.

[1615] The "user terminal transmission means" is a function for transmitting the try-on images generated by the server to the user's terminal.

[1616] The "catalog viewing means" is a function that allows the user to view the transmitted costume catalog on the terminal.

[1617] The "try-on reservation information sending means" is a function for sending reservation information for trying on the outfit selected by the user to the server.

[1618] The "reservation information storage means" is a function for storing reservation information received from a user in a database.

[1619] The "Sales / Rental Company Notification Method" is a function for notifying sales or rental companies of the user's reservation information.

[1620] The "prompt generation means" is a function for generating instructions to be sent to the image generation AI based on the user's characteristics.

[1621] The "image providing means" is a function for providing the generated image to the user.

[1622] As an embodiment of the present invention, a system for running a virtual fitting room application on a smartphone is constructed. The specific operation of this system will be described below.

[1623] First, the user launches the smartphone application and logs in. After logging in, the user can either take a photo of their face using the application's camera function or select and upload a photo from their existing photo library. This process is carried out by the image uploading means.

[1624] The server then processes the received facial photo using image analysis tools to extract facial features, using libraries such as OpenCV, and standardizes the analyzed facial photo, preparing it for the next step.

[1625] The server generates a prompt to be sent to the image generation AI based on the analysis results of the facial photo. For example, a prompt such as "Please generate multiple photos of a woman in her twenties wearing a casual dress" is created using the prompt generation means.

[1626] Based on the prompt, the image generation means uses algorithms such as GAN (generative adversarial network) using TensorFlow or PyTorch to generate images of the user wearing various costumes. The generated images are stored in a database and then managed by the database storage means.

[1627] The generated try-on images are sent to the smartphone app by the user terminal transmission means. The user can then use the catalog browsing means to view the sent try-on images on the app and select the outfit they like.

[1628] When a user makes a reservation to try on clothes, they fill out the reservation form in the app and send the information to the server. This is done by the try-on reservation information sending means. The server saves the received reservation information in the database saving means and simultaneously notifies the sales / rental business. The sales / rental business notifying means is used to notify the relevant parties of the user's reservation information, ensuring a smooth try-on experience.

[1629] This system not only allows users to easily make fitting reservations and select outfits, but also enables efficient operation for sales / rental businesses. This allows users to experience virtual fittings at home or anywhere and easily select the appropriate outfit.

[1630] For example, if a user wants to choose a furisode for their coming-of-age ceremony, they take a photo of their face directly into the app and upload it, and a prompt such as "Please generate photos of a woman in her 20s wearing multiple casual dresses" is sent to the generative AI model. This model then generates an image of the user wearing the furisode, and the user then selects a furisode based on that image and makes a reservation to try it on.

[1631] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1632] Step 1:

[1633] A user launches a smartphone application and logs in. They can then use the app's camera to take a photo of their face or select and upload a photo from their existing photo library.

[1634] Input: User's face photo

[1635] Output: Face photo data to the server

[1636] Specific operation: Send the facial photo data to the server using the image upload method.

[1637] Step 2:

[1638] The server processes the received facial photos using image analysis tools to extract facial features, performs facial recognition using libraries such as OpenCV, and converts them into a standardized format.

[1639] Input: User's face photo data

[1640] Output: Extracted facial feature data

[1641] Specific operation: The received facial photo is analyzed using OpenCV, and facial features are extracted and standardized.

[1642] Step 3:

[1643] The server generates a prompt sentence to be sent to the image generation AI based on the analysis results of the facial photo. Using the prompt sentence generation means, it creates an instruction sentence based on specific conditions.

[1644] Input: Extracted facial feature data

[1645] Output: prompt statement

[1646] Specific operation: Based on the analysis results, generate a prompt such as "Please generate multiple photos of a woman in her 20s wearing a casual dress."

[1647] Step 4:

[1648] The server sends prompts to an image generation AI model (e.g., GAN), which generates images of the user trying on various outfits.

[1649] Input: prompt statement

[1650] Output: Try-on image data

[1651] Specific operation: The image generation AI model is run based on the prompt sentence, and the generated image is obtained.

[1652] Step 5:

[1653] The generated try-on images are stored in a database by a database storage means.

[1654] Input: Try-on image data

[1655] Output: Try-on images stored in a database

[1656] Specific operation: Save the generated try-on images in an image database.

[1657] Step 6:

[1658] The server transmits the generated try-on images to a smartphone app via a user terminal transmission means, allowing the user to view them.

[1659] Input: Try-on images stored in the database

[1660] Output: Try-on image sent to user's device

[1661] Specific operation: The generated try-on image is sent to the user's device.

[1662] Step 7:

[1663] The user browses the sent catalog, selects a favorite outfit, and checks the try-on images using the catalog browsing means.

[1664] Input: Try-on image sent to user's device

[1665] Output: User selected outfit information

[1666] What it does: The user browses through try-on images within the app and selects the outfit that best suits them.

[1667] Step 8:

[1668] The user makes a reservation for trying on the selected outfit, and transmits the reservation information to the server using the try-on reservation information transmitting means.

[1669] Input: Information about the outfit selected by the user, fitting reservation information

[1670] Output: Reservation information sent to the server

[1671] Specific operation: Fill out the reservation form within the app and send the fitting reservation information to the server.

[1672] Step 9:

[1673] The server stores the received reservation information in a database using a database storage means, and notifies the sales / rental business of the reservation information.

[1674] Input: Reservation information sent to the server

[1675] Output: Reservation information stored in the database, notified businesses

[1676] What it does: Stores the reservation information and notifies the sales / rental business via email or notification system.

[1677] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1678] The system of the present invention generates images based on a user's facial photograph, making it appear as if the user is trying on various outfits, allowing the user to efficiently select outfits. Furthermore, the system incorporates an emotion engine that recognizes the user's emotions, adding a function that suggests optimal outfits based on the user's emotional state.

[1679] Specific operation of the system

[1680] 1. Registering a user's photo

[1681] The user launches the application and logs in. After logging in, they take a photo of themselves or select one from their existing photo library and upload it. They then press the "Upload" button to send the photo to the server.

[1682] Example: If a user wants to choose a furisode kimono for their Coming of Age Ceremony, they can take a photo of their face directly into the app and upload it.

[1683] 2. Processing facial photos

[1684] The server receives the face photo uploaded by the user, uses image analysis algorithms to extract facial features, crops the extracted face area, and standardizes the resolution and size.

[1685] Example: The server uses a library such as OpenCV to perform face recognition and convert the data into a standardized format.

[1686] 3. Clothing catalog generation using image generation AI

[1687] The server sends a request to the image generation AI based on the preprocessed facial image to generate images of the user wearing various costumes, which are then stored in a database.

[1688] Example: The server uses algorithms such as GAN to generate images of the user wearing a gorgeous furisode or an elegant wedding dress.

[1689] 4. Provision of catalogues

[1690] The server sends the generated costume catalog to the user's terminal, and the catalog data can be viewed by the user in real time.

[1691] Example: A user uses the app to view images of various furisode kimonos and compare multiple designs.

[1692] 5. Emotion Recognition by Emotion Engine

[1693] The server-based emotion engine analyzes the user's facial expressions and estimates their emotional state. This emotion data is collected in real time as the user browses through the clothing catalog.

[1694] Example: The server uses the webcam to capture and analyze facial expressions in real time while the user is viewing the image.

[1695] 6. Recommendations for the best costume

[1696] The server then suggests the most suitable outfits for the user based on the estimated emotional state, including selecting outfits that match the user's preferences and filtering colors and designs based on the emotional state.

[1697] Example: If a user shows a happy expression when seeing a particular furisode, the system will prioritize suggesting that outfit.

[1698] 7. Dynamic catalog updates

[1699] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the contents of the catalog in response to the changes.

[1700] Example: Prioritize a list of outfits that the user is interested in, and add related outfit recommendations to the next page they view.

[1701] 8. Reservations for fitting

[1702] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[1703] Example: To try on the furisode kimono of a user's choice, the user enters the desired date and time and store information to confirm the reservation.

[1704] 9. Submitting and Managing Reservation Information

[1705] The terminal sends the fitting reservation information entered by the user to the server, which stores the received reservation information in a database and notifies the corresponding sales / rental business.

[1706] Example: The server sends reservation information via the business's API or email, and after receiving confirmation from the business, notifies the user of the fitting date.

[1707] 10. Calculating and recording commissions

[1708] When the server confirms the successful transaction, it records the information in a database and calculates the reward according to the successful transaction.

[1709] Example: When a user rents a furisode after trying it on, the server records the data and calculates the fee based on the contract fee from the business operator.

[1710] In this way, the system of the present invention allows users to simulate trying on various outfits based on their own facial photographs, and by receiving optimal outfit suggestions based on their emotional state, they can efficiently and satisfactorily select outfits. The system also has the advantage for sales / rental businesses, as it allows them to efficiently approach customers by utilizing the user's emotions.

[1711] The processing flow will be explained below.

[1712] Step 1:

[1713] The user starts the application and logs in. They enter their user ID and password for authentication.

[1714] Step 2:

[1715] Users can take a photo of themselves using the camera function or select and upload a photo from their existing photo library, then press the "Upload" button to send the photo to the server.

[1716] Step 3:

[1717] The server receives the face photo uploaded by the user, processes the HTTP request, and temporarily stores the image data.

[1718] Step 4:

[1719] The server analyzes the received facial photos and uses a facial recognition algorithm to crop the facial area, including locating the face using the OpenCV library.

[1720] Step 5:

[1721] The server extracts the facial area and standardizes the resolution, size, etc. This preprocessing allows the image generation AI to generate stable, high-quality images.

[1722] Step 6:

[1723] The server sends the preprocessed facial image to the image generation AI and requests it to generate images of the user wearing various outfits. The image generation AI then uses techniques such as GAN to generate multiple outfit patterns.

[1724] Step 7:

[1725] The server receives the generated images, adds corresponding metadata (e.g., type of costume, color, design) to each, and stores them in a database.

[1726] Step 8:

[1727] The server sends the generated clothing catalog to the user's device, sending the constructed catalog data as an HTTP response, allowing the user to view it in real time.

[1728] Step 9:

[1729] Users can browse the costume catalog sent to them within the application and select the costume they like. They can also check detailed information about the costume and other users' ratings by touching the screen.

[1730] Step 10:

[1731] The emotion engine built into the server analyzes the user's facial expressions and estimates their emotional state. The emotion engine uses a webcam to capture and analyze facial expressions in real time while the user is viewing the image.

[1732] Step 11:

[1733] The server then suggests the most suitable outfits for the user based on the estimated emotional state, including selecting outfits that match the user's preferences and filtering colors and designs based on the emotional state.

[1734] Step 12:

[1735] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the catalog contents accordingly. It prioritizes the display of outfits that the user is interested in, and also adds related outfit recommendations to the next page viewed.

[1736] Step 13:

[1737] To make a reservation to try on the selected outfit, the user fills out the reservation form, enters the necessary information (desired date and time, store, etc.), and presses the "Make a reservation" button.

[1738] Step 14:

[1739] The device sends the fitting reservation information entered by the user to the server, generating an HTTP POST request and sending the reservation data to the server.

[1740] Step 15:

[1741] The server stores the received reservation information in a database, generates a reservation ID, and records the reservation details.

[1742] Step 16:

[1743] The server sends a fitting reservation notification to the corresponding costume sales / rental business via API or email.

[1744] Step 17:

[1745] The server receives the reservation confirmation from the business and notifies the user of the confirmation information. It also generates a notification message to inform the user of the fitting details.

[1746] Step 18:

[1747] When a transaction is confirmed, the server records the information in a database, saving data such as the transaction ID, transaction date and time, user ID, and business ID.

[1748] Step 19:

[1749] The server calculates the reward for the successful transaction and records the information. It calculates the reward amount based on the contract reward from the business operator and saves the corresponding record in the database.

[1750] This is the specific flow of the program processing of this system. By combining it with an emotion engine, users can receive optimal outfit suggestions based on their emotional state, realizing an efficient and satisfying outfit selection process.

[1751] Example 2

[1752] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1753] In conventional online shopping systems, it was difficult for users to check how clothes would look when tried on using their own photos. Furthermore, there was no system that could provide optimal suggestions based on the user's emotional state, which prevented users from increasing satisfaction. Furthermore, the means by which users could make reservations for try-ons was inefficient, and it was time-consuming for businesses to handle the process.

[1754] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an image uploading means for a user to register their own facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to store the generated images in a database and send them to a user terminal, a means for the user to browse the sent catalog and select a favorite outfit, an emotion recognition means for analyzing the user's facial expression and estimating their emotional state using an emotion engine built into the server, a means for the server to suggest optimal outfits based on the estimated emotional state, a means for the server to dynamically update the catalog content, a reservation information sending means for the user to make a try-on reservation and send the information to the server, and a means for the server to save the reservation information received from the user in a database and notify the sales / rental company. This allows the user to simulate trying on various outfits based on their own facial photo and further receive optimal outfit suggestions based on their emotional state, enabling efficient and satisfying outfit selection.

[1755] "User" refers to anyone who uses this system.

[1756] "Image upload means" refers to the means by which a user can register a photo of their face in the system.

[1757] A "server" refers to a computing device that centrally handles the processing of the entire system.

[1758] "Image analysis means" refers to a means for analyzing facial photographs received by the server and extracting facial features.

[1759] "Image generation means" refers to the means by which the server generates images of the user wearing various costumes based on the analyzed facial photograph.

[1760] A "database" refers to a collection of data that a system uses to efficiently store and manage data.

[1761] "User device" refers to the electronic device (e.g., smartphone, personal computer) used by the user for access.

[1762] "Catalogue" refers to the collection of images of various outfits that are generated and provided by the System and that can be viewed by Users.

[1763] "Emotion engine" refers to the component within the system that analyzes the user's facial expressions and infers their emotional state.

[1764] "Emotion recognition means" refers to a means for analyzing a user's facial expressions using an emotion engine and estimating their emotional state.

[1765] The "means for suggesting optimal outfits" refers to a means for suggesting optimal outfits to a user based on an estimated emotional state.

[1766] "Means for dynamic updating" refers to a means by which the server updates the catalog contents in real time.

[1767] The "reservation information transmission means" refers to a means for transmitting information to the server that allows a user to make a reservation for trying on clothes.

[1768] "Sales / Rental Business" refers to a business that offers costumes for sale or rental.

[1769] This invention relates to a system that generates images based on a user's facial photograph, allowing the user to try on various outfits and efficiently select outfits. This system also incorporates an emotion engine that recognizes the user's emotions, and includes a function to suggest the most suitable outfit based on the user's emotional state.

[1770] First, users launch the application and log in to register their own face photo. Then, they can take a photo of their face or select one from their existing photo library and upload it. The uploaded face photo is sent to the server and stored in a database. For this purpose, users take and upload photos using devices such as smartphones or computers.

[1771] The server then inputs the received facial photo into an image analysis algorithm to extract facial features. This analysis uses libraries such as OpenCV. The extracted facial features are cropped and standardized in resolution and size. The standardized image is then sent to the next processing step.

[1772] Next, the server inputs the preprocessed facial image into the image generation AI, and provides the following prompt to the generation AI model:

[1773] Use a photo of the user's face to generate an image of them wearing a gorgeous furisode or an elegant wedding dress.

[1774] The server uses algorithms such as GAN (Generative Adversarial Network) to generate images of the user's face dressed in various outfits. The generated images are stored in a database, along with associated metadata. This generated outfit catalog is then sent from the server to the user's device, where it can be viewed in real time.

[1775] While the user is browsing the clothing catalog, the emotion engine built into the server captures the user's facial expressions with a webcam and analyzes their emotions using a facial expression analysis algorithm. The emotion data is collected and the user's emotional state is estimated.

[1776] The server then suggests the most suitable outfits based on this emotional state. The suggestion algorithm selects the most suitable outfits by taking into account the user's emotional data and past preference data. The server also dynamically updates the catalog contents according to changes in emotions and sends them to the user's device in real time. This dynamic update allows the user to continue browsing outfits that interest them and related designs.

[1777] When a user finds an outfit they like, they can make a reservation to try it on. They fill out a reservation form within the app and submit information such as the desired date and time and the store. The device sends this information to the server, which stores the received reservation information in a database. The server then notifies the sales / rental company, and after receiving confirmation from the company, notifies the user of the try-on date and time.

[1778] Finally, if the user tries on the clothes and makes a purchase, the server records the information in a database, calculates the commission for the purchase, and notifies the business operator. This process allows users to efficiently choose clothes that satisfy them.

[1779] In this way, the system of the present invention enables a simulation of trying on clothes based on the user's facial photograph, and suggests optimal outfits based on the user's emotional state. It also efficiently handles everything from trying-on reservations to calculating and managing commissions, providing convenience to both users and businesses.

[1780] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1781] Step 1:

[1782] Registering a user's photo

[1783] Users launch the application, log in, and then click the "Upload Face Photo" button to either take a photo or select one from their existing photo library and upload it.

[1784] Input: User's face photo (either a newly taken photo or a photo selected from the library)

[1785] Output: The uploaded face photo is sent to the server and stored in the database.

[1786] Specific operation: The user uses a smartphone or computer to take or select a photo of their face and send it to the server via the app.

[1787] Step 2:

[1788] Face photo processing

[1789] The server inputs the facial photo received from the user into an image analysis algorithm to extract facial features. Facial recognition is performed using the OpenCV library.

[1790] Input: A photo of the face sent by the user

[1791] Output: extracted facial features and standardized face image

[1792] Specific operation: The server uses OpenCV to perform facial recognition and converts the facial image to a standard size of 500x500 pixels.

[1793] Step 3:

[1794] Clothing catalog generation using image generation AI

[1795] The server inputs the preprocessed facial image into the image generation AI, and gives the following prompt to the generation AI model:

[1796] Use a photo of the user's face to generate an image of them wearing a gorgeous furisode or an elegant wedding dress.

[1797] Input: Standardized face image and prompt text

[1798] Output: Images of the user wearing various generated outfits

[1799] How it works: The server uses a Generative Adversarial Network (GAN) to generate images of the user wearing gorgeous furisode or elegant dresses based on a photo of their face.

[1800] Step 4:

[1801] Providing a catalog

[1802] The server then sends the generated costume catalog to the user's device, where it can be viewed in real time.

[1803] Input: Generated outfit catalog

[1804] Output: Catalog display on user terminal

[1805] Specific operation: The server sends the generated image data to the user's terminal, allowing the user to view the images in real time on their smartphone or computer.

[1806] Step 5:

[1807] Emotion recognition by emotion engine

[1808] The emotion engine built into the server captures the user's facial expressions using a webcam and analyzes their emotions using an expression analysis algorithm.

[1809] Input: User's facial expressions (images and video captured in real time)

[1810] Output: Estimated emotional state data

[1811] Specific operation: While the user is browsing the catalog, the server captures the user's facial expressions via a webcam and performs emotion recognition.

[1812] Step 6:

[1813] Optimal costume suggestions

[1814] The server then suggests the best outfit for the user based on the estimated emotional state. The suggestion algorithm takes into account emotional data and past preference data.

[1815] Input: Estimated emotional state data and historical preference data

[1816] Output: Optimal costume suggestion data

[1817] Specific operation: The server selects the most suitable outfit based on the user's emotional state and past preference data and suggests it to the user.

[1818] Step 7:

[1819] Dynamic catalog updates

[1820] The server monitors changes in the emotional state estimated by the emotion engine in real time and dynamically updates the contents of the catalog in response to the changes.

[1821] Input: Real-time changing emotional state data

[1822] Output: Dynamically updated catalog

[1823] Specific operation: The server monitors changes in the user's emotional state in real time, and automatically updates the contents of the catalog and sends them to the user whenever the emotion changes.

[1824] Step 8:

[1825] Accepting reservations for try-on

[1826] To make a reservation to try on the selected outfit, the user fills out a reservation form within the app, inputting information such as the desired date and time and the store, and submitting it.

[1827] Input: Trial reservation information entered by the user

[1828] Output: Reservation information sent to the server

[1829] Specific operation: The user uses the app to enter the desired date and time and store information, and presses the send button to send the reservation information to the server.

[1830] Step 9:

[1831] Submit and manage your reservations

[1832] The server stores the received reservation information in a database and notifies the corresponding sales / rental business.

[1833] Input: Booking information received from the user

[1834] Output: Reservation information stored in the database and notifications to businesses

[1835] Specific operation: The server records the reservation information in a database, notifies the business via API or email, and once confirmation is received, notifies the user of the fitting date.

[1836] Step 10:

[1837] Calculating and recording commissions

[1838] When the server confirms that a contract has been concluded, it records the information in a database, calculates the contract fee, and notifies the business operator.

[1839] Input: Data related to the deal

[1840] Output: Calculated commission and notification

[1841] Specific operation: The server records the contract information in a database, calculates the contract fee, and notifies the business operator.

[1842] (Application example 2)

[1843] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1844] When choosing an outfit, users have to try on many outfits in person, which takes time and effort. Furthermore, when choosing outfits online, there is a risk of making the wrong choice because there are limited ways to check whether the outfit suits the user. Furthermore, outfits suggested without taking into account the user's emotional state may result in low satisfaction. A new system is needed to solve these problems.

[1845] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an image upload means for the user to register their own facial photo, a means for the server to receive the facial photo and perform image analysis, an image generation means for the server to generate images of the user wearing various outfits based on the analyzed facial photo, a means for the server to save the generated images in a database and send them to the user terminal, a means for the user to browse the sent catalog and select a favorite outfit, a reservation information sending means for the user to make a try-on reservation and send that information to the server, a means for recognizing the user's emotional state in real time using an emotion engine and suggesting the most suitable outfit based on the emotional state, and a means for the server to save the reservation information received from the user in a database and notify the sales / rental company. This allows the user to receive the most suitable outfit suggestions in real time based on their facial photo, realizing an efficient and satisfying outfit selection process.

[1846] A "user" is a person who uses this system to register a photo of their face, select an outfit, and make a fitting reservation.

[1847] The "image uploading means" refers to a method and device that allows a user to register a photograph of his or her face.

[1848] The "server" is a central system that analyzes facial photos received from users and performs image generation and emotion recognition.

[1849] "Means for performing image analysis" refers to a technology in which the server processes a facial photo sent by the user and extracts facial features.

[1850] The "image generation means" is a technology that generates virtual images of users wearing various costumes based on facial photographs analyzed by the server.

[1851] The "database" is an information management system for storing generated images and reservation information.

[1852] "User Terminal" means a device through which a User accesses and operates this System.

[1853] A "catalog" is a list of costumes generated in a format that can be viewed by users.

[1854] The "means for making a fitting reservation" refers to a method and device for making a reservation to actually try on the outfit selected by the user.

[1855] The "reservation information transmission means" is a technology that transmits the try-on reservation information entered by the user to the server.

[1856] The "Emotion Engine" is an artificial intelligence technology that analyzes a user's facial expressions and recognizes their emotional state in real time.

[1857] The "means for suggesting optimal outfits" is a technology that dynamically selects and presents optimal outfits to users based on emotional data recognized by the emotion engine.

[1858] The "sales / rental company" is a service provider that actually provides the costumes based on the generated images and fitting reservation information.

[1859] System Overview

[1860] The embodiment of this invention is a virtual store system where users can try on various outfits based on their facial photographs. This system is composed of a user terminal, a server, an image generation means, an emotion engine, a database, and a sales / rental agent.

[1861] Hardware used

[1862] User devices: smartphones, tablets, etc.

[1863] Server: High-performance computer (e.g., Amazon EC2)

[1864] Data storage: Cloud storage (e.g. Amazon S3)

[1865] Software used

[1866] Face Recognition Library: OpenCV

[1867] Emotion recognition engines: Affectiva and RealEyes

[1868] Image generation AI: GAN (Generative Adversarial Network)

[1869] Backend: Python (Flask / Django)

[1870] Data management: MySQL

[1871] Mobile App Development: React Native

[1872] Specific operation of the system

[1873] 1. User photo registration:

[1874] Users launch the smartphone app and log in. After logging in, they can take a photo of themselves or select one from their existing photo library and upload it. The uploaded image is sent to the server.

[1875] 2. Facial Photo Analysis:

[1876] The server analyzes the facial photo received from the user using OpenCV, extracts facial features, and converts them into a standardized format.

[1877] 3. Image generation:

[1878] The server uses GAN to generate images of the user wearing various costumes based on the analyzed facial images, and these generated images are stored in a database.

[1879] 4. Catalog provided by:

[1880] The server sends the generated clothing catalog to the user's device, where the user can browse the catalog and compare multiple clothing items.

[1881] 5. Emotion recognition:

[1882] The emotion engine analyzes users' facial expressions to recognize their emotional state in real time, thereby collecting emotional data as users browse the catalog.

[1883] 6. Best outfit suggestions:

[1884] The server then suggests the most suitable outfit for the user based on the emotional data obtained from the emotion engine. This suggestion is made in real time, increasing user satisfaction.

[1885] 7. Try-on appointment:

[1886] Users can make reservations to try on selected outfits. The reservation information is sent to the server and stored in a database. The server then notifies the sales / rental company of this information.

[1887] Specific examples

[1888] For example, consider a case where a user using a smartphone app is choosing a furisode for their coming-of-age ceremony. The user uploads a photo of their face into the app, which analyzes it and generates an image of them wearing the furisode. As the user browses the catalog, the emotion engine analyzes their facial expressions, and if they show a happy expression, that furisode will be suggested as a priority. If the user wants to try on the furisode, they can make a reservation through the app, and the sales / rental company will be notified.

[1889] Example of input prompt for generative AI model

[1890] "A 20-year-old woman is choosing a furisode for her Coming-of-Age Ceremony. Based on a photo of her face, if she is smiling, she will try on a red furisode; if she is relaxed, she will try on a blue furisode; if she is deep in thought, she will try on a white furisode."

[1891] In this way, the present invention provides a specific system operation method and technology that allows users to select outfits efficiently and satisfactorily.

[1892] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1893] Step 1:

[1894] User device: The user launches the application and logs in. The user takes a photo of their face with their smartphone camera or selects a photo from their existing photo library and sends it to the server using an image upload method.

[1895] Input: User's face photo

[1896] Output: Sending photo data to the server

[1897] Specific operation: The user takes a photo of their face with their smartphone and taps the upload button.

[1898] Step 2:

[1899] Server: The server receives the face photo sent by the user and analyzes the face image using OpenCV, extracts facial features, and standardizes the face image.

[1900] Input: Face photo sent to the server

[1901] Output: Standardized face image

[1902] What it does: The server applies a facial recognition algorithm to identify the location of facial contours, eyes, nose, etc. Based on the identified facial features, it resizes the image to a standard size (e.g., 224x224 pixels).

[1903] Step 3:

[1904] Server: The server uses GAN to generate images of the user wearing various costumes based on the analyzed standardized facial image. The generated images are stored in a database.

[1905] Input: Standardized face image

[1906] Output: Images of the user in various outfits

[1907] Specific operation: The server inputs a standardized face image into the GAN model, generates images of the person trying on multiple outfits, and stores each image in a database.

[1908] Step 4:

[1909] Server: The server sends the generated costume catalog to the user's device. The catalog contains images of the user wearing various costumes generated based on their facial photograph.

[1910] Input: Generated garment image

[1911] Output: Sending catalog data to the user's device

[1912] Specific operation: The server compiles the generated costume images into a catalog format and sends it to the user's device. The user can then view the catalog on their smartphone.

[1913] Step 5:

[1914] User device: The emotion engine monitors the user's facial expressions in real time and estimates their emotional state. Meanwhile, the user can browse the catalog and select their favorite outfit.

[1915] Input: User's facial expression data

[1916] Output: Estimated user emotional state

[1917] How it works: The smartphone camera is used to capture the user's facial expressions, and the emotion recognition engine analyzes the data to infer their emotional state. For example, if the user is smiling, the emotion engine will recognize it as "joy."

[1918] Step 6:

[1919] Server: Based on the emotional state estimated by the emotion engine, the server prioritizes and suggests the most suitable outfits for the user.

[1920] Input: Estimated user emotional state

[1921] Output: Recommendation of optimal outfits to the user

[1922] How it works: The server receives data from the emotion engine, re-ranks the outfits in the catalog based on the estimated emotional state, and displays them to the user. For example, if the emotion of "joy" is recognized, bright-colored outfits will be displayed preferentially.

[1923] Step 7:

[1924] User terminal: The user selects a favorite outfit and makes a reservation to try it on. The try-on reservation information is sent to the server using the reservation information sending means.

[1925] Input: User's fitting reservation information (desired date and time, store, etc.)

[1926] Output: Send reservation information to the server

[1927] Specific operation: If a user wants to try on an outfit selected from the catalog, they fill out the reservation form and tap the submit button.

[1928] Step 8:

[1929] Server: The server stores the reservation information received from the user in a database and notifies the sales / rental agent.

[1930] Input: Try-on reservation information

[1931] Output: Save to database and notify vendor

[1932] Specific operation: The server records the reservation information in a database and notifies the sales / rental agent of the information via API or email.

[1933] example

[1934] Example of an input prompt for a generative AI model:

[1935] "A 20-year-old woman is choosing a furisode for her Coming-of-Age Ceremony. Based on a photo of her face, if she is smiling, she will try on a red furisode; if she is relaxed, she will try on a blue furisode; if she is deep in thought, she will try on a white furisode."

[1936] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1937] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1938] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1939] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1940] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1941] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1942] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1943] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1944] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1945] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the e...

Claims

1. An image uploading means for users to register their own face photos; A server receives the facial photograph and performs image analysis; an image generation means for generating images of the user wearing various costumes based on the analyzed facial photograph; The server stores the generated image in a database and transmits it to a user terminal; A means for users to browse the submitted catalog and select their favorite outfits; A reservation information transmission means for a user to make a reservation for trying on clothes and transmitting the reservation information to a server; The server stores the reservation information received from the user in a database and notifies the sales / rental business. A system including:

2. 2. The system according to claim 1, further comprising means for the server to record information on the costumes that have been sold and to calculate a commission for the sale.

3. The system of claim 1 , wherein the server further comprises a pre-processing means for extracting facial features of the user using facial recognition technology and standardizing the facial image.

Images