System

Abstract

To provide a system for enabling a user to evaluate his / her own style and to easily select an appropriate fashion item.SOLUTION: A means in which a user uploads a whole body photograph, a means in which a server receives the uploaded whole body photograph, a means in which the server performs image analysis on the received whole body photograph, a means in which the server generates an improvement point of a user's fashion based on an image analysis result, and a means in which the server refers to past user photograph data; A system comprising: means for analyzing a user's preference; means for selecting a proposal item from a database of a shopping site affiliated with a server; means for generating a coordination example by combining the proposal item with a whole body photograph; means for transmitting the generated coordination example and an improvement point to a terminal of the user; means for viewing the improvement point and the proposal item through the terminal; and means for accessing the shopping site to purchase the proposal item.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many people today are very interested in fashion, but they often struggle with the common problems of not knowing how to coordinate their outfits and whether items purchased from online shops suit them. Furthermore, choosing the perfect item from the wide variety of fashion items available is not easy and takes time and effort. Therefore, there is a need for a system that allows users to improve their style and easily select the right items. [Means for solving the problem]

[0005] To solve this problem, we provide a system that includes the following means: a means for users to upload full-body photos, a means for a server to perform image analysis of the received photos, a means for generating improvements to the user's fashion based on the analysis results, a means for analyzing the user's past fashion data to understand their preferences, a means for selecting suggested items from affiliated shopping sites, a means for combining the suggested items with a full-body photo to generate coordination examples, and a means for transmitting the improvements and suggested items to the user's terminal.This system allows users to evaluate their own style and easily select appropriate fashion items.

[0006] A "user" is someone who uses the system to upload a full-body photo of themselves and receive suggestions for fashion improvements.

[0007] A "full-body photo" is image data that shows the user's entire body.

[0008] "Means for uploading" refers to a function that provides an interface for users to send full-body photos to an online server.

[0009] A "server" is a remote computer system that stores, analyzes, and transmits the results of full-body photographs received from users.

[0010] The "means for analyzing images" refers to an algorithm or system for analyzing the received full-body photograph and evaluating the characteristics of the fashion items and the overall coordination balance.

[0011] "Fashion improvements" refers to specific advice and suggestions for the user's outfits that are generated based on the results of image analysis.

[0012] "Past user photo data" refers to a database containing full-body photos previously uploaded by users and their analysis results.

[0013] The "means for analyzing user preferences" is a system that grasps a user's fashion trends and preferences based on past user photo data.

[0014] "Shopping site" refers to a website that sells fashion items online.

[0015] "Suggested items" are fashion items from the shopping site that the server selects based on the user's preferences and fashion improvements.

[0016] The "means for combining with a full-body photograph" refers to an algorithm or system that displays suggested items realistically on a full-body photograph of the user and generates specific coordination examples.

[0017] The "coordination example" is an image of a specific fashion coordination created by combining suggested items with a full-body photograph of the user.

[0018] The "transmission means" refers to a communication function for transmitting the generated coordination examples and fashion improvements to the user's terminal.

[0019] "Device" means the device used by a User to upload a full-body photo and receive and display analysis results and outfit examples. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] This invention relates to a system that suggests fashion improvements and additional items suitable for a user by having the user upload a selfie photo. The system includes the following specific procedures and functions.

[0042] User operations

[0043] First, users take a full-body selfie using a smartphone or digital camera, and then they can easily upload this full-body photo through a dedicated app or website.

[0044] Image reception and analysis

[0045] 1. The user takes a full-body photo and submits it using the app or website's upload function.

[0046] 2. The device sends the uploaded full-body photo to the server.

[0047] 3. The server preprocesses the received full-body photo and passes it to the image recognition module.

[0048] 4. The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (tops, bottoms, shoes, etc.), extracting features such as color, shape, and pattern of each fashion item.

[0049] Suggested improvements

[0050] 5. The server evaluates the overall fashion balance based on the extracted features and generates specific improvements, such as providing specific feedback such as "the color balance between the top and bottom is lacking."

[0051] 6. The server references the user's past photo database, learns the user's preferences and tendencies, and generates more personalized advice.

[0052] Item suggestions and synthesis

[0053] 7. The server selects appropriate fashion items from the database of affiliated shopping sites based on the user's preferences and fashion improvements.

[0054] 8. The server then synthesizes the suggested items onto the user's full-body photo to generate a specific coordination example. This composite image allows the user to visually confirm how the suggested items match their own style.

[0055] User Notification

[0056] 9. The server sends the generated feedback of improvements and coordination examples to the terminal.

[0057] 10. The device displays feedback and outfit examples to the user, and also provides links to the best items, allowing the user to go directly to a shopping site and purchase the suggested items.

[0058] Specific examples

[0059] For example, a user may take a full-body photo of themselves wearing a blue jacket, black jeans, and white sneakers. When the user uploads this photo to the system, the server first receives the photo and uses an image recognition module to identify the jacket, jeans, and sneakers. The server then evaluates the overall balance of the outfit and generates feedback such as, "The combination of the jacket and jeans is dark, so it would be good to pair it with a light-colored shirt."

[0060] The server then references the user's past outfit history and determines that the user has previously preferred white and gray shirts. Based on this, the server selects a light blue shirt from a partner shopping site and superimposes it onto the user's full-body photo. A link to the suggested item's details is then sent to the user's device, allowing the user to view and purchase the item directly if necessary.

[0061] In this way, the system can highly personalize fashion suggestions tailored to each user's individual style and preferences, and encourage users' purchasing behavior by providing visual coordination examples.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] The user takes a photo of themselves using a smartphone or digital camera.

[0065] Step 2:

[0066] Users open a dedicated app or website and use the interface to upload a full-body photo they have taken.

[0067] Step 3:

[0068] The device generates an HTTP request to send the uploaded photo file to the server.

[0069] Step 4:

[0070] The server receives the HTTP request, obtains the full-body photo file, and saves it.

[0071] Step 5:

[0072] The server performs pre-processing on the received full-body photos, including image normalization, resizing, and noise removal.

[0073] Step 6:

[0074] The server sends the preprocessed images to an image recognition module.

[0075] Step 7:

[0076] The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (e.g., jacket, pants, shoes, etc.). For each identified item, it extracts features such as color, shape, and pattern.

[0077] Step 8:

[0078] The server evaluates each item individually based on the extracted feature amounts, and evaluates the overall coordination balance.

[0079] Step 9:

[0080] The server calculates improvements to the fashion based on the evaluation results and generates specific feedback to the user, such as "The color combination of your jacket and jeans is too dark, so it would be better to pair it with a lighter colored shirt."

[0081] Step 10:

[0082] The server refers to a database of past photos of the user and analyzes their past fashion trends and preferences, allowing it to make suggestions that are faithful to the user's preferences.

[0083] Step 11:

[0084] The server searches the database of affiliated shopping sites for items that match the user's preferences and correspond to fashion improvements.

[0085] Step 12:

[0086] The server then combines the selected suggested items with a full-body photograph of the user to generate a specific coordination example, allowing the user to visually confirm how the suggested items will suit their style.

[0087] Step 13:

[0088] The server transmits the generated feedback of the improvements and the coordinate examples to the user's terminal.

[0089] Step 14:

[0090] The device displays the received feedback and outfit suggestions to the user, including a link to the newly suggested item.

[0091] Step 15:

[0092] Users can review the displayed feedback and outfit examples, click on the link for the suggested item to access the shopping site, and, if desired, purchase the suggested item directly.

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] Today's consumers need personalized fashion advice and suggestions, but providing them requires specialized knowledge and is time-consuming. Current systems lack the functionality to provide specific feedback or suggest new fashion items to help users review and improve their outfits. This makes it difficult for users to find the perfect fashion items to match their preferences and style.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes: means for a user to upload a full-body photograph; means for receiving the uploaded full-body photograph; means for preprocessing the received full-body photograph; means for image analysis of the preprocessed full-body photograph; means for an image recognition module to identify each fashion item from the full-body photograph; means for extracting features of the identified fashion items by the image recognition module; means for evaluating the user's fashion balance based on the extracted features and generating improvements; means for the server to refer to past user photograph data, analyze the user's preferences, and generate personalized advice; means for selecting suggested items from a database of a shopping site affiliated with the server; means for the server to synthesize the suggested items with the full-body photograph to generate a coordination example; means for transmitting the synthesized coordination example and improvements to the user's terminal; means for the user to view the improvements and suggested items via the terminal; and means for the user to access the shopping site to purchase the suggested items. This enables users to easily find fashion items that suit their style.

[0098] A "full-body photo" is an image taken by a user that includes the entire body from head to toe.

[0099] A "server" is a computer system that receives, processes, and transmits data.

[0100] "Uploading means" refers to a method or function that allows a user to send a full-body photo to a server via the Internet.

[0101] The "receiving means" refers to a method or function by which the server receives the full-body photo sent by the user.

[0102] The "preprocessing means" is a method or function for removing noise from the received full-body photograph and adjusting the size.

[0103] "Image analysis means" refers to a method or function for analyzing a full-body photograph to recognize individual fashion items.

[0104] An "image recognition module" is software or hardware for identifying each fashion item in a full-body photograph and extracting their features.

[0105] The "extracting means" is a method or function for extracting characteristics such as color, shape, pattern, etc. from the identified fashion items.

[0106] The "means for evaluating fashion balance" is a method or function for evaluating the quality of the overall coordination based on the extracted feature amounts.

[0107] The "means for generating improvements" is a method or function for generating specific suggestions for improving coordination based on the evaluation results of fashion balance.

[0108] The "means for analyzing preferences" is a method or function for analyzing past user photo data to learn user preferences and tendencies.

[0109] The "means for selecting suggested items" is a method or function for selecting appropriate fashion items from the database of the affiliated shopping site.

[0110] The "means for generating coordination examples by combining" is a method or function for combining selected suggested items with a full-body photograph to generate specific coordination examples.

[0111] The "means for sending feedback and coordination examples" is a method or function for sending the generated improvements and coordination examples to the user's terminal.

[0112] The "viewing means" is a method or function for a user to use a terminal to view improvements and suggested items.

[0113] "Means for accessing a shopping site" refers to the method or function by which a user accesses a shopping site on the Internet to purchase the suggested items.

[0114] This invention relates to a system that suggests fashion improvements and additional items that suit the user when the user uploads a full-body selfie photo. This system is specifically implemented using a "full-body photo," a "server," "uploading means," "receiving means," "preprocessing means," "image analysis means," an "image recognition module," "extraction means," "means for evaluating fashion balance," "means for generating improvements," "means for analyzing preferences," "means for selecting suggested items," "means for synthesizing and generating coordination examples," "means for sending feedback and coordination examples," "means for viewing," and "means for accessing a shopping site."

[0115] Hardware and Software Usage:

[0116] First, the user takes a full-body selfie using a smartphone or digital camera. Then, the user accesses a dedicated app or website and uploads the full-body photo. Dedicated apps and websites are often built using standard web technologies (HTML, CSS, JavaScript (registered trademark), etc.).

[0117] The user's device sends the uploaded full-body photo to a server. The server is a high-performance computing system that receives the uploaded full-body photo and performs pre-processing on the image. This pre-processing involves noise reduction and resizing using an image processing library such as OpenCV. The pre-processed full-body photo is then passed to an image recognition module. This module uses a deep learning library such as TENSORFLOW® to identify each fashion item (e.g., top, bottom, shoes) in the photo and extract features such as color, shape, and pattern.

[0118] The server evaluates the overall balance of the fashion based on the extracted features and generates specific improvements. This evaluation uses a machine learning model, which can provide specific feedback such as "the color balance between the top and bottom is lacking." The server also references a database of the user's past photos to learn the user's preferences and tendencies. This learning is done using a machine learning algorithm based on the user's past history data.

[0119] The server selects appropriate fashion items from the database of affiliated shopping sites based on the user's preferences and fashion improvements. This selection process accesses the shopping site's database via an API. The suggested items are then combined with the user's full-body photo to generate specific coordination examples. This combination is achieved using image processing tools such as Python's PIL (Pillow) library.

[0120] The generated coordination examples and improvements are sent from the server to the user's device, where they can be viewed through a dedicated app or website. Users can access the detailed link for the suggested items through feedback and purchase the suggested items directly on the shopping site.

[0121] Examples:

[0122] For example, consider the case where a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers and uploads it to a dedicated app. The server receives this photo, performs preprocessing, and then uses an image recognition module to identify each item. During this process, features such as the blue of the jacket, the black of the jeans, and the white of the sneakers are extracted. The server then evaluates the balance of the fashion and generates feedback such as, "The combination of the jacket and jeans is dark, so it would be good to pair it with a light-colored shirt."

[0123] The server also analyzes the user's past photos, learns that the user prefers white or gray shirts, and selects suggested items (e.g., light blue shirts) based on that. It then composites the selected shirt onto a full-body photo to generate a composite image. Finally, the server sends the generated feedback and composite image to the user's device, where the user can review it and directly access the shopping site using the purchase link for the suggested item.

[0124] Example prompt for a generative AI model:

[0125] "If a user takes a full-body photo that shows a blue jacket, black jeans, and white sneakers, the server analyzes the photo, suggests ways to improve the user's fashion, and selects appropriate items (e.g., a light blue shirt) from a partner shopping site. The server then combines the shirt with the full-body photo and presents it to the user as a specific coordination example."

[0126] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0127] Step 1:

[0128] Users take a full-body selfie using a smartphone or digital camera.

[0129] Specific Actions: The user sets the camera so that their entire body is visible and takes a photo of themselves wearing a blue jacket, black jeans, and white sneakers.

[0130] Input: User's full-body photo (image file)

[0131] Output: Full-body photo taken

[0132] Step 2:

[0133] Users access a dedicated app or website and upload a full-body photo.

[0134] Specific behavior: The user opens the app, clicks the "Upload Photo" button, selects a full-body photo they have taken, and submits it.

[0135] Input: Taken full-body photo (image file), user operation

[0136] Output: Uploaded full-body photo

[0137] Step 3:

[0138] The device then sends the uploaded full-body photo to the server.

[0139] Specific operation: The device sends the photo data to the specified endpoint on the server via an Internet connection.

[0140] Input: Uploaded full-body photo (image file)

[0141] Output: Full body photo sent to the server

[0142] Step 4:

[0143] The server pre-processes the received full-body photos.

[0144] Specific operation: OpenCV is used on the server to perform preprocessing such as image noise removal, size adjustment, and color correction.

[0145] Input: Full-body photo (image file) sent to the server

[0146] Output: Preprocessed full-body photo

[0147] Step 5:

[0148] The image recognition module in the server analyzes the pre-processed full-body photos and identifies each fashion item.

[0149] What it does: Uses TensorFlow to identify jackets, jeans, and sneakers in a full-body photo.

[0150] Input: Preprocessed full-body photo

[0151] Output: Identified fashion items (jacket, jeans, sneakers)

[0152] Step 6:

[0153] Based on the identification results, the server extracts features such as color, shape, and pattern of each item.

[0154] Specific operation: For each identified item, calculate the RGB color values, shape parameters, and pattern features.

[0155] Input: Identified fashion item data

[0156] Output: Extracted features (RGB color values, shape, pattern)

[0157] Step 7:

[0158] Based on the extracted features, the server evaluates the fashion balance and generates specific improvements.

[0159] Specific actions: Using a machine learning model, the extracted features are evaluated and specific improvements are suggested, such as "the overall color balance is dark, so add a bright color to the upper body."

[0160] Input: extracted features

[0161] Output: Fashion balance evaluation results and improvements

[0162] Step 8:

[0163] The server references the user's past photo database and learns the user's preferences and tendencies.

[0164] How it works: Past photo data is passed through a machine learning algorithm to identify the colors and styles the user prefers.

[0165] Input: User's past photo data

[0166] Output: Data about user preferences and tendencies

[0167] Step 9:

[0168] The server generates personalized advice that reflects the user's preferences and tendencies.

[0169] Specific operation: The fashion balance evaluation results are combined with the user's preference data to generate personalized advice such as "A light blue shirt is good."

[0170] Input: Fashion balance evaluation results, data on user preferences and trends

[0171] Output: Personalized advice

[0172] Step 10:

[0173] The server combines the suggested items with a full-body photo to generate specific coordination examples.

[0174] Specific behavior: Using Python's PIL (Pillow) library, a light blue shirt is superimposed onto a full-body photo of the user.

[0175] Input: personalized advice, full-body photo of user

[0176] Output: Composite coordinated image

[0177] Step 11:

[0178] The synthesized coordination examples and improvements are sent from the server to the user's device.

[0179] Specific operation: Data including coordination examples and improvements is sent from the server to the user's device.

[0180] Input: Synthesized coordinate image, improvement points

[0181] Output: Coordinate image sent to the user's device and improvements

[0182] Step 12:

[0183] The user's device will display feedback, outfit examples, and provide links to shopping sites.

[0184] How it works: A dedicated app or website provides feedback and visual examples of outfits, along with links to purchase suggested items.

[0185] Input: Coordinate image sent from the server and improvements

[0186] Output: A user-visible feedback screen and a purchase link

[0187] Step 13:

[0188] Users can check the suggested items through their device and access the shopping site if necessary.

[0189] What happens: The user reviews the details of the suggested item in the app and clicks the purchase link to visit the shopping site.

[0190] Input: Feedback screen and purchase link

[0191] Output: Visit to shopping site and purchase item

[0192] (Application example 1)

[0193] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0194] Conventional fashion suggestion systems lack real-time functionality when users receive advice and item suggestions to improve their outfits, making them particularly inconvenient in brick-and-mortar stores. This requires users to take photos every time they try on clothes, upload them, and wait for the results, which takes time and effort. Furthermore, it is difficult to receive personalized suggestions based on users' preferences and trends in real time, making it difficult to directly link these to purchasing behavior.

[0195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0196] In this invention, the server includes: means for a user to upload a full-body photo; means for receiving the uploaded full-body photo; means for image-analyzing the received full-body photo; means for generating improvements to the user's fashion based on the image analysis results; means for referencing past user photo data and analyzing the user's preferences; means for selecting suggested items from a database of a purchasing platform affiliated with the server; means for the server to generate outfit examples by combining the suggested items with the full-body photo; means for the server to transmit the generated outfit examples and improvements to the user's terminal; means for the user to view the improvements and suggested items through the terminal; means for the user to access the purchasing platform to purchase the suggested items; means for the user to take and upload a full-body photo in real time in a store using smart glasses or a head-mounted display; and means for the server to generate fashion analysis results and suggestions in real time and present them to the user. This enables users to receive fashion evaluations and improvement suggestions in real time even in physical stores, providing an efficient shopping experience.

[0197] "User" refers to a person who uses the system to improve their fashion or receive suggestions.

[0198] A "full-body photo" refers to a photo that includes the user's entire body from head to toe.

[0199] "Means for uploading" refers to the functionality that a user uses to send a full-body photo of themselves to the system.

[0200] "Server" refers to a computer system that receives, analyzes, and generates recommendations from user-submitted data.

[0201] "Image analysis" refers to the process in which the server receives a full-body photo and identifies fashion items and extracts features.

[0202] "Image analysis results" refers to the identification information and characteristics of a fashion item obtained through image analysis.

[0203] "Fashion improvements" refers to feedback that suggests a better fashion style for the user's current outfit.

[0204] "User preferences" refers to the user's fashion preferences and tendencies analyzed based on past databases.

[0205] "Buying Platform" refers to an online or offline sales system through which the proposed fashion items are available for purchase.

[0206] "Suggested items" refer to fashion items selected by the server based on the user's preferences and fashion improvements.

[0207] "Synthesis" refers to the process of displaying suggested items superimposed on a full-body photo of the user.

[0208] "Coordination example" refers to an example of a fashion style visually shown by combining suggested items with a full-body photograph of the user.

[0209] "Smart glasses" refers to an augmented reality (AR)-enabled eyeglass-type device that users can wear to take full-body photos and upload them to the system.

[0210] "Head-mounted display" refers to a head-mounted display device that a user wears and can take full-body photos and upload them to the system.

[0211] "Real-time" refers to the short time between when a user takes a full-body photo and when they receive feedback or suggestions from the server, and the time is immediate.

[0212] "Means for generating fashion analysis results and suggestions in real time and presenting them to users" refers to the function of instantly analyzing a full-body photo uploaded by a user to the system and displaying the results and suggestions to the user.

[0213] As an embodiment of the present invention, a series of steps in which a user takes a full-body photo and uploads it to a server in order to improve his or her fashion will be specifically described.

[0214] System Program

[0215] The program consists of the following major steps:

[0216] 1. Take and upload a full-body photo of the user:

[0217] Users take a full-body photo of themselves using a smartphone, smart glasses, or a head-mounted display.

[0218] The full-body photo taken is uploaded to the server via the application.

[0219] 2. Image reception and analysis by the server:

[0220] The server receives the uploaded full-body photograph.

[0221] The received images are preprocessed and passed to an image recognition model (a model using Keras / TensorFlow) for analysis.

[0222] Fashion items are identified and features are extracted using image analysis.

[0223] 3. Fashion Improvements and Item Suggestion Generation:

[0224] The server evaluates the fashion balance based on the analyzed features and generates improvements.

[0225] It learns user preferences by referencing past user data and selects appropriate suggested items from the database of affiliated purchasing platforms.

[0226] 4. Coordination example generation and synthesis:

[0227] The proposed items are combined with the user's full-body photo to generate visual coordination examples.

[0228] Coordination examples and improvements are sent to the user's device.

[0229] 5. User Notice and Shopping Support:

[0230] Users can check areas for improvement and suggested items through their devices.

[0231] Through the link to the suggested item, the user can access the purchasing platform and make a direct purchase.

[0232] Natural language explanation of the process

[0233] Hardware and software used:

[0234] Smartphones, smart glasses, head-mounted displays: devices that allow users to take full-body photos and upload them to a server through an application.

[0235] Keras / TensorFlow: Libraries for running image recognition models on the server.

[0236] OpenCV: A library for image preprocessing and analysis.

[0237] Server: A computer system that receives data, analyzes it, generates proposals, and notifies users.

[0238] Data processing and calculation:

[0239] Image preprocessing: The server resizes the images uploaded by the user and converts them into a format suitable for input to the image recognition model.

[0240] Image analysis: Using an image recognition model, clothing items are identified and their features are extracted.

[0241] Fashion balance evaluation: Based on the extracted features, the balance of the entire outfit is analyzed and improvements are identified.

[0242] Selection of suggested items: By referring to the user's past preference data, appropriate items are selected from the purchasing platform's database.

[0243] Coordination example generation: The proposed items are combined with a full-body photo to generate visually easy-to-understand coordination examples.

[0244] Notifications and shopping support: Send improvements and suggested items to your device and provide direct purchase links.

[0245] Specific examples

[0246] For example, if a user uploads a full-body photo wearing a blue jacket, black jeans, and white sneakers, the server analyzes it and provides feedback such as, "To improve the color balance, pair it with a light-colored shirt." If the user's past data indicates that they prefer light-colored shirts, the server will suggest a light blue shirt. The shirt is then superimposed on the full-body photo, and the result is immediately presented to the user. The user can then purchase the shirt directly on the purchasing platform via the provided link.

[0247] Prompt Sentence Examples

[0248] "Analyze a user's full-body photo, identify fashion items, evaluate their proportions, and generate suggestions for improvements and additional items."

[0249] As described above, this system allows users to receive real-time improvements and suggestions even in physical stores, allowing them to enjoy an efficient and personalized shopping experience.

[0250] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0251] Step 1:

[0252] The user takes a full-body photo and uploads it to the server through the application. The input is a full-body photo taken by the user with a smartphone, smart glasses, or head-mounted display, and the output is an image file sent to the server. The user takes a full-body photo using their device and sends it to the server using the application's upload function.

[0253] Step 2:

[0254] The server receives the uploaded full-body photo. The input is the image file sent by the user, and the output is the image data stored in the server's database. The server stores the received full-body photo in the database.

[0255] Step 3:

[0256] The server preprocesses the received full-body photo and inputs it into the image recognition model. The input is the image data of the saved full-body photo, and the output is image data converted into a format suitable for the model. The server uses OpenCV to perform preprocessing such as image resizing and noise removal, and converts it into a format suitable for input into the image recognition model.

[0257] Step 4:

[0258] The server inputs the preprocessed image data into an image recognition model (Keras / TensorFlow) to identify fashion items and extract their features. The input is the preprocessed image data, and the output is the identified fashion items and their features (color, shape, pattern, etc.). The server uses a generative AI model to perform image recognition, identify each fashion item, and extract its features.

[0259] Step 5:

[0260] The server evaluates the fashion balance based on the extracted features and generates improvements to the user's fashion. The input is the identified fashion items and their features, and the output is a fashion balance evaluation and feedback on improvements. The server analyzes the features, evaluates the overall coordination balance, and calculates specific improvements.

[0261] Step 6:

[0262] The server references the user's past database and learns the user's tastes and preferences. The input is the user's past fashion data, and the output is personalized information based on the user's preferences. The server uses past photo data for machine learning to analyze and learn the user's tastes and trends.

[0263] Step 7:

[0264] The server selects suggested items from the database of the affiliated purchasing platform. The input is the fashion balance rating, the user's preferences, and the purchasing platform's item database, and the output is a list of suggested items. The server selects appropriate items from the purchasing platform based on the user's preferences and fashion improvement points.

[0265] Step 8:

[0266] The server composites the suggested items onto a full-body photo to generate a coordination example. The input is the suggested items and the user's full-body photo, and the output is an image of the composite coordination example. The server composites the suggested items onto a full-body photo to generate a coordination example that can be visually confirmed.

[0267] Step 9:

[0268] The server sends the generated coordination examples and improvements to the user's device. The input is an image of the coordination example and feedback on improvements, and the output is the information displayed on the user's device. The server sends the generated coordination examples and improvements to the user's device and displays them.

[0269] Step 10:

[0270] The user views the improvements and suggested items through the terminal. The input is information sent from the server (coordination examples, improvements, suggested items), and the output is visual information that the user checks. The user uses the terminal to check the sent coordination examples, improvements, and suggested items.

[0271] Step 11:

[0272] The user accesses the purchasing platform to purchase the suggested item. The input is a link to the purchasing platform displayed on the terminal, and the output is the purchase of the suggested item. The user clicks the link through the terminal to access the purchasing platform and purchase the item.

[0273] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0274] This invention relates to a system that uses a full-body photograph of a user to suggest improvements to fashion and also recognizes the user's emotions to suggest personalized items. The following describes the procedures and functions for specifically implementing this system.

[0275] User operations

[0276] First, the user takes a full-body photo of themselves using a smartphone or digital camera. Emotional information, such as facial expressions and voice data, is also collected. This emotional information is recorded in the form of video or voice memos during shooting.

[0277] Receiving and analyzing image and emotion data

[0278] 1. The user opens a dedicated app or website and uploads a full-body photo and emotional information.

[0279] 2. The device sends the uploaded photo file and emotion information to the server via an HTTP request.

[0280] 3. The server receives the HTTP request, obtains the full-body photo and emotion information, and saves them.

[0281] 4. The server preprocesses the full-body photo and passes it to the image recognition module. Preprocessing includes normalizing, resizing, and removing noise from the image.

[0282] 5. The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (top, bottoms, shoes, etc.). It extracts features such as color, shape, and pattern for each identified item.

[0283] Emotional Data Analysis

[0284] 6. The server uses an emotion engine to recognize the user's emotions from the uploaded facial and voice data, for example, analyzing whether the user is smiling, surprised, relaxed, etc.

[0285] 7. The server generates feedback based on the recognized emotion data according to the user's emotions. For example, if the user is relaxed, it may suggest casual fashion.

[0286] Improvements and Item Suggestions

[0287] 8. The server evaluates the overall fashion balance based on the extracted features and emotion data, and calculates areas for improvement. For example, it provides specific feedback such as, "The jacket color is too dark, so it would be better to pair it with a lighter-colored shirt."

[0288] 9. The server refers to the user's past fashion database and analyzes the user's preferences and trends. Based on this, it suggests items that match the user's past preferences and are based on emotional data.

[0289] 10. The server searches and selects items that correspond to the user's preferences and fashion improvements from the database of affiliated shopping sites.

[0290] Combining items and notifying the user

[0291] 11. The server then synthesizes the selected suggested items onto the user's full-body photo to generate a specific coordination example. The synthesized image allows the user to visually confirm how the suggested items match their own style.

[0292] 12. The server sends the generated feedback of improvements and coordinate examples to the terminal.

[0293] 13. The device displays the received feedback and outfit examples to the user, including links to purchase the suggested items.

[0294] 14. Users can review the displayed feedback and outfit examples, click on the link for the suggested item to access the shopping site, and, if desired, purchase the suggested item directly.

[0295] Specific examples

[0296] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. If the user's emotional data indicates that they are in a relaxed state, the server receives and analyzes the photo and emotional data. The server identifies the jacket, jeans, and sneakers and extracts their features. Next, the server evaluates the overall balance of the outfit and calculates an improvement point, such as "matching it with a lighter-colored shirt." The server then suggests casual fashion items (e.g., a light blue shirt) that are appropriate for a relaxed state based on the emotional data. The system then composites the shirt into the full-body photo to generate a specific outfit example. A link to the composite image and the suggested items is sent to the user's device, where the user can view and purchase the items.

[0297] In this way, the system can provide highly personalized fashion suggestions that correspond to the user's emotions, thereby increasing user satisfaction.

[0298] The processing flow will be explained below.

[0299] Step 1:

[0300] Users take full-body photographs of themselves using a smartphone or digital camera, and emotional information such as facial expressions and voice data is also collected.

[0301] Step 2:

[0302] The user opens a dedicated app or website and navigates through a screen to upload a full-body photo and emotional information.

[0303] Step 3:

[0304] The device sends the uploaded full-body photo and emotion information to the server, packaging the photo data and emotion data as an HTTP request.

[0305] Step 4:

[0306] The server receives the HTTP request and stores the full-body photo and emotion data separately.

[0307] Step 5:

[0308] The server pre-processes the stored full-body photos and sends them to the image recognition module, which includes image normalization, resizing, and noise removal.

[0309] Step 6:

[0310] The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (jacket, pants, shoes, etc.), extracting features such as color, shape, and pattern for each identified item.

[0311] Step 7:

[0312] The server evaluates each fashion item based on the feature amount and analyzes the overall coordination balance.

[0313] Step 8:

[0314] The server calculates areas for improvement in the user's fashion based on the analyzed features and generates specific feedback.

[0315] Step 9:

[0316] The server uses an emotion engine to recognize emotions from the user's facial expressions and voice data, categorizing specific emotions such as smile, surprise, and relaxation.

[0317] Step 10:

[0318] The server then refers to the recognized emotion data and creates personalized feedback according to the user's emotions, for example, suggesting casual fashion items if the user is relaxed.

[0319] Step 11:

[0320] The server refers to the user's past photo database and analyzes the user's tastes and preferences, which clarifies past preference patterns.

[0321] Step 12:

[0322] The server searches and selects appropriate items from the database of affiliated shopping sites based on the user's preferences and emotional data.

[0323] Step 13:

[0324] The server then combines the selected suggested items with a full-body photograph of the user to generate specific coordination examples, allowing the user to visually confirm how the suggested items fit together.

[0325] Step 14:

[0326] The server transmits the generated feedback on the improvements, the coordination examples, and the links of the suggested items to the terminal.

[0327] Step 15:

[0328] The device displays the received feedback and outfit examples to the user, including links to purchase the suggested fashion items.

[0329] Step 16:

[0330] Users can check the displayed feedback and outfit examples, click on the link of the suggested item to access the shopping site, and purchase the suggested item directly if desired.

[0331] Specific examples

[0332] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. Also, suppose the user has a relaxed, smiling expression at the time of the photo. When this full-body photo and emotional information are uploaded to the system, the server receives and stores the data, and performs image and emotional recognition. The jacket, jeans, and sneakers are identified, their features are extracted, and the overall outfit is evaluated. A light blue shirt is then suggested as a casual item suitable for a relaxed state. This shirt is then superimposed onto the user's photo and sent to the user's device along with specific outfit examples. The user can then review the results and purchase the suggested shirt on a shopping site.

[0333] In this way, the system can achieve advanced personalization according to the user's emotions and make fashion suggestions.

[0334] Example 2

[0335] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0336] Conventional fashion suggestion systems have difficulty making suggestions that fully take into account the user's emotional state and preferences. This can lead to a decrease in user satisfaction and a decrease in their willingness to purchase the suggested items. Furthermore, due to a lack of functionality to effectively utilize the user's past data, it has been difficult to provide detailed fashion suggestions tailored to individual needs.

[0337] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0338] In this invention, the server includes: means for a user to upload a full-body photograph and emotional data; means for receiving the full-body photograph and emotional data; means for preprocessing the received full-body photograph; means for transmitting the preprocessed full-body photograph to an image recognition system and identifying each fashion item; means for extracting features of each identified fashion item; means for analyzing the emotional data to recognize the user's emotional state; means for generating improvements to the user's fashion based on the image recognition results and the emotional data; means for referencing past user photograph data and analyzing the user's preferences; means for selecting suggested items from a database of a shopping site affiliated with the server; means for generating outfit examples by combining the suggested items with the full-body photograph; means for transmitting the generated outfit examples and improvements to the user's terminal; means for the user to view the improvements and suggested items through the terminal; and means for the user to access the shopping site to purchase the suggested items. This makes it possible to provide personalized fashion suggestions that take the user's emotional state into consideration, thereby increasing satisfaction and encouraging purchases.

[0339] "User" refers to an individual who uses the Service and provides a full-body photo of themselves and emotional data.

[0340] A "full-body photo" is image data that captures the entire image of the user, and is used to make it easier to identify each fashion item.

[0341] "Emotion data" refers to information obtained from the user's facial expressions and voice, and is data for analyzing the user's psychological state.

[0342] "Dedicated app or website" refers to a software platform that allows users to upload full-body photos and emotional data.

[0343] "Server" refers to a computer system that processes and stores received data, performs various analyses, and generates composite images.

[0344] "Preprocessing" refers to the processes the server performs on the full-body photo, such as image normalization, resizing, and noise removal.

[0345] An "image recognition system" refers to technology that analyzes full-body photographs to identify fashion items and extract their features.

[0346] "Features" refers to data that indicates the characteristics of each fashion item, such as color, shape, and pattern.

[0347] An "emotion engine" refers to a system that analyzes collected emotional data and determines the user's psychological state.

[0348] "Fashion improvements" refers to modifications and advice for the user's fashion that the server suggests based on image recognition results and emotion data.

[0349] "Past user photo data" refers to full-body photo data previously uploaded by a user, and is used to analyze preference trends.

[0350] "Database of affiliated shopping sites" refers to the database of online shops that the server accesses to select suggested items.

[0351] "Suggested items" refer to fashion items selected by the server based on the user's preferences and areas for improvement.

[0352] "Composite image" refers to image data in which suggested items are combined with a full-body photograph to show specific coordination examples.

[0353] "Feedback" refers to specific advice or suggestions generated based on data provided by the user.

[0354] "Terminal" refers to the device that a user uses to view feedback and coordination examples from the server.

[0355] "Purchase Link" refers to the web link provided to purchase the suggested item.

[0356] An embodiment of this invention is a system that uses a user's full-body photo and emotional data to provide highly personalized fashion suggestions. This system mainly consists of a means for a user to upload a full-body photo and emotional data, a server to process and analyze the data, and an end device to display the results.

[0357] 1. User operations

[0358] First, the user takes a full-body photo using a smartphone or digital camera. Each fashion item (e.g., jacket, bottoms, shoes, etc.) should be visible in the photo. Emotional information, such as facial expressions and voice data, is also collected simultaneously. This emotional information can be recorded as a video or voice memo during shooting.

[0359] 2. Uploading images and emotion data

[0360] The user opens a dedicated application or website, where a screen for uploading a full-body photo and emotional information appears. The user uses this screen to send the full-body photo and emotional data to the server. This data is sent from the device to the server via an HTTP request.

[0361] 3. Receiving and Preprocessing Data by the Server

[0362] The server receives the full-body photo and emotion data sent by the user, and performs preprocessing such as normalization, resizing, and noise reduction on the received full-body photo. The preprocessed image data is then sent to the image recognition system.

[0363] 4. Analysis using an image recognition system

[0364] The image recognition system in the server analyzes the preprocessed full-body photograph and identifies each fashion item (tops, bottoms, shoes, etc.), extracting features such as color, shape, and pattern for each identified item.

[0365] 5. Emotional Data Analysis by Emotion Engine

[0366] The server uses an emotion engine to analyze the uploaded facial and voice data and identify the user's emotional state, such as relaxed, surprised, or happy, thereby understanding the user's psychological state.

[0367] 6. Generate fashion improvements and suggestions

[0368] The server evaluates the user's overall fashion balance based on the image recognition results and emotion data, and calculates areas for improvement. It also references the user's past fashion database to analyze past preferences and trends. Based on this information, it searches and selects the most suitable fashion items from the shopping site's database.

[0369] 7. Generating synthetic images and feedback

[0370] The proposed items are then combined with a full-body photo of the user to generate a composite image showing specific outfit examples, which allows the user to visually see how the outfit will look and is sent to the device along with feedback.

[0371] 8. User Notification and Item Purchase

[0372] The device displays the received feedback and outfit examples to the user, including a link to purchase the suggested items. The user can click this link to access a shopping site and purchase the suggested items if desired.

[0373] Specific examples

[0374] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. If the user's emotional data indicates that they are in a relaxed state, the server receives and analyzes the photo and emotional data. The server identifies the jacket, jeans, and sneakers and extracts their features. Next, the server evaluates the overall balance of the outfit and calculates an improvement point, such as "matching it with a lighter-colored shirt." The server then suggests casual fashion items (e.g., a light blue shirt) that are appropriate for a relaxed state based on the emotional data. The system then composites the shirt into the full-body photo to generate a specific outfit example. A link to the composite image and the suggested items is sent to the user's device, where the user can view and purchase the items.

[0375] Prompt Sentence Examples

[0376] Example prompts to be input to the generative AI model

[0377] Here's a full-body photo of me. I'm wearing a blue jacket, black jeans, and white sneakers. Can you suggest a casual outfit that shows me in a relaxed state? Please also tell me the specific items and why.

[0378] In this way, the system can provide highly personalized fashion suggestions that correspond to the user's emotions, thereby increasing user satisfaction.

[0379] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0380] Step 1:

[0381] A user takes a full-body photo of themselves using a smartphone or digital camera. The input is the user's full-body photo, and the output is a digital image file. The user also records facial expressions and audio data as videos or audio memos to collect emotional information. The input is the user's emotional data, and the output is a digital video or audio file.

[0382] Step 2:

[0383] The user opens a dedicated application or website and uploads the full-body photo they have taken and the collected emotion data. The input is the user's full-body photo and emotion data, and the output is a dataset for transmission. When the user clicks the upload button, the data is sent to the server.

[0384] Step 3:

[0385] The device sends a full-body photo and emotion data to the server via an HTTP request. The input is the user's full-body photo and emotion data, and the output is an HTTP request to the server.

[0386] Step 4:

[0387] The server receives HTTP requests sent from the device and acquires full-body photos and emotion data. The input is the HTTP request from the device, and the output is a dataset on the server. These data are stored in a database.

[0388] Step 5:

[0389] The server performs preprocessing on the received full-body photo. Preprocessing includes image normalization, resizing, and noise removal. The input is the full-body photo, and the output is the preprocessed image data. Specifically, the image is resized to a specific resolution and filtered to remove noise.

[0390] Step 6:

[0391] The server sends the preprocessed full-body photo to the image recognition system. The input is the preprocessed image data, and the output is the classification results. The image recognition system analyzes the full-body photo and identifies each fashion item (top, bottom, shoes, etc.). It extracts features such as color, shape, and pattern for each identified item.

[0392] Step 7:

[0393] The server uses an emotion engine to analyze the uploaded facial and voice data. The input is emotion data, and the output is the identified emotional state. The server recognizes emotions such as relaxed, surprised, happy, etc.

[0394] Step 8:

[0395] The server generates suggestions for improving the user's fashion based on the image recognition results and emotion data. The inputs are the image recognition results and emotion data, and the output is specific suggestions for improving the fashion. For example, it generates feedback such as "It would be better to wear a brighter colored shirt."

[0396] Step 9:

[0397] The server analyzes the user's preferences by referencing past user photo data. The input is the user's past fashion data, and the output is their taste trends. Based on this, the server suggests items that match the user's past preferences and suit their current emotional state.

[0398] Step 10:

[0399] The server selects suggested items from the database of affiliated shopping sites. The input is the user's preferences and fashion improvements, and the output is suggested items. Suggested items are selected by searching the database of the shopping site.

[0400] Step 11:

[0401] The server composites the selected suggested items onto a full-body photo to generate specific coordination examples. The input is a full-body photo and the suggested items, and the output is a composite image. By generating a composite image, users can visually confirm how the suggested items match their own style.

[0402] Step 12:

[0403] The server sends the generated feedback and coordination examples of the improvements to the terminal. The input is the feedback and coordination examples, and the output is data transmission to the terminal. The terminal displays the received feedback and coordination examples to the user.

[0404] Step 13:

[0405] The user checks the displayed feedback and outfit examples. The input is the feedback and outfit examples sent from the server, and the output is the user's response. The user can click on the link of the suggested item to access the shopping site and purchase the suggested item directly if desired.

[0406] (Application example 2)

[0407] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0408] Conventional fashion recommendation systems only make suggestions based on the user's static photos and past data, making it difficult to provide personalized suggestions that reflect real-time emotions and situations. Furthermore, there was a lack of a way to suggest improvements in real time while trying on clothes in a physical store. This resulted in the problem of users being unable to select the perfect fashion item in a physical store.

[0409] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting facial expression and voice data of the user and recognizing emotions, means for generating feedback based on the recognized emotion data, and means for collecting full-body photos and emotion data in real time in the store and making fashion suggestions based on them. This enables the user to receive appropriate fashion suggestions on the spot while trying on clothes in the physical store.

[0410] A "full-body photo" is a still image showing the user's entire body, and is data used to analyze fashion item coordination and areas for improvement.

[0411] "Emotional data" is information that indicates the user's emotional state, obtained from facial expressions and voice. Personalized fashion suggestions are based on this information.

[0412] "Image analysis" involves analyzing uploaded full-body photos and identifying each fashion item in the image, thereby extracting the features of specific clothing and accessories.

[0413] "Feedback" refers to suggestions and advice generated for users based on the results of image analysis and emotional data, including suggestions for fashion improvements and suggested items.

[0414] "Suggested items" are fashion items selected by the system based on the user's preferences and emotional data, including simulated images of how the items will look when worn.

[0415] "Smart glasses" are wearable devices that incorporate sensors such as cameras and microphones and can process the user's visual and audio information in real time, and are used for data collection and analysis.

[0416] "Real-time" means that data is processed and analyzed immediately at the moment it is collected, and the results are immediately provided to the user.

[0417] "Personalized" refers to suggestions and advice that are customized based on the preferences and emotional state of a specific user. Fashion suggestions are optimized for each individual user.

[0418] "Coordination" refers to combining suggested items with a full-body photo of the user to provide a specific styling example, allowing the user to visually confirm the result.

[0419] A "shopping site" is a website where fashion items can be purchased online. In this invention, purchase links for suggested items are provided.

[0420] The "server" is a computing system that receives, stores, analyzes, and processes data sent by users, and generates suggestions and feedback based on the analysis results.

[0421] The system of the present invention includes the following components: A user takes a full-body photo and uploads the photo, along with facial expression and voice data, to a server. The server analyzes the received full-body photo and emotional data, generates fashion improvements for the user, and selects suggested items. Furthermore, data collection and suggestions can be made in real time in a physical store using smart glasses.

[0422] Overall system configuration

[0423] 1. User device: Using a device such as a smartphone, tablet, or smart glasses, a full-body photo and emotional data are taken and sent to the server.

[0424] 2. Server: Responsible for data analysis and performs the following processes:

[0425] Image analysis: Identify fashion items and extract features from uploaded full-body photos.

[0426] Emotion recognition: Recognizes the user's emotions using facial and voice data.

[0427] Proposal generation: Based on the results of image analysis and emotion recognition, suggestions for improvements and items for the user's fashion are generated.

[0428] 3. Hardware and Software:

[0429] Smart glasses (e.g., general-purpose wearable devices)

[0430] Camera and microphone (built-in or external to smart glasses)

[0431] OpenCV (image processing library)

[0432] DeepFace (emotion recognition library)

[0433] An HTTP server (you can use Flask or Django to receive and process data)

[0434] Example of a system

[0435] The user puts on the smart glasses and takes a full-body photo while trying them on in a physical store. The camera built into the smart glasses captures the photo, and the microphone collects the user's voice. This data is immediately sent to the server. The server uses image analysis to identify the items the user is wearing, extracts their features, and evaluates the user's satisfaction and interest through emotion analysis.

[0436] Based on the extracted data, real-time feedback is generated and displayed on the user's smart glasses. For example, if the user is wearing a blue jacket that doesn't match their relaxed facial expression, the server will suggest a light blue shirt as an alternative and provide examples of how to coordinate it. The user can try on the item and receive another evaluation along with their emotional data. This allows the user to easily choose the best item.

[0437] Prompt Sentence Examples

[0438] "Your goal is to develop a system that allows users to take a full-body photo in a physical store using a general-purpose wearable device and receive personalized fashion recommendations by collecting real-time emotional data on the spot. Implement a function that enhances the process of users selecting the most suitable fashion items based on their emotions."

[0439] This invention allows users to receive fashion suggestions in real time even in physical stores, enabling them to select the most suitable items on the spot. Furthermore, by utilizing emotion data, the accuracy of suggestions can be improved, leading to increased user satisfaction.

[0440] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0441] Step 1: The user wears the smart glasses and takes a full-body photo. The image data captured by the camera built into the smart glasses is input. The output is full-body photo data.

[0442] Step 2: Collect the user's facial expression and voice data simultaneously. Facial expression and voice information are input via the microphone and camera built into the smart glasses. Emotion data is obtained as the output.

[0443] Step 3: The device sends the full-body photo data and emotion data to the server. The input is the full-body photo and emotion data, and the output is sent to the server via an HTTP request.

[0444] Step 4: The server performs image analysis on the received full-body photo. The input is the full-body photo data, and the output is the identification results and feature values ​​for each fashion item (tops, bottoms, shoes, etc.). Specifically, OpenCV is used to remove noise and resize the image, and then DeepFace is used to identify the items.

[0445] Step 5: The server analyzes the emotion data. The input is facial expressions and voice data, and the output is the user's emotional state (e.g., relaxed, happy, anxious, etc.). Specifically, emotion recognition is performed using the DeepFace library.

[0446] Step 6: The server generates feedback based on the image analysis and emotion recognition results. The input is the feature values ​​and emotion data of the fashion items, and the output is suggestions for improvements to the fashion and items. Specifically, the server references the user's past fashion database and generates a list of candidate items to suggest.

[0447] Step 7: The server synthesizes the suggested items onto a full-body photo to generate an example outfit. The input is the suggested items and a full-body photo, and the output is an image in which the suggested items are synthesized onto the user's full-body photo. Specifically, the server uses an image synthesis algorithm to create a visually easy-to-understand outfit image.

[0448] Step 8: The server sends the generated coordination examples and improvements to the user's device. The input is the coordination examples and improvements, and the output is displayed on the user's smart glasses.

[0449] Step 9: The user checks the displayed feedback and coordination examples. The input is the data displayed on the smart glasses, and the output is the user's perceived feedback.

[0450] Step 10: The user accesses the shopping site to purchase the suggested item that they are satisfied with. The input is the purchase link for the suggested item, and the output is access to the shopping site and the purchase procedure.

[0451] This series of steps allows users to receive optimal fashion suggestions in real time while trying on clothes in a physical store, and the feedback gained from this entire process can significantly improve user satisfaction.

[0452] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0453] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0454] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0455] [Second embodiment]

[0456] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0457] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0458] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0459] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0460] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0461] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0462] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0463] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0464] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0465] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0466] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0467] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0468] This invention relates to a system that suggests fashion improvements and additional items suitable for a user by having the user upload a selfie photo. The system includes the following specific procedures and functions.

[0469] User operations

[0470] First, users take a full-body selfie using a smartphone or digital camera, and then they can easily upload this full-body photo through a dedicated app or website.

[0471] Image reception and analysis

[0472] 1. The user takes a full-body photo and submits it using the app or website's upload function.

[0473] 2. The device sends the uploaded full-body photo to the server.

[0474] 3. The server preprocesses the received full-body photo and passes it to the image recognition module.

[0475] 4. The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (tops, bottoms, shoes, etc.), extracting features such as color, shape, and pattern of each fashion item.

[0476] Suggested improvements

[0477] 5. The server evaluates the overall fashion balance based on the extracted features and generates specific improvements, such as providing specific feedback such as "the color balance between the top and bottom is lacking."

[0478] 6. The server references the user's past photo database, learns the user's preferences and tendencies, and generates more personalized advice.

[0479] Item suggestions and synthesis

[0480] 7. The server selects appropriate fashion items from the database of affiliated shopping sites based on the user's preferences and fashion improvements.

[0481] 8. The server then synthesizes the suggested items onto the user's full-body photo to generate a specific coordination example. This composite image allows the user to visually confirm how the suggested items match their own style.

[0482] User Notification

[0483] 9. The server sends the generated feedback of improvements and coordination examples to the terminal.

[0484] 10. The device displays feedback and outfit examples to the user, and also provides links to the best items, allowing the user to go directly to a shopping site and purchase the suggested items.

[0485] Specific examples

[0486] For example, a user may take a full-body photo of themselves wearing a blue jacket, black jeans, and white sneakers. When the user uploads this photo to the system, the server first receives the photo and uses an image recognition module to identify the jacket, jeans, and sneakers. The server then evaluates the overall balance of the outfit and generates feedback such as, "The combination of the jacket and jeans is dark, so it would be good to pair it with a light-colored shirt."

[0487] The server then references the user's past outfit history and determines that the user has previously preferred white and gray shirts. Based on this, the server selects a light blue shirt from a partner shopping site and superimposes it onto the user's full-body photo. A link to the suggested item's details is then sent to the user's device, allowing the user to view and purchase the item directly if necessary.

[0488] In this way, the system can highly personalize fashion suggestions tailored to each user's individual style and preferences, and encourage users' purchasing behavior by providing visual coordination examples.

[0489] The processing flow will be explained below.

[0490] Step 1:

[0491] The user takes a photo of themselves using a smartphone or digital camera.

[0492] Step 2:

[0493] Users open a dedicated app or website and use the interface to upload a full-body photo they have taken.

[0494] Step 3:

[0495] The device generates an HTTP request to send the uploaded photo file to the server.

[0496] Step 4:

[0497] The server receives the HTTP request, obtains the full-body photo file, and saves it.

[0498] Step 5:

[0499] The server performs pre-processing on the received full-body photos, including image normalization, resizing, and noise removal.

[0500] Step 6:

[0501] The server sends the preprocessed images to an image recognition module.

[0502] Step 7:

[0503] The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (e.g., jacket, pants, shoes, etc.). For each identified item, it extracts features such as color, shape, and pattern.

[0504] Step 8:

[0505] The server evaluates each item individually based on the extracted feature amounts, and evaluates the overall coordination balance.

[0506] Step 9:

[0507] The server calculates improvements to the fashion based on the evaluation results and generates specific feedback to the user, such as "The color combination of your jacket and jeans is too dark, so it would be better to pair it with a lighter colored shirt."

[0508] Step 10:

[0509] The server refers to a database of past photos of the user and analyzes their past fashion trends and preferences, allowing it to make suggestions that are faithful to the user's preferences.

[0510] Step 11:

[0511] The server searches the database of affiliated shopping sites for items that match the user's preferences and correspond to fashion improvements.

[0512] Step 12:

[0513] The server then combines the selected suggested items with a full-body photograph of the user to generate a specific coordination example, allowing the user to visually confirm how the suggested items will suit their style.

[0514] Step 13:

[0515] The server transmits the generated feedback of the improvements and the coordinate examples to the user's terminal.

[0516] Step 14:

[0517] The device displays the received feedback and outfit suggestions to the user, including a link to the newly suggested item.

[0518] Step 15:

[0519] Users can review the displayed feedback and outfit examples, click on the link for the suggested item to access the shopping site, and, if desired, purchase the suggested item directly.

[0520] Example 1

[0521] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0522] Today's consumers need personalized fashion advice and suggestions, but providing them requires specialized knowledge and is time-consuming. Current systems lack the functionality to provide specific feedback or suggest new fashion items to help users review and improve their outfits. This makes it difficult for users to find the perfect fashion items to match their preferences and style.

[0523] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0524] In this invention, the server includes: means for a user to upload a full-body photograph; means for receiving the uploaded full-body photograph; means for preprocessing the received full-body photograph; means for image analysis of the preprocessed full-body photograph; means for an image recognition module to identify each fashion item from the full-body photograph; means for extracting features of the identified fashion items by the image recognition module; means for evaluating the user's fashion balance based on the extracted features and generating improvements; means for the server to refer to past user photograph data, analyze the user's preferences, and generate personalized advice; means for selecting suggested items from a database of a shopping site affiliated with the server; means for the server to synthesize the suggested items with the full-body photograph to generate a coordination example; means for transmitting the synthesized coordination example and improvements to the user's terminal; means for the user to view the improvements and suggested items via the terminal; and means for the user to access the shopping site to purchase the suggested items. This enables users to easily find fashion items that suit their style.

[0525] A "full-body photo" is an image taken by a user that includes the entire body from head to toe.

[0526] A "server" is a computer system that receives, processes, and transmits data.

[0527] "Uploading means" refers to a method or function that allows a user to send a full-body photo to a server via the Internet.

[0528] The "receiving means" refers to a method or function by which the server receives the full-body photo sent by the user.

[0529] The "preprocessing means" is a method or function for removing noise from the received full-body photograph and adjusting the size.

[0530] "Image analysis means" refers to a method or function for analyzing a full-body photograph to recognize individual fashion items.

[0531] An "image recognition module" is software or hardware for identifying each fashion item in a full-body photograph and extracting their features.

[0532] The "extracting means" is a method or function for extracting characteristics such as color, shape, pattern, etc. from the identified fashion items.

[0533] The "means for evaluating fashion balance" is a method or function for evaluating the quality of the overall coordination based on the extracted feature amounts.

[0534] The "means for generating improvements" is a method or function for generating specific suggestions for improving coordination based on the evaluation results of fashion balance.

[0535] The "means for analyzing preferences" is a method or function for analyzing past user photo data to learn user preferences and tendencies.

[0536] The "means for selecting suggested items" is a method or function for selecting appropriate fashion items from the database of the affiliated shopping site.

[0537] The "means for generating coordination examples by combining" is a method or function for combining selected suggested items with a full-body photograph to generate specific coordination examples.

[0538] The "means for sending feedback and coordination examples" is a method or function for sending the generated improvements and coordination examples to the user's terminal.

[0539] The "viewing means" is a method or function for a user to use a terminal to view improvements and suggested items.

[0540] "Means for accessing a shopping site" refers to the method or function by which a user accesses a shopping site on the Internet to purchase the suggested items.

[0541] This invention relates to a system that suggests fashion improvements and additional items that suit the user when the user uploads a full-body selfie photo. This system is specifically implemented using a "full-body photo," a "server," "uploading means," "receiving means," "preprocessing means," "image analysis means," an "image recognition module," "extraction means," "means for evaluating fashion balance," "means for generating improvements," "means for analyzing preferences," "means for selecting suggested items," "means for synthesizing and generating coordination examples," "means for sending feedback and coordination examples," "means for viewing," and "means for accessing a shopping site."

[0542] Hardware and Software Usage:

[0543] First, the user takes a full-body selfie using a smartphone or digital camera. Then, the user accesses a dedicated app or website and uploads the full-body photo. Dedicated apps and websites are often built using standard web technologies (HTML, CSS, JavaScript, etc.).

[0544] The user's device sends the uploaded full-body photo to a server. The server is a high-performance computing system that receives the uploaded full-body photo and performs preprocessing on the image. This preprocessing involves noise reduction and resizing using image processing libraries such as OpenCV. The preprocessed full-body photo is then passed to an image recognition module. This module uses deep learning libraries such as TensorFlow to identify each fashion item (top, bottom, shoes, etc.) in the photo and extract features such as color, shape, and pattern.

[0545] The server evaluates the overall balance of the fashion based on the extracted features and generates specific improvements. This evaluation uses a machine learning model, which can provide specific feedback such as "the color balance between the top and bottom is lacking." The server also references a database of the user's past photos to learn the user's preferences and tendencies. This learning is done using a machine learning algorithm based on the user's past history data.

[0546] The server selects appropriate fashion items from the database of affiliated shopping sites based on the user's preferences and fashion improvements. This selection process accesses the shopping site's database via an API. The suggested items are then combined with the user's full-body photo to generate specific coordination examples. This combination is achieved using image processing tools such as Python's PIL (Pillow) library.

[0547] The generated coordination examples and improvements are sent from the server to the user's device, where they can be viewed through a dedicated app or website. Users can access the detailed link for the suggested items through feedback and purchase the suggested items directly on the shopping site.

[0548] Examples:

[0549] For example, consider the case where a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers and uploads it to a dedicated app. The server receives this photo, performs preprocessing, and then uses an image recognition module to identify each item. During this process, features such as the blue of the jacket, the black of the jeans, and the white of the sneakers are extracted. The server then evaluates the balance of the fashion and generates feedback such as, "The combination of the jacket and jeans is dark, so it would be good to pair it with a light-colored shirt."

[0550] The server also analyzes the user's past photos, learns that the user prefers white or gray shirts, and selects suggested items (e.g., light blue shirts) based on that. It then composites the selected shirt onto a full-body photo to generate a composite image. Finally, the server sends the generated feedback and composite image to the user's device, where the user can review it and directly access the shopping site using the purchase link for the suggested item.

[0551] Example prompt for a generative AI model:

[0552] "If a user takes a full-body photo that shows a blue jacket, black jeans, and white sneakers, the server analyzes the photo, suggests ways to improve the user's fashion, and selects appropriate items (e.g., a light blue shirt) from a partner shopping site. The server then combines the shirt with the full-body photo and presents it to the user as a specific coordination example."

[0553] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0554] Step 1:

[0555] Users take a full-body selfie using a smartphone or digital camera.

[0556] Specific Actions: The user sets the camera so that their entire body is visible and takes a photo of themselves wearing a blue jacket, black jeans, and white sneakers.

[0557] Input: User's full-body photo (image file)

[0558] Output: Full-body photo taken

[0559] Step 2:

[0560] Users access a dedicated app or website and upload a full-body photo.

[0561] Specific behavior: The user opens the app, clicks the "Upload Photo" button, selects a full-body photo they have taken, and submits it.

[0562] Input: Taken full-body photo (image file), user operation

[0563] Output: Uploaded full-body photo

[0564] Step 3:

[0565] The device then sends the uploaded full-body photo to the server.

[0566] Specific operation: The device sends the photo data to the specified endpoint on the server via an Internet connection.

[0567] Input: Uploaded full-body photo (image file)

[0568] Output: Full body photo sent to the server

[0569] Step 4:

[0570] The server pre-processes the received full-body photos.

[0571] Specific operation: OpenCV is used on the server to perform preprocessing such as image noise removal, size adjustment, and color correction.

[0572] Input: Full-body photo (image file) sent to the server

[0573] Output: Preprocessed full-body photo

[0574] Step 5:

[0575] The image recognition module in the server analyzes the pre-processed full-body photos and identifies each fashion item.

[0576] What it does: Uses TensorFlow to identify jackets, jeans, and sneakers in a full-body photo.

[0577] Input: Preprocessed full-body photo

[0578] Output: Identified fashion items (jacket, jeans, sneakers)

[0579] Step 6:

[0580] Based on the identification results, the server extracts features such as color, shape, and pattern of each item.

[0581] Specific operation: For each identified item, calculate the RGB color values, shape parameters, and pattern features.

[0582] Input: Identified fashion item data

[0583] Output: Extracted features (RGB color values, shape, pattern)

[0584] Step 7:

[0585] Based on the extracted features, the server evaluates the fashion balance and generates specific improvements.

[0586] Specific actions: Using a machine learning model, the extracted features are evaluated and specific improvements are suggested, such as "the overall color balance is dark, so add a bright color to the upper body."

[0587] Input: extracted features

[0588] Output: Fashion balance evaluation results and improvements

[0589] Step 8:

[0590] The server references the user's past photo database and learns the user's preferences and tendencies.

[0591] How it works: Past photo data is passed through a machine learning algorithm to identify the colors and styles the user prefers.

[0592] Input: User's past photo data

[0593] Output: Data about user preferences and tendencies

[0594] Step 9:

[0595] The server generates personalized advice that reflects the user's preferences and tendencies.

[0596] Specific operation: The fashion balance evaluation results are combined with the user's preference data to generate personalized advice such as "A light blue shirt is good."

[0597] Input: Fashion balance evaluation results, data on user preferences and trends

[0598] Output: Personalized advice

[0599] Step 10:

[0600] The server combines the suggested items with a full-body photo to generate specific coordination examples.

[0601] Specific behavior: Using Python's PIL (Pillow) library, a light blue shirt is superimposed onto a full-body photo of the user.

[0602] Input: personalized advice, full-body photo of user

[0603] Output: Composite coordinated image

[0604] Step 11:

[0605] The synthesized coordination examples and improvements are sent from the server to the user's device.

[0606] Specific operation: Data including coordination examples and improvements is sent from the server to the user's device.

[0607] Input: Synthesized coordinate image, improvement points

[0608] Output: Coordinate image sent to the user's device and improvements

[0609] Step 12:

[0610] The user's device will display feedback, outfit examples, and provide links to shopping sites.

[0611] How it works: A dedicated app or website provides feedback and visual examples of outfits, along with links to purchase suggested items.

[0612] Input: Coordinate image sent from the server and improvements

[0613] Output: A user-visible feedback screen and a purchase link

[0614] Step 13:

[0615] Users can check the suggested items through their device and access the shopping site if necessary.

[0616] What happens: The user reviews the details of the suggested item in the app and clicks the purchase link to visit the shopping site.

[0617] Input: Feedback screen and purchase link

[0618] Output: Visit to shopping site and purchase item

[0619] (Application example 1)

[0620] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0621] Conventional fashion suggestion systems lack real-time functionality when users receive advice and item suggestions to improve their outfits, making them particularly inconvenient in brick-and-mortar stores. This requires users to take photos every time they try on clothes, upload them, and wait for the results, which takes time and effort. Furthermore, it is difficult to receive personalized suggestions based on users' preferences and trends in real time, making it difficult to directly link these to purchasing behavior.

[0622] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0623] In this invention, the server includes: means for a user to upload a full-body photo; means for receiving the uploaded full-body photo; means for image-analyzing the received full-body photo; means for generating improvements to the user's fashion based on the image analysis results; means for referencing past user photo data and analyzing the user's preferences; means for selecting suggested items from a database of a purchasing platform affiliated with the server; means for the server to generate outfit examples by combining the suggested items with the full-body photo; means for the server to transmit the generated outfit examples and improvements to the user's terminal; means for the user to view the improvements and suggested items through the terminal; means for the user to access the purchasing platform to purchase the suggested items; means for the user to take and upload a full-body photo in real time in a store using smart glasses or a head-mounted display; and means for the server to generate fashion analysis results and suggestions in real time and present them to the user. This enables users to receive fashion evaluations and improvement suggestions in real time even in physical stores, providing an efficient shopping experience.

[0624] "User" refers to a person who uses the system to improve their fashion or receive suggestions.

[0625] A "full-body photo" refers to a photo that includes the user's entire body from head to toe.

[0626] "Means for uploading" refers to the functionality that a user uses to send a full-body photo of themselves to the system.

[0627] "Server" refers to a computer system that receives, analyzes, and generates recommendations from user-submitted data.

[0628] "Image analysis" refers to the process in which the server receives a full-body photo and identifies fashion items and extracts features.

[0629] "Image analysis results" refers to the identification information and characteristics of a fashion item obtained through image analysis.

[0630] "Fashion improvements" refers to feedback that suggests a better fashion style for the user's current outfit.

[0631] "User preferences" refers to the user's fashion preferences and tendencies analyzed based on past databases.

[0632] "Buying Platform" refers to an online or offline sales system through which the proposed fashion items are available for purchase.

[0633] "Suggested items" refer to fashion items selected by the server based on the user's preferences and fashion improvements.

[0634] "Synthesis" refers to the process of displaying suggested items superimposed on a full-body photo of the user.

[0635] "Coordination example" refers to an example of a fashion style visually shown by combining suggested items with a full-body photograph of the user.

[0636] "Smart glasses" refers to an augmented reality (AR)-enabled eyeglass-type device that users can wear to take full-body photos and upload them to the system.

[0637] "Head-mounted display" refers to a head-mounted display device that a user wears and can take full-body photos and upload them to the system.

[0638] "Real-time" refers to the short time between when a user takes a full-body photo and when they receive feedback or suggestions from the server, and the time is immediate.

[0639] "Means for generating fashion analysis results and suggestions in real time and presenting them to users" refers to the function of instantly analyzing a full-body photo uploaded by a user to the system and displaying the results and suggestions to the user.

[0640] As an embodiment of the present invention, a series of steps in which a user takes a full-body photo and uploads it to a server in order to improve his or her fashion will be specifically described.

[0641] System Program

[0642] The program consists of the following major steps:

[0643] 1. Take and upload a full-body photo of the user:

[0644] Users take a full-body photo of themselves using a smartphone, smart glasses, or a head-mounted display.

[0645] The full-body photo taken is uploaded to the server via the application.

[0646] 2. Image reception and analysis by the server:

[0647] The server receives the uploaded full-body photograph.

[0648] The received images are preprocessed and passed to an image recognition model (a model using Keras / TensorFlow) for analysis.

[0649] Fashion items are identified and features are extracted using image analysis.

[0650] 3. Fashion Improvements and Item Suggestion Generation:

[0651] The server evaluates the fashion balance based on the analyzed features and generates improvements.

[0652] It learns user preferences by referencing past user data and selects appropriate suggested items from the database of affiliated purchasing platforms.

[0653] 4. Coordination example generation and synthesis:

[0654] The proposed items are combined with the user's full-body photo to generate visual coordination examples.

[0655] Coordination examples and improvements are sent to the user's device.

[0656] 5. User Notice and Shopping Support:

[0657] Users can check areas for improvement and suggested items through their devices.

[0658] Through the link to the suggested item, the user can access the purchasing platform and make a direct purchase.

[0659] Natural language explanation of the process

[0660] Hardware and software used:

[0661] Smartphones, smart glasses, head-mounted displays: devices that allow users to take full-body photos and upload them to a server through an application.

[0662] Keras / TensorFlow: Libraries for running image recognition models on the server.

[0663] OpenCV: A library for image preprocessing and analysis.

[0664] Server: A computer system that receives data, analyzes it, generates proposals, and notifies users.

[0665] Data processing and calculation:

[0666] Image preprocessing: The server resizes the images uploaded by the user and converts them into a format suitable for input to the image recognition model.

[0667] Image analysis: Using an image recognition model, clothing items are identified and their features are extracted.

[0668] Fashion balance evaluation: Based on the extracted features, the balance of the entire outfit is analyzed and improvements are identified.

[0669] Selection of suggested items: By referring to the user's past preference data, appropriate items are selected from the purchasing platform's database.

[0670] Coordination example generation: The proposed items are combined with a full-body photo to generate visually easy-to-understand coordination examples.

[0671] Notifications and shopping support: Send improvements and suggested items to your device and provide direct purchase links.

[0672] Specific examples

[0673] For example, if a user uploads a full-body photo wearing a blue jacket, black jeans, and white sneakers, the server analyzes it and provides feedback such as, "To improve the color balance, pair it with a light-colored shirt." If the user's past data indicates that they prefer light-colored shirts, the server will suggest a light blue shirt. The shirt is then superimposed on the full-body photo, and the result is immediately presented to the user. The user can then purchase the shirt directly on the purchasing platform via the provided link.

[0674] Prompt Sentence Examples

[0675] "Analyze a user's full-body photo, identify fashion items, evaluate their proportions, and generate suggestions for improvements and additional items."

[0676] As described above, this system allows users to receive real-time improvements and suggestions even in physical stores, allowing them to enjoy an efficient and personalized shopping experience.

[0677] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0678] Step 1:

[0679] The user takes a full-body photo and uploads it to the server through the application. The input is a full-body photo taken by the user with a smartphone, smart glasses, or head-mounted display, and the output is an image file sent to the server. The user takes a full-body photo using their device and sends it to the server using the application's upload function.

[0680] Step 2:

[0681] The server receives the uploaded full-body photo. The input is the image file sent by the user, and the output is the image data stored in the server's database. The server stores the received full-body photo in the database.

[0682] Step 3:

[0683] The server preprocesses the received full-body photo and inputs it into the image recognition model. The input is the image data of the saved full-body photo, and the output is image data converted into a format suitable for the model. The server uses OpenCV to perform preprocessing such as image resizing and noise removal, and converts it into a format suitable for input into the image recognition model.

[0684] Step 4:

[0685] The server inputs the preprocessed image data into an image recognition model (Keras / TensorFlow) to identify fashion items and extract their features. The input is the preprocessed image data, and the output is the identified fashion items and their features (color, shape, pattern, etc.). The server uses a generative AI model to perform image recognition, identify each fashion item, and extract its features.

[0686] Step 5:

[0687] The server evaluates the fashion balance based on the extracted features and generates improvements to the user's fashion. The input is the identified fashion items and their features, and the output is a fashion balance evaluation and feedback on improvements. The server analyzes the features, evaluates the overall coordination balance, and calculates specific improvements.

[0688] Step 6:

[0689] The server references the user's past database and learns the user's tastes and preferences. The input is the user's past fashion data, and the output is personalized information based on the user's preferences. The server uses past photo data for machine learning to analyze and learn the user's tastes and trends.

[0690] Step 7:

[0691] The server selects suggested items from the database of the affiliated purchasing platform. The input is the fashion balance rating, the user's preferences, and the purchasing platform's item database, and the output is a list of suggested items. The server selects appropriate items from the purchasing platform based on the user's preferences and fashion improvement points.

[0692] Step 8:

[0693] The server composites the suggested items onto a full-body photo to generate a coordination example. The input is the suggested items and the user's full-body photo, and the output is an image of the composite coordination example. The server composites the suggested items onto a full-body photo to generate a coordination example that can be visually confirmed.

[0694] Step 9:

[0695] The server sends the generated coordination examples and improvements to the user's device. The input is an image of the coordination example and feedback on improvements, and the output is the information displayed on the user's device. The server sends the generated coordination examples and improvements to the user's device and displays them.

[0696] Step 10:

[0697] The user views the improvements and suggested items through the terminal. The input is information sent from the server (coordination examples, improvements, suggested items), and the output is visual information that the user checks. The user uses the terminal to check the sent coordination examples, improvements, and suggested items.

[0698] Step 11:

[0699] The user accesses the purchasing platform to purchase the suggested item. The input is a link to the purchasing platform displayed on the terminal, and the output is the purchase of the suggested item. The user clicks the link through the terminal to access the purchasing platform and purchase the item.

[0700] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0701] This invention relates to a system that uses a full-body photograph of a user to suggest improvements to fashion and also recognizes the user's emotions to suggest personalized items. The following describes the procedures and functions for specifically implementing this system.

[0702] User operations

[0703] First, the user takes a full-body photo of themselves using a smartphone or digital camera. Emotional information, such as facial expressions and voice data, is also collected. This emotional information is recorded in the form of video or voice memos during shooting.

[0704] Receiving and analyzing image and emotion data

[0705] 1. The user opens a dedicated app or website and uploads a full-body photo and emotional information.

[0706] 2. The device sends the uploaded photo file and emotion information to the server via an HTTP request.

[0707] 3. The server receives the HTTP request, obtains the full-body photo and emotion information, and saves them.

[0708] 4. The server preprocesses the full-body photo and passes it to the image recognition module. Preprocessing includes normalizing, resizing, and removing noise from the image.

[0709] 5. The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (top, bottoms, shoes, etc.). It extracts features such as color, shape, and pattern for each identified item.

[0710] Emotional Data Analysis

[0711] 6. The server uses an emotion engine to recognize the user's emotions from the uploaded facial and voice data, for example, analyzing whether the user is smiling, surprised, relaxed, etc.

[0712] 7. The server generates feedback based on the recognized emotion data according to the user's emotions. For example, if the user is relaxed, it may suggest casual fashion.

[0713] Improvements and Item Suggestions

[0714] 8. The server evaluates the overall fashion balance based on the extracted features and emotion data, and calculates areas for improvement. For example, it provides specific feedback such as, "The jacket color is too dark, so it would be better to pair it with a lighter-colored shirt."

[0715] 9. The server refers to the user's past fashion database and analyzes the user's preferences and trends. Based on this, it suggests items that match the user's past preferences and are based on emotional data.

[0716] 10. The server searches and selects items that correspond to the user's preferences and fashion improvements from the database of affiliated shopping sites.

[0717] Combining items and notifying the user

[0718] 11. The server then synthesizes the selected suggested items onto the user's full-body photo to generate a specific coordination example. The synthesized image allows the user to visually confirm how the suggested items match their own style.

[0719] 12. The server sends the generated feedback of improvements and coordinate examples to the terminal.

[0720] 13. The device displays the received feedback and outfit examples to the user, including links to purchase the suggested items.

[0721] 14. Users can review the displayed feedback and outfit examples, click on the link for the suggested item to access the shopping site, and, if desired, purchase the suggested item directly.

[0722] Specific examples

[0723] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. If the user's emotional data indicates that they are in a relaxed state, the server receives and analyzes the photo and emotional data. The server identifies the jacket, jeans, and sneakers and extracts their features. Next, the server evaluates the overall balance of the outfit and calculates an improvement point, such as "matching it with a lighter-colored shirt." The server then suggests casual fashion items (e.g., a light blue shirt) that are appropriate for a relaxed state based on the emotional data. The system then composites the shirt into the full-body photo to generate a specific outfit example. A link to the composite image and the suggested items is sent to the user's device, where the user can view and purchase the items.

[0724] In this way, the system can provide highly personalized fashion suggestions that correspond to the user's emotions, thereby increasing user satisfaction.

[0725] The processing flow will be explained below.

[0726] Step 1:

[0727] Users take full-body photographs of themselves using a smartphone or digital camera, and emotional information such as facial expressions and voice data is also collected.

[0728] Step 2:

[0729] The user opens a dedicated app or website and navigates through a screen to upload a full-body photo and emotional information.

[0730] Step 3:

[0731] The device sends the uploaded full-body photo and emotion information to the server, packaging the photo data and emotion data as an HTTP request.

[0732] Step 4:

[0733] The server receives the HTTP request and stores the full-body photo and emotion data separately.

[0734] Step 5:

[0735] The server pre-processes the stored full-body photos and sends them to the image recognition module, which includes image normalization, resizing, and noise removal.

[0736] Step 6:

[0737] The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (jacket, pants, shoes, etc.), extracting features such as color, shape, and pattern for each identified item.

[0738] Step 7:

[0739] The server evaluates each fashion item based on the feature amount and analyzes the overall coordination balance.

[0740] Step 8:

[0741] The server calculates areas for improvement in the user's fashion based on the analyzed features and generates specific feedback.

[0742] Step 9:

[0743] The server uses an emotion engine to recognize emotions from the user's facial expressions and voice data, categorizing specific emotions such as smile, surprise, and relaxation.

[0744] Step 10:

[0745] The server then refers to the recognized emotion data and creates personalized feedback according to the user's emotions, for example, suggesting casual fashion items if the user is relaxed.

[0746] Step 11:

[0747] The server refers to the user's past photo database and analyzes the user's tastes and preferences, which clarifies past preference patterns.

[0748] Step 12:

[0749] The server searches and selects appropriate items from the database of affiliated shopping sites based on the user's preferences and emotional data.

[0750] Step 13:

[0751] The server then combines the selected suggested items with a full-body photograph of the user to generate specific coordination examples, allowing the user to visually confirm how the suggested items fit together.

[0752] Step 14:

[0753] The server transmits the generated feedback on the improvements, the coordination examples, and the links of the suggested items to the terminal.

[0754] Step 15:

[0755] The device displays the received feedback and outfit examples to the user, including links to purchase the suggested fashion items.

[0756] Step 16:

[0757] Users can check the displayed feedback and outfit examples, click on the link of the suggested item to access the shopping site, and purchase the suggested item directly if desired.

[0758] Specific examples

[0759] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. Also, suppose the user has a relaxed, smiling expression at the time of the photo. When this full-body photo and emotional information are uploaded to the system, the server receives and stores the data, and performs image and emotional recognition. The jacket, jeans, and sneakers are identified, their features are extracted, and the overall outfit is evaluated. A light blue shirt is then suggested as a casual item suitable for a relaxed state. This shirt is then superimposed onto the user's photo and sent to the user's device along with specific outfit examples. The user can then review the results and purchase the suggested shirt on a shopping site.

[0760] In this way, the system can achieve advanced personalization according to the user's emotions and make fashion suggestions.

[0761] Example 2

[0762] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0763] Conventional fashion suggestion systems have difficulty making suggestions that fully take into account the user's emotional state and preferences. This can lead to a decrease in user satisfaction and a decrease in their willingness to purchase the suggested items. Furthermore, due to a lack of functionality to effectively utilize the user's past data, it has been difficult to provide detailed fashion suggestions tailored to individual needs.

[0764] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0765] In this invention, the server includes: means for a user to upload a full-body photograph and emotional data; means for receiving the full-body photograph and emotional data; means for preprocessing the received full-body photograph; means for transmitting the preprocessed full-body photograph to an image recognition system and identifying each fashion item; means for extracting features of each identified fashion item; means for analyzing the emotional data to recognize the user's emotional state; means for generating improvements to the user's fashion based on the image recognition results and the emotional data; means for referencing past user photograph data and analyzing the user's preferences; means for selecting suggested items from a database of a shopping site affiliated with the server; means for generating outfit examples by combining the suggested items with the full-body photograph; means for transmitting the generated outfit examples and improvements to the user's terminal; means for the user to view the improvements and suggested items through the terminal; and means for the user to access the shopping site to purchase the suggested items. This makes it possible to provide personalized fashion suggestions that take the user's emotional state into consideration, thereby increasing satisfaction and encouraging purchases.

[0766] "User" refers to an individual who uses the Service and provides a full-body photo of themselves and emotional data.

[0767] A "full-body photo" is image data that captures the entire image of the user, and is used to make it easier to identify each fashion item.

[0768] "Emotion data" refers to information obtained from the user's facial expressions and voice, and is data for analyzing the user's psychological state.

[0769] "Dedicated app or website" refers to a software platform that allows users to upload full-body photos and emotional data.

[0770] "Server" refers to a computer system that processes and stores received data, performs various analyses, and generates composite images.

[0771] "Preprocessing" refers to the processes the server performs on the full-body photo, such as image normalization, resizing, and noise removal.

[0772] An "image recognition system" refers to technology that analyzes full-body photographs to identify fashion items and extract their features.

[0773] "Features" refers to data that indicates the characteristics of each fashion item, such as color, shape, and pattern.

[0774] An "emotion engine" refers to a system that analyzes collected emotional data and determines the user's psychological state.

[0775] "Fashion improvements" refers to modifications and advice for the user's fashion that the server suggests based on image recognition results and emotion data.

[0776] "Past user photo data" refers to full-body photo data previously uploaded by a user, and is used to analyze preference trends.

[0777] "Database of affiliated shopping sites" refers to the database of online shops that the server accesses to select suggested items.

[0778] "Suggested items" refer to fashion items selected by the server based on the user's preferences and areas for improvement.

[0779] "Composite image" refers to image data in which suggested items are combined with a full-body photograph to show specific coordination examples.

[0780] "Feedback" refers to specific advice or suggestions generated based on data provided by the user.

[0781] "Terminal" refers to the device that a user uses to view feedback and coordination examples from the server.

[0782] "Purchase Link" refers to the web link provided to purchase the suggested item.

[0783] An embodiment of this invention is a system that uses a user's full-body photo and emotional data to provide highly personalized fashion suggestions. This system mainly consists of a means for a user to upload a full-body photo and emotional data, a server to process and analyze the data, and an end device to display the results.

[0784] 1. User operations

[0785] First, the user takes a full-body photo using a smartphone or digital camera. Each fashion item (e.g., jacket, bottoms, shoes, etc.) should be visible in the photo. Emotional information, such as facial expressions and voice data, is also collected simultaneously. This emotional information can be recorded as a video or voice memo during shooting.

[0786] 2. Uploading images and emotion data

[0787] The user opens a dedicated application or website, where a screen for uploading a full-body photo and emotional information appears. The user uses this screen to send the full-body photo and emotional data to the server. This data is sent from the device to the server via an HTTP request.

[0788] 3. Receiving and Preprocessing Data by the Server

[0789] The server receives the full-body photo and emotion data sent by the user, and performs preprocessing such as normalization, resizing, and noise reduction on the received full-body photo. The preprocessed image data is then sent to the image recognition system.

[0790] 4. Analysis using an image recognition system

[0791] The image recognition system in the server analyzes the preprocessed full-body photograph and identifies each fashion item (tops, bottoms, shoes, etc.), extracting features such as color, shape, and pattern for each identified item.

[0792] 5. Emotional Data Analysis by Emotion Engine

[0793] The server uses an emotion engine to analyze the uploaded facial and voice data and identify the user's emotional state, such as relaxed, surprised, or happy, thereby understanding the user's psychological state.

[0794] 6. Generate fashion improvements and suggestions

[0795] The server evaluates the user's overall fashion balance based on the image recognition results and emotion data, and calculates areas for improvement. It also references the user's past fashion database to analyze past preferences and trends. Based on this information, it searches and selects the most suitable fashion items from the shopping site's database.

[0796] 7. Generating synthetic images and feedback

[0797] The proposed items are then combined with a full-body photo of the user to generate a composite image showing specific outfit examples, which allows the user to visually see how the outfit will look and is sent to the device along with feedback.

[0798] 8. User Notification and Item Purchase

[0799] The device displays the received feedback and outfit examples to the user, including a link to purchase the suggested items. The user can click this link to access a shopping site and purchase the suggested items if desired.

[0800] Specific examples

[0801] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. If the user's emotional data indicates that they are in a relaxed state, the server receives and analyzes the photo and emotional data. The server identifies the jacket, jeans, and sneakers and extracts their features. Next, the server evaluates the overall balance of the outfit and calculates an improvement point, such as "matching it with a lighter-colored shirt." The server then suggests casual fashion items (e.g., a light blue shirt) that are appropriate for a relaxed state based on the emotional data. The system then composites the shirt into the full-body photo to generate a specific outfit example. A link to the composite image and the suggested items is sent to the user's device, where the user can view and purchase the items.

[0802] Prompt Sentence Examples

[0803] Example prompts to be input to the generative AI model

[0804] Here's a full-body photo of me. I'm wearing a blue jacket, black jeans, and white sneakers. Can you suggest a casual outfit that shows me in a relaxed state? Please also tell me the specific items and why.

[0805] In this way, the system can provide highly personalized fashion suggestions that correspond to the user's emotions, thereby increasing user satisfaction.

[0806] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0807] Step 1:

[0808] A user takes a full-body photo of themselves using a smartphone or digital camera. The input is the user's full-body photo, and the output is a digital image file. The user also records facial expressions and audio data as videos or audio memos to collect emotional information. The input is the user's emotional data, and the output is a digital video or audio file.

[0809] Step 2:

[0810] The user opens a dedicated application or website and uploads the full-body photo they have taken and the collected emotion data. The input is the user's full-body photo and emotion data, and the output is a dataset for transmission. When the user clicks the upload button, the data is sent to the server.

[0811] Step 3:

[0812] The device sends a full-body photo and emotion data to the server via an HTTP request. The input is the user's full-body photo and emotion data, and the output is an HTTP request to the server.

[0813] Step 4:

[0814] The server receives HTTP requests sent from the device and acquires full-body photos and emotion data. The input is the HTTP request from the device, and the output is a dataset on the server. These data are stored in a database.

[0815] Step 5:

[0816] The server performs preprocessing on the received full-body photo. Preprocessing includes image normalization, resizing, and noise removal. The input is the full-body photo, and the output is the preprocessed image data. Specifically, the image is resized to a specific resolution and filtered to remove noise.

[0817] Step 6:

[0818] The server sends the preprocessed full-body photo to the image recognition system. The input is the preprocessed image data, and the output is the classification results. The image recognition system analyzes the full-body photo and identifies each fashion item (top, bottom, shoes, etc.). It extracts features such as color, shape, and pattern for each identified item.

[0819] Step 7:

[0820] The server uses an emotion engine to analyze the uploaded facial and voice data. The input is emotion data, and the output is the identified emotional state. The server recognizes emotions such as relaxed, surprised, happy, etc.

[0821] Step 8:

[0822] The server generates suggestions for improving the user's fashion based on the image recognition results and emotion data. The inputs are the image recognition results and emotion data, and the output is specific suggestions for improving the fashion. For example, it generates feedback such as "It would be better to wear a brighter colored shirt."

[0823] Step 9:

[0824] The server analyzes the user's preferences by referencing past user photo data. The input is the user's past fashion data, and the output is their taste trends. Based on this, the server suggests items that match the user's past preferences and suit their current emotional state.

[0825] Step 10:

[0826] The server selects suggested items from the database of affiliated shopping sites. The input is the user's preferences and fashion improvements, and the output is suggested items. Suggested items are selected by searching the database of the shopping site.

[0827] Step 11:

[0828] The server composites the selected suggested items onto a full-body photo to generate specific coordination examples. The input is a full-body photo and the suggested items, and the output is a composite image. By generating a composite image, users can visually confirm how the suggested items match their own style.

[0829] Step 12:

[0830] The server sends the generated feedback and coordination examples of the improvements to the terminal. The input is the feedback and coordination examples, and the output is data transmission to the terminal. The terminal displays the received feedback and coordination examples to the user.

[0831] Step 13:

[0832] The user checks the displayed feedback and outfit examples. The input is the feedback and outfit examples sent from the server, and the output is the user's response. The user can click on the link of the suggested item to access the shopping site and purchase the suggested item directly if desired.

[0833] (Application example 2)

[0834] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0835] Conventional fashion recommendation systems only make suggestions based on the user's static photos and past data, making it difficult to provide personalized suggestions that reflect real-time emotions and situations. Furthermore, there was a lack of a way to suggest improvements in real time while trying on clothes in a physical store. This resulted in the problem of users being unable to select the perfect fashion item in a physical store.

[0836] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting facial expression and voice data of the user and recognizing emotions, means for generating feedback based on the recognized emotion data, and means for collecting full-body photos and emotion data in real time in the store and making fashion suggestions based on them. This enables the user to receive appropriate fashion suggestions on the spot while trying on clothes in the physical store.

[0837] A "full-body photo" is a still image showing the user's entire body, and is data used to analyze fashion item coordination and areas for improvement.

[0838] "Emotional data" is information that indicates the user's emotional state, obtained from facial expressions and voice. Personalized fashion suggestions are based on this information.

[0839] "Image analysis" involves analyzing uploaded full-body photos and identifying each fashion item in the image, thereby extracting the features of specific clothing and accessories.

[0840] "Feedback" refers to suggestions and advice generated for users based on the results of image analysis and emotional data, including suggestions for fashion improvements and suggested items.

[0841] "Suggested items" are fashion items selected by the system based on the user's preferences and emotional data, including simulated images of how the items will look when worn.

[0842] "Smart glasses" are wearable devices that incorporate sensors such as cameras and microphones and can process the user's visual and audio information in real time, and are used for data collection and analysis.

[0843] "Real-time" means that data is processed and analyzed immediately at the moment it is collected, and the results are immediately provided to the user.

[0844] "Personalized" refers to suggestions and advice that are customized based on the preferences and emotional state of a specific user. Fashion suggestions are optimized for each individual user.

[0845] "Coordination" refers to combining suggested items with a full-body photo of the user to provide a specific styling example, allowing the user to visually confirm the result.

[0846] A "shopping site" is a website where fashion items can be purchased online. In this invention, purchase links for suggested items are provided.

[0847] The "server" is a computing system that receives, stores, analyzes, and processes data sent by users, and generates suggestions and feedback based on the analysis results.

[0848] The system of the present invention includes the following components: A user takes a full-body photo and uploads the photo, along with facial expression and voice data, to a server. The server analyzes the received full-body photo and emotional data, generates fashion improvements for the user, and selects suggested items. Furthermore, data collection and suggestions can be made in real time in a physical store using smart glasses.

[0849] Overall system configuration

[0850] 1. User device: Using a device such as a smartphone, tablet, or smart glasses, a full-body photo and emotional data are taken and sent to the server.

[0851] 2. Server: Responsible for data analysis and performs the following processes:

[0852] Image analysis: Identify fashion items and extract features from uploaded full-body photos.

[0853] Emotion recognition: Recognizes the user's emotions using facial and voice data.

[0854] Proposal generation: Based on the results of image analysis and emotion recognition, suggestions for improvements and items for the user's fashion are generated.

[0855] 3. Hardware and Software:

[0856] Smart glasses (e.g., general-purpose wearable devices)

[0857] Camera and microphone (built-in or external to smart glasses)

[0858] OpenCV (image processing library)

[0859] DeepFace (emotion recognition library)

[0860] An HTTP server (you can use Flask or Django to receive and process data)

[0861] Example of a system

[0862] The user puts on the smart glasses and takes a full-body photo while trying them on in a physical store. The camera built into the smart glasses captures the photo, and the microphone collects the user's voice. This data is immediately sent to the server. The server uses image analysis to identify the items the user is wearing, extracts their features, and evaluates the user's satisfaction and interest through emotion analysis.

[0863] Based on the extracted data, real-time feedback is generated and displayed on the user's smart glasses. For example, if the user is wearing a blue jacket that doesn't match their relaxed facial expression, the server will suggest a light blue shirt as an alternative and provide examples of how to coordinate it. The user can try on the item and receive another evaluation along with their emotional data. This allows the user to easily choose the best item.

[0864] Prompt Sentence Examples

[0865] "Your goal is to develop a system that allows users to take a full-body photo in a physical store using a general-purpose wearable device and receive personalized fashion recommendations by collecting real-time emotional data on the spot. Implement a function that enhances the process of users selecting the most suitable fashion items based on their emotions."

[0866] This invention allows users to receive fashion suggestions in real time even in physical stores, enabling them to select the most suitable items on the spot. Furthermore, by utilizing emotion data, the accuracy of suggestions can be improved, leading to increased user satisfaction.

[0867] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0868] Step 1: The user wears the smart glasses and takes a full-body photo. The image data captured by the camera built into the smart glasses is input. The output is full-body photo data.

[0869] Step 2: Collect the user's facial expression and voice data simultaneously. Facial expression and voice information are input via the microphone and camera built into the smart glasses. Emotion data is obtained as the output.

[0870] Step 3: The device sends the full-body photo data and emotion data to the server. The input is the full-body photo and emotion data, and the output is sent to the server via an HTTP request.

[0871] Step 4: The server performs image analysis on the received full-body photo. The input is the full-body photo data, and the output is the identification results and feature values ​​for each fashion item (tops, bottoms, shoes, etc.). Specifically, OpenCV is used to remove noise and resize the image, and then DeepFace is used to identify the items.

[0872] Step 5: The server analyzes the emotion data. The input is facial expressions and voice data, and the output is the user's emotional state (e.g., relaxed, happy, anxious, etc.). Specifically, emotion recognition is performed using the DeepFace library.

[0873] Step 6: The server generates feedback based on the image analysis and emotion recognition results. The input is the feature values ​​and emotion data of the fashion items, and the output is suggestions for improvements to the fashion and items. Specifically, the server references the user's past fashion database and generates a list of candidate items to suggest.

[0874] Step 7: The server synthesizes the suggested items onto a full-body photo to generate an example outfit. The input is the suggested items and a full-body photo, and the output is an image in which the suggested items are synthesized onto the user's full-body photo. Specifically, the server uses an image synthesis algorithm to create a visually easy-to-understand outfit image.

[0875] Step 8: The server sends the generated coordination examples and improvements to the user's device. The input is the coordination examples and improvements, and the output is displayed on the user's smart glasses.

[0876] Step 9: The user checks the displayed feedback and coordination examples. The input is the data displayed on the smart glasses, and the output is the user's perceived feedback.

[0877] Step 10: The user accesses the shopping site to purchase the suggested item that they are satisfied with. The input is the purchase link for the suggested item, and the output is access to the shopping site and the purchase procedure.

[0878] This series of steps allows users to receive optimal fashion suggestions in real time while trying on clothes in a physical store, and the feedback gained from this entire process can significantly improve user satisfaction.

[0879] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0880] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0881] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0882] [Third embodiment]

[0883] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0884] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0885] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0886] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0887] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0888] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0889] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0890] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0891] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0892] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0893] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0894] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0895] This invention relates to a system that suggests fashion improvements and additional items suitable for a user by having the user upload a selfie photo. The system includes the following specific procedures and functions.

[0896] User operations

[0897] First, users take a full-body selfie using a smartphone or digital camera, and then they can easily upload this full-body photo through a dedicated app or website.

[0898] Image reception and analysis

[0899] 1. The user takes a full-body photo and submits it using the app or website's upload function.

[0900] 2. The device sends the uploaded full-body photo to the server.

[0901] 3. The server preprocesses the received full-body photo and passes it to the image recognition module.

[0902] 4. The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (tops, bottoms, shoes, etc.), extracting features such as color, shape, and pattern of each fashion item.

[0903] Suggested improvements

[0904] 5. The server evaluates the overall fashion balance based on the extracted features and generates specific improvements, such as providing specific feedback such as "the color balance between the top and bottom is lacking."

[0905] 6. The server references the user's past photo database, learns the user's preferences and tendencies, and generates more personalized advice.

[0906] Item suggestions and synthesis

[0907] 7. The server selects appropriate fashion items from the database of affiliated shopping sites based on the user's preferences and fashion improvements.

[0908] 8. The server then synthesizes the suggested items onto the user's full-body photo to generate a specific coordination example. This composite image allows the user to visually confirm how the suggested items match their own style.

[0909] User Notification

[0910] 9. The server sends the generated feedback of improvements and coordination examples to the terminal.

[0911] 10. The device displays feedback and outfit examples to the user, and also provides links to the best items, allowing the user to go directly to a shopping site and purchase the suggested items.

[0912] Specific examples

[0913] For example, a user may take a full-body photo of themselves wearing a blue jacket, black jeans, and white sneakers. When the user uploads this photo to the system, the server first receives the photo and uses an image recognition module to identify the jacket, jeans, and sneakers. The server then evaluates the overall balance of the outfit and generates feedback such as, "The combination of the jacket and jeans is dark, so it would be good to pair it with a light-colored shirt."

[0914] The server then references the user's past outfit history and determines that the user has previously preferred white and gray shirts. Based on this, the server selects a light blue shirt from a partner shopping site and superimposes it onto the user's full-body photo. A link to the suggested item's details is then sent to the user's device, allowing the user to view and purchase the item directly if necessary.

[0915] In this way, the system can highly personalize fashion suggestions tailored to each user's individual style and preferences, and encourage users' purchasing behavior by providing visual coordination examples.

[0916] The processing flow will be explained below.

[0917] Step 1:

[0918] The user takes a photo of themselves using a smartphone or digital camera.

[0919] Step 2:

[0920] Users open a dedicated app or website and use the interface to upload a full-body photo they have taken.

[0921] Step 3:

[0922] The device generates an HTTP request to send the uploaded photo file to the server.

[0923] Step 4:

[0924] The server receives the HTTP request, obtains the full-body photo file, and saves it.

[0925] Step 5:

[0926] The server performs pre-processing on the received full-body photos, including image normalization, resizing, and noise removal.

[0927] Step 6:

[0928] The server sends the preprocessed images to an image recognition module.

[0929] Step 7:

[0930] The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (e.g., jacket, pants, shoes, etc.). For each identified item, it extracts features such as color, shape, and pattern.

[0931] Step 8:

[0932] The server evaluates each item individually based on the extracted feature amounts, and evaluates the overall coordination balance.

[0933] Step 9:

[0934] The server calculates improvements to the fashion based on the evaluation results and generates specific feedback to the user, such as "The color combination of your jacket and jeans is too dark, so it would be better to pair it with a lighter colored shirt."

[0935] Step 10:

[0936] The server refers to a database of past photos of the user and analyzes their past fashion trends and preferences, allowing it to make suggestions that are faithful to the user's preferences.

[0937] Step 11:

[0938] The server searches the database of affiliated shopping sites for items that match the user's preferences and correspond to fashion improvements.

[0939] Step 12:

[0940] The server then combines the selected suggested items with a full-body photograph of the user to generate a specific coordination example, allowing the user to visually confirm how the suggested items will suit their style.

[0941] Step 13:

[0942] The server transmits the generated feedback of the improvements and the coordinate examples to the user's terminal.

[0943] Step 14:

[0944] The device displays the received feedback and outfit suggestions to the user, including a link to the newly suggested item.

[0945] Step 15:

[0946] Users can review the displayed feedback and outfit examples, click on the link for the suggested item to access the shopping site, and, if desired, purchase the suggested item directly.

[0947] Example 1

[0948] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0949] Today's consumers need personalized fashion advice and suggestions, but providing them requires specialized knowledge and is time-consuming. Current systems lack the functionality to provide specific feedback or suggest new fashion items to help users review and improve their outfits. This makes it difficult for users to find the perfect fashion items to match their preferences and style.

[0950] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0951] In this invention, the server includes: means for a user to upload a full-body photograph; means for receiving the uploaded full-body photograph; means for preprocessing the received full-body photograph; means for image analysis of the preprocessed full-body photograph; means for an image recognition module to identify each fashion item from the full-body photograph; means for extracting features of the identified fashion items by the image recognition module; means for evaluating the user's fashion balance based on the extracted features and generating improvements; means for the server to refer to past user photograph data, analyze the user's preferences, and generate personalized advice; means for selecting suggested items from a database of a shopping site affiliated with the server; means for the server to synthesize the suggested items with the full-body photograph to generate a coordination example; means for transmitting the synthesized coordination example and improvements to the user's terminal; means for the user to view the improvements and suggested items via the terminal; and means for the user to access the shopping site to purchase the suggested items. This enables users to easily find fashion items that suit their style.

[0952] A "full-body photo" is an image taken by a user that includes the entire body from head to toe.

[0953] A "server" is a computer system that receives, processes, and transmits data.

[0954] "Uploading means" refers to a method or function that allows a user to send a full-body photo to a server via the Internet.

[0955] The "receiving means" refers to a method or function by which the server receives the full-body photo sent by the user.

[0956] The "preprocessing means" is a method or function for removing noise from the received full-body photograph and adjusting the size.

[0957] "Image analysis means" refers to a method or function for analyzing a full-body photograph to recognize individual fashion items.

[0958] An "image recognition module" is software or hardware for identifying each fashion item in a full-body photograph and extracting their features.

[0959] The "extracting means" is a method or function for extracting characteristics such as color, shape, pattern, etc. from the identified fashion items.

[0960] The "means for evaluating fashion balance" is a method or function for evaluating the quality of the overall coordination based on the extracted feature amounts.

[0961] The "means for generating improvements" is a method or function for generating specific suggestions for improving coordination based on the evaluation results of fashion balance.

[0962] The "means for analyzing preferences" is a method or function for analyzing past user photo data to learn user preferences and tendencies.

[0963] The "means for selecting suggested items" is a method or function for selecting appropriate fashion items from the database of the affiliated shopping site.

[0964] The "means for generating coordination examples by combining" is a method or function for combining selected suggested items with a full-body photograph to generate specific coordination examples.

[0965] The "means for sending feedback and coordination examples" is a method or function for sending the generated improvements and coordination examples to the user's terminal.

[0966] The "viewing means" is a method or function for a user to use a terminal to view improvements and suggested items.

[0967] "Means for accessing a shopping site" refers to the method or function by which a user accesses a shopping site on the Internet to purchase the suggested items.

[0968] This invention relates to a system that suggests fashion improvements and additional items that suit the user when the user uploads a full-body selfie photo. This system is specifically implemented using a "full-body photo," a "server," "uploading means," "receiving means," "preprocessing means," "image analysis means," an "image recognition module," "extraction means," "means for evaluating fashion balance," "means for generating improvements," "means for analyzing preferences," "means for selecting suggested items," "means for synthesizing and generating coordination examples," "means for sending feedback and coordination examples," "means for viewing," and "means for accessing a shopping site."

[0969] Hardware and Software Usage:

[0970] First, the user takes a full-body selfie using a smartphone or digital camera. Then, the user accesses a dedicated app or website and uploads the full-body photo. Dedicated apps and websites are often built using standard web technologies (HTML, CSS, JavaScript, etc.).

[0971] The user's device sends the uploaded full-body photo to a server. The server is a high-performance computing system that receives the uploaded full-body photo and performs preprocessing on the image. This preprocessing involves noise reduction and resizing using image processing libraries such as OpenCV. The preprocessed full-body photo is then passed to an image recognition module. This module uses deep learning libraries such as TensorFlow to identify each fashion item (top, bottom, shoes, etc.) in the photo and extract features such as color, shape, and pattern.

[0972] The server evaluates the overall balance of the fashion based on the extracted features and generates specific improvements. This evaluation uses a machine learning model, which can provide specific feedback such as "the color balance between the top and bottom is lacking." The server also references a database of the user's past photos to learn the user's preferences and tendencies. This learning is done using a machine learning algorithm based on the user's past history data.

[0973] The server selects appropriate fashion items from the database of affiliated shopping sites based on the user's preferences and fashion improvements. This selection process accesses the shopping site's database via an API. The suggested items are then combined with the user's full-body photo to generate specific coordination examples. This combination is achieved using image processing tools such as Python's PIL (Pillow) library.

[0974] The generated coordination examples and improvements are sent from the server to the user's device, where they can be viewed through a dedicated app or website. Users can access the detailed link for the suggested items through feedback and purchase the suggested items directly on the shopping site.

[0975] Examples:

[0976] For example, consider the case where a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers and uploads it to a dedicated app. The server receives this photo, performs preprocessing, and then uses an image recognition module to identify each item. During this process, features such as the blue of the jacket, the black of the jeans, and the white of the sneakers are extracted. The server then evaluates the balance of the fashion and generates feedback such as, "The combination of the jacket and jeans is dark, so it would be good to pair it with a light-colored shirt."

[0977] The server also analyzes the user's past photos, learns that the user prefers white or gray shirts, and selects suggested items (e.g., light blue shirts) based on that. It then composites the selected shirt onto a full-body photo to generate a composite image. Finally, the server sends the generated feedback and composite image to the user's device, where the user can review it and directly access the shopping site using the purchase link for the suggested item.

[0978] Example prompt for a generative AI model:

[0979] "If a user takes a full-body photo that shows a blue jacket, black jeans, and white sneakers, the server analyzes the photo, suggests ways to improve the user's fashion, and selects appropriate items (e.g., a light blue shirt) from a partner shopping site. The server then combines the shirt with the full-body photo and presents it to the user as a specific coordination example."

[0980] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0981] Step 1:

[0982] Users take a full-body selfie using a smartphone or digital camera.

[0983] Specific Actions: The user sets the camera so that their entire body is visible and takes a photo of themselves wearing a blue jacket, black jeans, and white sneakers.

[0984] Input: User's full-body photo (image file)

[0985] Output: Full-body photo taken

[0986] Step 2:

[0987] Users access a dedicated app or website and upload a full-body photo.

[0988] Specific behavior: The user opens the app, clicks the "Upload Photo" button, selects a full-body photo they have taken, and submits it.

[0989] Input: Taken full-body photo (image file), user operation

[0990] Output: Uploaded full-body photo

[0991] Step 3:

[0992] The device then sends the uploaded full-body photo to the server.

[0993] Specific operation: The device sends the photo data to the specified endpoint on the server via an Internet connection.

[0994] Input: Uploaded full-body photo (image file)

[0995] Output: Full body photo sent to the server

[0996] Step 4:

[0997] The server pre-processes the received full-body photos.

[0998] Specific operation: OpenCV is used on the server to perform preprocessing such as image noise removal, size adjustment, and color correction.

[0999] Input: Full-body photo (image file) sent to the server

[1000] Output: Preprocessed full-body photo

[1001] Step 5:

[1002] The image recognition module in the server analyzes the pre-processed full-body photos and identifies each fashion item.

[1003] What it does: Uses TensorFlow to identify jackets, jeans, and sneakers in a full-body photo.

[1004] Input: Preprocessed full-body photo

[1005] Output: Identified fashion items (jacket, jeans, sneakers)

[1006] Step 6:

[1007] Based on the identification results, the server extracts features such as color, shape, and pattern of each item.

[1008] Specific operation: For each identified item, calculate the RGB color values, shape parameters, and pattern features.

[1009] Input: Identified fashion item data

[1010] Output: Extracted features (RGB color values, shape, pattern)

[1011] Step 7:

[1012] Based on the extracted features, the server evaluates the fashion balance and generates specific improvements.

[1013] Specific actions: Using a machine learning model, the extracted features are evaluated and specific improvements are suggested, such as "the overall color balance is dark, so add a bright color to the upper body."

[1014] Input: extracted features

[1015] Output: Fashion balance evaluation results and improvements

[1016] Step 8:

[1017] The server references the user's past photo database and learns the user's preferences and tendencies.

[1018] How it works: Past photo data is passed through a machine learning algorithm to identify the colors and styles the user prefers.

[1019] Input: User's past photo data

[1020] Output: Data about user preferences and tendencies

[1021] Step 9:

[1022] The server generates personalized advice that reflects the user's preferences and tendencies.

[1023] Specific operation: The fashion balance evaluation results are combined with the user's preference data to generate personalized advice such as "A light blue shirt is good."

[1024] Input: Fashion balance evaluation results, data on user preferences and trends

[1025] Output: Personalized advice

[1026] Step 10:

[1027] The server combines the suggested items with a full-body photo to generate specific coordination examples.

[1028] Specific behavior: Using Python's PIL (Pillow) library, a light blue shirt is superimposed onto a full-body photo of the user.

[1029] Input: personalized advice, full-body photo of user

[1030] Output: Composite coordinated image

[1031] Step 11:

[1032] The synthesized coordination examples and improvements are sent from the server to the user's device.

[1033] Specific operation: Data including coordination examples and improvements is sent from the server to the user's device.

[1034] Input: Synthesized coordinate image, improvement points

[1035] Output: Coordinate image sent to the user's device and improvements

[1036] Step 12:

[1037] The user's device will display feedback, outfit examples, and provide links to shopping sites.

[1038] How it works: A dedicated app or website provides feedback and visual examples of outfits, along with links to purchase suggested items.

[1039] Input: Coordinate image sent from the server and improvements

[1040] Output: A user-visible feedback screen and a purchase link

[1041] Step 13:

[1042] Users can check the suggested items through their device and access the shopping site if necessary.

[1043] What happens: The user reviews the details of the suggested item in the app and clicks the purchase link to visit the shopping site.

[1044] Input: Feedback screen and purchase link

[1045] Output: Visit to shopping site and purchase item

[1046] (Application example 1)

[1047] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1048] Conventional fashion suggestion systems lack real-time functionality when users receive advice and item suggestions to improve their outfits, making them particularly inconvenient in brick-and-mortar stores. This requires users to take photos every time they try on clothes, upload them, and wait for the results, which takes time and effort. Furthermore, it is difficult to receive personalized suggestions based on users' preferences and trends in real time, making it difficult to directly link these to purchasing behavior.

[1049] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1050] In this invention, the server includes: means for a user to upload a full-body photo; means for receiving the uploaded full-body photo; means for image-analyzing the received full-body photo; means for generating improvements to the user's fashion based on the image analysis results; means for referencing past user photo data and analyzing the user's preferences; means for selecting suggested items from a database of a purchasing platform affiliated with the server; means for the server to generate outfit examples by combining the suggested items with the full-body photo; means for the server to transmit the generated outfit examples and improvements to the user's terminal; means for the user to view the improvements and suggested items through the terminal; means for the user to access the purchasing platform to purchase the suggested items; means for the user to take and upload a full-body photo in real time in a store using smart glasses or a head-mounted display; and means for the server to generate fashion analysis results and suggestions in real time and present them to the user. This enables users to receive fashion evaluations and improvement suggestions in real time even in physical stores, providing an efficient shopping experience.

[1051] "User" refers to a person who uses the system to improve their fashion or receive suggestions.

[1052] A "full-body photo" refers to a photo that includes the user's entire body from head to toe.

[1053] "Means for uploading" refers to the functionality that a user uses to send a full-body photo of themselves to the system.

[1054] "Server" refers to a computer system that receives, analyzes, and generates recommendations from user-submitted data.

[1055] "Image analysis" refers to the process in which the server receives a full-body photo and identifies fashion items and extracts features.

[1056] "Image analysis results" refers to the identification information and characteristics of a fashion item obtained through image analysis.

[1057] "Fashion improvements" refers to feedback that suggests a better fashion style for the user's current outfit.

[1058] "User preferences" refers to the user's fashion preferences and tendencies analyzed based on past databases.

[1059] "Buying Platform" refers to an online or offline sales system through which the proposed fashion items are available for purchase.

[1060] "Suggested items" refer to fashion items selected by the server based on the user's preferences and fashion improvements.

[1061] "Synthesis" refers to the process of displaying suggested items superimposed on a full-body photo of the user.

[1062] "Coordination example" refers to an example of a fashion style visually shown by combining suggested items with a full-body photograph of the user.

[1063] "Smart glasses" refers to an augmented reality (AR)-enabled eyeglass-type device that users can wear to take full-body photos and upload them to the system.

[1064] "Head-mounted display" refers to a head-mounted display device that a user wears and can take full-body photos and upload them to the system.

[1065] "Real-time" refers to the short time between when a user takes a full-body photo and when they receive feedback or suggestions from the server, and the time is immediate.

[1066] "Means for generating fashion analysis results and suggestions in real time and presenting them to users" refers to the function of instantly analyzing a full-body photo uploaded by a user to the system and displaying the results and suggestions to the user.

[1067] As an embodiment of the present invention, a series of steps in which a user takes a full-body photo and uploads it to a server in order to improve his or her fashion will be specifically described.

[1068] System Program

[1069] The program consists of the following major steps:

[1070] 1. Take and upload a full-body photo of the user:

[1071] Users take a full-body photo of themselves using a smartphone, smart glasses, or a head-mounted display.

[1072] The full-body photo taken is uploaded to the server via the application.

[1073] 2. Image reception and analysis by the server:

[1074] The server receives the uploaded full-body photograph.

[1075] The received images are preprocessed and passed to an image recognition model (a model using Keras / TensorFlow) for analysis.

[1076] Fashion items are identified and features are extracted using image analysis.

[1077] 3. Fashion Improvements and Item Suggestion Generation:

[1078] The server evaluates the fashion balance based on the analyzed features and generates improvements.

[1079] It learns user preferences by referencing past user data and selects appropriate suggested items from the database of affiliated purchasing platforms.

[1080] 4. Coordination example generation and synthesis:

[1081] The proposed items are combined with the user's full-body photo to generate visual coordination examples.

[1082] Coordination examples and improvements are sent to the user's device.

[1083] 5. User Notice and Shopping Support:

[1084] Users can check areas for improvement and suggested items through their devices.

[1085] Through the link to the suggested item, the user can access the purchasing platform and make a direct purchase.

[1086] Natural language explanation of the process

[1087] Hardware and software used:

[1088] Smartphones, smart glasses, head-mounted displays: devices that allow users to take full-body photos and upload them to a server through an application.

[1089] Keras / TensorFlow: Libraries for running image recognition models on the server.

[1090] OpenCV: A library for image preprocessing and analysis.

[1091] Server: A computer system that receives data, analyzes it, generates proposals, and notifies users.

[1092] Data processing and calculation:

[1093] Image preprocessing: The server resizes the images uploaded by the user and converts them into a format suitable for input to the image recognition model.

[1094] Image analysis: Using an image recognition model, clothing items are identified and their features are extracted.

[1095] Fashion balance evaluation: Based on the extracted features, the balance of the entire outfit is analyzed and improvements are identified.

[1096] Selection of suggested items: By referring to the user's past preference data, appropriate items are selected from the purchasing platform's database.

[1097] Coordination example generation: The proposed items are combined with a full-body photo to generate visually easy-to-understand coordination examples.

[1098] Notifications and shopping support: Send improvements and suggested items to your device and provide direct purchase links.

[1099] Specific examples

[1100] For example, if a user uploads a full-body photo wearing a blue jacket, black jeans, and white sneakers, the server analyzes it and provides feedback such as, "To improve the color balance, pair it with a light-colored shirt." If the user's past data indicates that they prefer light-colored shirts, the server will suggest a light blue shirt. The shirt is then superimposed on the full-body photo, and the result is immediately presented to the user. The user can then purchase the shirt directly on the purchasing platform via the provided link.

[1101] Prompt Sentence Examples

[1102] "Analyze a user's full-body photo, identify fashion items, evaluate their proportions, and generate suggestions for improvements and additional items."

[1103] As described above, this system allows users to receive real-time improvements and suggestions even in physical stores, allowing them to enjoy an efficient and personalized shopping experience.

[1104] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1105] Step 1:

[1106] The user takes a full-body photo and uploads it to the server through the application. The input is a full-body photo taken by the user with a smartphone, smart glasses, or head-mounted display, and the output is an image file sent to the server. The user takes a full-body photo using their device and sends it to the server using the application's upload function.

[1107] Step 2:

[1108] The server receives the uploaded full-body photo. The input is the image file sent by the user, and the output is the image data stored in the server's database. The server stores the received full-body photo in the database.

[1109] Step 3:

[1110] The server preprocesses the received full-body photo and inputs it into the image recognition model. The input is the image data of the saved full-body photo, and the output is image data converted into a format suitable for the model. The server uses OpenCV to perform preprocessing such as image resizing and noise removal, and converts it into a format suitable for input into the image recognition model.

[1111] Step 4:

[1112] The server inputs the preprocessed image data into an image recognition model (Keras / TensorFlow) to identify fashion items and extract their features. The input is the preprocessed image data, and the output is the identified fashion items and their features (color, shape, pattern, etc.). The server uses a generative AI model to perform image recognition, identify each fashion item, and extract its features.

[1113] Step 5:

[1114] The server evaluates the fashion balance based on the extracted features and generates improvements to the user's fashion. The input is the identified fashion items and their features, and the output is a fashion balance evaluation and feedback on improvements. The server analyzes the features, evaluates the overall coordination balance, and calculates specific improvements.

[1115] Step 6:

[1116] The server references the user's past database and learns the user's tastes and preferences. The input is the user's past fashion data, and the output is personalized information based on the user's preferences. The server uses past photo data for machine learning to analyze and learn the user's tastes and trends.

[1117] Step 7:

[1118] The server selects suggested items from the database of the affiliated purchasing platform. The input is the fashion balance rating, the user's preferences, and the purchasing platform's item database, and the output is a list of suggested items. The server selects appropriate items from the purchasing platform based on the user's preferences and fashion improvement points.

[1119] Step 8:

[1120] The server composites the suggested items onto a full-body photo to generate a coordination example. The input is the suggested items and the user's full-body photo, and the output is an image of the composite coordination example. The server composites the suggested items onto a full-body photo to generate a coordination example that can be visually confirmed.

[1121] Step 9:

[1122] The server sends the generated coordination examples and improvements to the user's device. The input is an image of the coordination example and feedback on improvements, and the output is the information displayed on the user's device. The server sends the generated coordination examples and improvements to the user's device and displays them.

[1123] Step 10:

[1124] The user views the improvements and suggested items through the terminal. The input is information sent from the server (coordination examples, improvements, suggested items), and the output is visual information that the user checks. The user uses the terminal to check the sent coordination examples, improvements, and suggested items.

[1125] Step 11:

[1126] The user accesses the purchasing platform to purchase the suggested item. The input is a link to the purchasing platform displayed on the terminal, and the output is the purchase of the suggested item. The user clicks the link through the terminal to access the purchasing platform and purchase the item.

[1127] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1128] This invention relates to a system that uses a full-body photograph of a user to suggest improvements to fashion and also recognizes the user's emotions to suggest personalized items. The following describes the procedures and functions for specifically implementing this system.

[1129] User operations

[1130] First, the user takes a full-body photo of themselves using a smartphone or digital camera. Emotional information, such as facial expressions and voice data, is also collected. This emotional information is recorded in the form of video or voice memos during shooting.

[1131] Receiving and analyzing image and emotion data

[1132] 1. The user opens a dedicated app or website and uploads a full-body photo and emotional information.

[1133] 2. The device sends the uploaded photo file and emotion information to the server via an HTTP request.

[1134] 3. The server receives the HTTP request, obtains the full-body photo and emotion information, and saves them.

[1135] 4. The server preprocesses the full-body photo and passes it to the image recognition module. Preprocessing includes normalizing, resizing, and removing noise from the image.

[1136] 5. The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (top, bottoms, shoes, etc.). It extracts features such as color, shape, and pattern for each identified item.

[1137] Emotional Data Analysis

[1138] 6. The server uses an emotion engine to recognize the user's emotions from the uploaded facial and voice data, for example, analyzing whether the user is smiling, surprised, relaxed, etc.

[1139] 7. The server generates feedback based on the recognized emotion data according to the user's emotions. For example, if the user is relaxed, it may suggest casual fashion.

[1140] Improvements and Item Suggestions

[1141] 8. The server evaluates the overall fashion balance based on the extracted features and emotion data, and calculates areas for improvement. For example, it provides specific feedback such as, "The jacket color is too dark, so it would be better to pair it with a lighter-colored shirt."

[1142] 9. The server refers to the user's past fashion database and analyzes the user's preferences and trends. Based on this, it suggests items that match the user's past preferences and are based on emotional data.

[1143] 10. The server searches and selects items that correspond to the user's preferences and fashion improvements from the database of affiliated shopping sites.

[1144] Combining items and notifying the user

[1145] 11. The server then synthesizes the selected suggested items onto the user's full-body photo to generate a specific coordination example. The synthesized image allows the user to visually confirm how the suggested items match their own style.

[1146] 12. The server sends the generated feedback of improvements and coordinate examples to the terminal.

[1147] 13. The device displays the received feedback and outfit examples to the user, including links to purchase the suggested items.

[1148] 14. Users can review the displayed feedback and outfit examples, click on the link for the suggested item to access the shopping site, and, if desired, purchase the suggested item directly.

[1149] Specific examples

[1150] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. If the user's emotional data indicates that they are in a relaxed state, the server receives and analyzes the photo and emotional data. The server identifies the jacket, jeans, and sneakers and extracts their features. Next, the server evaluates the overall balance of the outfit and calculates an improvement point, such as "matching it with a lighter-colored shirt." The server then suggests casual fashion items (e.g., a light blue shirt) that are appropriate for a relaxed state based on the emotional data. The system then composites the shirt into the full-body photo to generate a specific outfit example. A link to the composite image and the suggested items is sent to the user's device, where the user can view and purchase the items.

[1151] In this way, the system can provide highly personalized fashion suggestions that correspond to the user's emotions, thereby increasing user satisfaction.

[1152] The processing flow will be explained below.

[1153] Step 1:

[1154] Users take full-body photographs of themselves using a smartphone or digital camera, and emotional information such as facial expressions and voice data is also collected.

[1155] Step 2:

[1156] The user opens a dedicated app or website and navigates through a screen to upload a full-body photo and emotional information.

[1157] Step 3:

[1158] The device sends the uploaded full-body photo and emotion information to the server, packaging the photo data and emotion data as an HTTP request.

[1159] Step 4:

[1160] The server receives the HTTP request and stores the full-body photo and emotion data separately.

[1161] Step 5:

[1162] The server pre-processes the stored full-body photos and sends them to the image recognition module, which includes image normalization, resizing, and noise removal.

[1163] Step 6:

[1164] The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (jacket, pants, shoes, etc.), extracting features such as color, shape, and pattern for each identified item.

[1165] Step 7:

[1166] The server evaluates each fashion item based on the feature amount and analyzes the overall coordination balance.

[1167] Step 8:

[1168] The server calculates areas for improvement in the user's fashion based on the analyzed features and generates specific feedback.

[1169] Step 9:

[1170] The server uses an emotion engine to recognize emotions from the user's facial expressions and voice data, categorizing specific emotions such as smile, surprise, and relaxation.

[1171] Step 10:

[1172] The server then refers to the recognized emotion data and creates personalized feedback according to the user's emotions, for example, suggesting casual fashion items if the user is relaxed.

[1173] Step 11:

[1174] The server refers to the user's past photo database and analyzes the user's tastes and preferences, which clarifies past preference patterns.

[1175] Step 12:

[1176] The server searches and selects appropriate items from the database of affiliated shopping sites based on the user's preferences and emotional data.

[1177] Step 13:

[1178] The server then combines the selected suggested items with a full-body photograph of the user to generate specific coordination examples, allowing the user to visually confirm how the suggested items fit together.

[1179] Step 14:

[1180] The server transmits the generated feedback on the improvements, the coordination examples, and the links of the suggested items to the terminal.

[1181] Step 15:

[1182] The device displays the received feedback and outfit examples to the user, including links to purchase the suggested fashion items.

[1183] Step 16:

[1184] Users can check the displayed feedback and outfit examples, click on the link of the suggested item to access the shopping site, and purchase the suggested item directly if desired.

[1185] Specific examples

[1186] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. Also, suppose the user has a relaxed, smiling expression at the time of the photo. When this full-body photo and emotional information are uploaded to the system, the server receives and stores the data, and performs image and emotional recognition. The jacket, jeans, and sneakers are identified, their features are extracted, and the overall outfit is evaluated. A light blue shirt is then suggested as a casual item suitable for a relaxed state. This shirt is then superimposed onto the user's photo and sent to the user's device along with specific outfit examples. The user can then review the results and purchase the suggested shirt on a shopping site.

[1187] In this way, the system can achieve advanced personalization according to the user's emotions and make fashion suggestions.

[1188] Example 2

[1189] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1190] Conventional fashion suggestion systems have difficulty making suggestions that fully take into account the user's emotional state and preferences. This can lead to a decrease in user satisfaction and a decrease in their willingness to purchase the suggested items. Furthermore, due to a lack of functionality to effectively utilize the user's past data, it has been difficult to provide detailed fashion suggestions tailored to individual needs.

[1191] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1192] In this invention, the server includes: means for a user to upload a full-body photograph and emotional data; means for receiving the full-body photograph and emotional data; means for preprocessing the received full-body photograph; means for transmitting the preprocessed full-body photograph to an image recognition system and identifying each fashion item; means for extracting features of each identified fashion item; means for analyzing the emotional data to recognize the user's emotional state; means for generating improvements to the user's fashion based on the image recognition results and the emotional data; means for referencing past user photograph data and analyzing the user's preferences; means for selecting suggested items from a database of a shopping site affiliated with the server; means for generating outfit examples by combining the suggested items with the full-body photograph; means for transmitting the generated outfit examples and improvements to the user's terminal; means for the user to view the improvements and suggested items through the terminal; and means for the user to access the shopping site to purchase the suggested items. This makes it possible to provide personalized fashion suggestions that take the user's emotional state into consideration, thereby increasing satisfaction and encouraging purchases.

[1193] "User" refers to an individual who uses the Service and provides a full-body photo of themselves and emotional data.

[1194] A "full-body photo" is image data that captures the entire image of the user, and is used to make it easier to identify each fashion item.

[1195] "Emotion data" refers to information obtained from the user's facial expressions and voice, and is data for analyzing the user's psychological state.

[1196] "Dedicated app or website" refers to a software platform that allows users to upload full-body photos and emotional data.

[1197] "Server" refers to a computer system that processes and stores received data, performs various analyses, and generates composite images.

[1198] "Preprocessing" refers to the processes the server performs on the full-body photo, such as image normalization, resizing, and noise removal.

[1199] An "image recognition system" refers to technology that analyzes full-body photographs to identify fashion items and extract their features.

[1200] "Features" refers to data that indicates the characteristics of each fashion item, such as color, shape, and pattern.

[1201] An "emotion engine" refers to a system that analyzes collected emotional data and determines the user's psychological state.

[1202] "Fashion improvements" refers to modifications and advice for the user's fashion that the server suggests based on image recognition results and emotion data.

[1203] "Past user photo data" refers to full-body photo data previously uploaded by a user, and is used to analyze preference trends.

[1204] "Database of affiliated shopping sites" refers to the database of online shops that the server accesses to select suggested items.

[1205] "Suggested items" refer to fashion items selected by the server based on the user's preferences and areas for improvement.

[1206] "Composite image" refers to image data in which suggested items are combined with a full-body photograph to show specific coordination examples.

[1207] "Feedback" refers to specific advice or suggestions generated based on data provided by the user.

[1208] "Terminal" refers to the device that a user uses to view feedback and coordination examples from the server.

[1209] "Purchase Link" refers to the web link provided to purchase the suggested item.

[1210] An embodiment of this invention is a system that uses a user's full-body photo and emotional data to provide highly personalized fashion suggestions. This system mainly consists of a means for a user to upload a full-body photo and emotional data, a server to process and analyze the data, and an end device to display the results.

[1211] 1. User operations

[1212] First, the user takes a full-body photo using a smartphone or digital camera. Each fashion item (e.g., jacket, bottoms, shoes, etc.) should be visible in the photo. Emotional information, such as facial expressions and voice data, is also collected simultaneously. This emotional information can be recorded as a video or voice memo during shooting.

[1213] 2. Uploading images and emotion data

[1214] The user opens a dedicated application or website, where a screen for uploading a full-body photo and emotional information appears. The user uses this screen to send the full-body photo and emotional data to the server. This data is sent from the device to the server via an HTTP request.

[1215] 3. Receiving and Preprocessing Data by the Server

[1216] The server receives the full-body photo and emotion data sent by the user, and performs preprocessing such as normalization, resizing, and noise reduction on the received full-body photo. The preprocessed image data is then sent to the image recognition system.

[1217] 4. Analysis using an image recognition system

[1218] The image recognition system in the server analyzes the preprocessed full-body photograph and identifies each fashion item (tops, bottoms, shoes, etc.), extracting features such as color, shape, and pattern for each identified item.

[1219] 5. Emotional Data Analysis by Emotion Engine

[1220] The server uses an emotion engine to analyze the uploaded facial and voice data and identify the user's emotional state, such as relaxed, surprised, or happy, thereby understanding the user's psychological state.

[1221] 6. Generate fashion improvements and suggestions

[1222] The server evaluates the user's overall fashion balance based on the image recognition results and emotion data, and calculates areas for improvement. It also references the user's past fashion database to analyze past preferences and trends. Based on this information, it searches and selects the most suitable fashion items from the shopping site's database.

[1223] 7. Generating synthetic images and feedback

[1224] The proposed items are then combined with a full-body photo of the user to generate a composite image showing specific outfit examples, which allows the user to visually see how the outfit will look and is sent to the device along with feedback.

[1225] 8. User Notification and Item Purchase

[1226] The device displays the received feedback and outfit examples to the user, including a link to purchase the suggested items. The user can click this link to access a shopping site and purchase the suggested items if desired.

[1227] Specific examples

[1228] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. If the user's emotional data indicates that they are in a relaxed state, the server receives and analyzes the photo and emotional data. The server identifies the jacket, jeans, and sneakers and extracts their features. Next, the server evaluates the overall balance of the outfit and calculates an improvement point, such as "matching it with a lighter-colored shirt." The server then suggests casual fashion items (e.g., a light blue shirt) that are appropriate for a relaxed state based on the emotional data. The system then composites the shirt into the full-body photo to generate a specific outfit example. A link to the composite image and the suggested items is sent to the user's device, where the user can view and purchase the items.

[1229] Prompt Sentence Examples

[1230] Example prompts to be input to the generative AI model

[1231] Here's a full-body photo of me. I'm wearing a blue jacket, black jeans, and white sneakers. Can you suggest a casual outfit that shows me in a relaxed state? Please also tell me the specific items and why.

[1232] In this way, the system can provide highly personalized fashion suggestions that correspond to the user's emotions, thereby increasing user satisfaction.

[1233] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1234] Step 1:

[1235] A user takes a full-body photo of themselves using a smartphone or digital camera. The input is the user's full-body photo, and the output is a digital image file. The user also records facial expressions and audio data as videos or audio memos to collect emotional information. The input is the user's emotional data, and the output is a digital video or audio file.

[1236] Step 2:

[1237] The user opens a dedicated application or website and uploads the full-body photo they have taken and the collected emotion data. The input is the user's full-body photo and emotion data, and the output is a dataset for transmission. When the user clicks the upload button, the data is sent to the server.

[1238] Step 3:

[1239] The device sends a full-body photo and emotion data to the server via an HTTP request. The input is the user's full-body photo and emotion data, and the output is an HTTP request to the server.

[1240] Step 4:

[1241] The server receives HTTP requests sent from the device and acquires full-body photos and emotion data. The input is the HTTP request from the device, and the output is a dataset on the server. These data are stored in a database.

[1242] Step 5:

[1243] The server performs preprocessing on the received full-body photo. Preprocessing includes image normalization, resizing, and noise removal. The input is the full-body photo, and the output is the preprocessed image data. Specifically, the image is resized to a specific resolution and filtered to remove noise.

[1244] Step 6:

[1245] The server sends the preprocessed full-body photo to the image recognition system. The input is the preprocessed image data, and the output is the classification results. The image recognition system analyzes the full-body photo and identifies each fashion item (top, bottom, shoes, etc.). It extracts features such as color, shape, and pattern for each identified item.

[1246] Step 7:

[1247] The server uses an emotion engine to analyze the uploaded facial and voice data. The input is emotion data, and the output is the identified emotional state. The server recognizes emotions such as relaxed, surprised, happy, etc.

[1248] Step 8:

[1249] The server generates suggestions for improving the user's fashion based on the image recognition results and emotion data. The inputs are the image recognition results and emotion data, and the output is specific suggestions for improving the fashion. For example, it generates feedback such as "It would be better to wear a brighter colored shirt."

[1250] Step 9:

[1251] The server analyzes the user's preferences by referencing past user photo data. The input is the user's past fashion data, and the output is their taste trends. Based on this, the server suggests items that match the user's past preferences and suit their current emotional state.

[1252] Step 10:

[1253] The server selects suggested items from the database of affiliated shopping sites. The input is the user's preferences and fashion improvements, and the output is suggested items. Suggested items are selected by searching the database of the shopping site.

[1254] Step 11:

[1255] The server composites the selected suggested items onto a full-body photo to generate specific coordination examples. The input is a full-body photo and the suggested items, and the output is a composite image. By generating a composite image, users can visually confirm how the suggested items match their own style.

[1256] Step 12:

[1257] The server sends the generated feedback and coordination examples of the improvements to the terminal. The input is the feedback and coordination examples, and the output is data transmission to the terminal. The terminal displays the received feedback and coordination examples to the user.

[1258] Step 13:

[1259] The user checks the displayed feedback and outfit examples. The input is the feedback and outfit examples sent from the server, and the output is the user's response. The user can click on the link of the suggested item to access the shopping site and purchase the suggested item directly if desired.

[1260] (Application example 2)

[1261] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1262] Conventional fashion recommendation systems only make suggestions based on the user's static photos and past data, making it difficult to provide personalized suggestions that reflect real-time emotions and situations. Furthermore, there was a lack of a way to suggest improvements in real time while trying on clothes in a physical store. This resulted in the problem of users being unable to select the perfect fashion item in a physical store.

[1263] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting facial expression and voice data of the user and recognizing emotions, means for generating feedback based on the recognized emotion data, and means for collecting full-body photos and emotion data in real time in the store and making fashion suggestions based on them. This enables the user to receive appropriate fashion suggestions on the spot while trying on clothes in the physical store.

[1264] A "full-body photo" is a still image showing the user's entire body, and is data used to analyze fashion item coordination and areas for improvement.

[1265] "Emotional data" is information that indicates the user's emotional state, obtained from facial expressions and voice. Personalized fashion suggestions are based on this information.

[1266] "Image analysis" involves analyzing uploaded full-body photos and identifying each fashion item in the image, thereby extracting the features of specific clothing and accessories.

[1267] "Feedback" refers to suggestions and advice generated for users based on the results of image analysis and emotional data, including suggestions for fashion improvements and suggested items.

[1268] "Suggested items" are fashion items selected by the system based on the user's preferences and emotional data, including simulated images of how the items will look when worn.

[1269] "Smart glasses" are wearable devices that incorporate sensors such as cameras and microphones and can process the user's visual and audio information in real time, and are used for data collection and analysis.

[1270] "Real-time" means that data is processed and analyzed immediately at the moment it is collected, and the results are immediately provided to the user.

[1271] "Personalized" refers to suggestions and advice that are customized based on the preferences and emotional state of a specific user. Fashion suggestions are optimized for each individual user.

[1272] "Coordination" refers to combining suggested items with a full-body photo of the user to provide a specific styling example, allowing the user to visually confirm the result.

[1273] A "shopping site" is a website where fashion items can be purchased online. In this invention, purchase links for suggested items are provided.

[1274] The "server" is a computing system that receives, stores, analyzes, and processes data sent by users, and generates suggestions and feedback based on the analysis results.

[1275] The system of the present invention includes the following components: A user takes a full-body photo and uploads the photo, along with facial expression and voice data, to a server. The server analyzes the received full-body photo and emotional data, generates fashion improvements for the user, and selects suggested items. Furthermore, data collection and suggestions can be made in real time in a physical store using smart glasses.

[1276] Overall system configuration

[1277] 1. User device: Using a device such as a smartphone, tablet, or smart glasses, a full-body photo and emotional data are taken and sent to the server.

[1278] 2. Server: Responsible for data analysis and performs the following processes:

[1279] Image analysis: Identify fashion items and extract features from uploaded full-body photos.

[1280] Emotion recognition: Recognizes the user's emotions using facial and voice data.

[1281] Proposal generation: Based on the results of image analysis and emotion recognition, suggestions for improvements and items for the user's fashion are generated.

[1282] 3. Hardware and Software:

[1283] Smart glasses (e.g., general-purpose wearable devices)

[1284] Camera and microphone (built-in or external to smart glasses)

[1285] OpenCV (image processing library)

[1286] DeepFace (emotion recognition library)

[1287] An HTTP server (you can use Flask or Django to receive and process data)

[1288] Example of a system

[1289] The user puts on the smart glasses and takes a full-body photo while trying them on in a physical store. The camera built into the smart glasses captures the photo, and the microphone collects the user's voice. This data is immediately sent to the server. The server uses image analysis to identify the items the user is wearing, extracts their features, and evaluates the user's satisfaction and interest through emotion analysis.

[1290] Based on the extracted data, real-time feedback is generated and displayed on the user's smart glasses. For example, if the user is wearing a blue jacket that doesn't match their relaxed facial expression, the server will suggest a light blue shirt as an alternative and provide examples of how to coordinate it. The user can try on the item and receive another evaluation along with their emotional data. This allows the user to easily choose the best item.

[1291] Prompt Sentence Examples

[1292] "Your goal is to develop a system that allows users to take a full-body photo in a physical store using a general-purpose wearable device and receive personalized fashion recommendations by collecting real-time emotional data on the spot. Implement a function that enhances the process of users selecting the most suitable fashion items based on their emotions."

[1293] This invention allows users to receive fashion suggestions in real time even in physical stores, enabling them to select the most suitable items on the spot. Furthermore, by utilizing emotion data, the accuracy of suggestions can be improved, leading to increased user satisfaction.

[1294] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1295] Step 1: The user wears the smart glasses and takes a full-body photo. The image data captured by the camera built into the smart glasses is input. The output is full-body photo data.

[1296] Step 2: Collect the user's facial expression and voice data simultaneously. Facial expression and voice information are input via the microphone and camera built into the smart glasses. Emotion data is obtained as the output.

[1297] Step 3: The device sends the full-body photo data and emotion data to the server. The input is the full-body photo and emotion data, and the output is sent to the server via an HTTP request.

[1298] Step 4: The server performs image analysis on the received full-body photo. The input is the full-body photo data, and the output is the identification results and feature values ​​for each fashion item (tops, bottoms, shoes, etc.). Specifically, OpenCV is used to remove noise and resize the image, and then DeepFace is used to identify the items.

[1299] Step 5: The server analyzes the emotion data. The input is facial expressions and voice data, and the output is the user's emotional state (e.g., relaxed, happy, anxious, etc.). Specifically, emotion recognition is performed using the DeepFace library.

[1300] Step 6: The server generates feedback based on the image analysis and emotion recognition results. The input is the feature values ​​and emotion data of the fashion items, and the output is suggestions for improvements to the fashion and items. Specifically, the server references the user's past fashion database and generates a list of candidate items to suggest.

[1301] Step 7: The server synthesizes the suggested items onto a full-body photo to generate an example outfit. The input is the suggested items and a full-body photo, and the output is an image in which the suggested items are synthesized onto the user's full-body photo. Specifically, the server uses an image synthesis algorithm to create a visually easy-to-understand outfit image.

[1302] Step 8: The server sends the generated coordination examples and improvements to the user's device. The input is the coordination examples and improvements, and the output is displayed on the user's smart glasses.

[1303] Step 9: The user checks the displayed feedback and coordination examples. The input is the data displayed on the smart glasses, and the output is the user's perceived feedback.

[1304] Step 10: The user accesses the shopping site to purchase the suggested item that they are satisfied with. The input is the purchase link for the suggested item, and the output is access to the shopping site and the purchase procedure.

[1305] This series of steps allows users to receive optimal fashion suggestions in real time while trying on clothes in a physical store, and the feedback gained from this entire process can significantly improve user satisfaction.

[1306] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1307] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1308] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1309] [Fourth embodiment]

[1310] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1311] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1312] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1313] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1314] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1315] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1316] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1317] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1318] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1319] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1320] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1321] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1322] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1323] This invention relates to a system that suggests fashion improvements and additional items suitable for a user by having the user upload a selfie photo. The system includes the following specific procedures and functions.

[1324] User operations

[1325] First, users take a full-body selfie using a smartphone or digital camera, and then they can easily upload this full-body photo through a dedicated app or website.

[1326] Image reception and analysis

[1327] 1. The user takes a full-body photo and submits it using the app or website's upload function.

[1328] 2. The device sends the uploaded full-body photo to the server.

[1329] 3. The server preprocesses the received full-body photo and passes it to the image recognition module.

[1330] 4. The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (tops, bottoms, shoes, etc.), extracting features such as color, shape, and pattern of each fashion item.

[1331] Suggested improvements

[1332] 5. The server evaluates the overall fashion balance based on the extracted features and generates specific improvements, such as providing specific feedback such as "the color balance between the top and bottom is lacking."

[1333] 6. The server references the user's past photo database, learns the user's preferences and tendencies, and generates more personalized advice.

[1334] Item suggestions and synthesis

[1335] 7. The server selects appropriate fashion items from the database of affiliated shopping sites based on the user's preferences and fashion improvements.

[1336] 8. The server then synthesizes the suggested items onto the user's full-body photo to generate a specific coordination example. This composite image allows the user to visually confirm how the suggested items match their own style.

[1337] User Notification

[1338] 9. The server sends the generated feedback of improvements and coordination examples to the terminal.

[1339] 10. The device displays feedback and outfit examples to the user, and also provides links to the best items, allowing the user to go directly to a shopping site and purchase the suggested items.

[1340] Specific examples

[1341] For example, a user may take a full-body photo of themselves wearing a blue jacket, black jeans, and white sneakers. When the user uploads this photo to the system, the server first receives the photo and uses an image recognition module to identify the jacket, jeans, and sneakers. The server then evaluates the overall balance of the outfit and generates feedback such as, "The combination of the jacket and jeans is dark, so it would be good to pair it with a light-colored shirt."

[1342] The server then references the user's past outfit history and determines that the user has previously preferred white and gray shirts. Based on this, the server selects a light blue shirt from a partner shopping site and superimposes it onto the user's full-body photo. A link to the suggested item's details is then sent to the user's device, allowing the user to view and purchase the item directly if necessary.

[1343] In this way, the system can highly personalize fashion suggestions tailored to each user's individual style and preferences, and encourage users' purchasing behavior by providing visual coordination examples.

[1344] The processing flow will be explained below.

[1345] Step 1:

[1346] The user takes a photo of themselves using a smartphone or digital camera.

[1347] Step 2:

[1348] Users open a dedicated app or website and use the interface to upload a full-body photo they have taken.

[1349] Step 3:

[1350] The device generates an HTTP request to send the uploaded photo file to the server.

[1351] Step 4:

[1352] The server receives the HTTP request, obtains the full-body photo file, and saves it.

[1353] Step 5:

[1354] The server performs pre-processing on the received full-body photos, including image normalization, resizing, and noise removal.

[1355] Step 6:

[1356] The server sends the preprocessed images to an image recognition module.

[1357] Step 7:

[1358] The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (e.g., jacket, pants, shoes, etc.). For each identified item, it extracts features such as color, shape, and pattern.

[1359] Step 8:

[1360] The server evaluates each item individually based on the extracted feature amounts, and evaluates the overall coordination balance.

[1361] Step 9:

[1362] The server calculates improvements to the fashion based on the evaluation results and generates specific feedback to the user, such as "The color combination of your jacket and jeans is too dark, so it would be better to pair it with a lighter colored shirt."

[1363] Step 10:

[1364] The server refers to a database of past photos of the user and analyzes their past fashion trends and preferences, allowing it to make suggestions that are faithful to the user's preferences.

[1365] Step 11:

[1366] The server searches the database of affiliated shopping sites for items that match the user's preferences and correspond to fashion improvements.

[1367] Step 12:

[1368] The server then combines the selected suggested items with a full-body photograph of the user to generate a specific coordination example, allowing the user to visually confirm how the suggested items will suit their style.

[1369] Step 13:

[1370] The server transmits the generated feedback of the improvements and the coordinate examples to the user's terminal.

[1371] Step 14:

[1372] The device displays the received feedback and outfit suggestions to the user, including a link to the newly suggested item.

[1373] Step 15:

[1374] Users can review the displayed feedback and outfit examples, click on the link for the suggested item to access the shopping site, and, if desired, purchase the suggested item directly.

[1375] Example 1

[1376] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1377] Today's consumers need personalized fashion advice and suggestions, but providing them requires specialized knowledge and is time-consuming. Current systems lack the functionality to provide specific feedback or suggest new fashion items to help users review and improve their outfits. This makes it difficult for users to find the perfect fashion items to match their preferences and style.

[1378] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1379] In this invention, the server includes: means for a user to upload a full-body photograph; means for receiving the uploaded full-body photograph; means for preprocessing the received full-body photograph; means for image analysis of the preprocessed full-body photograph; means for an image recognition module to identify each fashion item from the full-body photograph; means for extracting features of the identified fashion items by the image recognition module; means for evaluating the user's fashion balance based on the extracted features and generating improvements; means for the server to refer to past user photograph data, analyze the user's preferences, and generate personalized advice; means for selecting suggested items from a database of a shopping site affiliated with the server; means for the server to synthesize the suggested items with the full-body photograph to generate a coordination example; means for transmitting the synthesized coordination example and improvements to the user's terminal; means for the user to view the improvements and suggested items via the terminal; and means for the user to access the shopping site to purchase the suggested items. This enables users to easily find fashion items that suit their style.

[1380] A "full-body photo" is an image taken by a user that includes the entire body from head to toe.

[1381] A "server" is a computer system that receives, processes, and transmits data.

[1382] "Uploading means" refers to a method or function that allows a user to send a full-body photo to a server via the Internet.

[1383] The "receiving means" refers to a method or function by which the server receives the full-body photo sent by the user.

[1384] The "preprocessing means" is a method or function for removing noise from the received full-body photograph and adjusting the size.

[1385] "Image analysis means" refers to a method or function for analyzing a full-body photograph to recognize individual fashion items.

[1386] An "image recognition module" is software or hardware for identifying each fashion item in a full-body photograph and extracting their features.

[1387] The "extracting means" is a method or function for extracting characteristics such as color, shape, pattern, etc. from the identified fashion items.

[1388] The "means for evaluating fashion balance" is a method or function for evaluating the quality of the overall coordination based on the extracted feature amounts.

[1389] The "means for generating improvements" is a method or function for generating specific suggestions for improving coordination based on the evaluation results of fashion balance.

[1390] The "means for analyzing preferences" is a method or function for analyzing past user photo data to learn user preferences and tendencies.

[1391] The "means for selecting suggested items" is a method or function for selecting appropriate fashion items from the database of the affiliated shopping site.

[1392] The "means for generating coordination examples by combining" is a method or function for combining selected suggested items with a full-body photograph to generate specific coordination examples.

[1393] The "means for sending feedback and coordination examples" is a method or function for sending the generated improvements and coordination examples to the user's terminal.

[1394] The "viewing means" is a method or function for a user to use a terminal to view improvements and suggested items.

[1395] "Means for accessing a shopping site" refers to the method or function by which a user accesses a shopping site on the Internet to purchase the suggested items.

[1396] This invention relates to a system that suggests fashion improvements and additional items that suit the user when the user uploads a full-body selfie photo. This system is specifically implemented using a "full-body photo," a "server," "uploading means," "receiving means," "preprocessing means," "image analysis means," an "image recognition module," "extraction means," "means for evaluating fashion balance," "means for generating improvements," "means for analyzing preferences," "means for selecting suggested items," "means for synthesizing and generating coordination examples," "means for sending feedback and coordination examples," "means for viewing," and "means for accessing a shopping site."

[1397] Hardware and Software Usage:

[1398] First, the user takes a full-body selfie using a smartphone or digital camera. Then, the user accesses a dedicated app or website and uploads the full-body photo. Dedicated apps and websites are often built using standard web technologies (HTML, CSS, JavaScript, etc.).

[1399] The user's device sends the uploaded full-body photo to a server. The server is a high-performance computing system that receives the uploaded full-body photo and performs preprocessing on the image. This preprocessing involves noise reduction and resizing using image processing libraries such as OpenCV. The preprocessed full-body photo is then passed to an image recognition module. This module uses deep learning libraries such as TensorFlow to identify each fashion item (top, bottom, shoes, etc.) in the photo and extract features such as color, shape, and pattern.

[1400] The server evaluates the overall balance of the fashion based on the extracted features and generates specific improvements. This evaluation uses a machine learning model, which can provide specific feedback such as "the color balance between the top and bottom is lacking." The server also references a database of the user's past photos to learn the user's preferences and tendencies. This learning is done using a machine learning algorithm based on the user's past history data.

[1401] The server selects appropriate fashion items from the database of affiliated shopping sites based on the user's preferences and fashion improvements. This selection process accesses the shopping site's database via an API. The suggested items are then combined with the user's full-body photo to generate specific coordination examples. This combination is achieved using image processing tools such as Python's PIL (Pillow) library.

[1402] The generated coordination examples and improvements are sent from the server to the user's device, where they can be viewed through a dedicated app or website. Users can access the detailed link for the suggested items through feedback and purchase the suggested items directly on the shopping site.

[1403] Examples:

[1404] For example, consider the case where a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers and uploads it to a dedicated app. The server receives this photo, performs preprocessing, and then uses an image recognition module to identify each item. During this process, features such as the blue of the jacket, the black of the jeans, and the white of the sneakers are extracted. The server then evaluates the balance of the fashion and generates feedback such as, "The combination of the jacket and jeans is dark, so it would be good to pair it with a light-colored shirt."

[1405] The server also analyzes the user's past photos, learns that the user prefers white or gray shirts, and selects suggested items (e.g., light blue shirts) based on that. It then composites the selected shirt onto a full-body photo to generate a composite image. Finally, the server sends the generated feedback and composite image to the user's device, where the user can review it and directly access the shopping site using the purchase link for the suggested item.

[1406] Example prompt for a generative AI model:

[1407] "If a user takes a full-body photo that shows a blue jacket, black jeans, and white sneakers, the server analyzes the photo, suggests ways to improve the user's fashion, and selects appropriate items (e.g., a light blue shirt) from a partner shopping site. The server then combines the shirt with the full-body photo and presents it to the user as a specific coordination example."

[1408] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1409] Step 1:

[1410] Users take a full-body selfie using a smartphone or digital camera.

[1411] Specific Actions: The user sets the camera so that their entire body is visible and takes a photo of themselves wearing a blue jacket, black jeans, and white sneakers.

[1412] Input: User's full-body photo (image file)

[1413] Output: Full-body photo taken

[1414] Step 2:

[1415] Users access a dedicated app or website and upload a full-body photo.

[1416] Specific behavior: The user opens the app, clicks the "Upload Photo" button, selects a full-body photo they have taken, and submits it.

[1417] Input: Taken full-body photo (image file), user operation

[1418] Output: Uploaded full-body photo

[1419] Step 3:

[1420] The device then sends the uploaded full-body photo to the server.

[1421] Specific operation: The device sends the photo data to the specified endpoint on the server via an Internet connection.

[1422] Input: Uploaded full-body photo (image file)

[1423] Output: Full body photo sent to the server

[1424] Step 4:

[1425] The server pre-processes the received full-body photos.

[1426] Specific operation: OpenCV is used on the server to perform preprocessing such as image noise removal, size adjustment, and color correction.

[1427] Input: Full-body photo (image file) sent to the server

[1428] Output: Preprocessed full-body photo

[1429] Step 5:

[1430] The image recognition module in the server analyzes the pre-processed full-body photos and identifies each fashion item.

[1431] What it does: Uses TensorFlow to identify jackets, jeans, and sneakers in a full-body photo.

[1432] Input: Preprocessed full-body photo

[1433] Output: Identified fashion items (jacket, jeans, sneakers)

[1434] Step 6:

[1435] Based on the identification results, the server extracts features such as color, shape, and pattern of each item.

[1436] Specific operation: For each identified item, calculate the RGB color values, shape parameters, and pattern features.

[1437] Input: Identified fashion item data

[1438] Output: Extracted features (RGB color values, shape, pattern)

[1439] Step 7:

[1440] Based on the extracted features, the server evaluates the fashion balance and generates specific improvements.

[1441] Specific actions: Using a machine learning model, the extracted features are evaluated and specific improvements are suggested, such as "the overall color balance is dark, so add a bright color to the upper body."

[1442] Input: extracted features

[1443] Output: Fashion balance evaluation results and improvements

[1444] Step 8:

[1445] The server references the user's past photo database and learns the user's preferences and tendencies.

[1446] How it works: Past photo data is passed through a machine learning algorithm to identify the colors and styles the user prefers.

[1447] Input: User's past photo data

[1448] Output: Data about user preferences and tendencies

[1449] Step 9:

[1450] The server generates personalized advice that reflects the user's preferences and tendencies.

[1451] Specific operation: The fashion balance evaluation results are combined with the user's preference data to generate personalized advice such as "A light blue shirt is good."

[1452] Input: Fashion balance evaluation results, data on user preferences and trends

[1453] Output: Personalized advice

[1454] Step 10:

[1455] The server combines the suggested items with a full-body photo to generate specific coordination examples.

[1456] Specific behavior: Using Python's PIL (Pillow) library, a light blue shirt is superimposed onto a full-body photo of the user.

[1457] Input: personalized advice, full-body photo of user

[1458] Output: Composite coordinated image

[1459] Step 11:

[1460] The synthesized coordination examples and improvements are sent from the server to the user's device.

[1461] Specific operation: Data including coordination examples and improvements is sent from the server to the user's device.

[1462] Input: Synthesized coordinate image, improvement points

[1463] Output: Coordinate image sent to the user's device and improvements

[1464] Step 12:

[1465] The user's device will display feedback, outfit examples, and provide links to shopping sites.

[1466] How it works: A dedicated app or website provides feedback and visual examples of outfits, along with links to purchase suggested items.

[1467] Input: Coordinate image sent from the server and improvements

[1468] Output: A user-visible feedback screen and a purchase link

[1469] Step 13:

[1470] Users can check the suggested items through their device and access the shopping site if necessary.

[1471] What happens: The user reviews the details of the suggested item in the app and clicks the purchase link to visit the shopping site.

[1472] Input: Feedback screen and purchase link

[1473] Output: Visit to shopping site and purchase item

[1474] (Application example 1)

[1475] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1476] Conventional fashion suggestion systems lack real-time functionality when users receive advice and item suggestions to improve their outfits, making them particularly inconvenient in brick-and-mortar stores. This requires users to take photos every time they try on clothes, upload them, and wait for the results, which takes time and effort. Furthermore, it is difficult to receive personalized suggestions based on users' preferences and trends in real time, making it difficult to directly link these to purchasing behavior.

[1477] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1478] In this invention, the server includes: means for a user to upload a full-body photo; means for receiving the uploaded full-body photo; means for image-analyzing the received full-body photo; means for generating improvements to the user's fashion based on the image analysis results; means for referencing past user photo data and analyzing the user's preferences; means for selecting suggested items from a database of a purchasing platform affiliated with the server; means for the server to generate outfit examples by combining the suggested items with the full-body photo; means for the server to transmit the generated outfit examples and improvements to the user's terminal; means for the user to view the improvements and suggested items through the terminal; means for the user to access the purchasing platform to purchase the suggested items; means for the user to take and upload a full-body photo in real time in a store using smart glasses or a head-mounted display; and means for the server to generate fashion analysis results and suggestions in real time and present them to the user. This enables users to receive fashion evaluations and improvement suggestions in real time even in physical stores, providing an efficient shopping experience.

[1479] "User" refers to a person who uses the system to improve their fashion or receive suggestions.

[1480] A "full-body photo" refers to a photo that includes the user's entire body from head to toe.

[1481] "Means for uploading" refers to the functionality that a user uses to send a full-body photo of themselves to the system.

[1482] "Server" refers to a computer system that receives, analyzes, and generates recommendations from user-submitted data.

[1483] "Image analysis" refers to the process in which the server receives a full-body photo and identifies fashion items and extracts features.

[1484] "Image analysis results" refers to the identification information and characteristics of a fashion item obtained through image analysis.

[1485] "Fashion improvements" refers to feedback that suggests a better fashion style for the user's current outfit.

[1486] "User preferences" refers to the user's fashion preferences and tendencies analyzed based on past databases.

[1487] "Buying Platform" refers to an online or offline sales system through which the proposed fashion items are available for purchase.

[1488] "Suggested items" refer to fashion items selected by the server based on the user's preferences and fashion improvements.

[1489] "Synthesis" refers to the process of displaying suggested items superimposed on a full-body photo of the user.

[1490] "Coordination example" refers to an example of a fashion style visually shown by combining suggested items with a full-body photograph of the user.

[1491] "Smart glasses" refers to an augmented reality (AR)-enabled eyeglass-type device that users can wear to take full-body photos and upload them to the system.

[1492] "Head-mounted display" refers to a head-mounted display device that a user wears and can take full-body photos and upload them to the system.

[1493] "Real-time" refers to the short time between when a user takes a full-body photo and when they receive feedback or suggestions from the server, and the time is immediate.

[1494] "Means for generating fashion analysis results and suggestions in real time and presenting them to users" refers to the function of instantly analyzing a full-body photo uploaded by a user to the system and displaying the results and suggestions to the user.

[1495] As an embodiment of the present invention, a series of steps in which a user takes a full-body photo and uploads it to a server in order to improve his or her fashion will be specifically described.

[1496] System Program

[1497] The program consists of the following major steps:

[1498] 1. Take and upload a full-body photo of the user:

[1499] Users take a full-body photo of themselves using a smartphone, smart glasses, or a head-mounted display.

[1500] The full-body photo taken is uploaded to the server via the application.

[1501] 2. Image reception and analysis by the server:

[1502] The server receives the uploaded full-body photograph.

[1503] The received images are preprocessed and passed to an image recognition model (a model using Keras / TensorFlow) for analysis.

[1504] Fashion items are identified and features are extracted using image analysis.

[1505] 3. Fashion Improvements and Item Suggestion Generation:

[1506] The server evaluates the fashion balance based on the analyzed features and generates improvements.

[1507] It learns user preferences by referencing past user data and selects appropriate suggested items from the database of affiliated purchasing platforms.

[1508] 4. Coordination example generation and synthesis:

[1509] The proposed items are combined with the user's full-body photo to generate visual coordination examples.

[1510] Coordination examples and improvements are sent to the user's device.

[1511] 5. User Notice and Shopping Support:

[1512] Users can check areas for improvement and suggested items through their devices.

[1513] Through the link to the suggested item, the user can access the purchasing platform and make a direct purchase.

[1514] Natural language explanation of the process

[1515] Hardware and software used:

[1516] Smartphones, smart glasses, head-mounted displays: devices that allow users to take full-body photos and upload them to a server through an application.

[1517] Keras / TensorFlow: Libraries for running image recognition models on the server.

[1518] OpenCV: A library for image preprocessing and analysis.

[1519] Server: A computer system that receives data, analyzes it, generates proposals, and notifies users.

[1520] Data processing and calculation:

[1521] Image preprocessing: The server resizes the images uploaded by the user and converts them into a format suitable for input to the image recognition model.

[1522] Image analysis: Using an image recognition model, clothing items are identified and their features are extracted.

[1523] Fashion balance evaluation: Based on the extracted features, the balance of the entire outfit is analyzed and improvements are identified.

[1524] Selection of suggested items: By referring to the user's past preference data, appropriate items are selected from the purchasing platform's database.

[1525] Coordination example generation: The proposed items are combined with a full-body photo to generate visually easy-to-understand coordination examples.

[1526] Notifications and shopping support: Send improvements and suggested items to your device and provide direct purchase links.

[1527] Specific examples

[1528] For example, if a user uploads a full-body photo wearing a blue jacket, black jeans, and white sneakers, the server analyzes it and provides feedback such as, "To improve the color balance, pair it with a light-colored shirt." If the user's past data indicates that they prefer light-colored shirts, the server will suggest a light blue shirt. The shirt is then superimposed on the full-body photo, and the result is immediately presented to the user. The user can then purchase the shirt directly on the purchasing platform via the provided link.

[1529] Prompt Sentence Examples

[1530] "Analyze a user's full-body photo, identify fashion items, evaluate their proportions, and generate suggestions for improvements and additional items."

[1531] As described above, this system allows users to receive real-time improvements and suggestions even in physical stores, allowing them to enjoy an efficient and personalized shopping experience.

[1532] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1533] Step 1:

[1534] The user takes a full-body photo and uploads it to the server through the application. The input is a full-body photo taken by the user with a smartphone, smart glasses, or head-mounted display, and the output is an image file sent to the server. The user takes a full-body photo using their device and sends it to the server using the application's upload function.

[1535] Step 2:

[1536] The server receives the uploaded full-body photo. The input is the image file sent by the user, and the output is the image data stored in the server's database. The server stores the received full-body photo in the database.

[1537] Step 3:

[1538] The server preprocesses the received full-body photo and inputs it into the image recognition model. The input is the image data of the saved full-body photo, and the output is image data converted into a format suitable for the model. The server uses OpenCV to perform preprocessing such as image resizing and noise removal, and converts it into a format suitable for input into the image recognition model.

[1539] Step 4:

[1540] The server inputs the preprocessed image data into an image recognition model (Keras / TensorFlow) to identify fashion items and extract their features. The input is the preprocessed image data, and the output is the identified fashion items and their features (color, shape, pattern, etc.). The server uses a generative AI model to perform image recognition, identify each fashion item, and extract its features.

[1541] Step 5:

[1542] The server evaluates the fashion balance based on the extracted features and generates improvements to the user's fashion. The input is the identified fashion items and their features, and the output is a fashion balance evaluation and feedback on improvements. The server analyzes the features, evaluates the overall coordination balance, and calculates specific improvements.

[1543] Step 6:

[1544] The server references the user's past database and learns the user's tastes and preferences. The input is the user's past fashion data, and the output is personalized information based on the user's preferences. The server uses past photo data for machine learning to analyze and learn the user's tastes and trends.

[1545] Step 7:

[1546] The server selects suggested items from the database of the affiliated purchasing platform. The input is the fashion balance rating, the user's preferences, and the purchasing platform's item database, and the output is a list of suggested items. The server selects appropriate items from the purchasing platform based on the user's preferences and fashion improvement points.

[1547] Step 8:

[1548] The server composites the suggested items onto a full-body photo to generate a coordination example. The input is the suggested items and the user's full-body photo, and the output is an image of the composite coordination example. The server composites the suggested items onto a full-body photo to generate a coordination example that can be visually confirmed.

[1549] Step 9:

[1550] The server sends the generated coordination examples and improvements to the user's device. The input is an image of the coordination example and feedback on improvements, and the output is the information displayed on the user's device. The server sends the generated coordination examples and improvements to the user's device and displays them.

[1551] Step 10:

[1552] The user views the improvements and suggested items through the terminal. The input is information sent from the server (coordination examples, improvements, suggested items), and the output is visual information that the user checks. The user uses the terminal to check the sent coordination examples, improvements, and suggested items.

[1553] Step 11:

[1554] The user accesses the purchasing platform to purchase the suggested item. The input is a link to the purchasing platform displayed on the terminal, and the output is the purchase of the suggested item. The user clicks the link through the terminal to access the purchasing platform and purchase the item.

[1555] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1556] This invention relates to a system that uses a full-body photograph of a user to suggest improvements to fashion and also recognizes the user's emotions to suggest personalized items. The following describes the procedures and functions for specifically implementing this system.

[1557] User operations

[1558] First, the user takes a full-body photo of themselves using a smartphone or digital camera. Emotional information, such as facial expressions and voice data, is also collected. This emotional information is recorded in the form of video or voice memos during shooting.

[1559] Receiving and analyzing image and emotion data

[1560] 1. The user opens a dedicated app or website and uploads a full-body photo and emotional information.

[1561] 2. The device sends the uploaded photo file and emotion information to the server via an HTTP request.

[1562] 3. The server receives the HTTP request, obtains the full-body photo and emotion information, and saves them.

[1563] 4. The server preprocesses the full-body photo and passes it to the image recognition module. Preprocessing includes normalizing, resizing, and removing noise from the image.

[1564] 5. The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (top, bottoms, shoes, etc.). It extracts features such as color, shape, and pattern for each identified item.

[1565] Emotional Data Analysis

[1566] 6. The server uses an emotion engine to recognize the user's emotions from the uploaded facial and voice data, for example, analyzing whether the user is smiling, surprised, relaxed, etc.

[1567] 7. The server generates feedback based on the recognized emotion data according to the user's emotions. For example, if the user is relaxed, it may suggest casual fashion.

[1568] Improvements and Item Suggestions

[1569] 8. The server evaluates the overall fashion balance based on the extracted features and emotion data, and calculates areas for improvement. For example, it provides specific feedback such as, "The jacket color is too dark, so it would be better to pair it with a lighter-colored shirt."

[1570] 9. The server refers to the user's past fashion database and analyzes the user's preferences and trends. Based on this, it suggests items that match the user's past preferences and are based on emotional data.

[1571] 10. The server searches and selects items that correspond to the user's preferences and fashion improvements from the database of affiliated shopping sites.

[1572] Combining items and notifying the user

[1573] 11. The server then synthesizes the selected suggested items onto the user's full-body photo to generate a specific coordination example. The synthesized image allows the user to visually confirm how the suggested items match their own style.

[1574] 12. The server sends the generated feedback of improvements and coordinate examples to the terminal.

[1575] 13. The device displays the received feedback and outfit examples to the user, including links to purchase the suggested items.

[1576] 14. Users can review the displayed feedback and outfit examples, click on the link for the suggested item to access the shopping site, and, if desired, purchase the suggested item directly.

[1577] Specific examples

[1578] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. If the user's emotional data indicates that they are in a relaxed state, the server receives and analyzes the photo and emotional data. The server identifies the jacket, jeans, and sneakers and extracts their features. Next, the server evaluates the overall balance of the outfit and calculates an improvement point, such as "matching it with a lighter-colored shirt." The server then suggests casual fashion items (e.g., a light blue shirt) that are appropriate for a relaxed state based on the emotional data. The system then composites the shirt into the full-body photo to generate a specific outfit example. A link to the composite image and the suggested items is sent to the user's device, where the user can view and purchase the items.

[1579] In this way, the system can provide highly personalized fashion suggestions that correspond to the user's emotions, thereby increasing user satisfaction.

[1580] The processing flow will be explained below.

[1581] Step 1:

[1582] Users take full-body photographs of themselves using a smartphone or digital camera, and emotional information such as facial expressions and voice data is also collected.

[1583] Step 2:

[1584] The user opens a dedicated app or website and navigates through a screen to upload a full-body photo and emotional information.

[1585] Step 3:

[1586] The device sends the uploaded full-body photo and emotion information to the server, packaging the photo data and emotion data as an HTTP request.

[1587] Step 4:

[1588] The server receives the HTTP request and stores the full-body photo and emotion data separately.

[1589] Step 5:

[1590] The server pre-processes the stored full-body photos and sends them to the image recognition module, which includes image normalization, resizing, and noise removal.

[1591] Step 6:

[1592] The image recognition module (in the server) analyzes the full-body photo and identifies each fashion item (jacket, pants, shoes, etc.), extracting features such as color, shape, and pattern for each identified item.

[1593] Step 7:

[1594] The server evaluates each fashion item based on the feature amount and analyzes the overall coordination balance.

[1595] Step 8:

[1596] The server calculates areas for improvement in the user's fashion based on the analyzed features and generates specific feedback.

[1597] Step 9:

[1598] The server uses an emotion engine to recognize emotions from the user's facial expressions and voice data, categorizing specific emotions such as smile, surprise, and relaxation.

[1599] Step 10:

[1600] The server then refers to the recognized emotion data and creates personalized feedback according to the user's emotions, for example, suggesting casual fashion items if the user is relaxed.

[1601] Step 11:

[1602] The server refers to the user's past photo database and analyzes the user's tastes and preferences, which clarifies past preference patterns.

[1603] Step 12:

[1604] The server searches and selects appropriate items from the database of affiliated shopping sites based on the user's preferences and emotional data.

[1605] Step 13:

[1606] The server then combines the selected suggested items with a full-body photograph of the user to generate specific coordination examples, allowing the user to visually confirm how the suggested items fit together.

[1607] Step 14:

[1608] The server transmits the generated feedback on the improvements, the coordination examples, and the links of the suggested items to the terminal.

[1609] Step 15:

[1610] The device displays the received feedback and outfit examples to the user, including links to purchase the suggested fashion items.

[1611] Step 16:

[1612] Users can check the displayed feedback and outfit examples, click on the link of the suggested item to access the shopping site, and purchase the suggested item directly if desired.

[1613] Specific examples

[1614] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. Also, suppose the user has a relaxed, smiling expression at the time of the photo. When this full-body photo and emotional information are uploaded to the system, the server receives and stores the data, and performs image and emotional recognition. The jacket, jeans, and sneakers are identified, their features are extracted, and the overall outfit is evaluated. A light blue shirt is then suggested as a casual item suitable for a relaxed state. This shirt is then superimposed onto the user's photo and sent to the user's device along with specific outfit examples. The user can then review the results and purchase the suggested shirt on a shopping site.

[1615] In this way, the system can achieve advanced personalization according to the user's emotions and make fashion suggestions.

[1616] Example 2

[1617] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1618] Conventional fashion suggestion systems have difficulty making suggestions that fully take into account the user's emotional state and preferences. This can lead to a decrease in user satisfaction and a decrease in their willingness to purchase the suggested items. Furthermore, due to a lack of functionality to effectively utilize the user's past data, it has been difficult to provide detailed fashion suggestions tailored to individual needs.

[1619] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1620] In this invention, the server includes: means for a user to upload a full-body photograph and emotional data; means for receiving the full-body photograph and emotional data; means for preprocessing the received full-body photograph; means for transmitting the preprocessed full-body photograph to an image recognition system and identifying each fashion item; means for extracting features of each identified fashion item; means for analyzing the emotional data to recognize the user's emotional state; means for generating improvements to the user's fashion based on the image recognition results and the emotional data; means for referencing past user photograph data and analyzing the user's preferences; means for selecting suggested items from a database of a shopping site affiliated with the server; means for generating outfit examples by combining the suggested items with the full-body photograph; means for transmitting the generated outfit examples and improvements to the user's terminal; means for the user to view the improvements and suggested items through the terminal; and means for the user to access the shopping site to purchase the suggested items. This makes it possible to provide personalized fashion suggestions that take the user's emotional state into consideration, thereby increasing satisfaction and encouraging purchases.

[1621] "User" refers to an individual who uses the Service and provides a full-body photo of themselves and emotional data.

[1622] A "full-body photo" is image data that captures the entire image of the user, and is used to make it easier to identify each fashion item.

[1623] "Emotion data" refers to information obtained from the user's facial expressions and voice, and is data for analyzing the user's psychological state.

[1624] "Dedicated app or website" refers to a software platform that allows users to upload full-body photos and emotional data.

[1625] "Server" refers to a computer system that processes and stores received data, performs various analyses, and generates composite images.

[1626] "Preprocessing" refers to the processes the server performs on the full-body photo, such as image normalization, resizing, and noise removal.

[1627] An "image recognition system" refers to technology that analyzes full-body photographs to identify fashion items and extract their features.

[1628] "Features" refers to data that indicates the characteristics of each fashion item, such as color, shape, and pattern.

[1629] An "emotion engine" refers to a system that analyzes collected emotional data and determines the user's psychological state.

[1630] "Fashion improvements" refers to modifications and advice for the user's fashion that the server suggests based on image recognition results and emotion data.

[1631] "Past user photo data" refers to full-body photo data previously uploaded by a user, and is used to analyze preference trends.

[1632] "Database of affiliated shopping sites" refers to the database of online shops that the server accesses to select suggested items.

[1633] "Suggested items" refer to fashion items selected by the server based on the user's preferences and areas for improvement.

[1634] "Composite image" refers to image data in which suggested items are combined with a full-body photograph to show specific coordination examples.

[1635] "Feedback" refers to specific advice or suggestions generated based on data provided by the user.

[1636] "Terminal" refers to the device that a user uses to view feedback and coordination examples from the server.

[1637] "Purchase Link" refers to the web link provided to purchase the suggested item.

[1638] An embodiment of this invention is a system that uses a user's full-body photo and emotional data to provide highly personalized fashion suggestions. This system mainly consists of a means for a user to upload a full-body photo and emotional data, a server to process and analyze the data, and an end device to display the results.

[1639] 1. User operations

[1640] First, the user takes a full-body photo using a smartphone or digital camera. Each fashion item (e.g., jacket, bottoms, shoes, etc.) should be visible in the photo. Emotional information, such as facial expressions and voice data, is also collected simultaneously. This emotional information can be recorded as a video or voice memo during shooting.

[1641] 2. Uploading images and emotion data

[1642] The user opens a dedicated application or website, where a screen for uploading a full-body photo and emotional information appears. The user uses this screen to send the full-body photo and emotional data to the server. This data is sent from the device to the server via an HTTP request.

[1643] 3. Receiving and Preprocessing Data by the Server

[1644] The server receives the full-body photo and emotion data sent by the user, and performs preprocessing such as normalization, resizing, and noise reduction on the received full-body photo. The preprocessed image data is then sent to the image recognition system.

[1645] 4. Analysis using an image recognition system

[1646] The image recognition system in the server analyzes the preprocessed full-body photograph and identifies each fashion item (tops, bottoms, shoes, etc.), extracting features such as color, shape, and pattern for each identified item.

[1647] 5. Emotional Data Analysis by Emotion Engine

[1648] The server uses an emotion engine to analyze the uploaded facial and voice data and identify the user's emotional state, such as relaxed, surprised, or happy, thereby understanding the user's psychological state.

[1649] 6. Generate fashion improvements and suggestions

[1650] The server evaluates the user's overall fashion balance based on the image recognition results and emotion data, and calculates areas for improvement. It also references the user's past fashion database to analyze past preferences and trends. Based on this information, it searches and selects the most suitable fashion items from the shopping site's database.

[1651] 7. Generating synthetic images and feedback

[1652] The proposed items are then combined with a full-body photo of the user to generate a composite image showing specific outfit examples, which allows the user to visually see how the outfit will look and is sent to the device along with feedback.

[1653] 8. User Notification and Item Purchase

[1654] The device displays the received feedback and outfit examples to the user, including a link to purchase the suggested items. The user can click this link to access a shopping site and purchase the suggested items if desired.

[1655] Specific examples

[1656] For example, suppose a user takes a full-body photo wearing a blue jacket, black jeans, and white sneakers. If the user's emotional data indicates that they are in a relaxed state, the server receives and analyzes the photo and emotional data. The server identifies the jacket, jeans, and sneakers and extracts their features. Next, the server evaluates the overall balance of the outfit and calculates an improvement point, such as "matching it with a lighter-colored shirt." The server then suggests casual fashion items (e.g., a light blue shirt) that are appropriate for a relaxed state based on the emotional data. The system then composites the shirt into the full-body photo to generate a specific outfit example. A link to the composite image and the suggested items is sent to the user's device, where the user can view and purchase the items.

[1657] Prompt Sentence Examples

[1658] Example prompts to be input to the generative AI model

[1659] Here's a full-body photo of me. I'm wearing a blue jacket, black jeans, and white sneakers. Can you suggest a casual outfit that shows me in a relaxed state? Please also tell me the specific items and why.

[1660] In this way, the system can provide highly personalized fashion suggestions that correspond to the user's emotions, thereby increasing user satisfaction.

[1661] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1662] Step 1:

[1663] A user takes a full-body photo of themselves using a smartphone or digital camera. The input is the user's full-body photo, and the output is a digital image file. The user also records facial expressions and audio data as videos or audio memos to collect emotional information. The input is the user's emotional data, and the output is a digital video or audio file.

[1664] Step 2:

[1665] The user opens a dedicated application or website and uploads the full-body photo they have taken and the collected emotion data. The input is the user's full-body photo and emotion data, and the output is a dataset for transmission. When the user clicks the upload button, the data is sent to the server.

[1666] Step 3:

[1667] The device sends a full-body photo and emotion data to the server via an HTTP request. The input is the user's full-body photo and emotion data, and the output is an HTTP request to the server.

[1668] Step 4:

[1669] The server receives HTTP requests sent from the device and acquires full-body photos and emotion data. The input is the HTTP request from the device, and the output is a dataset on the server. These data are stored in a database.

[1670] Step 5:

[1671] The server performs preprocessing on the received full-body photo. Preprocessing includes image normalization, resizing, and noise removal. The input is the full-body photo, and the output is the preprocessed image data. Specifically, the image is resized to a specific resolution and filtered to remove noise.

[1672] Step 6:

[1673] The server sends the preprocessed full-body photo to the image recognition system. The input is the preprocessed image data, and the output is the classification results. The image recognition system analyzes the full-body photo and identifies each fashion item (top, bottom, shoes, etc.). It extracts features such as color, shape, and pattern for each identified item.

[1674] Step 7:

[1675] The server uses an emotion engine to analyze the uploaded facial and voice data. The input is emotion data, and the output is the identified emotional state. The server recognizes emotions such as relaxed, surprised, happy, etc.

[1676] Step 8:

[1677] The server generates suggestions for improving the user's fashion based on the image recognition results and emotion data. The inputs are the image recognition results and emotion data, and the output is specific suggestions for improving the fashion. For example, it generates feedback such as "It would be better to wear a brighter colored shirt."

[1678] Step 9:

[1679] The server analyzes the user's preferences by referencing past user photo data. The input is the user's past fashion data, and the output is their taste trends. Based on this, the server suggests items that match the user's past preferences and suit their current emotional state.

[1680] Step 10:

[1681] The server selects suggested items from the database of affiliated shopping sites. The input is the user's preferences and fashion improvements, and the output is suggested items. Suggested items are selected by searching the database of the shopping site.

[1682] Step 11:

[1683] The server composites the selected suggested items onto a full-body photo to generate specific coordination examples. The input is a full-body photo and the suggested items, and the output is a composite image. By generating a composite image, users can visually confirm how the suggested items match their own style.

[1684] Step 12:

[1685] The server sends the generated feedback and coordination examples of the improvements to the terminal. The input is the feedback and coordination examples, and the output is data transmission to the terminal. The terminal displays the received feedback and coordination examples to the user.

[1686] Step 13:

[1687] The user checks the displayed feedback and outfit examples. The input is the feedback and outfit examples sent from the server, and the output is the user's response. The user can click on the link of the suggested item to access the shopping site and purchase the suggested item directly if desired.

[1688] (Application example 2)

[1689] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1690] Conventional fashion recommendation systems only make suggestions based on the user's static photos and past data, making it difficult to provide personalized suggestions that reflect real-time emotions and situations. Furthermore, there was a lack of a way to suggest improvements in real time while trying on clothes in a physical store. This resulted in the problem of users being unable to select the perfect fashion item in a physical store.

[1691] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting facial expression and voice data of the user and recognizing emotions, means for generating feedback based on the recognized emotion data, and means for collecting full-body photos and emotion data in real time in the store and making fashion suggestions based on them. This enables the user to receive appropriate fashion suggestions on the spot while trying on clothes in the physical store.

[1692] A "full-body photo" is a still image showing the user's entire body, and is data used to analyze fashion item coordination and areas for improvement.

[1693] "Emotional data" is information that indicates the user's emotional state, obtained from facial expressions and voice. Personalized fashion suggestions are based on this information.

[1694] "Image analysis" involves analyzing uploaded full-body photos and identifying each fashion item in the image, thereby extracting the features of specific clothing and accessories.

[1695] "Feedback" refers to suggestions and advice generated for users based on the results of image analysis and emotional data, including suggestions for fashion improvements and suggested items.

[1696] "Suggested items" are fashion items selected by the system based on the user's preferences and emotional data, including simulated images of how the items will look when worn.

[1697] "Smart glasses" are wearable devices that incorporate sensors such as cameras and microphones and can process the user's visual and audio information in real time, and are used for data collection and analysis.

[1698] "Real-time" means that data is processed and analyzed immediately at the moment it is collected, and the results are immediately provided to the user.

[1699] "Personalized" refers to suggestions and advice that are customized based on the preferences and emotional state of a specific user. Fashion suggestions are optimized for each individual user.

[1700] "Coordination" refers to combining suggested items with a full-body photo of the user to provide a specific styling example, allowing the user to visually confirm the result.

[1701] A "shopping site" is a website where fashion items can be purchased online. In this invention, purchase links for suggested items are provided.

[1702] The "server" is a computing system that receives, stores, analyzes, and processes data sent by users, and generates suggestions and feedback based on the analysis results.

[1703] The system of the present invention includes the following components: A user takes a full-body photo and uploads the photo, along with facial expression and voice data, to a server. The server analyzes the received full-body photo and emotional data, generates fashion improvements for the user, and selects suggested items. Furthermore, data collection and suggestions can be made in real time in a physical store using smart glasses.

[1704] Overall system configuration

[1705] 1. User device: Using a device such as a smartphone, tablet, or smart glasses, a full-body photo and emotional data are taken and sent to the server.

[1706] 2. Server: Responsible for data analysis and performs the following processes:

[1707] Image analysis: Identify fashion items and extract features from uploaded full-body photos.

[1708] Emotion recognition: Recognizes the user's emotions using facial and voice data.

[1709] Proposal generation: Based on the results of image analysis and emotion recognition, suggestions for improvements and items for the user's fashion are generated.

[1710] 3. Hardware and Software:

[1711] Smart glasses (e.g., general-purpose wearable devices)

[1712] Camera and microphone (built-in or external to smart glasses)

[1713] OpenCV (image processing library)

[1714] DeepFace (emotion recognition library)

[1715] An HTTP server (you can use Flask or Django to receive and process data)

[1716] Example of a system

[1717] The user puts on the smart glasses and takes a full-body photo while trying them on in a physical store. The camera built into the smart glasses captures the photo, and the microphone collects the user's voice. This data is immediately sent to the server. The server uses image analysis to identify the items the user is wearing, extracts their features, and evaluates the user's satisfaction and interest through emotion analysis.

[1718] Based on the extracted data, real-time feedback is generated and displayed on the user's smart glasses. For example, if the user is wearing a blue jacket that doesn't match their relaxed facial expression, the server will suggest a light blue shirt as an alternative and provide examples of how to coordinate it. The user can try on the item and receive another evaluation along with their emotional data. This allows the user to easily choose the best item.

[1719] Prompt Sentence Examples

[1720] "Your goal is to develop a system that allows users to take a full-body photo in a physical store using a general-purpose wearable device and receive personalized fashion recommendations by collecting real-time emotional data on the spot. Implement a function that enhances the process of users selecting the most suitable fashion items based on their emotions."

[1721] This invention allows users to receive fashion suggestions in real time even in physical stores, enabling them to select the most suitable items on the spot. Furthermore, by utilizing emotion data, the accuracy of suggestions can be improved, leading to increased user satisfaction.

[1722] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1723] Step 1: The user wears the smart glasses and takes a full-body photo. The image data captured by the camera built into the smart glasses is input. The output is full-body photo data.

[1724] Step 2: Collect the user's facial expression and voice data simultaneously. Facial expression and voice information are input via the microphone and camera built into the smart glasses. Emotion data is obtained as the output.

[1725] Step 3: The device sends the full-body photo data and emotion data to the server. The input is the full-body photo and emotion data, and the output is sent to the server via an HTTP request.

[1726] Step 4: The server performs image analysis on the received full-body photo. The input is the full-body photo data, and the output is the identification results and feature values ​​for each fashion item (tops, bottoms, shoes, etc.). Specifically, OpenCV is used to remove noise and resize the image, and then DeepFace is used to identify the items.

[1727] Step 5: The server analyzes the emotion data. The input is facial expressions and voice data, and the output is the user's emotional state (e.g., relaxed, happy, anxious, etc.). Specifically, emotion recognition is performed using the DeepFace library.

[1728] Step 6: The server generates feedback based on the image analysis and emotion recognition results. The input is the feature values ​​and emotion data of the fashion items, and the output is suggestions for improvements to the fashion and items. Specifically, the server references the user's past fashion database and generates a list of candidate items to suggest.

[1729] Step 7: The server synthesizes the suggested items onto a full-body photo to generate an example outfit. The input is the suggested items and a full-body photo, and the output is an image in which the suggested items are synthesized onto the user's full-body photo. Specifically, the server uses an image synthesis algorithm to create a visually easy-to-understand outfit image.

[1730] Step 8: The server sends the generated coordination examples and improvements to the user's device. The input is the coordination examples and improvements, and the output is displayed on the user's smart glasses.

[1731] Step 9: The user checks the displayed feedback and coordination examples. The input is the data displayed on the smart glasses, and the output is the user's perceived feedback.

[1732] Step 10: The user accesses the shopping site to purchase the suggested item that they are satisfied with. The input is the purchase link for the suggested item, and the output is access to the shopping site and the purchase procedure.

[1733] This series of steps allows users to receive optimal fashion suggestions in real time while trying on clothes in a physical store, and the feedback gained from this entire process can significantly improve user satisfaction.

[1734] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1735] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1736] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1737] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1738] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1739] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1740] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in...

Claims

1. a means for a user to upload a full-body photograph; a means for the server to receive the uploaded full-body photograph; A means for performing image analysis on the full-body photograph received by the server; A means for the server to generate improvements to the user's fashion based on the image analysis results; A means for the server to refer to past user photo data and analyze user preferences; A means for selecting suggested items from a database of a shopping site affiliated with the server; A means for the server to synthesize the suggested items with a full-body photograph to generate coordination examples; A means for the server to transmit the generated coordination example and improvements to the user's terminal; A way for users to view improvements and suggested items through their devices; A means for the user to access a shopping site to purchase the suggested items; A system including:

2. a means for transmitting the received full-body photograph to an image recognition system and identifying each fashion item; A means for extracting features of each fashion item; A means of evaluating the overall fashion balance, The system of claim 1 , comprising:

3. A means for the server to refer to a database of past fashions of the user and learn the user's taste trends; a means for suggesting items based on past preferences; The system of claim 1 , comprising:

Images