System

Abstract

A system is provided.SOLUTION: A system comprising: means for photographing fashion items owned by a user and inputting item data; means for analyzing the item data and assigning classification information; means for receiving a coordination condition input from the user; means for searching the item data of the user based on the coordination condition and generating a coordination proposal; means for visually displaying the coordination proposal; and means for detecting a missing item and generating purchase recommendation data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many people often struggle with styling their outfits for everyday occasions and special events. Furthermore, those lacking fashion knowledge and etiquette can feel anxious. The present invention aims to alleviate users' fashion-related worries and concerns by providing them with fashion item suggestions, coordination examples, lifestyle information, and more through a virtual stylist who interacts with them as if they were a friend. [Means for solving the problem]

[0005] The system according to the present invention includes the following means.

[0006] 1. A means for users to take photos of fashion items they own and enter item data.

[0007] 2. A means of analyzing item data and assigning classification information.

[0008] 3. A means for accepting coordination condition input from the user.

[0009] 4. A means for searching user item data and generating coordination suggestions based on coordination conditions.

[0010] 5. A means of visually displaying coordination suggestions.

[0011] 6. A means of detecting missing items and generating purchasing recommendations.

[0012] Furthermore, the system may include means for suggesting places and event information for the user based on the outfit suggestions, means for visually displaying the suggested outfits applied to the user's avatar, means for generating a notification suggesting the recycling of unwanted items, and means for providing information on style and etiquette for ceremonial occasions. This makes it possible to comprehensively support and resolve the user's worries and anxieties about fashion.

[0013] "User" means an individual who uses the Virtual Stylist System.

[0014] "Fashion items" are fashion-related products such as clothing and accessories owned by users.

[0015] "Item data" refers to images and information of fashion items that users register in the system.

[0016] "Coordination conditions" are set conditions for the coordination desired by the user, such as season, scene, style, etc.

[0017] "Classification information" refers to information such as category, color, material, etc. that is generated based on item data.

[0018] "Coordination suggestions" are styling suggestions generated based on the user's item data and coordination conditions.

[0019] "Purchase recommendation data" is purchase recommendation information for items that are missing from a coordination proposal, and includes images, brands, prices, and links to where they can be purchased.

[0020] An "avatar" is a virtual character that appears within the system as the user's alter ego.

[0021] A "recycling suggestion notification" is a notification that suggests to the user how to recycle fashion items that are no longer needed.

[0022] "Weddings, funerals, and other special events" are weddings, funerals, and festivals.

[0023] The "Virtual Stylist System" is a system that provides styling suggestions and purchasing advice based on the fashion items owned by the user. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0025] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0026] First, the terms used in the following description will be explained.

[0027] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0028] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0029] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0030] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0032] [First embodiment]

[0033] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0034] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0037] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0040] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0041] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0042] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0043] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0044] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0045] Specific processes and operations of the virtual stylist system of the present invention will be described below. Note that in the following description, the server, the terminal, and the user are used as the subject.

[0046] 1. Initial Setup and User Registration

[0047] user:

[0048] Users download and install the app on their smartphone or other device.

[0049] When the app is launched, an account creation screen appears, where users enter basic information such as their name, email address, and password to create an account.

[0050] Users then customize their profile information and avatar.

[0051] Device:

[0052] The entered information is sent to the server.

[0053] server:

[0054] Store the received information in a database and create a user account.

[0055] 2. Registering fashion items

[0056] user:

[0057] Take a photo of a fashion item you own with your smartphone camera, such as a sweater, shirt, or jeans.

[0058] Upload the captured images through the app's interface.

[0059] Device:

[0060] Sends the uploaded image to the server.

[0061] server:

[0062] The received image data is analyzed and automatically tagged with classification information such as the item's category, color, and material.

[0063] Stores item data in a database, associated with the user's account information.

[0064] 3. Coordination suggestions

[0065] user:

[0066] Enter the conditions for which you would like to receive outfit suggestions (season, occasion, style, etc.) into the app.

[0067] Device:

[0068] The entered condition data is sent to the server.

[0069] server:

[0070] The system selects registered fashion items that meet the conditions and runs an algorithm to generate coordination suggestions.

[0071] In order to visually display the generated coordination proposal, image data to be applied to an avatar is generated.

[0072] Device:

[0073] The completed coordination proposal is displayed to the user.

[0074] user:

[0075] Review the proposed coordination and provide feedback as needed.

[0076] 4. Suggestion and purchase of missing items

[0077] server:

[0078] Based on the suggested outfit, it will detect missing items. For example, if the user does not own "leather shoes," it will detect this.

[0079] Generates purchasing recommendation data and provides purchasing information (product images, brand, price, purchase link, etc.).

[0080] Device:

[0081] Display a list of missing items to the user with a link to purchase them.

[0082] user:

[0083] Click on the purchase link and complete the purchase process as needed.

[0084] 5. Collaborative projects and promotions

[0085] server:

[0086] We regularly collect information on collaborations with popular fashion magazines and famous stylists.

[0087] Set collaboration information as a push notification and generate data to notify the user.

[0088] Device:

[0089] Receive push notifications and display them to the user.

[0090] user:

[0091] Check your push notifications to view collaborations and promotions that interest you.

[0092] 6. Recycling suggestions for unwanted items

[0093] server:

[0094] Analyzes the user's item database to detect items that have not been used for a long time.

[0095] For unwanted items, suggestion notification data is generated recommending recycling or using them at a flea market.

[0096] Device:

[0097] Show users a suggestion notification to guide them on how to recycle unwanted items.

[0098] user:

[0099] Click the link to begin recycling your unwanted items.

[0100] 7. Suggestions for places to go

[0101] server:

[0102] Based on the suggested outfits, we collect information on events, travel destinations, restaurants, etc. that may interest the user.

[0103] The proposed information and coordinated images are integrated to generate data to be provided to the user.

[0104] Device:

[0105] Showing users suggested places and events.

[0106] user:

[0107] Check the event information that interests you and find out more details.

[0108] 8. Support for style and etiquette for ceremonial occasions

[0109] server:

[0110] Maintain a database of information on style and general etiquette related to weddings, funerals, and other ceremonies.

[0111] Generate data that provides appropriate style and manner information in response to user requests.

[0112] Device:

[0113] The suggested style and manner information is displayed to the user, and a screen showing the style applied to the user's avatar is displayed.

[0114] user:

[0115] Check the suggested style and etiquette information to alleviate any concerns.

[0116] Specific examples

[0117] user:

[0118] In the app, set the conditions "commute," "autumn," and "business casual" and request outfit suggestions.

[0119] Check the outfit suggested by the server: gray sweater, white shirt, and dark blue jeans.

[0120] Check the proposal and give feedback such as "Leather shoes are needed."

[0121] server:

[0122] Based on the suggested outfit, it detects that "leather shoes" are in short supply and generates purchasing information.

[0123] Device:

[0124] Present the user with a purchase link and provide them with a link to complete the purchase.

[0125] user:

[0126] Click on the provided link and complete the process to purchase the "leather shoes."

[0127] The system of the present invention allows users to effectively utilize their fashion items and eliminates the anxiety of coordinating outfits for everyday wear and special events.

[0128] The processing flow will be explained below.

[0129] Program processing flow

[0130] 1. Initial Setup and User Registration

[0131] Step 1:

[0132] User: Downloads and installs the app on a smartphone or other device.

[0133] Step 2:

[0134] User: Launch the app and enter basic information (name, email address, password) on the new account creation screen.

[0135] Step 3:

[0136] Terminal: Sends the entered information to the server.

[0137] Step 4:

[0138] Server: Stores the received user information in a database and creates a user account.

[0139] Step 5:

[0140] Users: Customize your profile information and avatar.

[0141] 2. Registering fashion items

[0142] Step 1:

[0143] User: Take a photo of a fashion item they own with their smartphone camera, such as a sweater, shirt, or jeans.

[0144] Step 2:

[0145] User: Register the captured image within the app.

[0146] Step 3:

[0147] On the device: Sends the uploaded image to the server.

[0148] Step 4:

[0149] Server: Analyzes the received image data and automatically tags it with classification information such as item category, color, and material.

[0150] Step 5:

[0151] Server: Stores tagged item data in a database, associating it with user account information.

[0152] 3. Coordination suggestions

[0153] Step 1:

[0154] User: Enter the conditions for which you would like the app to suggest outfits (season, occasion, style, etc.).

[0155] Step 2:

[0156] Terminal: Sends the entered condition data to the server.

[0157] Step 3:

[0158] Server: Selects registered fashion items that match the criteria and runs an algorithm to generate coordination suggestions.

[0159] Step 4:

[0160] Server: Generates image data to be applied to an avatar to visually display the generated coordination proposal.

[0161] Step 5:

[0162] Device: Displays the completed outfit suggestions to the user.

[0163] Step 6:

[0164] User: Review the proposed outfits and provide feedback.

[0165] 4. Suggestion and purchase of missing items

[0166] Step 1:

[0167] Server: Detects missing items based on the proposed outfit. For example, if the user does not have "leather shoes," it will detect this.

[0168] Step 2:

[0169] Server: Generates purchasing recommendation data (product images, brand, price, purchase link, etc.).

[0170] Step 3:

[0171] On the device: Display a list of missing items to the user with a link to purchase them.

[0172] Step 4:

[0173] User: Clicks on the purchase link if necessary and completes the purchase.

[0174] 5. Collaborative projects and promotions

[0175] Step 1:

[0176] Server: Regularly collects information on collaborations with popular fashion magazines and famous stylists.

[0177] Step 2:

[0178] Server: Sets collaboration information as a push notification and generates data to notify the user.

[0179] Step 3:

[0180] Device: Receives the push notification and displays it to the user.

[0181] Step 4:

[0182] User: Check push notifications to view collaborations and promotions that interest them.

[0183] 6. Recycling suggestions for unwanted items

[0184] Step 1:

[0185] Server: Analyzes the user's item database and detects items that have not been used for a long time.

[0186] Step 2:

[0187] Server: Generates notification data suggesting recycling or selling unwanted items at flea markets.

[0188] Step 3:

[0189] On the device: Show users a suggestion notification to guide them on how to recycle unwanted items.

[0190] Step 4:

[0191] User: Clicks on a link to recycle unwanted items.

[0192] 7. Suggestions for places to go

[0193] Step 1:

[0194] Server: Based on the suggested outfits, collects information on events, travel destinations, restaurants, etc. that the user may be interested in.

[0195] Step 2:

[0196] Server: Integrates the recommendation information and coordinated images to generate data to provide to users.

[0197] Step 3:

[0198] On your device: Show suggested places and events to the user.

[0199] Step 4:

[0200] Users: View events that interest them and learn more.

[0201] 8. Support for style and etiquette for ceremonial occasions

[0202] Step 1:

[0203] Server: Manages a database of information on style and general etiquette related to ceremonial occasions.

[0204] Step 2:

[0205] Server: Generates data to provide appropriate style and manner information in response to user requests.

[0206] Step 3:

[0207] On the device: Show suggested styles and mannerisms to the user and display the results on the user's avatar.

[0208] Step 4:

[0209] Users: Review suggested style and etiquette information to ease concerns.

[0210] This will enable us to provide comprehensive support to users regarding their fashion concerns.

[0211] Example 1

[0212] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0213] Today's consumers face daily fashion-related worries and difficulties in coordinating outfits. They particularly need a way to efficiently coordinate outfits for each season and occasion while making the most of the items they own. Furthermore, there is no comprehensive fashion management system that can suggest and purchase items they are running low on, provide information on collaborations with the fashion industry, or even recycle unwanted items. This often results in users wasting time and effort choosing their daily fashion, which can be stressful.

[0214] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0215] In this invention, the server includes means for photographing fashion items owned by the user and inputting item data, means for analyzing the item data and assigning classification information, means for accepting coordination conditions input from the user, means for searching the user's item data based on the coordination conditions and generating coordination suggestions, means for visually displaying the coordination suggestions, means for detecting missing items and generating purchase recommendation data, means for collecting collaboration information with specific fashion magazines and stylists and generating push notifications, means for analyzing the user's item database and detecting items that have not been used for a long time and suggesting recycling, and means for providing information on styles and general etiquette related to ceremonial occasions. This allows users to effectively utilize their own fashion items, reduces anxiety about coordinating outfits for everyday use and special events, and enables comprehensive fashion management.

[0216] "User" refers to any individual or organization that uses this system.

[0217] "Fashion items" refers to fashion-related items such as clothing, accessories, and shoes owned by a user.

[0218] "Item data" refers to data that includes information such as images, categories, colors, and materials of fashion items.

[0219] "Coordination conditions" refer to conditions such as season, scene, and style specified by the user.

[0220] "Coordination suggestions" refer to style suggestions generated based on the user's fashion items and coordination conditions.

[0221] "Purchase recommendation data" refers to data that identifies missing items and includes related purchasing information (such as product images, brand, price, and purchase links).

[0222] "Collaboration Information" refers to information provided in collaboration with specific fashion magazines or stylists.

[0223] "Push notification" refers to a notification sent from a server to a user.

[0224] "Recycling suggestions" refer to suggestions that encourage users to recycle or sell fashion items that they have not used for a long time.

[0225] "Style information" refers to information about fashion styles and general manners related to ceremonial occasions.

[0226] The specific processing and operation of the virtual stylist system of the present invention will be described below. The present invention manages the user's fashion items and suggests coordination suggestions by exchanging data between the user, the terminal, and the server.

[0227] Initial Setup and User Registration

[0228] User: First, the user downloads and installs a dedicated app onto their smartphone or other device. When the app is launched, an account creation screen appears. The user enters basic information such as name, email address, and password to create an account. They can then customize their profile information and avatar.

[0229] Terminal: This input information is sent from the terminal to the server.

[0230] Server: The server stores the received information in a database and creates a user account.

[0231] Fashion item registration

[0232] User: The user takes a photo of a fashion item they own, such as a sweater, shirt, or jeans, with their smartphone camera and uploads the image through the app's interface.

[0233] On your device: Uploaded images are sent to the server.

[0234] Server: The received image data is analyzed using services such as the Google Cloud Vision API. After analysis, classification information such as the item's category, color, and material is automatically tagged and stored in a database in association with the user's account information.

[0235] Coordination suggestions

[0236] User: The user inputs the conditions for which they would like to receive outfit suggestions (for example, season, occasion, style, etc.) into the app.

[0237] Terminal: The entered condition data is sent to the server.

[0238] Server: The server selects registered fashion items that match the input criteria and runs an algorithm to generate outfit suggestions. This algorithm also takes into account the user's past preferences and trend information to suggest optimal outfits. The generated outfit suggestions are prepared in a form that can be displayed visually, and image data to be applied to the user's avatar is generated.

[0239] Terminal: The completed outfit suggestions are displayed to the user, who can then review the suggestions and provide feedback if necessary.

[0240] Suggestion and purchase of missing items

[0241] Server: Detects missing items based on the suggested outfit. For example, if the user does not own "leather shoes," it detects this. To fill this gap, it generates purchasing recommendation data and provides detailed information such as product images, brand, price, and purchase links.

[0242] On the device: Display a list of missing items to the user with a link to purchase them.

[0243] User: Clicks on the purchase link if necessary and completes the purchase.

[0244] Collaborative projects and promotions

[0245] Server: Periodically collects information about collaborations with specific fashion magazines and famous stylists. The collected information is sent to users via push notifications.

[0246] Device: Receives the push notification and displays it to the user.

[0247] User: Check push notifications to view collaborations and promotions that interest them.

[0248] Recycling suggestions for unwanted items

[0249] Server: Analyzes the user's item database and detects items that have not been used for a long time. Based on this, it generates notification data recommending recycling or selling them at a flea market.

[0250] On the device: Show users a suggestion notification to guide them on how to recycle unwanted items.

[0251] User: Clicks on a link to recycle unwanted items.

[0252] Suggestions for places to go

[0253] Server: Based on the suggested outfits, the server collects information on events, travel destinations, restaurants, etc. that the user may be interested in. The server then integrates the suggested information with the outfit images to generate data to be provided to the user.

[0254] On your device: Show suggested places and events to the user.

[0255] Users: View events that interest them and learn more.

[0256] Style and etiquette support for ceremonial occasions

[0257] Server: Manages a database of information on style and general etiquette related to ceremonial occasions. Generates data to provide appropriate style and etiquette information in response to user requests.

[0258] On the device: Show suggested styles and mannerisms to the user, and show the styled screen on the user's avatar.

[0259] Users: Review suggested style and etiquette information to ease concerns.

[0260] Specific examples

[0261] As an example of a prompt, if the user sets the conditions "commute," "autumn," and "business casual" and requests outfit suggestions, the server will suggest an outfit of "gray sweater, white shirt, and dark blue denim." If the user provides feedback that "I need leather shoes," the server will detect the shortage and generate purchasing information, and the device will present the user with a purchase link.

[0262] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0263] Step 1:

[0264] User: Download and install the Virtual Stylist System app from the smartphone app store. Launch the app and proceed to the account creation screen. Here, enter information such as your name, email address, and password to create an account. The entered information is sent to your device.

[0265] Input: User's basic information (name, email address, password)

[0266] Output: User information is sent to the terminal

[0267] Step 2:

[0268] Terminal: Sends the account information entered by the user to the server.

[0269] Input: User information (name, email address, password)

[0270] Output: User information is sent to the server

[0271] Step 3:

[0272] Server: The received user information is saved in the database and a user account is created. This completes the user registration.

[0273] Input: User information sent from the device

[0274] Output: User information stored in database, account created

[0275] Step 4:

[0276] User: After creating an account, customize your profile information and avatar. Enter the customized information and send it to your device.

[0277] Input: Profile information, avatar setting information

[0278] Output: Profile information and avatar settings are sent to the device.

[0279] Step 5:

[0280] Device: Sends user-customized information to the server.

[0281] Input: Profile information, avatar setting information

[0282] Output: Customization information sent to the server

[0283] Step 6:

[0284] Server: Save the received customization information in the database. This completes the initial setup and user registration.

[0285] Input: Customization information sent from the device

[0286] Output: Customization information stored in the database

[0287] Step 7:

[0288] Users take photos of their fashion items with their smartphone camera and upload the images through the app's interface.

[0289] Input: Photographed fashion item images

[0290] Output: The image is captured to the device.

[0291] Step 8:

[0292] On the device: Sends the uploaded image to the server.

[0293] Input: Photographed fashion item images

[0294] Output: Image data sent to the server

[0295] Step 9:

[0296] Server: Analyzes the received image data using services such as Google Cloud Vision API to generate classification information such as item category, color, material, etc. The classification information is then tagged to the item data and stored in a database.

[0297] Input: Image data sent from the device

[0298] Output: Item data associated with classification information, item data stored in a database

[0299] Step 10:

[0300] User: Enter the conditions for which they would like to receive outfit suggestions (e.g., season, occasion, style, etc.) into the app and send the results to their device.

[0301] Input: Coordination conditions

[0302] Output: Coordinate conditions are captured in the terminal

[0303] Step 11:

[0304] Terminal: Sends the entered coordination condition data to the server.

[0305] Input: Coordination condition data

[0306] Output: Coordinate condition data sent to the server

[0307] Step 12:

[0308] Server: Runs an algorithm to select registered fashion items that match coordination criteria. Based on the selected items, it generates coordination suggestions and generates image data to be applied to the user's avatar.

[0309] Input: Registered fashion item data, coordination condition data

[0310] Output: Coordination suggestions and image data to be applied to the avatar

[0311] Step 13:

[0312] Device: Displays the completed outfit suggestions to the user.

[0313] Input: Coordination suggestions and image data sent from the server

[0314] Output: Outfit suggestions displayed on the user's device

[0315] Step 14:

[0316] User: Check the suggested outfit and provide feedback if necessary. For example, enter a comment such as "I need leather shoes" and send that information to the device.

[0317] Input: Feedback information (e.g., I need leather shoes)

[0318] Output: Feedback information captured in terminal

[0319] Step 15:

[0320] Device: Sends user feedback information to the server.

[0321] Input: Feedback information

[0322] Output: Feedback information sent to the server

[0323] Step 16:

[0324] Server: Based on the proposed outfit, detects missing items (e.g., leather shoes) and generates purchasing recommendations, including product images, brand, price, and purchase link details.

[0325] Input: User feedback information, suggested coordinate data

[0326] Output: Purchase recommendation data

[0327] Step 17:

[0328] Device: Display purchase recommendation data to the user and provide a purchase link.

[0329] Input: Purchase recommendation data sent from the server

[0330] Output: Purchase recommendation data displayed on the user's device

[0331] Step 18:

[0332] User: Clicks on the purchase link and completes the purchase.

[0333] Input: Purchase link click

[0334] Output: Completed checkout

[0335] Step 19:

[0336] Server: Periodically collects collaboration information with specific fashion magazines and stylists and generates push notifications.

[0337] Input: Information collected from fashion magazines and stylists

[0338] Output: Generates push notification data

[0339] Step 20:

[0340] Device: Receives the push notification and displays it to the user.

[0341] Input: Push notification data sent from the server

[0342] Output: A push notification that appears on the user's device

[0343] Step 21:

[0344] User: Check push notifications to view collaborations and promotions that interest them.

[0345] Input: Push notification click

[0346] Output: Collaboration and promotion information to be displayed

[0347] Step 22:

[0348] Server: Analyzes the user's item database and detects items that have not been used for a long time. Generates notification data suggesting recycling or selling items at a flea market.

[0349] Input: Item database

[0350] Output: Recycling suggestion notification data

[0351] Step 23:

[0352] Device: Show recycling suggestion notification to user.

[0353] Input: Recycle proposal notification data sent from the server

[0354] Output: A recycling suggestion notification displayed on the user's device.

[0355] Step 24:

[0356] User: Clicks on a link to recycle unwanted items.

[0357] Input: Click on recycle link

[0358] Output: Start of recycling procedure

[0359] Step 25:

[0360] Server: Based on the suggested outfits, collects information on events, travel destinations, and restaurants that the user may be interested in, and generates suggestion data.

[0361] Input: Proposed coordinate data

[0362] Output: Event and location suggestions

[0363] Step 26:

[0364] On your device: Show suggested places and events to the user.

[0365] Input: Proposal data sent from the server

[0366] Output: Event and location information displayed on the user's device

[0367] Step 27:

[0368] Users: View events that interest them and learn more.

[0369] Input: Click on event information

[0370] Output: Detailed information about the event displayed

[0371] Step 28:

[0372] Server: Manages a database of information on style and general etiquette related to ceremonial occasions, and provides data in response to user requests.

[0373] Input: User request

[0374] Output: Style and manner information provided

[0375] Step 29:

[0376] On the device: Show suggested styles and mannerisms to the user, and show the styled screen on the user's avatar.

[0377] Input: Style and manner information sent from the server

[0378] Output: Style and manner information displayed on the user's device

[0379] Step 30:

[0380] Users: Review suggested style and etiquette information to ease concerns.

[0381] Input: Check style and manner information

[0382] Output: User concerns resolved

[0383] In this way, users can effectively manage their fashion items through every step, eliminating the worry of choosing everyday fashion or coordinating outfits for special events.

[0384] (Application example 1)

[0385] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0386] Problems faced by users of conventional fashion coordination systems include the complicated process of suggesting outfits in real time and purchasing missing items, making it difficult to use on the go. Furthermore, there are limited ways for users to visually check the results of applying suggested outfits to their owned fashion items, limiting the user experience. The problem this invention aims to solve is to enable real-time suggestions and purchasing procedures, thereby improving user convenience and satisfaction.

[0387] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0388] In this invention, the server includes means for photographing fashion items owned by the user and inputting item data, means for analyzing the item data and assigning classification information, means for accepting coordination condition input from the user, means for searching the user's item data based on the coordination conditions and generating coordination suggestions, means for visually displaying the coordination suggestions, means for detecting missing items and generating purchase recommendation data, means for allowing the user to check the suggested coordination in real time and proceed with purchasing the missing items, and means for visually displaying the results via smart glasses or a head-mounted display, thereby enabling the user to check the coordination suggestions in real time and immediately proceed with purchasing the missing items.

[0389] "User" refers to an individual consumer who uses the system.

[0390] "Fashion items" refer to items that make up an individual's appearance, such as clothing, accessories, and shoes.

[0391] "Item data" refers to images of each fashion item owned by the user and their characteristic information.

[0392] "Classification information" refers to information that describes characteristics such as the category, color, material, and style of a fashion item.

[0393] "Coordination conditions" refer to the desired scene, season, style, and other conditions that the user enters into the app.

[0394] "Coordination suggestions" refer to fashion combinations generated based on the user's item data and coordination conditions.

[0395] "Visual display means" refers to the method by which visual information is presented to the user through an application or device.

[0396] "Missing items" refer to items that are necessary for the proposed outfit but that the user does not own.

[0397] "Purchase recommendation data" refers to product information and links provided to help users purchase items they are missing.

[0398] "Means for checking in real time" refers to a method in which suggested outfits are instantly presented to users, allowing them to provide feedback and complete the purchase process.

[0399] "Smart glasses" are a type of wearable device that displays information in the user's field of vision.

[0400] A "head-mounted display" is a display device that is worn on the user's head and has the function of providing visual information.

[0401] "On-the-go use" refers to a user using the system's functions outside of their home.

[0402] The following describes in detail the embodiments of the present invention: First, an overview of the entire system and specific examples of each processing step are shown.

[0403] System Configuration

[0404] This invention consists of a user, a terminal (smart glasses or a head-mounted display), and a server.

[0405] Hardware and Software

[0406] 1. Hardware:

[0407] Smart glasses (e.g., smart glasses, head-mounted displays)

[0408] Device (e.g. smartphone, tablet)

[0409] 2. Software:

[0410] Image analysis algorithms (e.g., OpenCV)

[0411] Deep learning models (e.g. TensorFlow, PyTorch)

[0412] Cloud services (e.g., Google Cloud Vision API, Amazon Rekognition)

[0413] Database (e.g. Firebase Cloud Firestore)

[0414] API (e.g., online shopping API)

[0415] Program processing description

[0416] The program for this system is designed to efficiently manage the fashion items owned by the user, provide real-time coordination suggestions, and assist in purchasing missing items. Each processing step and its operation are explained in detail below.

[0417] 1. Register user item data:

[0418] Users use their device's camera function to take photos of fashion items they own and upload the image data to the app.

[0419] The terminal transmits the image data received from the user to the server.

[0420] The server uses Google Cloud Vision API and Amazon Rekognition to analyze images and generate classification information such as item category, color, material, etc. This information is stored in Firebase Cloud Firestore.

[0421] 2. Coordination suggestions:

[0422] Users input coordination criteria (e.g., season, occasion, style) through the application.

[0423] The terminal transmits the input condition data to the server.

[0424] The server searches the user's item data and uses deep learning models to generate optimal outfits, which are visually displayed through smart glasses or a head-mounted display.

[0425] 3. Missing item detection and purchasing recommendations:

[0426] The server detects missing items based on the proposed coordinates, which are identified by comparing them with a database.

[0427] The server generates purchasing recommendation data (product information, purchase links, etc.) via an internet shopping API and sends it to the terminal.

[0428] The terminal displays the information to the user through smart glasses or a head-mounted display, allowing the user to complete the purchase process in real time.

[0429] Examples of specific examples and prompts

[0430] Specific examples

[0431] The user puts on the smart glasses and inputs a suggestion for an outfit for "office casual" and "winter" into the app. The server references the user's item data and suggests an outfit of "black knit, blue jeans, and down jacket." The user reviews the suggested outfit and gives feedback that "I need winter boots." The server detects any missing items and provides a purchase link via an online shopping API. The user uses the link to complete the purchase process.

[0432] Prompt Sentence Examples

[0433] Use smart glasses to take a photo of the items the user is wearing and suggest a casual office outfit based on that image. Also, detect missing items and provide a link to purchase them.

[0434] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0435] Step 1: User Registration

[0436] A user installs the app on their device and enters the information required to create an account (such as name, email address, and password). The device sends this information to the server, which stores it in a database and creates a user account.

[0437] Step 2: Enter item data

[0438] Users take a photo of their fashion items using the camera function of their smart glasses or device. The device then uploads the captured image data to the app and sends it to the server. After receiving the image data, the server analyzes the image using Google Cloud Vision API or Amazon Rekognition to generate classification information such as the item's category, color, and material. This classification information is then stored in a database as item data.

[0439] Step 3: Enter coordination conditions

[0440] The user inputs the conditions necessary for coordinating outfits (e.g., season, occasion, style, etc.) through the application. The device sends this coordination condition data to the server. The server begins searching for the user's item data based on the received coordination conditions.

[0441] Step 4: Generate outfit suggestions

[0442] The server generates the optimal outfit based on the outfit conditions and item data using a deep learning model (e.g., TensorFlow, PyTorch). The generated outfit is output as image data and applied to the user's avatar. The result is sent to the device.

[0443] Step 5: View outfit suggestions

[0444] The device then displays the received outfit suggestions on the screen of the smart glasses or head-mounted display, allowing the user to visually check the outfits.

[0445] Step 6: Detect missing items and make purchasing recommendations

[0446] The server compares the proposed coordinates with a database to detect missing items. For the detected missing items, the server generates purchase recommendation data (product information, purchase links, etc.) using an online shopping API. This purchase recommendation data is sent to the device.

[0447] Step 7: Checkout

[0448] The device displays the recommended purchase data to the user through smart glasses or a head-mounted display, and the user can click on the provided link to go to an online shopping site and complete the purchase process for the desired item.

[0449] Step 8: Push notifications and location suggestions

[0450] The server collects information about places and events that the user may be interested in based on the coordination suggestions, generates this information as a push notification, and sends it to the device. The device then presents the push notification to the user, allowing the user to check the details.

[0451] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0452] The present invention is directed to a virtual stylist system incorporating an emotion engine. The following describes the specific processing steps for suggesting coordination and lifestyle based on the fashion items owned by the user, and for making suggestions according to the user's emotional state.

[0453] 1. Initial Setup and User Registration

[0454] user:

[0455] Download and install the app on your smartphone or other device.

[0456] Launch the app and enter your basic information (name, email address, password) on the new account creation screen.

[0457] Customize your profile information and avatar.

[0458] Device:

[0459] The entered information is sent to the server.

[0460] server:

[0461] The received user information is stored in a database and a user account is created.

[0462] 2. Registering fashion items

[0463] user:

[0464] Take a photo of a fashion item you own with your smartphone camera, such as a sweater, shirt, or jeans.

[0465] Register the captured image in the app.

[0466] Device:

[0467] Sends the uploaded image to the server.

[0468] server:

[0469] The received image data is analyzed and automatically tagged with classification information such as the item's category, color, and material.

[0470] The tagged item data is stored in a database in association with the user's account information.

[0471] 3. Coordination suggestions

[0472] user:

[0473] Enter the conditions for which you would like to receive outfit suggestions (season, occasion, style, etc.) into the app.

[0474] Device:

[0475] The entered condition data is sent to the server.

[0476] server:

[0477] The system selects registered fashion items that meet the conditions and runs an algorithm to generate coordination suggestions.

[0478] server:

[0479] The emotion engine analyzes the user's emotions and adjusts the coordination suggestions based on the results.

[0480] server:

[0481] In order to visually display the generated coordination proposal, image data to be applied to an avatar is generated.

[0482] Device:

[0483] The completed coordination proposal is displayed to the user.

[0484] user:

[0485] Review the proposed coordination and provide feedback as needed.

[0486] 4. Suggestion and purchase of missing items

[0487] server:

[0488] Based on the suggested outfit, it will detect missing items. For example, if the user does not own "leather shoes," it will detect this.

[0489] Generate purchasing recommendation data (product images, brand, price, purchase link, etc.).

[0490] Device:

[0491] Display a list of missing items to the user with a link to purchase them.

[0492] user:

[0493] Click on the purchase link and complete the purchase process as needed.

[0494] 5. Collaborative projects and promotions

[0495] server:

[0496] We regularly collect information on collaborations with popular fashion magazines and famous stylists.

[0497] Set collaboration information as a push notification and generate data to notify the user.

[0498] Device:

[0499] Receive push notifications and display them to the user.

[0500] user:

[0501] Check your push notifications to view collaborations and promotions that interest you.

[0502] 6. Recycling suggestions for unwanted items

[0503] server:

[0504] Analyzes the user's item database to detect items that have not been used for a long time.

[0505] Notification data is generated that suggests recycling or using unwanted items at a flea market.

[0506] Device:

[0507] Show users a suggestion notification to guide them on how to recycle unwanted items.

[0508] user:

[0509] Click the link to begin recycling your unwanted items.

[0510] 7. Suggestions for places to go

[0511] server:

[0512] Based on the suggested outfits, we collect information on events, travel destinations, restaurants, etc. that may interest the user.

[0513] The proposed information and coordinated images are integrated to generate data to be provided to the user.

[0514] Device:

[0515] Showing users suggested places and events.

[0516] user:

[0517] Check the event information that interests you and find out more details.

[0518] 8. Support for style and etiquette for ceremonial occasions

[0519] server:

[0520] Maintain a database of information on style and general etiquette related to weddings, funerals, and other ceremonies.

[0521] Generate data that provides appropriate style and manner information in response to user requests.

[0522] Device:

[0523] The suggested style and manner information is displayed to the user, and a screen showing the style and manner information applied to the user's avatar is displayed.

[0524] user:

[0525] Check the suggested style and etiquette information to alleviate any concerns.

[0526] Specific examples using emotion engines

[0527] user:

[0528] In the app, users set the conditions of "commute," "autumn," and "business casual" and request outfit suggestions. The emotion engine then uses facial recognition to read the user's current emotions (e.g., nervousness, joy, fatigue).

[0529] server:

[0530] Items that meet the conditions are selected from the registered fashion items, and coordination suggestions are generated.

[0531] Based on the user's emotions recognized by the emotion engine, for example, if the user is feeling nervous, the system will suggest outfits that include items in relaxing colors and materials.

[0532] Device:

[0533] The coordinated suggestions applied to the user's avatar are visually displayed.

[0534] user:

[0535] The user checks the suggested outfits, selects them if satisfied, and then carries out the purchasing process for any missing items.

[0536] The system of the present invention allows users to effectively utilize their own fashion items and receive suggestions for appropriate coordination and event information according to their emotional state, thereby providing comprehensive support for users' fashion-related concerns and improving user satisfaction.

[0537] The processing flow will be explained below.

[0538] Specific processing flow of the virtual stylist system using the emotion engine

[0539] 1. Initial Setup and User Registration

[0540] Step 1:

[0541] User: Downloads and installs the app on a smartphone or other device.

[0542] Step 2:

[0543] User: Launch the app and enter basic information (name, email address, password) on the new account creation screen.

[0544] Step 3:

[0545] Terminal: Sends the entered information to the server.

[0546] Step 4:

[0547] Server: Stores the received user information in a database and creates a user account.

[0548] Step 5:

[0549] Users: Customize your profile information and avatar.

[0550] 2. Registering fashion items

[0551] Step 1:

[0552] User: Take a photo of a fashion item they own with their smartphone camera, such as a sweater, shirt, or jeans.

[0553] Step 2:

[0554] User: Register the captured image within the app.

[0555] Step 3:

[0556] On the device: Sends the uploaded image to the server.

[0557] Step 4:

[0558] Server: Analyzes the received image data and automatically tags it with classification information such as item category, color, and material.

[0559] Step 5:

[0560] Server: Stores tagged item data in a database, associating it with user account information.

[0561] 3. Coordination suggestions

[0562] Step 1:

[0563] User: Enter the conditions for which you would like the app to suggest outfits (season, occasion, style, etc.).

[0564] Step 2:

[0565] Device: Facial recognition is performed using the smartphone camera to analyze the user's emotional state along with their input criteria in the emotion engine.

[0566] Step 3:

[0567] Terminal: Sends the emotion data and coordination conditions obtained through the emotion engine to the server.

[0568] Step 4:

[0569] Server: Selects registered fashion items that match the criteria and runs an algorithm to generate coordination suggestions.

[0570] Step 5:

[0571] Server: Adjusts outfit suggestions based on the user's emotions recognized by the emotion engine. For example, if the user is feeling nervous, the server suggests outfits that include items with relaxing colors and materials.

[0572] Step 6:

[0573] Server: Generates image data to be applied to an avatar to visually display the generated coordination proposal.

[0574] Step 7:

[0575] Device: Displays the completed outfit suggestions to the user.

[0576] Step 8:

[0577] User: Review the proposed coordination and provide feedback if necessary.

[0578] 4. Suggestion and purchase of missing items

[0579] Step 1:

[0580] Server: Detects missing items based on the proposed outfit. For example, if the user does not have "leather shoes," it will detect this.

[0581] Step 2:

[0582] Server: Generates purchasing recommendation data (product images, brand, price, purchase link, etc.).

[0583] Step 3:

[0584] On the device: Display a list of missing items to the user with a link to purchase them.

[0585] Step 4:

[0586] User: Clicks on the purchase link if necessary and completes the purchase.

[0587] 5. Collaborative projects and promotions

[0588] Step 1:

[0589] Server: Regularly collects information on collaborations with popular fashion magazines and famous stylists.

[0590] Step 2:

[0591] Server: Sets collaboration information as a push notification and generates data to notify the user.

[0592] Step 3:

[0593] Device: Receives the push notification and displays it to the user.

[0594] Step 4:

[0595] User: Check push notifications to view collaborations and promotions that interest them.

[0596] 6. Recycling suggestions for unwanted items

[0597] Step 1:

[0598] Server: Analyzes the user's item database and detects items that have not been used for a long time.

[0599] Step 2:

[0600] Server: Generates notification data suggesting recycling or selling unwanted items at flea markets.

[0601] Step 3:

[0602] On the device: Show users a suggestion notification to guide them on how to recycle unwanted items.

[0603] Step 4:

[0604] User: Clicks on a link to recycle unwanted items.

[0605] 7. Suggestions for places to go

[0606] Step 1:

[0607] Server: Based on the suggested outfits, collects information on events, travel destinations, restaurants, etc. that the user may be interested in.

[0608] Step 2:

[0609] Server: Integrates the recommendation information and coordinated images to generate data to provide to users.

[0610] Step 3:

[0611] On your device: Show suggested places and events to the user.

[0612] Step 4:

[0613] Users: View events that interest them and learn more.

[0614] 8. Support for style and etiquette for ceremonial occasions

[0615] Step 1:

[0616] Server: Manages a database of information on style and general etiquette related to ceremonial occasions.

[0617] Step 2:

[0618] Server: Generates data to provide appropriate style and manner information in response to user requests.

[0619] Step 3:

[0620] On the device: Show suggested styles and mannerisms to the user and display the results on the user's avatar.

[0621] Step 4:

[0622] Users: Review suggested style and etiquette information to ease concerns.

[0623] Specific examples using emotion engines

[0624] Step 1:

[0625] User: The app sets the conditions "Commute," "Autumn," and "Business Casual," and uses the smartphone camera to recognize the user's face and analyze their emotional state.

[0626] Step 2:

[0627] Device: Sends the facial recognition results to the emotion engine to analyze the current emotion (e.g., tension, joy, fatigue).

[0628] Step 3:

[0629] Terminal: Sends analyzed emotion data to the server.

[0630] Step 4:

[0631] Server: Searches the fashion item database and selects items that match the conditions.

[0632] Step 5:

[0633] Server: Adjusts outfit suggestions based on the results of the emotion engine. For example, if the customer is feeling nervous, the server will suggest items with relaxing materials and colors.

[0634] Step 6:

[0635] Server: Visualizes the coordinated outfit suggestions after adjustments and generates image data to apply to the avatar.

[0636] Step 7:

[0637] On device: Display coordinated outfit suggestions to the user.

[0638] Step 8:

[0639] User: Check the suggested outfit and, if satisfied, click the link to purchase the missing items and purchase the items they need.

[0640] The system of the present invention comprehensively supports users' fashion concerns by making effective use of their fashion items and providing appropriate coordination suggestions according to their emotional state.

[0641] Example 2

[0642] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0643] Conventional fashion coordination systems have problems with low user satisfaction because they do not make full use of the clothes a user owns and do not take into account the user's emotional state when making suggestions. It is also difficult for users to grasp what items they are lacking, making it difficult to effectively purchase the items they need. Furthermore, they lack support for overall lifestyles, such as suggesting places to go or event information.

[0644] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for photographing clothes owned by the user and inputting item information, means for analyzing the item information and assigning classification data, means for inputting coordination conditions from the user, means for searching the user's item information based on the coordination conditions and generating coordination suggestions, means for visually displaying the coordination suggestions, means for detecting missing items and generating purchase recommendation data, and means for recognizing the user's emotional state and adjusting the coordination suggestions based on the results. This allows the user to more effectively utilize their own clothes and receive support for coordination according to their emotional state and overall lifestyle.

[0645] "User" refers to someone who uses this system to register fashion items and receive coordination suggestions.

[0646] "Clothing" refers to fashion items that users own, photograph, and enter.

[0647] "Item information" is data about clothing, and includes detailed information such as category, color, and material.

[0648] "Classification data" refers to classification information such as category, color, material, etc. obtained by analyzing item information.

[0649] "Coordination" refers to clothing combinations suggested based on the user's item information and specific conditions.

[0650] "Conditions" are elements that are taken into consideration when creating a coordination, and include the season, the scene, the style, and the like.

[0651] "Coordination suggestions" refer to outfit combinations generated by the server based on conditions entered by the user.

[0652] The "means for visually displaying" refers to a function for visually displaying the generated coordination proposal so that the user can confirm it.

[0653] "Purchase recommendation data" refers to information (product images, brand, price, purchase link, etc.) that detects missing items and encourages their purchase.

[0654] "Emotional state" refers to the user's current emotions (e.g., tension, joy, fatigue), and is a factor reflected in outfit suggestions.

[0655] An "avatar" refers to a digital character that appears as a user's alter ego and is used to visually confirm coordination suggestions.

[0656] The present invention relates to a virtual stylist system incorporating an emotion engine. This system proposes coordination and lifestyle ideas based on the clothes owned by the user, and also makes suggestions according to the user's emotional state.

[0657] Overall system configuration

[0658] The system is implemented in the following way.

[0659] 1. The user takes a photo of the clothing they own and enters the item information.

[0660] 2. Analyze the entered item information and assign classification data.

[0661] 3. Enter the coordination requirements from the user.

[0662] 4. Based on the coordination conditions, search for the user's item information and generate coordination suggestions.

[0663] 5. Visually display coordination suggestions.

[0664] 6. Detect missing items and generate purchasing recommendations.

[0665] 7. Based on the suggested outfit, the app will suggest places for the user to go and event information.

[0666] 8. Apply the suggested outfit to the user's avatar and visually display it.

[0667] 9. Recognize the user's emotional state and tailor outfit suggestions accordingly.

[0668] Initial Setup and User Registration

[0669] user:

[0670] Users download and install the app on their smartphone, launch it, and enter their name, email address, and password on the new account creation screen, then customize their profile information and avatar.

[0671] Device:

[0672] The terminal sends the entered information to the server via an "HTTP POST request."

[0673] server:

[0674] The server saves the received user information in a MySQL database and creates a user account. The server performs validation to ensure there are no duplicate email addresses.

[0675] Fashion item registration

[0676] user:

[0677] Users take a photo of their clothing with their smartphone camera and register the image in the app, along with the item name, category, and tags (e.g., color, material).

[0678] Device:

[0679] The device uploads the captured image to cloud storage (e.g., AWS S3), generates a URL for the image, and sends it to the server.

[0680] server:

[0681] The server analyzes the received image URL using the Google Cloud Vision API, extracts classification information such as the item's category, color, and material, associates it with the user's account, and stores it in a database.

[0682] Coordination suggestions

[0683] user:

[0684] Users can request outfit suggestions by inputting conditions such as season, occasion, and style, and the system also uses facial recognition to detect the user's emotions.

[0685] Device:

[0686] The terminal sends the entered condition data to the server via an "HTTP POST request."

[0687] server:

[0688] The server selects items that match the criteria from the registered item information and generates coordination suggestions. It also analyzes the user's emotions using an emotion engine (e.g., Azure Emotion API) and adjusts the coordination suggestions based on the results.

[0689] server:

[0690] In order to visually display the generated coordination proposal, image data to be applied to the user's avatar is generated.

[0691] Device:

[0692] The terminal displays the completed coordination proposal to the user.

[0693] Suggestion and purchase of missing items

[0694] server:

[0695] The server detects missing items based on the suggested outfits and generates purchasing recommendation data (product images, brand, price, purchase links, etc.).

[0696] Device:

[0697] The device will display a list of missing items to the user along with a link to purchase them.

[0698] user:

[0699] The user clicks on the purchase link as needed and completes the purchase process.

[0700] Collaborative projects and promotions

[0701] server:

[0702] The server collects collaboration information with popular media and experts, updates it periodically, and generates data to notify users of the collaboration information as push notifications.

[0703] Device:

[0704] The device receives the push notification and displays it to the user.

[0705] user:

[0706] Users can check push notifications and view collaboration projects and promotional information that interest them.

[0707] Recycling suggestions for unwanted items

[0708] server:

[0709] The server analyzes the user's item information, detects items that have not been used for a long time, and generates notification data for recycling suggestions.

[0710] Device:

[0711] The device will display a recycling suggestion notification to the user and guide them on how to recycle.

[0712] user:

[0713] The user clicks on the suggested link to complete the recycling process.

[0714] Suggestions for places to go

[0715] server:

[0716] Based on the user's coordination suggestions, the server collects information on events, travel destinations, restaurants, etc. that may be of interest to the user and generates proposal data.

[0717] Device:

[0718] The device will display suggested locations and event information to the user.

[0719] user:

[0720] Users can check the event information of their interest and check the details.

[0721] Specific examples using emotion engines

[0722] user:

[0723] Users can set the conditions of "commute," "autumn," and "business casual" and request outfit suggestions. At the same time, the emotion engine uses facial recognition to read the user's current emotions (e.g., nervousness, joy, fatigue).

[0724] server:

[0725] The server selects registered fashion items that match the user's criteria and generates outfit suggestions. For example, if the user is feeling nervous, the server will suggest outfits that include items in relaxing colors and materials based on the user's emotions as recognized by the emotion engine.

[0726] Device:

[0727] The terminal visually displays the coordination suggestions applied to the user's avatar.

[0728] user:

[0729] The user checks the suggested outfits, selects them if satisfied, and then proceeds to purchase any missing items.

[0730] Specific examples and prompts for the generative AI model

[0731] Examples:

[0732] "I'm looking for some fall clothes to wear to work. I'd prefer business casual, but I'm a little nervous."

[0733] Prompt for the generative AI model:

[0734] "Users are looking for fall clothing for work, so suggest business casual outfits. Also, users are feeling stressed, so create outfits that include relaxing colors and materials."

[0735] The system of the present invention allows users to effectively utilize their clothing and receive suggestions for outfits and event information that match their emotional state, thereby providing comprehensive support for fashion-related concerns and improving user satisfaction.

[0736] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0737] Step 1: User registration and initial setup

[0738] user:

[0739] Download and install the app on your smartphone.

[0740] Launch the app, enter your name, email address, and password on the new account creation screen, and press the "Submit" button.

[0741] Input: Name, Email Address, Password

[0742] Output: Registration request data

[0743] Device:

[0744] The registration request data is sent to the server via an HTTP POST request.

[0745] Input: Registration request data

[0746] Output: Request to server

[0747] server:

[0748] The received registration request data undergoes validation checks (such as checking for duplicate accounts), is saved in a MySQL database, and a user account is created.

[0749] Input: Registration request data, database

[0750] Output: New user account

[0751] Step 2: Register your fashion items

[0752] user:

[0753] Use your smartphone camera to take a photo of your clothing.

[0754] Select the image you took within the app, enter the item name, category, and tags (color, material, etc.), and register the image.

[0755] Input: Clothing photo, item name, category, tags (color, material, etc.)

[0756] Output: Registration request data

[0757] Device:

[0758] The captured image is uploaded to cloud storage (e.g. AWS S3) and a URL for the image is generated.

[0759] The tagged information along with the image URL is sent to the server via an HTTP POST request.

[0760] Input: Clothing photo, item name, category, tags (color, material, etc.)

[0761] Output: Request to server

[0762] server:

[0763] The received image URL is analyzed using the Google Cloud Vision API, and classification information such as the item's category, color, and material is extracted, associated with the user's account, and stored in a database.

[0764] Input: Image URL, tagging information

[0765] Output: Classification data, database user information

[0766] Step 3: Enter coordination conditions

[0767] user:

[0768] Enter the conditions for which you would like the app to suggest outfits (season, occasion, style, etc.).

[0769] Use facial recognition to detect emotions.

[0770] Input: Coordination condition data, face recognition data

[0771] Output: Coordinated condition request

[0772] Device:

[0773] The entered condition data and facial recognition data are sent to the server via an "HTTP POST request."

[0774] Input: Coordination condition data, face recognition data

[0775] Output: Request to server

[0776] Step 4: Generate outfit suggestions

[0777] server:

[0778] From the registered fashion items, items that match the inputted coordination conditions are selected.

[0779] An emotion engine (e.g., Azure Emotion API) is used to analyze the user's emotions and tailor outfit suggestions based on the results.

[0780] Input: Item information, coordination conditions, emotion data

[0781] Output: Coordination suggestion data

[0782] Step 5: View outfit suggestions

[0783] server:

[0784] In order to visually display the generated coordination proposal, image data to be applied to the user's avatar is generated.

[0785] Input: Coordination suggestion data

[0786] Output: Coordinate image data

[0787] Device:

[0788] Coordinate image data is received and displayed to the user.

[0789] Input: Coordinate image data

[0790] Output: what is displayed to the user

[0791] user:

[0792] Review the proposed coordination and provide feedback as needed.

[0793] Input: Coordination image

[0794] Output: Feedback data

[0795] Step 6: Suggest and purchase missing items

[0796] server:

[0797] Detect missing items based on suggested coordinates.

[0798] Generate purchasing recommendation data (product images, brand, price, purchase link, etc.) for items in short supply.

[0799] Input: Coordination suggestions, user item data

[0800] Output: Purchase recommendation data

[0801] Device:

[0802] Display a list of missing items to the user with a link to purchase them.

[0803] Input: Purchase recommendation data

[0804] Output: what is displayed to the user

[0805] user:

[0806] Click on the purchase link and complete the purchase process as needed.

[0807] Enter: Purchase Link

[0808] Output: Purchase confirmation data

[0809] Step 7: Suggest places to go

[0810] server:

[0811] Based on the suggested outfits, we collect information on events, travel destinations, restaurants, etc. that may interest the user.

[0812] Input: Coordination suggestions

[0813] Output: Event proposal data

[0814] Device:

[0815] Showing users suggested places and events.

[0816] Input: Event proposal data

[0817] Output: what is displayed to the user

[0818] user:

[0819] Check the event information that interests you and find out more details.

[0820] Input: Event information

[0821] Output: Event details display

[0822] Step 8: Recognizing and Coordinating Emotional States

[0823] user:

[0824] Using facial recognition, the emotion engine reads the user's current emotions (e.g., nervousness, joy, fatigue).

[0825] Input: Facial recognition data

[0826] Output: Emotion data

[0827] server:

[0828] Coordination suggestions are adjusted based on emotional data recognized by the emotion engine.

[0829] Input: Emotion data

[0830] Output: Adjusted outfit suggestions

[0831] Device:

[0832] The adjusted coordination suggestions are applied to the user's avatar and visually displayed.

[0833] Input: Adjusted outfit suggestions

[0834] Output: Adjusted coordinate image

[0835] user:

[0836] Check the suggested outfits, purchase the items you need, and provide feedback.

[0837] Input: Adjusted coordinate image

[0838] Output: Purchase confirmation data, feedback data

[0839] (Application example 2)

[0840] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0841] Conventional fashion coordination systems simply make suggestions based on the user's owned fashion items and basic requirements, without considering the user's emotional state. As a result, they are unable to make suggestions that match the user's current emotions and mood, making it difficult to increase satisfaction. Furthermore, they lack an efficient means to encourage the purchase of missing items in the suggested coordination.

[0842] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for photographing fashion items owned by the user and inputting item data, a means for analyzing the item data and assigning classification information, a means for accepting coordination conditions input from the user, a means for searching the user's item data based on the coordination conditions and generating coordination suggestions, a means for recognizing the user's emotions and adjusting the coordination suggestions based on the emotions, and a means for applying the suggested coordination to the user's avatar and visually displaying it. This makes it possible to propose optimal fashion coordination that takes emotions into consideration. Furthermore, the server can also improve the user's purchasing experience by detecting missing items and generating purchase recommendation data.

[0843] "Fashion items" is a general term for fashion-related possessions such as clothing, accessories, and shoes.

[0844] "Item data" refers to attribute information such as images, categories, colors, and materials of fashion items owned by a user.

[0845] "Coordination conditions" are requirements such as season, occasion, style, etc. that are input by the user when requesting suggestions.

[0846] A "coordination suggestion" is a specific fashion combination generated based on coordination conditions.

[0847] "Emotion" is a general term for a user's current psychological state or mood (e.g., joy, tension, fatigue, etc.).

[0848] An "avatar" is a digital persona or character that virtually represents a user.

[0849] "Facial recognition" is a technology that analyzes a user's facial image and reads emotions from their facial expressions.

[0850] "Purchase recommendation data" is information such as the brand, price, and purchase link of missing items based on the suggested outfit.

[0851] This embodiment of the present invention describes a system for suggesting outfits based on data on fashion items owned by a user. This system proposes outfits taking into account the emotional state of the user and recommends the purchase of missing items as needed, thereby increasing user satisfaction.

[0852] Initial Setup and User Registration

[0853] Users download a dedicated application onto their smartphone or smart glasses and create a new account. Once the application is launched, users enter basic information (name, email address, password) and customize their profile information and avatar.

[0854] Fashion item registration

[0855] Users take photos of their own fashion items, such as sweaters, shirts, and jeans. The images are then uploaded to the server by the application. The server analyzes the image data and automatically tags the items with classification information such as category, color, and material. This classification information is then associated with the user's account information and stored in a database.

[0856] Coordination suggestions

[0857] When a user wants to receive outfit suggestions, they input the season, occasion, style, and other conditions into the application. Furthermore, the smart glasses' built-in facial recognition function is used to read the user's emotions. The emotion engine analyzes the user's current emotions (e.g., nervousness, joy, fatigue, etc.).

[0858] The server proposes optimal outfits from registered fashion items based on the user's conditions and emotional state. The proposed outfits are applied to the user's avatar and visually displayed on the smart glasses display. The user can review the proposed outfits and provide feedback if necessary.

[0859] Suggestion and purchase of missing items

[0860] The server detects missing items based on the suggested outfit. For example, if the user does not own "leather shoes," it will detect this. It then generates purchasing recommendation data for the missing items (product images, brand, price, purchasing links, etc.). The terminal displays a list of missing items and purchase links to the user. The user can click the purchase link and complete the purchase process as needed.

[0861] Emotion recognition and concrete examples

[0862] The application uses the smart glasses' camera to capture the user's face and analyzes their emotions using an emotion engine. For example, if a user sets the conditions "commute," "autumn," and "business casual," and the emotion engine detects "tension" through facial recognition, it will suggest outfits that include items with relaxing colors and materials.

[0863] In this case, an example of a prompt sentence input to the generative AI model is as follows:

[0864] The user sets the conditions of "commute," "autumn," and "business casual," and emotion recognition detects "tension." Based on these conditions, please suggest the best outfit for the user based on their fashion items.

[0865] In this way, the suggested coordination takes into account the user's emotional state and can improve user satisfaction.

[0866] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0867] Step 1:

[0868] Users download a dedicated application onto a device such as a smartphone or smart glasses and create a new account. The user enters basic information (name, email address, password) and customizes their profile information and avatar. The entered information is sent by the device to a server, which stores the received information in a database and creates a user account.

[0869] Step 2:

[0870] The user takes a photo of a fashion item they own using the smart glasses' camera. For example, a sweater, shirt, or jeans. The device then uploads the image to a server. The server analyzes the image data and automatically tags the item with classification information such as category, color, and material. This tagged information is then stored in a database along with the user's account information.

[0871] Step 3:

[0872] The user inputs the conditions for which they would like to receive outfit suggestions (season, occasion, style, etc.) into the application. The smart glasses' built-in facial recognition function is used to capture the user's face. The device then sends the conditions data entered by the user and the captured facial image data to the server. The server then uses an emotion engine to analyze the facial image and recognize the user's current emotion (e.g., nervousness, joy, fatigue, etc.).

[0873] Step 4:

[0874] The server generates outfit suggestions based on the user's conditions and emotional state. It selects registered fashion items that match the conditions and then runs an algorithm to suggest optimal outfits that take emotions into consideration. The outfit suggestions are generated and applied to the user's avatar. The generated outfit suggestions are sent to the device and visually displayed on the smart glasses' display.

[0875] Step 5:

[0876] Based on the suggested outfit, the server detects any missing items. For example, if the user does not own "leather shoes," the server detects those items. For each missing item, the server generates purchasing recommendation data (product image, brand, price, purchase link, etc.). The device displays a list of missing items and a purchase link to the user.

[0877] Step 6:

[0878] The user can click on the purchase link provided to complete the purchase of the missing item. Once the user completes the purchase, the information is updated by the server and stored in the database.

[0879] Step 7:

[0880] The application allows users to provide feedback on the proposed coordination. The user enters the feedback and the device sends it to the server. The server receives the feedback and uses it to improve the coordination algorithm and update personal data.

[0881] These steps enable the system to suggest optimal fashion coordination that takes into account the user's emotional state, improving the user's shopping experience.

[0882] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0883] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0884] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0885] [Second embodiment]

[0886] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0887] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0888] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0889] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0890] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0891] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0892] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0893] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0894] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0895] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0896] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0897] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0898] Specific processes and operations of the virtual stylist system of the present invention will be described below. Note that in the following description, the server, the terminal, and the user are used as the subject.

[0899] 1. Initial Setup and User Registration

[0900] user:

[0901] Users download and install the app on their smartphone or other device.

[0902] When the app is launched, an account creation screen appears, where users enter basic information such as their name, email address, and password to create an account.

[0903] Users then customize their profile information and avatar.

[0904] Device:

[0905] The entered information is sent to the server.

[0906] server:

[0907] Store the received information in a database and create a user account.

[0908] 2. Registering fashion items

[0909] user:

[0910] Take a photo of a fashion item you own with your smartphone camera, such as a sweater, shirt, or jeans.

[0911] Upload the captured images through the app's interface.

[0912] Device:

[0913] Sends the uploaded image to the server.

[0914] server:

[0915] The received image data is analyzed and automatically tagged with classification information such as the item's category, color, and material.

[0916] Stores item data in a database, associated with the user's account information.

[0917] 3. Coordination suggestions

[0918] user:

[0919] Enter the conditions for which you would like to receive outfit suggestions (season, occasion, style, etc.) into the app.

[0920] Device:

[0921] The entered condition data is sent to the server.

[0922] server:

[0923] The system selects registered fashion items that meet the conditions and runs an algorithm to generate coordination suggestions.

[0924] In order to visually display the generated coordination proposal, image data to be applied to an avatar is generated.

[0925] Device:

[0926] The completed coordination proposal is displayed to the user.

[0927] user:

[0928] Review the proposed coordination and provide feedback as needed.

[0929] 4. Suggestion and purchase of missing items

[0930] server:

[0931] Based on the suggested outfit, it will detect missing items. For example, if the user does not own "leather shoes," it will detect this.

[0932] Generates purchasing recommendation data and provides purchasing information (product images, brand, price, purchase link, etc.).

[0933] Device:

[0934] Display a list of missing items to the user with a link to purchase them.

[0935] user:

[0936] Click on the purchase link and complete the purchase process as needed.

[0937] 5. Collaborative projects and promotions

[0938] server:

[0939] We regularly collect information on collaborations with popular fashion magazines and famous stylists.

[0940] Set collaboration information as a push notification and generate data to notify the user.

[0941] Device:

[0942] Receive push notifications and display them to the user.

[0943] user:

[0944] Check your push notifications to view collaborations and promotions that interest you.

[0945] 6. Recycling suggestions for unwanted items

[0946] server:

[0947] Analyzes the user's item database to detect items that have not been used for a long time.

[0948] For unwanted items, suggestion notification data is generated recommending recycling or using them at a flea market.

[0949] Device:

[0950] Show users a suggestion notification to guide them on how to recycle unwanted items.

[0951] user:

[0952] Click the link to begin recycling your unwanted items.

[0953] 7. Suggestions for places to go

[0954] server:

[0955] Based on the suggested outfits, we collect information on events, travel destinations, restaurants, etc. that may interest the user.

[0956] The proposed information and coordinated images are integrated to generate data to be provided to the user.

[0957] Device:

[0958] Showing users suggested places and events.

[0959] user:

[0960] Check the event information that interests you and find out more details.

[0961] 8. Support for style and etiquette for ceremonial occasions

[0962] server:

[0963] Maintain a database of information on style and general etiquette related to weddings, funerals, and other ceremonies.

[0964] Generate data that provides appropriate style and manner information in response to user requests.

[0965] Device:

[0966] The suggested style and manner information is displayed to the user, and a screen showing the style applied to the user's avatar is displayed.

[0967] user:

[0968] Check the suggested style and etiquette information to alleviate any concerns.

[0969] Specific examples

[0970] user:

[0971] In the app, set the conditions "commute," "autumn," and "business casual" and request outfit suggestions.

[0972] Check the outfit suggested by the server: gray sweater, white shirt, and dark blue jeans.

[0973] Check the proposal and give feedback such as "Leather shoes are needed."

[0974] server:

[0975] Based on the suggested outfit, it detects that "leather shoes" are in short supply and generates purchasing information.

[0976] Device:

[0977] Present the user with a purchase link and provide them with a link to complete the purchase.

[0978] user:

[0979] Click on the provided link and complete the process to purchase the "leather shoes."

[0980] The system of the present invention allows users to effectively utilize their fashion items and eliminates the anxiety of coordinating outfits for everyday wear and special events.

[0981] The processing flow will be explained below.

[0982] Program processing flow

[0983] 1. Initial Setup and User Registration

[0984] Step 1:

[0985] User: Downloads and installs the app on a smartphone or other device.

[0986] Step 2:

[0987] User: Launch the app and enter basic information (name, email address, password) on the new account creation screen.

[0988] Step 3:

[0989] Terminal: Sends the entered information to the server.

[0990] Step 4:

[0991] Server: Stores the received user information in a database and creates a user account.

[0992] Step 5:

[0993] Users: Customize your profile information and avatar.

[0994] 2. Registering fashion items

[0995] Step 1:

[0996] User: Take a photo of a fashion item they own with their smartphone camera, such as a sweater, shirt, or jeans.

[0997] Step 2:

[0998] User: Register the captured image within the app.

[0999] Step 3:

[1000] On the device: Sends the uploaded image to the server.

[1001] Step 4:

[1002] Server: Analyzes the received image data and automatically tags it with classification information such as item category, color, and material.

[1003] Step 5:

[1004] Server: Stores tagged item data in a database, associating it with user account information.

[1005] 3. Coordination suggestions

[1006] Step 1:

[1007] User: Enter the conditions for which you would like the app to suggest outfits (season, occasion, style, etc.).

[1008] Step 2:

[1009] Terminal: Sends the entered condition data to the server.

[1010] Step 3:

[1011] Server: Selects registered fashion items that match the criteria and runs an algorithm to generate coordination suggestions.

[1012] Step 4:

[1013] Server: Generates image data to be applied to an avatar to visually display the generated coordination proposal.

[1014] Step 5:

[1015] Device: Displays the completed outfit suggestions to the user.

[1016] Step 6:

[1017] User: Review the proposed outfits and provide feedback.

[1018] 4. Suggestion and purchase of missing items

[1019] Step 1:

[1020] Server: Detects missing items based on the proposed outfit. For example, if the user does not have "leather shoes," it will detect this.

[1021] Step 2:

[1022] Server: Generates purchasing recommendation data (product images, brand, price, purchase link, etc.).

[1023] Step 3:

[1024] On the device: Display a list of missing items to the user with a link to purchase them.

[1025] Step 4:

[1026] User: Clicks on the purchase link if necessary and completes the purchase.

[1027] 5. Collaborative projects and promotions

[1028] Step 1:

[1029] Server: Regularly collects information on collaborations with popular fashion magazines and famous stylists.

[1030] Step 2:

[1031] Server: Sets collaboration information as a push notification and generates data to notify the user.

[1032] Step 3:

[1033] Device: Receives the push notification and displays it to the user.

[1034] Step 4:

[1035] User: Check push notifications to view collaborations and promotions that interest them.

[1036] 6. Recycling suggestions for unwanted items

[1037] Step 1:

[1038] Server: Analyzes the user's item database and detects items that have not been used for a long time.

[1039] Step 2:

[1040] Server: Generates notification data suggesting recycling or selling unwanted items at flea markets.

[1041] Step 3:

[1042] On the device: Show users a suggestion notification to guide them on how to recycle unwanted items.

[1043] Step 4:

[1044] User: Clicks on a link to recycle unwanted items.

[1045] 7. Suggestions for places to go

[1046] Step 1:

[1047] Server: Based on the suggested outfits, collects information on events, travel destinations, restaurants, etc. that the user may be interested in.

[1048] Step 2:

[1049] Server: Integrates the recommendation information and coordinated images to generate data to provide to users.

[1050] Step 3:

[1051] On your device: Show suggested places and events to the user.

[1052] Step 4:

[1053] Users: View events that interest them and learn more.

[1054] 8. Support for style and etiquette for ceremonial occasions

[1055] Step 1:

[1056] Server: Manages a database of information on style and general etiquette related to ceremonial occasions.

[1057] Step 2:

[1058] Server: Generates data to provide appropriate style and manner information in response to user requests.

[1059] Step 3:

[1060] On the device: Show suggested styles and mannerisms to the user and display the results on the user's avatar.

[1061] Step 4:

[1062] Users: Review suggested style and etiquette information to ease concerns.

[1063] This will enable us to provide comprehensive support to users regarding their fashion concerns.

[1064] Example 1

[1065] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1066] Today's consumers face daily fashion-related worries and difficulties in coordinating outfits. They particularly need a way to efficiently coordinate outfits for each season and occasion while making the most of the items they own. Furthermore, there is no comprehensive fashion management system that can suggest and purchase items they are running low on, provide information on collaborations with the fashion industry, or even recycle unwanted items. This often results in users wasting time and effort choosing their daily fashion, which can be stressful.

[1067] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1068] In this invention, the server includes means for photographing fashion items owned by the user and inputting item data, means for analyzing the item data and assigning classification information, means for accepting coordination conditions input from the user, means for searching the user's item data based on the coordination conditions and generating coordination suggestions, means for visually displaying the coordination suggestions, means for detecting missing items and generating purchase recommendation data, means for collecting collaboration information with specific fashion magazines and stylists and generating push notifications, means for analyzing the user's item database and detecting items that have not been used for a long time and suggesting recycling, and means for providing information on styles and general etiquette related to ceremonial occasions. This allows users to effectively utilize their own fashion items, reduces anxiety about coordinating outfits for everyday use and special events, and enables comprehensive fashion management.

[1069] "User" refers to any individual or organization that uses this system.

[1070] "Fashion items" refers to fashion-related items such as clothing, accessories, and shoes owned by a user.

[1071] "Item data" refers to data that includes information such as images, categories, colors, and materials of fashion items.

[1072] "Coordination conditions" refer to conditions such as season, scene, and style specified by the user.

[1073] "Coordination suggestions" refer to style suggestions generated based on the user's fashion items and coordination conditions.

[1074] "Purchase recommendation data" refers to data that identifies missing items and includes related purchasing information (such as product images, brand, price, and purchase links).

[1075] "Collaboration Information" refers to information provided in collaboration with specific fashion magazines or stylists.

[1076] "Push notification" refers to a notification sent from a server to a user.

[1077] "Recycling suggestions" refer to suggestions that encourage users to recycle or sell fashion items that they have not used for a long time.

[1078] "Style information" refers to information about fashion styles and general manners related to ceremonial occasions.

[1079] The specific processing and operation of the virtual stylist system of the present invention will be described below. The present invention manages the user's fashion items and suggests coordination suggestions by exchanging data between the user, the terminal, and the server.

[1080] Initial Setup and User Registration

[1081] User: First, the user downloads and installs a dedicated app onto their smartphone or other device. When the app is launched, an account creation screen appears. The user enters basic information such as name, email address, and password to create an account. They can then customize their profile information and avatar.

[1082] Terminal: This input information is sent from the terminal to the server.

[1083] Server: The server stores the received information in a database and creates a user account.

[1084] Fashion item registration

[1085] User: The user takes a photo of a fashion item they own, such as a sweater, shirt, or jeans, with their smartphone camera and uploads the image through the app's interface.

[1086] On your device: Uploaded images are sent to the server.

[1087] Server: The received image data is analyzed using services such as the Google Cloud Vision API. After analysis, classification information such as the item's category, color, and material is automatically tagged and stored in a database in association with the user's account information.

[1088] Coordination suggestions

[1089] User: The user inputs the conditions for which they would like to receive outfit suggestions (for example, season, occasion, style, etc.) into the app.

[1090] Terminal: The entered condition data is sent to the server.

[1091] Server: The server selects registered fashion items that match the input criteria and runs an algorithm to generate outfit suggestions. This algorithm also takes into account the user's past preferences and trend information to suggest optimal outfits. The generated outfit suggestions are prepared in a form that can be displayed visually, and image data to be applied to the user's avatar is generated.

[1092] Terminal: The completed outfit suggestions are displayed to the user, who can then review the suggestions and provide feedback if necessary.

[1093] Suggestion and purchase of missing items

[1094] Server: Detects missing items based on the suggested outfit. For example, if the user does not own "leather shoes," it detects this. To fill this gap, it generates purchasing recommendation data and provides detailed information such as product images, brand, price, and purchase links.

[1095] On the device: Display a list of missing items to the user with a link to purchase them.

[1096] User: Clicks on the purchase link if necessary and completes the purchase.

[1097] Collaborative projects and promotions

[1098] Server: Periodically collects information about collaborations with specific fashion magazines and famous stylists. The collected information is sent to users via push notifications.

[1099] Device: Receives the push notification and displays it to the user.

[1100] User: Check push notifications to view collaborations and promotions that interest them.

[1101] Recycling suggestions for unwanted items

[1102] Server: Analyzes the user's item database and detects items that have not been used for a long time. Based on this, it generates notification data recommending recycling or selling them at a flea market.

[1103] On the device: Show users a suggestion notification to guide them on how to recycle unwanted items.

[1104] User: Clicks on a link to recycle unwanted items.

[1105] Suggestions for places to go

[1106] Server: Based on the suggested outfits, the server collects information on events, travel destinations, restaurants, etc. that the user may be interested in. The server then integrates the suggested information with the outfit images to generate data to be provided to the user.

[1107] On your device: Show suggested places and events to the user.

[1108] Users: View events that interest them and learn more.

[1109] Style and etiquette support for ceremonial occasions

[1110] Server: Manages a database of information on style and general etiquette related to ceremonial occasions. Generates data to provide appropriate style and etiquette information in response to user requests.

[1111] On the device: Show suggested styles and mannerisms to the user, and show the styled screen on the user's avatar.

[1112] Users: Review suggested style and etiquette information to ease concerns.

[1113] Specific examples

[1114] As an example of a prompt, if the user sets the conditions "commute," "autumn," and "business casual" and requests outfit suggestions, the server will suggest an outfit of "gray sweater, white shirt, and dark blue denim." If the user provides feedback that "I need leather shoes," the server will detect the shortage and generate purchasing information, and the device will present the user with a purchase link.

[1115] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1116] Step 1:

[1117] User: Download and install the Virtual Stylist System app from the smartphone app store. Launch the app and proceed to the account creation screen. Here, enter information such as your name, email address, and password to create an account. The entered information is sent to your device.

[1118] Input: User's basic information (name, email address, password)

[1119] Output: User information is sent to the terminal

[1120] Step 2:

[1121] Terminal: Sends the account information entered by the user to the server.

[1122] Input: User information (name, email address, password)

[1123] Output: User information is sent to the server

[1124] Step 3:

[1125] Server: The received user information is saved in the database and a user account is created. This completes the user registration.

[1126] Input: User information sent from the device

[1127] Output: User information stored in database, account created

[1128] Step 4:

[1129] User: After creating an account, customize your profile information and avatar. Enter the customized information and send it to your device.

[1130] Input: Profile information, avatar setting information

[1131] Output: Profile information and avatar settings are sent to the device.

[1132] Step 5:

[1133] Device: Sends user-customized information to the server.

[1134] Input: Profile information, avatar setting information

[1135] Output: Customization information sent to the server

[1136] Step 6:

[1137] Server: Save the received customization information in the database. This completes the initial setup and user registration.

[1138] Input: Customization information sent from the device

[1139] Output: Customization information stored in the database

[1140] Step 7:

[1141] Users take photos of their fashion items with their smartphone camera and upload the images through the app's interface.

[1142] Input: Photographed fashion item images

[1143] Output: The image is captured to the device.

[1144] Step 8:

[1145] On the device: Sends the uploaded image to the server.

[1146] Input: Photographed fashion item images

[1147] Output: Image data sent to the server

[1148] Step 9:

[1149] Server: Analyzes the received image data using services such as Google Cloud Vision API to generate classification information such as item category, color, material, etc. The classification information is then tagged to the item data and stored in a database.

[1150] Input: Image data sent from the device

[1151] Output: Item data associated with classification information, item data stored in a database

[1152] Step 10:

[1153] User: Enter the conditions for which they would like to receive outfit suggestions (e.g., season, occasion, style, etc.) into the app and send the results to their device.

[1154] Input: Coordination conditions

[1155] Output: Coordinate conditions are captured in the terminal

[1156] Step 11:

[1157] Terminal: Sends the entered coordination condition data to the server.

[1158] Input: Coordination condition data

[1159] Output: Coordinate condition data sent to the server

[1160] Step 12:

[1161] Server: Runs an algorithm to select registered fashion items that match coordination criteria. Based on the selected items, it generates coordination suggestions and generates image data to be applied to the user's avatar.

[1162] Input: Registered fashion item data, coordination condition data

[1163] Output: Coordination suggestions and image data to be applied to the avatar

[1164] Step 13:

[1165] Device: Displays the completed outfit suggestions to the user.

[1166] Input: Coordination suggestions and image data sent from the server

[1167] Output: Outfit suggestions displayed on the user's device

[1168] Step 14:

[1169] User: Check the suggested outfit and provide feedback if necessary. For example, enter a comment such as "I need leather shoes" and send that information to the device.

[1170] Input: Feedback information (e.g., I need leather shoes)

[1171] Output: Feedback information captured in terminal

[1172] Step 15:

[1173] Device: Sends user feedback information to the server.

[1174] Input: Feedback information

[1175] Output: Feedback information sent to the server

[1176] Step 16:

[1177] Server: Based on the proposed outfit, detects missing items (e.g., leather shoes) and generates purchasing recommendations, including product images, brand, price, and purchase link details.

[1178] Input: User feedback information, suggested coordinate data

[1179] Output: Purchase recommendation data

[1180] Step 17:

[1181] Device: Display purchase recommendation data to the user and provide a purchase link.

[1182] Input: Purchase recommendation data sent from the server

[1183] Output: Purchase recommendation data displayed on the user's device

[1184] Step 18:

[1185] User: Clicks on the purchase link and completes the purchase.

[1186] Input: Purchase link click

[1187] Output: Completed checkout

[1188] Step 19:

[1189] Server: Periodically collects collaboration information with specific fashion magazines and stylists and generates push notifications.

[1190] Input: Information collected from fashion magazines and stylists

[1191] Output: Generates push notification data

[1192] Step 20:

[1193] Device: Receives the push notification and displays it to the user.

[1194] Input: Push notification data sent from the server

[1195] Output: A push notification that appears on the user's device

[1196] Step 21:

[1197] User: Check push notifications to view collaborations and promotions that interest them.

[1198] Input: Push notification click

[1199] Output: Collaboration and promotion information to be displayed

[1200] Step 22:

[1201] Server: Analyzes the user's item database and detects items that have not been used for a long time. Generates notification data suggesting recycling or selling items at a flea market.

[1202] Input: Item database

[1203] Output: Recycling suggestion notification data

[1204] Step 23:

[1205] Device: Show recycling suggestion notification to user.

[1206] Input: Recycle proposal notification data sent from the server

[1207] Output: A recycling suggestion notification displayed on the user's device.

[1208] Step 24:

[1209] User: Clicks on a link to recycle unwanted items.

[1210] Input: Click on recycle link

[1211] Output: Start of recycling procedure

[1212] Step 25:

[1213] Server: Based on the suggested outfits, collects information on events, travel destinations, and restaurants that the user may be interested in, and generates suggestion data.

[1214] Input: Proposed coordinate data

[1215] Output: Event and location suggestions

[1216] Step 26:

[1217] On your device: Show suggested places and events to the user.

[1218] Input: Proposal data sent from the server

[1219] Output: Event and location information displayed on the user's device

[1220] Step 27:

[1221] Users: View events that interest them and learn more.

[1222] Input: Click on event information

[1223] Output: Detailed information about the event displayed

[1224] Step 28:

[1225] Server: Manages a database of information on style and general etiquette related to ceremonial occasions, and provides data in response to user requests.

[1226] Input: User request

[1227] Output: Style and manner information provided

[1228] Step 29:

[1229] On the device: Show suggested styles and mannerisms to the user, and show the styled screen on the user's avatar.

[1230] Input: Style and manner information sent from the server

[1231] Output: Style and manner information displayed on the user's device

[1232] Step 30:

[1233] Users: Review suggested style and etiquette information to ease concerns.

[1234] Input: Check style and manner information

[1235] Output: User concerns resolved

[1236] In this way, users can effectively manage their fashion items through every step, eliminating the worry of choosing everyday fashion or coordinating outfits for special events.

[1237] (Application example 1)

[1238] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1239] Problems faced by users of conventional fashion coordination systems include the complicated process of suggesting outfits in real time and purchasing missing items, making it difficult to use on the go. Furthermore, there are limited ways for users to visually check the results of applying suggested outfits to their owned fashion items, limiting the user experience. The problem this invention aims to solve is to enable real-time suggestions and purchasing procedures, thereby improving user convenience and satisfaction.

[1240] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1241] In this invention, the server includes means for photographing fashion items owned by the user and inputting item data, means for analyzing the item data and assigning classification information, means for accepting coordination condition input from the user, means for searching the user's item data based on the coordination conditions and generating coordination suggestions, means for visually displaying the coordination suggestions, means for detecting missing items and generating purchase recommendation data, means for allowing the user to check the suggested coordination in real time and proceed with purchasing the missing items, and means for visually displaying the results via smart glasses or a head-mounted display, thereby enabling the user to check the coordination suggestions in real time and immediately proceed with purchasing the missing items.

[1242] "User" refers to an individual consumer who uses the system.

[1243] "Fashion items" refer to items that make up an individual's appearance, such as clothing, accessories, and shoes.

[1244] "Item data" refers to images of each fashion item owned by the user and their characteristic information.

[1245] "Classification information" refers to information that describes characteristics such as the category, color, material, and style of a fashion item.

[1246] "Coordination conditions" refer to the desired scene, season, style, and other conditions that the user enters into the app.

[1247] "Coordination suggestions" refer to fashion combinations generated based on the user's item data and coordination conditions.

[1248] "Visual display means" refers to the method by which visual information is presented to the user through an application or device.

[1249] "Missing items" refer to items that are necessary for the proposed outfit but that the user does not own.

[1250] "Purchase recommendation data" refers to product information and links provided to help users purchase items they are missing.

[1251] "Means for checking in real time" refers to a method in which suggested outfits are instantly presented to users, allowing them to provide feedback and complete the purchase process.

[1252] "Smart glasses" are a type of wearable device that displays information in the user's field of vision.

[1253] A "head-mounted display" is a display device that is worn on the user's head and has the function of providing visual information.

[1254] "On-the-go use" refers to a user using the system's functions outside of their home.

[1255] The following describes in detail the embodiments of the present invention: First, an overview of the entire system and specific examples of each processing step are shown.

[1256] System Configuration

[1257] This invention consists of a user, a terminal (smart glasses or a head-mounted display), and a server.

[1258] Hardware and Software

[1259] 1. Hardware:

[1260] Smart glasses (e.g., smart glasses, head-mounted displays)

[1261] Device (e.g. smartphone, tablet)

[1262] 2. Software:

[1263] Image analysis algorithms (e.g., OpenCV)

[1264] Deep learning models (e.g. TensorFlow, PyTorch)

[1265] Cloud services (e.g., Google Cloud Vision API, Amazon Rekognition)

[1266] Database (e.g. Firebase Cloud Firestore)

[1267] API (e.g., online shopping API)

[1268] Program processing description

[1269] The program for this system is designed to efficiently manage the fashion items owned by the user, provide real-time coordination suggestions, and assist in purchasing missing items. Each processing step and its operation are explained in detail below.

[1270] 1. Register user item data:

[1271] Users use their device's camera function to take photos of fashion items they own and upload the image data to the app.

[1272] The terminal transmits the image data received from the user to the server.

[1273] The server uses Google Cloud Vision API and Amazon Rekognition to analyze images and generate classification information such as item category, color, material, etc. This information is stored in Firebase Cloud Firestore.

[1274] 2. Coordination suggestions:

[1275] Users input coordination criteria (e.g., season, occasion, style) through the application.

[1276] The terminal transmits the input condition data to the server.

[1277] The server searches the user's item data and uses deep learning models to generate optimal outfits, which are visually displayed through smart glasses or a head-mounted display.

[1278] 3. Missing item detection and purchasing recommendations:

[1279] The server detects missing items based on the proposed coordinates, which are identified by comparing them with a database.

[1280] The server generates purchasing recommendation data (product information, purchase links, etc.) via an internet shopping API and sends it to the terminal.

[1281] The terminal displays the information to the user through smart glasses or a head-mounted display, allowing the user to complete the purchase process in real time.

[1282] Examples of specific examples and prompts

[1283] Specific examples

[1284] The user puts on the smart glasses and inputs a suggestion for an outfit for "office casual" and "winter" into the app. The server references the user's item data and suggests an outfit of "black knit, blue jeans, and down jacket." The user reviews the suggested outfit and gives feedback that "I need winter boots." The server detects any missing items and provides a purchase link via an online shopping API. The user uses the link to complete the purchase process.

[1285] Prompt Sentence Examples

[1286] Use smart glasses to take a photo of the items the user is wearing and suggest a casual office outfit based on that image. Also, detect missing items and provide a link to purchase them.

[1287] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1288] Step 1: User Registration

[1289] A user installs the app on their device and enters the information required to create an account (such as name, email address, and password). The device sends this information to the server, which stores it in a database and creates a user account.

[1290] Step 2: Enter item data

[1291] Users take a photo of their fashion items using the camera function of their smart glasses or device. The device then uploads the captured image data to the app and sends it to the server. After receiving the image data, the server analyzes the image using Google Cloud Vision API or Amazon Rekognition to generate classification information such as the item's category, color, and material. This classification information is then stored in a database as item data.

[1292] Step 3: Enter coordination conditions

[1293] The user inputs the conditions necessary for coordinating outfits (e.g., season, occasion, style, etc.) through the application. The device sends this coordination condition data to the server. The server begins searching for the user's item data based on the received coordination conditions.

[1294] Step 4: Generate outfit suggestions

[1295] The server generates the optimal outfit based on the outfit conditions and item data using a deep learning model (e.g., TensorFlow, PyTorch). The generated outfit is output as image data and applied to the user's avatar. The result is sent to the device.

[1296] Step 5: View outfit suggestions

[1297] The device then displays the received outfit suggestions on the screen of the smart glasses or head-mounted display, allowing the user to visually check the outfits.

[1298] Step 6: Detect missing items and make purchasing recommendations

[1299] The server compares the proposed coordinates with a database to detect missing items. For the detected missing items, the server generates purchase recommendation data (product information, purchase links, etc.) using an online shopping API. This purchase recommendation data is sent to the device.

[1300] Step 7: Checkout

[1301] The device displays the recommended purchase data to the user through smart glasses or a head-mounted display, and the user can click on the provided link to go to an online shopping site and complete the purchase process for the desired item.

[1302] Step 8: Push notifications and location suggestions

[1303] The server collects information about places and events that the user may be interested in based on the coordination suggestions, generates this information as a push notification, and sends it to the device. The device then presents the push notification to the user, allowing the user to check the details.

[1304] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1305] The present invention is directed to a virtual stylist system incorporating an emotion engine. The following describes the specific processing steps for suggesting coordination and lifestyle based on the fashion items owned by the user, and for making suggestions according to the user's emotional state.

[1306] 1. Initial Setup and User Registration

[1307] user:

[1308] Download and install the app on your smartphone or other device.

[1309] Launch the app and enter your basic information (name, email address, password) on the new account creation screen.

[1310] Customize your profile information and avatar.

[1311] Device:

[1312] The entered information is sent to the server.

[1313] server:

[1314] The received user information is stored in a database and a user account is created.

[1315] 2. Registering fashion items

[1316] user:

[1317] Take a photo of a fashion item you own with your smartphone camera, such as a sweater, shirt, or jeans.

[1318] Register the captured image in the app.

[1319] Device:

[1320] Sends the uploaded image to the server.

[1321] server:

[1322] The received image data is analyzed and automatically tagged with classification information such as the item's category, color, and material.

[1323] The tagged item data is stored in a database in association with the user's account information.

[1324] 3. Coordination suggestions

[1325] user:

[1326] Enter the conditions for which you would like to receive outfit suggestions (season, occasion, style, etc.) into the app.

[1327] Device:

[1328] The entered condition data is sent to the server.

[1329] server:

[1330] The system selects registered fashion items that meet the conditions and runs an algorithm to generate coordination suggestions.

[1331] server:

[1332] The emotion engine analyzes the user's emotions and adjusts the coordination suggestions based on the results.

[1333] server:

[1334] In order to visually display the generated coordination proposal, image data to be applied to an avatar is generated.

[1335] Device:

[1336] The completed coordination proposal is displayed to the user.

[1337] user:

[1338] Review the proposed coordination and provide feedback as needed.

[1339] 4. Suggestion and purchase of missing items

[1340] server:

[1341] Based on the suggested outfit, it will detect missing items. For example, if the user does not own "leather shoes," it will detect this.

[1342] Generate purchasing recommendation data (product images, brand, price, purchase link, etc.).

[1343] Device:

[1344] Display a list of missing items to the user with a link to purchase them.

[1345] user:

[1346] Click on the purchase link and complete the purchase process as needed.

[1347] 5. Collaborative projects and promotions

[1348] server:

[1349] We regularly collect information on collaborations with popular fashion magazines and famous stylists.

[1350] Set collaboration information as a push notification and generate data to notify the user.

[1351] Device:

[1352] Receive push notifications and display them to the user.

[1353] user:

[1354] Check your push notifications to view collaborations and promotions that interest you.

[1355] 6. Recycling suggestions for unwanted items

[1356] server:

[1357] Analyzes the user's item database to detect items that have not been used for a long time.

[1358] Notification data is generated that suggests recycling or using unwanted items at a flea market.

[1359] Device:

[1360] Show users a suggestion notification to guide them on how to recycle unwanted items.

[1361] user:

[1362] Click the link to begin recycling your unwanted items.

[1363] 7. Suggestions for places to go

[1364] server:

[1365] Based on the suggested outfits, we collect information on events, travel destinations, restaurants, etc. that may interest the user.

[1366] The proposed information and coordinated images are integrated to generate data to be provided to the user.

[1367] Device:

[1368] Showing users suggested places and events.

[1369] user:

[1370] Check the event information that interests you and find out more details.

[1371] 8. Support for style and etiquette for ceremonial occasions

[1372] server:

[1373] Maintain a database of information on style and general etiquette related to weddings, funerals, and other ceremonies.

[1374] Generate data that provides appropriate style and manner information in response to user requests.

[1375] Device:

[1376] The suggested style and manner information is displayed to the user, and a screen showing the style and manner information applied to the user's avatar is displayed.

[1377] user:

[1378] Check the suggested style and etiquette information to alleviate any concerns.

[1379] Specific examples using emotion engines

[1380] user:

[1381] In the app, users set the conditions of "commute," "autumn," and "business casual" and request outfit suggestions. The emotion engine then uses facial recognition to read the user's current emotions (e.g., nervousness, joy, fatigue).

[1382] server:

[1383] Items that meet the conditions are selected from the registered fashion items, and coordination suggestions are generated.

[1384] Based on the user's emotions recognized by the emotion engine, for example, if the user is feeling nervous, the system will suggest outfits that include items in relaxing colors and materials.

[1385] Device:

[1386] The coordinated suggestions applied to the user's avatar are visually displayed.

[1387] user:

[1388] The user checks the suggested outfits, selects them if satisfied, and then carries out the purchasing process for any missing items.

[1389] The system of the present invention allows users to effectively utilize their own fashion items and receive suggestions for appropriate coordination and event information according to their emotional state, thereby providing comprehensive support for users' fashion-related concerns and improving user satisfaction.

[1390] The processing flow will be explained below.

[1391] Specific processing flow of the virtual stylist system using the emotion engine

[1392] 1. Initial Setup and User Registration

[1393] Step 1:

[1394] User: Downloads and installs the app on a smartphone or other device.

[1395] Step 2:

[1396] User: Launch the app and enter basic information (name, email address, password) on the new account creation screen.

[1397] Step 3:

[1398] Terminal: Sends the entered information to the server.

[1399] Step 4:

[1400] Server: Stores the received user information in a database and creates a user account.

[1401] Step 5:

[1402] Users: Customize your profile information and avatar.

[1403] 2. Registering fashion items

[1404] Step 1:

[1405] User: Take a photo of a fashion item they own with their smartphone camera, such as a sweater, shirt, or jeans.

[1406] Step 2:

[1407] User: Register the captured image within the app.

[1408] Step 3:

[1409] On the device: Sends the uploaded image to the server.

[1410] Step 4:

[1411] Server: Analyzes the received image data and automatically tags it with classification information such as item category, color, and material.

[1412] Step 5:

[1413] Server: Stores tagged item data in a database, associating it with user account information.

[1414] 3. Coordination suggestions

[1415] Step 1:

[1416] User: Enter the conditions for which you would like the app to suggest outfits (season, occasion, style, etc.).

[1417] Step 2:

[1418] Device: Facial recognition is performed using the smartphone camera to analyze the user's emotional state along with their input criteria in the emotion engine.

[1419] Step 3:

[1420] Terminal: Sends the emotion data and coordination conditions obtained through the emotion engine to the server.

[1421] Step 4:

[1422] Server: Selects registered fashion items that match the criteria and runs an algorithm to generate coordination suggestions.

[1423] Step 5:

[1424] Server: Adjusts outfit suggestions based on the user's emotions recognized by the emotion engine. For example, if the user is feeling nervous, the server suggests outfits that include items with relaxing colors and materials.

[1425] Step 6:

[1426] Server: Generates image data to be applied to an avatar to visually display the generated coordination proposal.

[1427] Step 7:

[1428] Device: Displays the completed outfit suggestions to the user.

[1429] Step 8:

[1430] User: Review the proposed coordination and provide feedback if necessary.

[1431] 4. Suggestion and purchase of missing items

[1432] Step 1:

[1433] Server: Detects missing items based on the proposed outfit. For example, if the user does not have "leather shoes," it will detect this.

[1434] Step 2:

[1435] Server: Generates purchasing recommendation data (product images, brand, price, purchase link, etc.).

[1436] Step 3:

[1437] On the device: Display a list of missing items to the user with a link to purchase them.

[1438] Step 4:

[1439] User: Clicks on the purchase link if necessary and completes the purchase.

[1440] 5. Collaborative projects and promotions

[1441] Step 1:

[1442] Server: Regularly collects information on collaborations with popular fashion magazines and famous stylists.

[1443] Step 2:

[1444] Server: Sets collaboration information as a push notification and generates data to notify the user.

[1445] Step 3:

[1446] Device: Receives the push notification and displays it to the user.

[1447] Step 4:

[1448] User: Check push notifications to view collaborations and promotions that interest them.

[1449] 6. Recycling suggestions for unwanted items

[1450] Step 1:

[1451] Server: Analyzes the user's item database and detects items that have not been used for a long time.

[1452] Step 2:

[1453] Server: Generates notification data suggesting recycling or selling unwanted items at flea markets.

[1454] Step 3:

[1455] On the device: Show users a suggestion notification to guide them on how to recycle unwanted items.

[1456] Step 4:

[1457] User: Clicks on a link to recycle unwanted items.

[1458] 7. Suggestions for places to go

[1459] Step 1:

[1460] Server: Based on the suggested outfits, collects information on events, travel destinations, restaurants, etc. that the user may be interested in.

[1461] Step 2:

[1462] Server: Integrates the recommendation information and coordinated images to generate data to provide to users.

[1463] Step 3:

[1464] On your device: Show suggested places and events to the user.

[1465] Step 4:

[1466] Users: View events that interest them and learn more.

[1467] 8. Support for style and etiquette for ceremonial occasions

[1468] Step 1:

[1469] Server: Manages a database of information on style and general etiquette related to ceremonial occasions.

[1470] Step 2:

[1471] Server: Generates data to provide appropriate style and manner information in response to user requests.

[1472] Step 3:

[1473] On the device: Show suggested styles and mannerisms to the user and display the results on the user's avatar.

[1474] Step 4:

[1475] Users: Review suggested style and etiquette information to ease concerns.

[1476] Specific examples using emotion engines

[1477] Step 1:

[1478] User: The app sets the conditions "Commute," "Autumn," and "Business Casual," and uses the smartphone camera to recognize the user's face and analyze their emotional state.

[1479] Step 2:

[1480] Device: Sends the facial recognition results to the emotion engine to analyze the current emotion (e.g., tension, joy, fatigue).

[1481] Step 3:

[1482] Terminal: Sends analyzed emotion data to the server.

[1483] Step 4:

[1484] Server: Searches the fashion item database and selects items that match the conditions.

[1485] Step 5:

[1486] Server: Adjusts outfit suggestions based on the results of the emotion engine. For example, if the customer is feeling nervous, the server will suggest items with relaxing materials and colors.

[1487] Step 6:

[1488] Server: Visualizes the coordinated outfit suggestions after adjustments and generates image data to apply to the avatar.

[1489] Step 7:

[1490] On device: Display coordinated outfit suggestions to the user.

[1491] Step 8:

[1492] User: Check the suggested outfit and, if satisfied, click the link to purchase the missing items and purchase the items they need.

[1493] The system of the present invention comprehensively supports users' fashion concerns by making effective use of their fashion items and providing appropriate coordination suggestions according to their emotional state.

[1494] Example 2

[1495] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1496] Conventional fashion coordination systems have problems with low user satisfaction because they do not make full use of the clothes a user owns and do not take into account the user's emotional state when making suggestions. It is also difficult for users to grasp what items they are lacking, making it difficult to effectively purchase the items they need. Furthermore, they lack support for overall lifestyles, such as suggesting places to go or event information.

[1497] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for photographing clothes owned by the user and inputting item information, means for analyzing the item information and assigning classification data, means for inputting coordination conditions from the user, means for searching the user's item information based on the coordination conditions and generating coordination suggestions, means for visually displaying the coordination suggestions, means for detecting missing items and generating purchase recommendation data, and means for recognizing the user's emotional state and adjusting the coordination suggestions based on the results. This allows the user to more effectively utilize their own clothes and receive support for coordination according to their emotional state and overall lifestyle.

[1498] "User" refers to someone who uses this system to register fashion items and receive coordination suggestions.

[1499] "Clothing" refers to fashion items that users own, photograph, and enter.

[1500] "Item information" is data about clothing, and includes detailed information such as category, color, and material.

[1501] "Classification data" refers to classification information such as category, color, material, etc. obtained by analyzing item information.

[1502] "Coordination" refers to clothing combinations suggested based on the user's item information and specific conditions.

[1503] "Conditions" are elements that are taken into consideration when creating a coordination, and include the season, the scene, the style, and the like.

[1504] "Coordination suggestions" refer to outfit combinations generated by the server based on conditions entered by the user.

[1505] The "means for visually displaying" refers to a function for visually displaying the generated coordination proposal so that the user can confirm it.

[1506] "Purchase recommendation data" refers to information (product images, brand, price, purchase link, etc.) that detects missing items and encourages their purchase.

[1507] "Emotional state" refers to the user's current emotions (e.g., tension, joy, fatigue), and is a factor reflected in outfit suggestions.

[1508] An "avatar" refers to a digital character that appears as a user's alter ego and is used to visually confirm coordination suggestions.

[1509] The present invention relates to a virtual stylist system incorporating an emotion engine. This system proposes coordination and lifestyle ideas based on the clothes owned by the user, and also makes suggestions according to the user's emotional state.

[1510] Overall system configuration

[1511] The system is implemented in the following way.

[1512] 1. The user takes a photo of the clothing they own and enters the item information.

[1513] 2. Analyze the entered item information and assign classification data.

[1514] 3. Enter the coordination requirements from the user.

[1515] 4. Based on the coordination conditions, search for the user's item information and generate coordination suggestions.

[1516] 5. Visually display coordination suggestions.

[1517] 6. Detect missing items and generate purchasing recommendations.

[1518] 7. Based on the suggested outfit, the app will suggest places for the user to go and event information.

[1519] 8. Apply the suggested outfit to the user's avatar and visually display it.

[1520] 9. Recognize the user's emotional state and tailor outfit suggestions accordingly.

[1521] Initial Setup and User Registration

[1522] user:

[1523] Users download and install the app on their smartphone, launch it, and enter their name, email address, and password on the new account creation screen, then customize their profile information and avatar.

[1524] Device:

[1525] The terminal sends the entered information to the server via an "HTTP POST request."

[1526] server:

[1527] The server saves the received user information in a MySQL database and creates a user account. The server performs validation to ensure there are no duplicate email addresses.

[1528] Fashion item registration

[1529] user:

[1530] Users take a photo of their clothing with their smartphone camera and register the image in the app, along with the item name, category, and tags (e.g., color, material).

[1531] Device:

[1532] The device uploads the captured image to cloud storage (e.g., AWS S3), generates a URL for the image, and sends it to the server.

[1533] server:

[1534] The server analyzes the received image URL using the Google Cloud Vision API, extracts classification information such as the item's category, color, and material, associates it with the user's account, and stores it in a database.

[1535] Coordination suggestions

[1536] user:

[1537] Users can request outfit suggestions by inputting conditions such as season, occasion, and style, and the system also uses facial recognition to detect the user's emotions.

[1538] Device:

[1539] The terminal sends the entered condition data to the server via an "HTTP POST request."

[1540] server:

[1541] The server selects items that match the criteria from the registered item information and generates coordination suggestions. It also analyzes the user's emotions using an emotion engine (e.g., Azure Emotion API) and adjusts the coordination suggestions based on the results.

[1542] server:

[1543] In order to visually display the generated coordination proposal, image data to be applied to the user's avatar is generated.

[1544] Device:

[1545] The terminal displays the completed coordination proposal to the user.

[1546] Suggestion and purchase of missing items

[1547] server:

[1548] The server detects missing items based on the suggested outfits and generates purchasing recommendation data (product images, brand, price, purchase links, etc.).

[1549] Device:

[1550] The device will display a list of missing items to the user along with a link to purchase them.

[1551] user:

[1552] The user clicks on the purchase link as needed and completes the purchase process.

[1553] Collaborative projects and promotions

[1554] server:

[1555] The server collects collaboration information with popular media and experts, updates it periodically, and generates data to notify users of the collaboration information as push notifications.

[1556] Device:

[1557] The device receives the push notification and displays it to the user.

[1558] user:

[1559] Users can check push notifications and view collaboration projects and promotional information that interest them.

[1560] Recycling suggestions for unwanted items

[1561] server:

[1562] The server analyzes the user's item information, detects items that have not been used for a long time, and generates notification data for recycling suggestions.

[1563] Device:

[1564] The device will display a recycling suggestion notification to the user and guide them on how to recycle.

[1565] user:

[1566] The user clicks on the suggested link to complete the recycling process.

[1567] Suggestions for places to go

[1568] server:

[1569] Based on the user's coordination suggestions, the server collects information on events, travel destinations, restaurants, etc. that may be of interest to the user and generates proposal data.

[1570] Device:

[1571] The device will display suggested locations and event information to the user.

[1572] user:

[1573] Users can check the event information of their interest and check the details.

[1574] Specific examples using emotion engines

[1575] user:

[1576] Users can set the conditions of "commute," "autumn," and "business casual" and request outfit suggestions. At the same time, the emotion engine uses facial recognition to read the user's current emotions (e.g., nervousness, joy, fatigue).

[1577] server:

[1578] The server selects registered fashion items that match the user's criteria and generates outfit suggestions. For example, if the user is feeling nervous, the server will suggest outfits that include items in relaxing colors and materials based on the user's emotions as recognized by the emotion engine.

[1579] Device:

[1580] The terminal visually displays the coordination suggestions applied to the user's avatar.

[1581] user:

[1582] The user checks the suggested outfits, selects them if satisfied, and then proceeds to purchase any missing items.

[1583] Specific examples and prompts for the generative AI model

[1584] Examples:

[1585] "I'm looking for some fall clothes to wear to work. I'd prefer business casual, but I'm a little nervous."

[1586] Prompt for the generative AI model:

[1587] "Users are looking for fall clothing for work, so suggest business casual outfits. Also, users are feeling stressed, so create outfits that include relaxing colors and materials."

[1588] The system of the present invention allows users to effectively utilize their clothing and receive suggestions for outfits and event information that match their emotional state, thereby providing comprehensive support for fashion-related concerns and improving user satisfaction.

[1589] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1590] Step 1: User registration and initial setup

[1591] user:

[1592] Download and install the app on your smartphone.

[1593] Launch the app, enter your name, email address, and password on the new account creation screen, and press the "Submit" button.

[1594] Input: Name, Email Address, Password

[1595] Output: Registration request data

[1596] Device:

[1597] The registration request data is sent to the server via an HTTP POST request.

[1598] Input: Registration request data

[1599] Output: Request to server

[1600] server:

[1601] The received registration request data undergoes validation checks (such as checking for duplicate accounts), is saved in a MySQL database, and a user account is created.

[1602] Input: Registration request data, database

[1603] Output: New user account

[1604] Step 2: Register your fashion items

[1605] user:

[1606] Use your smartphone camera to take a photo of your clothing.

[1607] Select the image you took within the app, enter the item name, category, and tags (color, material, etc.), and register the image.

[1608] Input: Clothing photo, item name, category, tags (color, material, etc.)

[1609] Output: Registration request data

[1610] Device:

[1611] The captured image is uploaded to cloud storage (e.g. AWS S3) and a URL for the image is generated.

[1612] The tagged information along with the image URL is sent to the server via an HTTP POST request.

[1613] Input: Clothing photo, item name, category, tags (color, material, etc.)

[1614] Output: Request to server

[1615] server:

[1616] The received image URL is analyzed using the Google Cloud Vision API, and classification information such as the item's category, color, and material is extracted, associated with the user's account, and stored in a database.

[1617] Input: Image URL, tagging information

[1618] Output: Classification data, database user information

[1619] Step 3: Enter coordination conditions

[1620] user:

[1621] Enter the conditions for which you would like the app to suggest outfits (season, occasion, style, etc.).

[1622] Use facial recognition to detect emotions.

[1623] Input: Coordination condition data, face recognition data

[1624] Output: Coordinated condition request

[1625] Device:

[1626] The entered condition data and facial recognition data are sent to the server via an "HTTP POST request."

[1627] Input: Coordination condition data, face recognition data

[1628] Output: Request to server

[1629] Step 4: Generate outfit suggestions

[1630] server:

[1631] From the registered fashion items, items that match the inputted coordination conditions are selected.

[1632] An emotion engine (e.g., Azure Emotion API) is used to analyze the user's emotions and tailor outfit suggestions based on the results.

[1633] Input: Item information, coordination conditions, emotion data

[1634] Output: Coordination suggestion data

[1635] Step 5: View outfit suggestions

[1636] server:

[1637] In order to visually display the generated coordination proposal, image data to be applied to the user's avatar is generated.

[1638] Input: Coordination suggestion data

[1639] Output: Coordinate image data

[1640] Device:

[1641] Coordinate image data is received and displayed to the user.

[1642] Input: Coordinate image data

[1643] Output: what is displayed to the user

[1644] user:

[1645] Review the proposed coordination and provide feedback as needed.

[1646] Input: Coordination image

[1647] Output: Feedback data

[1648] Step 6: Suggest and purchase missing items

[1649] server:

[1650] Detect missing items based on suggested coordinates.

[1651] Generate purchasing recommendation data (product images, brand, price, purchase link, etc.) for items in short supply.

[1652] Input: Coordination suggestions, user item data

[1653] Output: Purchase recommendation data

[1654] Device:

[1655] Display a list of missing items to the user with a link to purchase them.

[1656] Input: Purchase recommendation data

[1657] Output: what is displayed to the user

[1658] user:

[1659] Click on the purchase link and complete the purchase process as needed.

[1660] Enter: Purchase Link

[1661] Output: Purchase confirmation data

[1662] Step 7: Suggest places to go

[1663] server:

[1664] Based on the suggested outfits, we collect information on events, travel destinations, restaurants, etc. that may interest the user.

[1665] Input: Coordination suggestions

[1666] Output: Event proposal data

[1667] Device:

[1668] Showing users suggested places and events.

[1669] Input: Event proposal data

[1670] Output: what is displayed to the user

[1671] user:

[1672] Check the event information that interests you and find out more details.

[1673] Input: Event information

[1674] Output: Event details display

[1675] Step 8: Recognizing and Coordinating Emotional States

[1676] user:

[1677] Using facial recognition, the emotion engine reads the user's current emotions (e.g., nervousness, joy, fatigue).

[1678] Input: Facial recognition data

[1679] Output: Emotion data

[1680] server:

[1681] Coordination suggestions are adjusted based on emotional data recognized by the emotion engine.

[1682] Input: Emotion data

[1683] Output: Adjusted outfit suggestions

[1684] Device:

[1685] The adjusted coordination suggestions are applied to the user's avatar and visually displayed.

[1686] Input: Adjusted outfit suggestions

[1687] Output: Adjusted coordinate image

[1688] user:

[1689] Check the suggested outfits, purchase the items you need, and provide feedback.

[1690] Input: Adjusted coordinate image

[1691] Output: Purchase confirmation data, feedback data

[1692] (Application example 2)

[1693] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1694] Conventional fashion coordination systems simply make suggestions based on the user's owned fashion items and basic requirements, without considering the user's emotional state. As a result, they are unable to make suggestions that match the user's current emotions and mood, making it difficult to increase satisfaction. Furthermore, they lack an efficient means to encourage the purchase of missing items in the suggested coordination.

[1695] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for photographing fashion items owned by the user and inputting item data, a means for analyzing the item data and assigning classification information, a means for accepting coordination conditions input from the user, a means for searching the user's item data based on the coordination conditions and generating coordination suggestions, a means for recognizing the user's emotions and adjusting the coordination suggestions based on the emotions, and a means for applying the suggested coordination to the user's avatar and visually displaying it. This makes it possible to propose optimal fashion coordination that takes emotions into consideration. Furthermore, the server can also improve the user's purchasing experience by detecting missing items and generating purchase recommendation data.

[1696] "Fashion items" is a general term for fashion-related possessions such as clothing, accessories, and shoes.

[1697] "Item data" refers to attribute information such as images, categories, colors, and materials of fashion items owned by a user.

[1698] "Coordination conditions" are requirements such as season, occasion, style, etc. that are input by the user when requesting suggestions.

[1699] A "coordination suggestion" is a specific fashion combination generated based on coordination conditions.

[1700] "Emotion" is a general term for a user's current psychological state or mood (e.g., joy, tension, fatigue, etc.).

[1701] An "avatar" is a digital persona or character that virtually represents a user.

[1702] "Facial recognition" is a technology that analyzes a user's facial image and reads emotions from their facial expressions.

[1703] "Purchase recommendation data" is information such as the brand, price, and purchase link of missing items based on the suggested outfit.

[1704] This embodiment of the present invention describes a system for suggesting outfits based on data on fashion items owned by a user. This system proposes outfits taking into account the emotional state of the user and recommends the purchase of missing items as needed, thereby increasing user satisfaction.

[1705] Initial Setup and User Registration

[1706] Users download a dedicated application onto their smartphone or smart glasses and create a new account. Once the application is launched, users enter basic information (name, email address, password) and customize their profile information and avatar.

[1707] Fashion item registration

[1708] Users take photos of their own fashion items, such as sweaters, shirts, and jeans. The images are then uploaded to the server by the application. The server analyzes the image data and automatically tags the items with classification information such as category, color, and material. This classification information is then associated with the user's account information and stored in a database.

[1709] Coordination suggestions

[1710] When a user wants to receive outfit suggestions, they input the season, occasion, style, and other conditions into the application. Furthermore, the smart glasses' built-in facial recognition function is used to read the user's emotions. The emotion engine analyzes the user's current emotions (e.g., nervousness, joy, fatigue, etc.).

[1711] The server proposes optimal outfits from registered fashion items based on the user's conditions and emotional state. The proposed outfits are applied to the user's avatar and visually displayed on the smart glasses display. The user can review the proposed outfits and provide feedback if necessary.

[1712] Suggestion and purchase of missing items

[1713] The server detects missing items based on the suggested outfit. For example, if the user does not own "leather shoes," it will detect this. It then generates purchasing recommendation data for the missing items (product images, brand, price, purchasing links, etc.). The terminal displays a list of missing items and purchase links to the user. The user can click the purchase link and complete the purchase process as needed.

[1714] Emotion recognition and concrete examples

[1715] The application uses the smart glasses' camera to capture the user's face and analyzes their emotions using an emotion engine. For example, if a user sets the conditions "commute," "autumn," and "business casual," and the emotion engine detects "tension" through facial recognition, it will suggest outfits that include items with relaxing colors and materials.

[1716] In this case, an example of a prompt sentence input to the generative AI model is as follows:

[1717] The user sets the conditions of "commute," "autumn," and "business casual," and emotion recognition detects "tension." Based on these conditions, please suggest the best outfit for the user from their fashion items.

[1718] In this way, the suggested coordination takes into account the user's emotional state and can improve user satisfaction.

[1719] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1720] Step 1:

[1721] Users download a dedicated application onto a device such as a smartphone or smart glasses and create a new account. The user enters basic information (name, email address, password) and customizes their profile information and avatar. The entered information is sent by the device to a server, which stores the received information in a database and creates a user account.

[1722] Step 2:

[1723] The user takes a photo of a fashion item they own using the smart glasses' camera. For example, a sweater, shirt, or jeans. The device then uploads the image to a server. The server analyzes the image data and automatically tags the item with classification information such as category, color, and material. This tagged information is then stored in a database along with the user's account information.

[1724] Step 3:

[1725] The user inputs the conditions for which they would like to receive outfit suggestions (season, occasion, style, etc.) into the application. The smart glasses' built-in facial recognition function is used to capture the user's face. The device then sends the conditions data entered by the user and the captured facial image data to the server. The server then uses an emotion engine to analyze the facial image and recognize the user's current emotion (e.g., nervousness, joy, fatigue, etc.).

[1726] Step 4:

[1727] The server generates outfit suggestions based on the user's conditions and emotional state. It selects registered fashion items that match the conditions and then runs an algorithm to suggest optimal outfits that take emotions into consideration. The outfit suggestions are generated and applied to the user's avatar. The generated outfit suggestions are sent to the device and visually displayed on the smart glasses' display.

[1728] Step 5:

[1729] Based on the suggested outfit, the server detects any missing items. For example, if the user does not own "leather shoes," the server detects those items. For each missing item, the server generates purchasing recommendation data (product image, brand, price, purchase link, etc.). The device displays a list of missing items and a purchase link to the user.

[1730] Step 6:

[1731] The user can click on the purchase link provided to complete the purchase of the missing item. Once the user completes the purchase, the information is updated by the server and stored in the database.

[1732] Step 7:

[1733] The application allows users to provide feedback on the proposed coordination. The user enters the feedback and the device sends it to the server. The server receives the feedback and uses it to improve the coordination algorithm and update personal data.

[1734] These steps enable the system to suggest optimal fashion coordination that takes into account the user's emotional state, improving the user's shopping experience.

[1735] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1736] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1737] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1738] [Third embodiment]

[1739] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1740] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1741] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1742] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1743] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1744] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1745] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1746] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1747] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1748] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1749] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1750] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1751] Specific processes and operations of the virtual stylist system of the present invention will be described below. Note that in the following description, the server, the terminal, and the user are used as the subject.

[1752] 1. Initial Setup and User Registration

[1753] user:

[1754] Users download and install the app on their smartphone or other device.

[1755] When the app is launched, an account creation screen appears, where users enter basic information such as their name, email address, and password to create an account.

[1756] Users then customize their profile information and avatar.

[1757] Device:

[1758] The entered information is sent to the server.

[1759] server:

[1760] Store the received information in a database and create a user account.

[1761] 2. Registering fashion items

[1762] user:

[1763] Take a photo of a fashion item you own with your smartphone camera, such as a sweater, shirt, or jeans.

[1764] Upload the captured images through the app's interface.

[1765] Device:

[1766] Sends the uploaded image to the server.

[1767] server:

[1768] The received image data is analyzed and automatically tagged with classification information such as the item's category, color, and material.

[1769] Stores item data in a database, associated with the user's account information.

[1770] 3. Coordination suggestions

[1771] user:

[1772] Enter the conditions for which you would like to receive outfit suggestions (season, occasion, style, etc.) into the app.

[1773] Device:

[1774] The entered condition data is sent to the server.

[1775] server:

[1776] The system selects registered fashion items that meet the conditions and runs an algorithm to generate coordination suggestions.

[1777] In order to visually display the generated coordination proposal, image data to be applied to an avatar is generated.

[1778] Device:

[1779] The completed coordination proposal is displayed to the user.

[1780] user:

[1781] Review the proposed coordination and provide feedback as needed.

[1782] 4. Suggestion and purchase of missing items

[1783] server:

[1784] Based on the suggested outfit, it will detect missing items. For example, if the user does not own "leather shoes," it will detect this.

[1785] Generates purchasing recommendation data and provides purchasing information (product images, brand, price, purchase link, etc.).

[1786] Device:

[1787] Display a list of missing items to the user with a link to purchase them.

[1788] user:

[1789] Click on the purchase link and complete the purchase process as needed.

[1790] 5. Collaborative projects and promotions

[1791] server:

[1792] We regularly collect information on collaborations with popular fashion magazines and famous stylists.

[1793] Set collaboration information as a push notification and generate data to notify the user.

[1794] Device:

[1795] Receive push notifications and display them to the user.

[1796] user:

[1797] Check your push notifications to view collaborations and promotions that interest you.

[1798] 6. Recycling suggestions for unwanted items

[1799] server:

[1800] Analyzes the user's item database to detect items that have not been used for a long time.

[1801] For unwanted items, suggestion notification data is generated recommending recycling or using them at a flea market.

[1802] Device:

[1803] Show users a suggestion notification to guide them on how to recycle unwanted items.

[1804] user:

[1805] Click the link to begin recycling your unwanted items.

[1806] 7. Suggestions for places to go

[1807] server:

[1808] Based on the suggested outfits, we collect information on events, travel destinations, restaurants, etc. that may interest the user.

[1809] The proposed information and coordinated images are integrated to generate data to be provided to the user.

[1810] Device:

[1811] Showing users suggested places and events.

[1812] user:

[1813] Check the event information that interests you and find out more details.

[1814] 8. Support for style and etiquette for ceremonial occasions

[1815] server:

[1816] Maintain a database of information on style and general etiquette related to weddings, funerals, and other ceremonies.

[1817] Generate data that provides appropriate style and manner information in response to user requests.

[1818] Device:

[1819] The suggested style and manner information is displayed to the user, and a screen showing the style applied to the user's avatar is displayed.

[1820] user:

[1821] Check the suggested style and etiquette information to alleviate any concerns.

[1822] Specific examples

[1823] user:

[1824] In the app, set the conditions "commute," "autumn," and "business casual" and request outfit suggestions.

[1825] Check the outfit suggested by the server: gray sweater, white shirt, and dark blue jeans.

[1826] Check the proposal and give feedback such as "Leather shoes are needed."

[1827] server:

[1828] Based on the suggested outfit, it detects that "leather shoes" are in short supply and generates purchasing information.

[1829] Device:

[1830] Present the user with a purchase link and provide them with a link to complete the purchase.

[1831] user:

[1832] Click on the provided link and complete the process to purchase the "leather shoes."

[1833] The system of the present invention allows users to effectively utilize their fashion items and eliminates the anxiety of coordinating outfits for everyday wear and special events.

[1834] The processing flow will be explained below.

[1835] Program processing flow

[1836] 1. Initial Setup and User Registration

[1837] Step 1:

[1838] User: Downloads and installs the app on a smartphone or other device.

[1839] Step 2:

[1840] User: Launch the app and enter basic information (name, email address, password) on the new account creation screen.

[1841] Step 3:

[1842] Terminal: Sends the entered information to the server.

[1843] Step 4:

[1844] Server: Stores the received user information in a database and creates a user account.

[1845] Step 5:

[1846] Users: Customize your profile information and avatar.

[1847] 2. Registering fashion items

[1848] Step 1:

[1849] User: Take a photo of a fashion item they own with their smartphone camera, such as a sweater, shirt, or jeans.

[1850] Step 2:

[1851] User: Register the captured image within the app.

[1852] Step 3:

[1853] On the device: Sends the uploaded image to the server.

[1854] Step 4:

[1855] Server: Analyzes the received image data and automatically tags it with classification information such as item category, color, and material.

[1856] Step 5:

[1857] Server: Stores tagged item data in a database, associating it with user account information.

[1858] 3. Coordination suggestions

[1859] Step 1:

[1860] User: Enter the conditions for which you would like the app to suggest outfits (season, occasion, style, etc.).

[1861] Step 2:

[1862] Terminal: Sends the entered condition data to the server.

[1863] Step 3:

[1864] Server: Selects registered fashion items that match the criteria and runs an algorithm to generate coordination suggestions.

[1865] Step 4:

[1866] Server: Generates image data to be applied to an avatar to visually display the generated coordination proposal.

[1867] Step 5:

[1868] Device: Displays the completed outfit suggestions to the user.

[1869] Step 6:

[1870] User: Review the proposed outfits and provide feedback.

[1871] 4. Suggestion and purchase of missing items

[1872] Step 1:

[1873] Server: Detects missing items based on the proposed outfit. For example, if the user does not have "leather shoes," it will detect this.

[1874] Step 2:

[1875] Server: Generates purchasing recommendation data (product images, brand, price, purchase link, etc.).

[1876] Step 3:

[1877] On the device: Display a list of missing items to the user with a link to purchase them.

[1878] Step 4:

[1879] User: Clicks on the purchase link if necessary and completes the purchase.

[1880] 5. Collaborative projects and promotions

[1881] Step 1:

[1882] Server: Regularly collects information on collaborations with popular fashion magazines and famous stylists.

[1883] Step 2:

[1884] Server: Sets collaboration information as a push notification and generates data to notify the user.

[1885] Step 3:

[1886] Device: Receives the push notification and displays it to the user.

[1887] Step 4:

[1888] User: Check push notifications to view collaborations and promotions that interest them.

[1889] 6. Recycling suggestions for unwanted items

[1890] Step 1:

[1891] Server: Analyzes the user's item database and detects items that have not been used for a long time.

[1892] Step 2:

[1893] Server: Generates notification data suggesting recycling or selling unwanted items at flea markets.

[1894] Step 3:

[1895] On the device: Show users a suggestion notification to guide them on how to recycle unwanted items.

[1896] Step 4:

[1897] User: Clicks on a link to recycle unwanted items.

[1898] 7. Suggestions for places to go

[1899] Step 1:

[1900] Server: Based on the suggested outfits, collects information on events, travel destinations, restaurants, etc. that the user may be interested in.

[1901] Step 2:

[1902] Server: Integrates the recommendation information and coordinated images to generate data to provide to users.

[1903] Step 3:

[1904] On your device: Show suggested places and events to the user.

[1905] Step 4:

[1906] Users: View events that interest them and learn more.

[1907] 8. Support for style and etiquette for ceremonial occasions

[1908] Step 1:

[1909] Server: Manages a database of information on style and general etiquette related to ceremonial occasions.

[1910] Step 2:

[1911] Server: Generates data to provide appropriate style and manner information in response to user requests.

[1912] Step 3:

[1913] On the device: Show suggested styles and mannerisms to the user and display the results on the user's avatar.

[1914] Step 4:

[1915] Users: Review suggested style and etiquette information to ease concerns.

[1916] This will enable us to provide comprehensive support to users regarding their fashion concerns.

[1917] Example 1

[1918] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1919] Today's consumers face daily fashion-related worries and difficulties in coordinating outfits. They particularly need a way to efficiently coordinate outfits for each season and occasion while making the most of the items they own. Furthermore, there is no comprehensive fashion management system that can suggest and purchase items they are running low on, provide information on collaborations with the fashion industry, or even recycle unwanted items. This often results in users wasting time and effort choosing their daily fashion, which can be stressful.

[1920] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1921] In this invention, the server includes means for photographing fashion items owned by the user and inputting item data, means for analyzing the item data and assigning classification information, means for accepting coordination conditions input from the user, means for searching the user's item data based on the coordination conditions and generating coordination suggestions, means for visually displaying the coordination suggestions, means for detecting missing items and generating purchase recommendation data, means for collecting collaboration information with specific fashion magazines and stylists and generating push notifications, means for analyzing the user's item database and detecting items that have not been used for a long time and suggesting recycling, and means for providing information on styles and general etiquette related to ceremonial occasions. This allows users to effectively utilize their own fashion items, reduces anxiety about coordinating outfits for everyday use and special events, and enables comprehensive fashion management.

[1922] "User" refers to any individual or organization that uses this system.

[1923] "Fashion items" refers to fashion-related items such as clothing, accessories, and shoes owned by a user.

[1924] "Item data" refers to data that includes information such as images, categories, colors, and materials of fashion items.

[1925] "Coordination conditions" refer to conditions such as season, scene, and style specified by the user.

[1926] "Coordination suggestions" refer to style suggestions generated based on the user's fashion items and coordination conditions.

[1927] "Purchase recommendation data" refers to data that identifies missing items and includes related purchasing information (such as product images, brand, price, and purchase links).

[1928] "Collaboration Information" refers to information provided in collaboration with specific fashion magazines or stylists.

[1929] "Push notification" refers to a notification sent from a server to a user.

[1930] "Recycling suggestions" refer to suggestions that encourage users to recycle or sell fashion items that they have not used for a long time.

[1931] "Style information" refers to information about fashion styles and general manners related to ceremonial occasions.

[1932] The specific processing and operation of the virtual stylist system of the present invention will be described below. The present invention manages the user's fashion items and suggests coordination suggestions by exchanging data between the user, the terminal, and the server.

[1933] Initial Setup and User Registration

[1934] User: First, the user downloads and installs a dedicated app onto their smartphone or other device. When the app is launched, an account creation screen appears. The user enters basic information such as name, email address, and password to create an account. They can then customize their profile information and avatar.

[1935] Terminal: This input information is sent from the terminal to the server.

[1936] Server: The server stores the received information in a database and creates a user account.

[1937] Fashion item registration

[1938] User: The user takes a photo of a fashion item they own, such as a sweater, shirt, or jeans, with their smartphone camera and uploads the image through the app's interface.

[1939] On your device: Uploaded images are sent to the server.

[1940] Server: The received image data is analyzed using services such as the Google Cloud Vision API. After analysis, classification information such as the item's category, color, and material is automatically tagged and stored in a database in association with the user's account information.

[1941] Coordination suggestions

[1942] User: The user inputs the conditions for which they would like to receive outfit suggestions (for example, season, occasion, style, etc.) into the app.

[1943] Terminal: The entered condition data is sent to the server.

[1944] Server: The server selects registered fashion items that match the input criteria and runs an algorithm to generate outfit suggestions. This algorithm also takes into account the user's past preferences and trend information to suggest optimal outfits. The generated outfit suggestions are prepared in a form that can be displayed visually, and image data to be applied to the user's avatar is generated.

[1945] Terminal: The completed outfit suggestions are displayed to the user, who can then review the suggestions and provide feedback if necessary.

[1946] Suggestion and purchase of missing items

[1947] Server: Detects missing items based on the suggested outfit. For example, if the user does not own "leather shoes," it detects this. To fill this gap, it generates purchasing recommendation data and provides detailed information such as product images, brand, price, and purchase links.

[1948] On the device: Display a list of missing items to the user with a link to purchase them.

[1949] User: Clicks on the purchase link if necessary and completes the purchase.

[1950] Collaborative projects and promotions

[1951] Server: Periodically collects information about collaborations with specific fashion magazines and famous stylists. The collected information is sent to users via push notifications.

[1952] Device: Receives the push notification and displays it to the user.

[1953] User: Check push notifications to view collaborations and promotions that interest them.

[1954] Recycling suggestions for unwanted items

[1955] Server: Analyzes the user's item database and detects items that have not been used for a long time. Based on this, it generates notification data recommending recycling or selling them at a flea market.

[1956] On the device: Show users a suggestion notification to guide them on how to recycle unwanted items.

[1957] User: Clicks on a link to recycle unwanted items.

[1958] Suggestions for places to go

[1959] Server: Based on the suggested outfits, the server collects information on events, travel destinations, restaurants, etc. that the user may be interested in. The server then integrates the suggested information with the outfit images to generate data to be provided to the user.

[1960] On your device: Show suggested places and events to the user.

[1961] Users: View events that interest them and learn more.

[1962] Style and etiquette support for ceremonial occasions

[1963] Server: Manages a database of information on style and general etiquette related to ceremonial occasions. Generates data to provide appropriate style and etiquette information in response to user requests.

[1964] On the device: Show suggested styles and mannerisms to the user, and show the styled screen on the user's avatar.

[1965] Users: Review suggested style and etiquette information to ease concerns.

[1966] Specific examples

[1967] As an example of a prompt, if the user sets the conditions "commute," "autumn," and "business casual" and requests outfit suggestions, the server will suggest an outfit of "gray sweater, white shirt, and dark blue denim." If the user provides feedback that "I need leather shoes," the server will detect the shortage and generate purchasing information, and the device will present the user with a purchase link.

[1968] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1969] Step 1:

[1970] User: Download and install the Virtual Stylist System app from the smartphone app store. Launch the app and proceed to the account creation screen. Here, enter information such as your name, email address, and password to create an account. The entered information is sent to your device.

[1971] Input: User's basic information (name, email address, password)

[1972] Output: User information is sent to the terminal

[1973] Step 2:

[1974] Terminal: Sends the account information entered by the user to the server.

[1975] Input: User information (name, email address, password)

[1976] Output: User information is sent to the server

[1977] Step 3:

[1978] Server: The received user information is saved in the database and a user account is created. This completes the user registration.

[1979] Input: User information sent from the device

[1980] Output: User information stored in database, account created

[1981] Step 4:

[1982] User: After creating an account, customize your profile information and avatar. Enter the customized information and send it to your device.

[1983] Input: Profile information, avatar setting information

[1984] Output: Profile information and avatar settings are sent to the device.

[1985] Step 5:

[1986] Device: Sends user-customized information to the server.

[1987] Input: Profile information, avatar setting information

[1988] Output: Customization information sent to the server

[1989] Step 6:

[1990] Server: Save the received customization information in the database. This completes the initial setup and user registration.

[1991] Input: Customization information sent from the device

[1992] Output: Customization information stored in the database

[1993] Step 7:

[1994] Users take photos of their fashion items with their smartphone camera and upload the images through the app's interface.

[1995] Input: Photographed fashion item images

[1996] Output: The image is captured to the device.

[1997] Step 8:

[1998] On the device: Sends the uploaded image to the server.

[1999] Input: Photographed fashion item images

[2000] Output: Image data sent to the server

[2001] Step 9:

[2002] Server: Analyzes the received image data using services such as Google Cloud Vision API to generate classification information such as item category, color, material, etc. The classification information is then tagged to the item data and stored in a database.

[2003] Input: Image data sent from the device

[2004] Output: Item data associated with classification information, item data stored in a database

[2005] Step 10:

[2006] User: Enter the conditions for which they would like to receive outfit suggestions (e.g., season, occasion, style, etc.) into the app and send the results to their device.

[2007] Input: Coordination conditions

[2008] Output: Coordinate conditions are captured in the terminal

[2009] Step 11:

[2010] Terminal: Sends the entered coordination condition data to the server.

[2011] Input: Coordination condition data

[2012] Output: Coordinate condition data sent to the server

[2013] Step 12:

[2014] Server: Runs an algorithm to select registered fashion items that match coordination criteria. Based on the selected items, it generates coordination suggestions and generates image data to be applied to the user's avatar.

[2015] Input: Registered fashion item data, coordination condition data

[2016] Output: Coordination suggestions and image data to be applied to the avatar

[2017] Step 13:

[2018] Device: Displays the completed outfit suggestions to the user.

[2019] Input: Coordination suggestions and image data sent from the server

[2020] Output: Outfit suggestions displayed on the user's device

[2021] Step 14:

[2022] User: Check the suggested outfit and provide feedback if necessary. For example, enter a comment such as "I need leather shoes" and send that information to the device.

[2023] Input: Feedback information (e.g., I need leather shoes)

[2024] Output: Feedback information captured in terminal

[2025] Step 15:

[2026] Device: Sends user feedback information to the server.

[2027] Input: Feedback information

[2028] Output: Feedback information sent to the server

[2029] Step 16:

[2030] Server: Based on the proposed outfit, detects missing items (e.g., leather shoes) and generates purchasing recommendations, including product images, brand, price, and purchase link details.

[2031] Input: User feedback information, suggested coordinate data

[2032] Output: Purchase recommendation data

[2033] Step 17:

[2034] Device: Display purchase recommendation data to the user and provide a purchase link.

[2035] Input: Purchase recommendation data sent from the server

[2036] Output: Purchase recommendation data displayed on the user's device

[2037] Step 18:

[2038] User: Clicks on the purchase link and completes the purchase.

[2039] Input: Purchase link click

[2040] Output: Completed checkout

[2041] Step 19:

[2042] Server: Periodically collects collaboration information with specific fashion magazines and stylists and generates push notifications.

[2043] Input: Information collected from fashion magazines and stylists

[2044] Output: Generates push notification data

[2045] Step 20:

[2046] Device: Receives the push notification and displays it to the user.

[2047] Input: Push notification data sent from the server

[2048] Output: A push notification that appears on the user's device

[2049] Step 21:

[2050] User: Check push notifications to view collaborations and promotions that interest them.

[2051] Input: Push notification click

[2052] Output: Collaboration and promotion information to be displayed

[2053] Step 22:

[2054] Server: Analyzes the user's item database and detects items that have not been used for a long time. Generates notification data suggesting recycling or selling items at a flea market.

[2055] Input: Item database

[2056] Output: Recycling suggestion notification data

[2057] Step 23:

[2058] Device: Show recycling suggestion notification to user.

[2059] Input: Recycle proposal notification data sent from the server

[2060] Output: A recycling suggestion notification displayed on the user's device.

[2061] Step 24:

[2062] User: Clicks on a link to recycle unwanted items.

[2063] Input: Click on recycle link

[2064] Output: Start of recycling procedure

[2065] Step 25:

[2066] Server: Based on the suggested outfits, collects information on events, travel destinations, and restaurants that the user may be interested in, and generates suggestion data.

[2067] Input: Proposed coordinate data

[2068] Output: Event and location suggestions

[2069] Step 26:

[2070] On your device: Show suggested places and events to the user.

[2071] Input: Proposal data sent from the server

[2072] Output: Event and location information displayed on the user's device

[2073] Step 27:

[2074] Users: View events that interest them and learn more.

[2075] Input: Click on event information

[2076] Output: Detailed information about the event displayed

[2077] Step 28:

[2078] Server: Manages a database of information on style and general etiquette related to ceremonial occasions, and provides data in response to user requests.

[2079] Input: User request

[2080] Output: Style and manner information provided

[2081] Step 29:

[2082] On the device: Show suggested styles and mannerisms to the user, and show the styled screen on the user's avatar.

[2083] Input: Style and manner information sent from the server

[2084] Output: Style and manner information displayed on the user's device

[2085] Step 30:

[2086] Users: Review suggested style and etiquette information to ease concerns.

[2087] Input: Check style and manner information

[2088] Output: User concerns resolved

[2089] In this way, users can effectively manage their fashion items through every step, eliminating the worry of choosing everyday fashion or coordinating outfits for special events.

[2090] (Application example 1)

[2091] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2092] Problems faced by users of conventional fashion coordination systems include the complicated process of suggesting outfits in real time and purchasing missing items, making it difficult to use on the go. Furthermore, there are limited ways for users to visually check the results of applying suggested outfits to their owned fashion items, limiting the user experience. The problem this invention aims to solve is to enable real-time suggestions and purchasing procedures, thereby improving user convenience and satisfaction.

[2093] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2094] In this invention, the server includes means for photographing fashion items owned by the user and inputting item data, means for analyzing the item data and assigning classification information, means for accepting coordination condition input from the user, means for searching the user's item data based on the coordination conditions and generating coordination suggestions, means for visually displaying the coordination suggestions, means for detecting missing items and generating purchase recommendation data, means for allowing the user to check the suggested coordination in real time and proceed with purchasing the missing items, and means for visually displaying the results via smart glasses or a head-mounted display, thereby enabling the user to check the coordination suggestions in real time and immediately proceed with purchasing the missing items.

[2095] "User" refers to an individual consumer who uses the system.

[2096] "Fashion items" refer to items that make up an individual's appearance, such as clothing, accessories, and shoes.

[2097] "Item data" refers to images of each fashion item owned by the user and their characteristic information.

[2098] "Classification information" refers to information that describes characteristics such as the category, color, material, and style of a fashion item.

[2099] "Coordination conditions" refer to the desired scene, season, style, and other conditions that the user enters into the app.

[2100] "Coordination suggestions" refer to fashion combinations generated based on the user's item data and coordination conditions.

[2101] "Visual display means" refers to the method by which visual information is presented to the user through an application or device.

[2102] "Missing items" refer to items that are necessary for the proposed outfit but that the user does not own.

[2103] "Purchase recommendation data" refers to product information and links provided to help users purchase items they are missing.

[2104] "Means for checking in real time" refers to a method in which suggested outfits are instantly presented to users, allowing them to provide feedback and complete the purchase process.

[2105] "Smart glasses" are a type of wearable device that displays information in the user's field of vision.

[2106] A "head-mounted display" is a display device that is worn on the user's head and has the function of providing visual information.

[2107] "On-the-go use" refers to a user using the system's functions outside of their home.

[2108] The following describes in detail the embodiments of the present invention: First, an overview of the entire system and specific examples of each processing step are shown.

[2109] System Configuration

[2110] This invention consists of a user, a terminal (smart glasses or a head-mounted display), and a server.

[2111] Hardware and Software

[2112] 1. Hardware:

[2113] Smart glasses (e.g., smart glasses, head-mounted displays)

[2114] Device (e.g. smartphone, tablet)

[2115] 2. Software:

[2116] Image analysis algorithms (e.g., OpenCV)

[2117] Deep learning models (e.g. TensorFlow, PyTorch)

[2118] Cloud services (e.g., Google Cloud Vision API, Amazon Rekognition)

[2119] Database (e.g. Firebase Cloud Firestore)

[2120] API (e.g., online shopping API)

[2121] Program processing description

[2122] The program for this system is designed to efficiently manage the fashion items owned by the user, provide real-time coordination suggestions, and assist in purchasing missing items. Each processing step and its operation are explained in detail below.

[2123] 1. Register user item data:

[2124] Users use their device's camera function to take photos of fashion items they own and upload the image data to the app.

[2125] The terminal transmits the image data received from the user to the server.

[2126] The server uses Google Cloud Vision API and Amazon Rekognition to analyze images and generate classification information such as item category, color, material, etc. This information is stored in Firebase Cloud Firestore.

[2127] 2. Coordination suggestions:

[2128] Users input coordination criteria (e.g., season, occasion, style) through the application.

[2129] The terminal transmits the input condition data to the server.

[2130] The server searches the user's item data and uses deep learning models to generate optimal outfits, which are visually displayed through smart glasses or a head-mounted display.

[2131] 3. Missing item detection and purchasing recommendations:

[2132] The server detects missing items based on the proposed coordinates, which are identified by comparing them with a database.

[2133] The server generates purchasing recommendation data (product information, purchase links, etc.) via an internet shopping API and sends it to the terminal.

[2134] The terminal displays the information to the user through smart glasses or a head-mounted display, allowing the user to complete the purchase process in real time.

[2135] Examples of specific examples and prompts

[2136] Specific examples

[2137] The user puts on the smart glasses and inputs a suggestion for an outfit for "office casual" and "winter" into the app. The server references the user's item data and suggests an outfit of "black knit, blue jeans, and down jacket." The user reviews the suggested outfit and gives feedback that "I need winter boots." The server detects any missing items and provides a purchase link via an online shopping API. The user uses the link to complete the purchase process.

[2138] Prompt Sentence Examples

[2139] Use smart glasses to take a photo of the items the user is wearing and suggest a casual office outfit based on that image. Also, detect missing items and provide a link to purchase them.

[2140] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2141] Step 1: User Registration

[2142] A user installs the app on their device and enters the information required to create an account (such as name, email address, and password). The device sends this information to the server, which stores it in a database and creates a user account.

[2143] Step 2: Enter item data

[2144] Users take a photo of their fashion items using the camera function of their smart glasses or device. The device then uploads the captured image data to the app and sends it to the server. After receiving the image data, the server analyzes the image using Google Cloud Vision API or Amazon Rekognition to generate classification information such as the item's category, color, and material. This classification information is then stored in a database as item data.

[2145] Step 3: Enter coordination conditions

[2146] The user inputs the conditions necessary for coordinating outfits (e.g., season, occasion, style, etc.) through the application. The device sends this coordination condition data to the server. The server begins searching for the user's item data based on the received coordination conditions.

[2147] Step 4: Generate outfit suggestions

[2148] The server generates the optimal outfit based on the outfit conditions and item data using a deep learning model (e.g., TensorFlow, PyTorch). The generated outfit is output as image data and applied to the user's avatar. The result is sent to the device.

[2149] Step 5: View outfit suggestions

[2150] The device then displays the received outfit suggestions on the screen of the smart glasses or head-mounted display, allowing the user to visually check the outfits.

[2151] Step 6: Detect missing items and make purchasing recommendations

[2152] The server compares the proposed coordinates with a database to detect missing items. For the detected missing items, the server generates purchase recommendation data (product information, purchase links, etc.) using an online shopping API. This purchase recommendation data is sent to the device.

[2153] Step 7: Checkout

[2154] The device displays the recommended purchase data to the user through smart glasses or a head-mounted display, and the user can click on the provided link to go to an online shopping site and complete the purchase process for the desired item.

[2155] Step 8: Push notifications and location suggestions

[2156] The server collects information about places and events that the user may be interested in based on the coordination suggestions, generates this information as a push notification, and sends it to the device. The device then presents the push notification to the user, allowing the user to check the details.

[2157] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2158] The present invention is directed to a virtual stylist system incorporating an emotion engine. The following describes the specific processing steps for suggesting coordination and lifestyle based on the fashion items owned by the user, and for making suggestions according to the user's emotional state.

[2159] 1. Initial Setup and User Registration

[2160] user:

[2161] Download and install the app on your smartphone or other device.

[2162] Launch the app and enter your basic information (name, email address, password) on the new account creation screen.

[2163] Customize your profile information and avatar.

[2164] Device:

[2165] The entered information is sent to the server.

[2166] server:

[2167] The received user information is stored in a database and a user account is created.

[2168] 2. Registering fashion items

[2169] user:

[2170] Take a photo of a fashion item you own with your smartphone camera, such as a sweater, shirt, or jeans.

[2171] Register the captured image in the app.

[2172] Device:

[2173] Sends the uploaded image to the server.

[2174] server:

[2175] The received image data is analyzed and automatically tagged with classification information such as the item's category, color, and material.

[2176] The tagged item data is stored in a database in association with the user's account information.

[2177] 3. Coordination suggestions

[2178] user:

[2179] Enter the conditions for which you would like to receive outfit suggestions (season, occasion, style, etc.) into the app.

[2180] Device:

[2181] The entered condition data is sent to the server.

[2182] server:

[2183] The system selects registered fashion items that meet the conditions and runs an algorithm to generate coordination suggestions.

[2184] server:

[2185] The emotion engine analyzes the user's emotions and adjusts the coordination suggestions based on the results.

[2186] server:

[2187] In order to visually display the generated coordination proposal, image data to be applied to an avatar is generated.

[2188] Device:

[2189] The completed coordination proposal is displayed to the user.

[2190] user:

[2191] Review the proposed coordination and provide feedback as needed.

[2192] 4. Suggestion and purchase of missing items

[2193] server:

[2194] Based on the suggested outfit, it will detect missing items. For example, if the user does not own "leather shoes," it will detect this.

[2195] Generate purchasing recommendation data (product images, brand, price, purchase link, etc.).

[2196] Device:

[2197] Display a list of missing items to the user with a link to purchase them.

[2198] user:

[2199] Click on the purchase link and complete the purchase process as needed.

[2200] 5. Collaborative projects and promotions

[2201] server:

[2202] We regularly collect information on collaborations with popular fashion magazines and famous stylists.

[2203] Set collaboration information as a push notification and generate data to notify the user.

[2204] Device:

[2205] Receive push notifications and display them to the user.

[2206] user:

[2207] Check your push notifications to view collaborations and promotions that interest you.

[2208] 6. Recycling suggestions for unwanted items

[2209] server:

[2210] Analyzes the user's item database to detect items that have not been used for a long time.

[2211] Notification data is generated that suggests recycling or using unwanted items at a flea market.

[2212] Device:

[2213] Show users a suggestion notification to guide them on how to recycle unwanted items.

[2214] user:

[2215] Click the link to begin recycling your unwanted items.

[2216] 7. Suggestions for places to go

[2217] server:

[2218] Based on the suggested outfits, we collect information on events, travel destinations, restaurants, etc. that may interest the user.

[2219] The proposed information and coordinated images are integrated to generate data to be provided to the user.

[2220] Device:

[2221] Showing users suggested places and events.

[2222] user:

[2223] Check the event information that interests you and find out more details.

[2224] 8. Support for style and etiquette for ceremonial occasions

[2225] server:

[2226] Maintain a database of information on style and general etiquette related to weddings, funerals, and other ceremonies.

[2227] Generate data that provides appropriate style and manner information in response to user requests.

[2228] Device:

[2229] The suggested style and manner information is displayed to the user, and a screen showing the style and manner information applied to the user's avatar is displayed.

[2230] user:

[2231] Check the suggested style and etiquette information to alleviate any concerns.

[2232] Specific examples using emotion engines

[2233] user:

[2234] In the app, users set the conditions of "commute," "autumn," and "business casual" and request outfit suggestions. The emotion engine then uses facial recognition to read the user's current emotions (e.g., nervousness, joy, fatigue).

[2235] server:

[2236] Items that meet the conditions are selected from the registered fashion items, and coordination suggestions are generated.

[2237] Based on the user's emotions recognized by the emotion engine, for example, if the user is feeling nervous, the system will suggest outfits that include items in relaxing colors and materials.

[2238] Device:

[2239] The coordinated suggestions applied to the user's avatar are visually displayed.

[2240] user:

[2241] The user checks the suggested outfits, selects them if satisfied, and then carries out the purchasing process for any missing items.

[2242] The system of the present invention allows users to effectively utilize their own fashion items and receive suggestions for appropriate coordination and event information according to their emotional state, thereby providing comprehensive support for users' fashion-related concerns and improving user satisfaction.

[2243] The processing flow will be explained below.

[2244] Specific processing flow of the virtual stylist system using the emotion engine

[2245] 1. Initial Setup and User Registration

[2246] Step 1:

[2247] User: Downloads and installs the app on a smartphone or other device.

[2248] Step 2:

[2249] User: Launch the app and enter basic information (name, email address, password) on the new account creation screen.

[2250] Step 3:

[2251] Terminal: Sends the entered information to the server.

[2252] Step 4:

[2253] Server: Stores the received user information in a database and creates a user account.

[2254] Step 5:

[2255] Users: Customize your profile information and avatar.

[2256] 2. Registering fashion items

[2257] Step 1:

[2258] User: Take a photo of a fashion item they own with their smartphone camera, such as a sweater, shirt, or jeans.

[2259] Step 2:

[2260] User: Register the captured image within the app.

[2261] Step 3:

[2262] On the device: Sends the uploaded image to the server.

[2263] Step 4:

[2264] Server: Analyzes the received image data and automatically tags it with classification information such as item category, color, and material.

[2265] Step 5:

[2266] Server: Stores tagged item data in a database, associating it with user account information.

[2267] 3. Coordination suggestions

[2268] Step 1:

[2269] User: Enter the conditions for which you would like the app to suggest outfits (season, occasion, style, etc.).

[2270] Step 2:

[2271] Device: Facial recognition is performed using the smartphone camera to analyze the user's emotional state along with their input criteria in the emotion engine.

[2272] Step 3:

[2273] Terminal: Sends the emotion data and coordination conditions obtained through the emotion engine to the server.

[2274] Step 4:

[2275] Server: Selects registered fashion items that match the criteria and runs an algorithm to generate coordination suggestions.

[2276] Step 5:

[2277] Server: Adjusts outfit suggestions based on the user's emotions recognized by the emotion engine. For example, if the user is feeling nervous, the server suggests outfits that include items with relaxing colors and materials.

[2278] Step 6:

[2279] Server: Generates image data to be applied to an avatar to visually display the generated coordination proposal.

[2280] Step 7:

[2281] Device: Displays the completed outfit suggestions to the user.

[2282] Step 8:

[2283] User: Review the proposed coordination and provide feedback if necessary.

[2284] 4. Suggestion and purchase of missing items

[2285] Step 1:

[2286] Server: Detects missing items based on the proposed outfit. For example, if the user does not have "leather shoes," it will detect this.

[2287] Step 2:

[2288] Server: Generates purchasing recommendation data (product images, brand, price, purchase link, etc.).

[2289] Step 3:

[2290] On the device: Display a list of missing items to the user with a link to purchase them.

[2291] Step 4:

[2292] User: Clicks on the purchase link if necessary and completes the purchase.

[2293] 5. Collaborative projects and promotions

[2294] Step 1:

[2295] Server: Regularly collects information on collaborations with popular fashion magazines and famous stylists.

[2296] Step 2:

[2297] Server: Sets collaboration information as a push notification and generates data to notify the user.

[2298] Step 3:

[2299] Device: Receives the push notification and displays it to the user.

[2300] Step 4:

[2301] User: Check push notifications to view collaborations and promotions that interest them.

[2302] 6. Recycling suggestions for unwanted items

[2303] Step 1:

[2304] Server: Analyzes the user's item database and detects items that have not been used for a long time.

[2305] Step 2:

[2306] Server: Generates notification data suggesting recycling or selling unwanted items at flea markets.

[2307] Step 3:

[2308] On the device: Show users a suggestion notification to guide them on how to recycle unwanted items.

[2309] Step 4:

[2310] User: Clicks on a link to recycle unwanted items.

[2311] 7. Suggestions for places to go

[2312] Step 1:

[2313] Server: Based on the suggested outfits, collects information on events, travel destinations, restaurants, etc. that the user may be interested in.

[2314] Step 2:

[2315] Server: Integrates the recommendation information and coordinated images to generate data to provide to users.

[2316] Step 3:

[2317] On your device: Show suggested places and events to the user.

[2318] Step 4:

[2319] Users: View events that interest them and learn more.

[2320] 8. Support for style and etiquette for ceremonial occasions

[2321] Step 1:

[2322] Server: Manages a database of information on style and general etiquette related to ceremonial occasions.

[2323] Step 2:

[2324] Server: Generates data to provide appropriate style and manner information in response to user requests.

[2325] Step 3:

[2326] On the device: Show suggested styles and mannerisms to the user and display the results on the user's avatar.

[2327] Step 4:

[2328] Users: Review suggested style and etiquette information to ease concerns.

[2329] Specific examples using emotion engines

[2330] Step 1:

[2331] User: The app sets the conditions "Commute," "Autumn," and "Business Casual," and uses the smartphone camera to recognize the user's face and analyze their emotional state.

[2332] Step 2:

[2333] Device: Sends the facial recognition results to the emotion engine to analyze the current emotion (e.g., tension, joy, fatigue).

[2334] Step 3:

[2335] Terminal: Sends analyzed emotion data to the server.

[2336] Step 4:

[2337] Server: Searches the fashion item database and selects items that match the conditions.

[2338] Step 5:

[2339] Server: Adjusts outfit suggestions based on the results of the emotion engine. For example, if the customer is feeling nervous, the server will suggest items with relaxing materials and colors.

[2340] Step 6:

[2341] Server: Visualizes the coordinated outfit suggestions after adjustments and generates image data to apply to the avatar.

[2342] Step 7:

[2343] On device: Display coordinated outfit suggestions to the user.

[2344] Step 8:

[2345] User: Check the suggested outfit and, if satisfied, click the link to purchase the missing items and purchase the items they need.

[2346] The system of the present invention comprehensively supports users' fashion concerns by making effective use of their fashion items and providing appropriate coordination suggestions according to their emotional state.

[2347] Example 2

[2348] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2349] Conventional fashion coordination systems have problems with low user satisfaction because they do not make full use of the clothes a user owns and do not take into account the user's emotional state when making suggestions. It is also difficult for users to grasp what items they are lacking, making it difficult to effectively purchase the items they need. Furthermore, they lack support for overall lifestyles, such as suggesting places to go or event information.

[2350] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for photographing clothes owned by the user and inputting item information, means for analyzing the item information and assigning classification data, means for inputting coordination conditions from the user, means for searching the user's item information based on the coordination conditions and generating coordination suggestions, means for visually displaying the coordination suggestions, means for detecting missing items and generating purchase recommendation data, and means for recognizing the user's emotional state and adjusting the coordination suggestions based on the results. This allows the user to more effectively utilize their own clothes and receive support for coordination according to their emotional state and overall lifestyle.

[2351] "User" refers to someone who uses this system to register fashion items and receive coordination suggestions.

[2352] "Clothing" refers to fashion items that users own, photograph, and enter.

[2353] "Item information" is data about clothing, and includes detailed information such as category, color, and material.

[2354] "Classification data" refers to classification information such as category, color, material, etc. obtained by analyzing item information.

[2355] "Coordination" refers to clothing combinations suggested based on the user's item information and specific conditions.

[2356] "Conditions" are elements that are taken into consideration when creating a coordination, and include the season, the scene, the style, and the like.

[2357] "Coordination suggestions" refer to outfit combinations generated by the server based on conditions entered by the user.

[2358] The "means for visually displaying" refers to a function for visually displaying the generated coordination proposal so that the user can confirm it.

[2359] "Purchase recommendation data" refers to information (product images, brand, price, purchase link, etc.) that detects missing items and encourages their purchase.

[2360] "Emotional state" refers to the user's current emotions (e.g., tension, joy, fatigue), and is a factor reflected in outfit suggestions.

[2361] An "avatar" refers to a digital character that appears as a user's alter ego and is used to visually confirm coordination suggestions.

[2362] The present invention relates to a virtual stylist system incorporating an emotion engine. This system proposes coordination and lifestyle ideas based on the clothes owned by the user, and also makes suggestions according to the user's emotional state.

[2363] Overall system configuration

[2364] The system is implemented in the following way.

[2365] 1. The user takes a photo of the clothing they own and enters the item information.

[2366] 2. Analyze the entered item information and assign classification data.

[2367] 3. Enter the coordination requirements from the user.

[2368] 4. Based on the coordination conditions, search for the user's item information and generate coordination suggestions.

[2369] 5. Visually display coordination suggestions.

[2370] 6. Detect missing items and generate purchasing recommendations.

[2371] 7. Based on the suggested outfit, the app will suggest places for the user to go and event information.

[2372] 8. Apply the suggested outfit to the user's avatar and visually display it.

[2373] 9. Recognize the user's emotional state and tailor outfit suggestions accordingly.

[2374] Initial Setup and User Registration

[2375] user:

[2376] Users download and install the app on their smartphone, launch it, and enter their name, email address, and password on the new account creation screen, then customize their profile information and avatar.

[2377] Device:

[2378] The terminal sends the entered information to the server via an "HTTP POST request."

[2379] server:

[2380] The server saves the received user information in a MySQL database and creates a user account. The server performs validation to ensure there are no duplicate email addresses.

[2381] Fashion item registration

[2382] user:

[2383] Users take a photo of their clothing with their smartphone camera and register the image in the app, along with the item name, category, and tags (e.g., color, material).

[2384] Device:

[2385] The device uploads the captured image to cloud storage (e.g., AWS S3), generates a URL for the image, and sends it to the server.

[2386] server:

[2387] The server analyzes the received image URL using the Google Cloud Vision API, extracts classification information such as the item's category, color, and material, associates it with the user's account, and stores it in a database.

[2388] Coordination suggestions

[2389] user:

[2390] Users can request outfit suggestions by inputting conditions such as season, occasion, and style, and the system also uses facial recognition to detect the user's emotions.

[2391] Device:

[2392] The terminal sends the entered condition data to the server via an "HTTP POST request."

[2393] server:

[2394] The server selects items that match the criteria from the registered item information and generates coordination suggestions. It also analyzes the user's emotions using an emotion engine (e.g., Azure Emotion API) and adjusts the coordination suggestions based on the results.

[2395] server:

[2396] In order to visually display the generated coordination proposal, image data to be applied to the user's avatar is generated.

[2397] Device:

[2398] The terminal displays the completed coordination proposal to the user.

[2399] Suggestion and purchase of missing items

[2400] server:

[2401] The server detects missing items based on the suggested outfits and generates purchasing recommendation data (product images, brand, price, purchase links, etc.).

[2402] Device:

[2403] The device will display a list of missing items to the user along with a link to purchase them.

[2404] user:

[2405] The user clicks on the purchase link as needed and completes the purchase process.

[2406] Collaborative projects and promotions

[2407] server:

[2408] The server collects collaboration information with popular media and experts, updates it periodically, and generates data to notify users of the collaboration information as push notifications.

[2409] Device:

[2410] The device receives the push notification and displays it to the user.

[2411] user:

[2412] Users can check push notifications and view collaboration projects and promotional information that interest them.

[2413] Recycling suggestions for unwanted items

[2414] server:

[2415] The server analyzes the user's item information, detects items that have not been used for a long time, and generates notification data for recycling suggestions.

[2416] Device:

[2417] The device will display a recycling suggestion notification to the user and guide them on how to recycle.

[2418] user:

[2419] The user clicks on the suggested link to complete the recycling process.

[2420] Suggestions for places to go

[2421] server:

[2422] Based on the user's coordination suggestions, the server collects information on events, travel destinations, restaurants, etc. that may be of interest to the user and generates proposal data.

[2423] Device:

[2424] The device will display suggested locations and event information to the user.

[2425] user:

[2426] Users can check the event information of their interest and check the details.

[2427] Specific examples using emotion engines

[2428] user:

[2429] Users can set the conditions of "commute," "autumn," and "business casual" and request outfit suggestions. At the same time, the emotion engine uses facial recognition to read the user's current emotions (e.g., nervousness, joy, fatigue).

[2430] server:

[2431] The server selects registered fashion items that match the user's criteria and generates outfit suggestions. For example, if the user is feeling nervous, the server will suggest outfits that include items in relaxing colors and materials based on the user's emotions as recognized by the emotion engine.

[2432] Device:

[2433] The terminal visually displays the coordination suggestions applied to the user's avatar.

[2434] user:

[2435] The user checks the suggested outfits, selects them if satisfied, and then proceeds to purchase any missing items.

[2436] Specific examples and prompts for the generative AI model

[2437] Examples:

[2438] "I'm looking for some fall clothes to wear to work. I'd prefer business casual, but I'm a little nervous."

[2439] Prompt for the generative AI model:

[2440] "Users are looking for fall clothing for work, so suggest business casual outfits. Also, users are feeling stressed, so create outfits that include relaxing colors and materials."

[2441] The system of the present invention allows users to effectively utilize their clothing and receive suggestions for outfits and event information that match their emotional state, thereby providing comprehensive support for fashion-related concerns and improving user satisfaction.

[2442] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2443] Step 1: User registration and initial setup

[2444] user:

[2445] Download and install the app on your smartphone.

[2446] Launch the app, enter your name, email address, and password on the new account creation screen, and press the "Submit" button.

[2447] Input: Name, Email Address, Password

[2448] Output: Registration request data

[2449] Device:

[2450] The registration request data is sent to the server via an HTTP POST request.

[2451] Input: Registration request data

[2452] Output: Request to server

[2453] server:

[2454] The received registration request data undergoes validation checks (such as checking for duplicate accounts), is saved in a MySQL database, and a user account is created.

[2455] Input: Registration request data, database

[2456] Output: New user account

[2457] Step 2: Register your fashion items

[2458] user:

[2459] Use your smartphone camera to take a photo of your clothing.

[2460] Select the image you took within the app, enter the item name, category, and tags (color, material, etc.), and register the image.

[2461] Input: Clothing photo, item name, category, tags (color, material, etc.)

[2462] Output: Registration request data

[2463] Device:

[2464] The captured image is uploaded to cloud storage (e.g. AWS S3) and a URL for the image is generated.

[2465] The tagged information along with the image URL is sent to the server via an HTTP POST request.

[2466] Input: Clothing photo, item name, category, tags (color, material, etc.)

[2467] Output: Request to server

[2468] server:

[2469] The received image URL is analyzed using the Google Cloud Vision API, and classification information such as the item's category, color, and material is extracted, associated with the user's account, and stored in a database.

[2470] Input: Image URL, tagging information

[2471] Output: Classification data, database user information

[2472] Step 3: Enter coordination conditions

[2473] user:

[2474] Enter the conditions for which you would like the app to suggest outfits (season, occasion, style, etc.).

[2475] Use facial recognition to detect emotions.

[2476] Input: Coordination condition data, face recognition data

[2477] Output: Coordinated condition request

[2478] Device:

[2479] The entered condition data and facial recognition data are sent to the server via an "HTTP POST request."

[2480] Input: Coordination condition data, face recognition data

[2481] Output: Request to server

[2482] Step 4: Generate outfit suggestions

[2483] server:

[2484] From the registered fashion items, items that match the inputted coordination conditions are selected.

[2485] An emotion engine (e.g., Azure Emotion API) is used to analyze the user's emotions and tailor outfit suggestions based on the results.

[2486] Input: Item information, coordination conditions, emotion data

[2487] Output: Coordination suggestion data

[2488] Step 5: View outfit suggestions

[2489] server:

[2490] In order to visually display the generated coordination proposal, image data to be applied to the user's avatar is generated.

[2491] Input: Coordination suggestion data

[2492] Output: Coordinate image data

[2493] Device:

[2494] Coordinate image data is received and displayed to the user.

[2495] Input: Coordinate image data

[2496] Output: what is displayed to the user

[2497] user:

[2498] Review the proposed coordination and provide feedback as needed.

[2499] Input: Coordination image

[2500] Output: Feedback data

[2501] Step 6: Suggest and purchase missing items

[2502] server:

[2503] Detect missing items based on suggested coordinates.

[2504] Generate purchasing recommendation data (product images, brand, price, purchase link, etc.) for items in short supply.

[2505] Input: Coordination suggestions, user item data

[2506] Output: Purchase recommendation data

[2507] Device:

[2508] Display a list of missing items to the user with a link to purchase them.

[2509] Input: Purchase recommendation data

[2510] Output: what is displayed to the user

[2511] user:

[2512] Click on the purchase link and complete the purchase process as needed.

[2513] Enter: Purchase Link

[2514] Output: Purchase confirmation data

[2515] Step 7: Suggest places to go

[2516] server:

[2517] Based on the suggested outfits, we collect information on events, travel destinations, restaurants, etc. that may interest the user.

[2518] Input: Coordination suggestions

[2519] Output: Event proposal data

[2520] Device:

[2521] Showing users suggested places and events.

[2522] Input: Event proposal data

[2523] Output: what is displayed to the user

[2524] user:

[2525] Check the event information that interests you and find out more details.

[2526] Input: Event information

[2527] Output: Event details display

[2528] Step 8: Recognizing and Coordinating Emotional States

[2529] user:

[2530] Using facial recognition, the emotion engine reads the user's current emotions (e.g., nervousness, joy, fatigue).

[2531] Input: Facial recognition data

[2532] Output: Emotion data

[2533] server:

[2534] Coordination suggestions are adjusted based on emotional data recognized by the emotion engine.

[2535] Input: Emotion data

[2536] Output: Adjusted outfit suggestions

[2537] Device:

[2538] The adjusted coordination suggestions are applied to the user's avatar and visually displayed.

[2539] Input: Adjusted outfit suggestions

[2540] Output: Adjusted coordinate image

[2541] user:

[2542] Check the suggested outfits, purchase the items you need, and provide feedback.

[2543] Input: Adjusted coordinate image

[2544] Output: Purchase confirmation data, feedback data

[2545] (Application example 2)

[2546] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2547] Conventional fashion coordination systems simply make suggestions based on the user's owned fashion items and basic requirements, without considering the user's emotional state. As a result, they are unable to make suggestions that match the user's current emotions and mood, making it difficult to increase satisfaction. Furthermore, they lack an efficient means to encourage the purchase of missing items in the suggested coordination.

[2548] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for photographing fashion items owned by the user and inputting item data, a means for analyzing the item data and assigning classification information, a means for accepting coordination conditions input from the user, a means for searching the user's item data based on the coordination conditions and generating coordination suggestions, a means for recognizing the user's emotions and adjusting the coordination suggestions based on the emotions, and a means for applying the suggested coordination to the user's avatar and visually displaying it. This makes it possible to propose optimal fashion coordination that takes emotions into consideration. Furthermore, the server can also improve the user's purchasing experience by detecting missing items and generating purchase recommendation data.

[2549] "Fashion items" is a general term for fashion-related possessions such as clothing, accessories, and shoes.

[2550] "Item data" refers to attribute information such as images, categories, colors, and materials of fashion items owned by a user.

[2551] "Coordination conditions" are requirements such as season, occasion, style, etc. that are input by the user when requesting suggestions.

[2552] A "coordination suggestion" is a specific fashion combination generated based on coordination conditions.

[2553] "Emotion" is a general term for a user's current psychological state or mood (e.g., joy, tension, fatigue, etc.).

[2554] An "avatar" is a digital persona or character that virtually represents a user.

[2555] "Facial recognition" is a technology that analyzes a user's facial image and reads emotions from their facial expressions.

[2556] "Purchase recommendation data" is information such as the brand, price, and purchase link of missing items based on the suggested outfit.

[2557] This embodiment of the present invention describes a system for suggesting outfits based on data on fashion items owned by a user. This system proposes outfits taking into account the emotional state of the user and recommends the purchase of missing items as needed, thereby increasing user satisfaction.

[2558] Initial Setup and User Registration

[2559] Users download a dedicated application onto their smartphone or smart glasses and create a new account. Once the application is launched, users enter basic information (name, email address, password) and customize their profile information and avatar.

[2560] Fashion item registration

[2561] Users take photos of their own fashion items, such as sweaters, shirts, and jeans. The images are then uploaded to the server by the application. The server analyzes the image data and automatically tags the items with classification information such as category, color, and material. This classification information is then associated with the user's account information and stored in a database.

[2562] Coordination suggestions

[2563] When a user wants to receive outfit suggestions, they input the season, occasion, style, and other conditions into the application. Furthermore, the smart glasses' built-in facial recognition function is used to read the user's emotions. The emotion engine analyzes the user's current emotions (e.g., nervousness, joy, fatigue, etc.).

[2564] The server proposes optimal outfits from registered fashion items based on the user's conditions and emotional state. The proposed outfits are applied to the user's avatar and visually displayed on the smart glasses display. The user can review the proposed outfits and provide feedback if necessary.

[2565] Suggestion and purchase of missing items

[2566] The server detects missing items based on the suggested outfit. For example, if the user does not own "leather shoes," it will detect this. It then generates purchasing recommendation data for the missing items (product images, brand, price, purchasing links, etc.). The terminal displays a list of missing items and purchase links to the user. The user can click the purchase link and complete the purchase process as needed.

[2567] Emotion recognition and concrete examples

[2568] The application uses the smart glasses' camera to capture the user's face and analyzes their emotions using an emotion engine. For example, if a user sets the conditions "commute," "autumn," and "business casual," and the emotion engine detects "tension" through facial recognition, it will suggest outfits that include items with relaxing colors and materials.

[2569] In this case, an example of a prompt sentence input to the generative AI model is as follows:

[2570] The user sets the conditions of "commute," "autumn," and "business casual," and emotion recognition detects "tension." Based on these conditions, please suggest the best outfit for the user from their fashion items.

[2571] In this way, the suggested coordination takes into account the user's emotional state and can improve user satisfaction.

[2572] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2573] Step 1:

[2574] Users download a dedicated application onto a device such as a smartphone or smart glasses and create a new account. The user enters basic information (name, email address, password) and customizes their profile information and avatar. The entered information is sent by the device to a server, which stores the received information in a database and creates a user account.

[2575] Step 2:

[2576] The user takes a photo of a fashion item they own using the smart glasses' camera. For example, a sweater, shirt, or jeans. The device then uploads the image to a server. The server analyzes the image data and automatically tags the item with classification information such as category, color, and material. This tagged information is then stored in a database along with the user's account information.

[2577] Step 3:

[2578] The user inputs the conditions for which they would like to receive outfit suggestions (season, occasion, style, etc.) into the application. The smart glasses' built-in facial recognition function is used to capture the user's face. The device then sends the conditions data entered by the user and the captured facial image data to the server. The server then uses an emotion engine to analyze the facial image and recognize the user's current emotion (e.g., nervousness, joy, fatigue, etc.).

[2579] Step 4:

[2580] The server generates outfit suggestions based on the user's conditions and emotional state. It selects registered fashion items that match the conditions and then runs an algorithm to suggest optimal outfits that take emotions into consideration. The outfit suggestions are generated and applied to the user's avatar. The generated outfit suggestions are sent to the device and visually displayed on the smart glasses' display.

[2581] Step 5:

[2582] Based on the suggested outfit, the server detects any missing items. For example, if the user does not own "leather shoes," the server detects those items. For each missing item, the server generates purchasing recommendation data (product image, brand, price, purchase link, etc.). The device displays a list of missing items and a purchase link to the user.

[2583] Step 6:

[2584] The user can click on the purchase link provided to complete the purchase of the missing item. Once the user completes the purchase, the information is updated by the server and stored in the database.

[2585] Step 7:

[2586] The application allows users to provide feedback on the proposed coordination. The user enters the feedback and the device sends it to the server. The server receives the feedback and uses it to improve the coordination algorithm and update personal data.

[2587] These steps enable the system to suggest optimal fashion coordination that takes into account the user's emotional state, improving the user's shopping experience.

[2588] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2589] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2590] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2591] [Fourth embodiment]

[2592] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2593] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2594] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2595] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2596] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2597] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2598] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2599] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2600] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2601] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2602] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2603] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2604] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2605] Specific processes and operations of the virtual stylist system of the present invention will be described below. Note that in the following description, the server, the terminal, and the user are used as the subject.

[2606] 1. Initial Setup and User Registration

[2607] user:

[2608] Users download and install the app on their smartphone or other device.

[2609] When the app is launched, an account creation screen appears, where users enter basic information such as their name, email address, and password to create an account.

[2610] Users then customize their profile information and avatar.

[2611] Device:

[2612] The entered information is sent to the server.

[2613] server:

[2614] Store the received information in a database and create a user account.

[2615] 2. Registering fashion items

[2616] user:

[2617] Take a photo of a fashion item you own with your smartphone camera, such as a sweater, shirt, or jeans.

[2618] Upload the captured images through the app's interface.

[2619] Device:

[2620] Sends the uploaded image to the server.

[2621] server:

[2622] The received image data is analyzed and automatically tagged with classification information such as the item's category, color, and material.

[2623] Stores item data in a database, associated with the user's account information.

[2624] 3. Coordination suggestions

[2625] user:

[2626] Enter the conditions for which you would like to receive outfit suggestions (season, occasion, style, etc.) into the app.

[2627] Device:

[2628] The entered condition data is sent to the server.

[2629] server:

[2630] The system selects registered fashion items that meet the conditions and runs an algorithm to generate coordination suggestions.

[2631] In order to visually display the generated coordination proposal, image data to be applied to an avatar is generated.

[2632] Device:

[2633] The completed coordination proposal is displayed to the user.

[2634] user:

[2635] Review the proposed coordination and provide feedback as needed.

[2636] 4. Suggestion and purchase of missing items

[2637] server:

[2638] Based on the suggested outfit, it will detect missing items. For example, if the user does not own "leather shoes," it will detect this.

[2639] Generates purchasing recommendation data and provides purchasing information (product images, brand, price, purchase link, etc.).

[2640] Device:

[2641] Display a list of missing items to the user with a link to purchase them.

[2642] user:

[2643] Click on the purchase link and complete the purchase process as needed.

[2644] 5. Collaborative projects and promotions

[2645] server:

[2646] We regularly collect information on collaborations with popular fashion magazines and famous stylists.

[2647] Set collaboration information as a push notification and generate data to notify the user.

[2648] Device:

[2649] Receive push notifications and display them to the user.

[2650] user:

[2651] Check your push notifications to view collaborations and promotions that interest you.

[2652] 6. Recycling suggestions for unwanted items

[2653] server:

[2654] Analyzes the user's item database to detect items that have not been used for a long time.

[2655] For unwanted items, suggestion notification data is generated recommending recycling or using them at a flea market.

[2656] Device:

[2657] Show users a suggestion notification to guide them on how to recycle unwanted items.

[2658] user:

[2659] Click the link to begin recycling your unwanted items.

[2660] 7. Suggestions for places to go

[2661] server:

[2662] Based on the suggested outfits, we collect information on events, travel destinations, restaurants, etc. that may interest the user.

[2663] The proposed information and coordinated images are integrated to generate data to be provided to the user.

[2664] Device:

[2665] Showing users suggested places and events.

[2666] user:

[2667] Check the event information that interests you and find out more details.

[2668] 8. Support for style and etiquette for ceremonial occasions

[2669] server:

[2670] Maintain a database of information on style and general etiquette related to weddings, funerals, and other ceremonies.

[2671] Generate data that provides appropriate style and manner information in response to user requests.

[2672] Device:

[2673] The suggested style and manner information is displayed to the user, and a screen showing the style applied to the user's avatar is displayed.

[2674] user:

[2675] Check the suggested style and etiquette information to alleviate any concerns.

[2676] Specific examples

[2677] user:

[2678] In the app, set the conditions "commute," "autumn," and "business casual" and request outfit suggestions.

[2679] Check the outfit suggested by the server: gray sweater, white shirt, and dark blue jeans.

[2680] Check the proposal and give feedback such as "Leather shoes are needed."

[2681] server:

[2682] Based on the suggested outfit, it detects that "leather shoes" are in short supply and generates purchasing information.

[2683] Device:

[2684] Present the user with a purchase link and provide them with a link to complete the purchase.

[2685] user:

[2686] Click on the provided link and complete the process to purchase the "leather shoes."

[2687] The system of the present invention allows users to effectively utilize their fashion items and eliminates the anxiety of coordinating outfits for everyday wear and special events.

[2688] The processing flow will be explained below.

[2689] Program processing flow

[2690] 1. Initial Setup and User Registration

[2691] Step 1:

[2692] User: Downloads and installs the app on a smartphone or other device.

[2693] Step 2:

[2694] User: Launch the app and enter basic information (name, email address, password) on the new account creation screen.

[2695] Step 3:

[2696] Terminal: Sends the entered information to the server.

[2697] Step 4:

[2698] Server: Stores the received user information in a database and creates a user account.

[2699] Step 5:

[2700] Users: Customize your profile information and avatar.

[2701] 2. Registering fashion items

[2702] Step 1:

[2703] User: Take a photo of a fashion item they own with their smartphone camera, such as a sweater, shirt, or jeans.

[2704] Step 2:

[2705] User: Register the captured image within the app.

[2706] Step 3:

[2707] On the device: Sends the uploaded image to the server.

[2708] Step 4:

[2709] Server: Analyzes the received image data and automatically tags it with classification information such as item category, color, and material.

[2710] Step 5:

[2711] Server: Stores tagged item data in a database, associating it with user account information.

[2712] 3. Coordination suggestions

[2713] Step 1:

[2714] User: Enter the conditions for which you would like the app to suggest outfits (season, occasion, style, etc.).

[2715] Step 2:

[2716] Terminal: Sends the entered condition data to the server.

[2717] Step 3:

[2718] Server: Selects registered fashion items that match the criteria and runs an algorithm to generate coordination suggestions.

[2719] Step 4:

[2720] Server: Generates image data to be applied to an avatar to visually display the generated coordination proposal.

[2721] Step 5:

[2722] Device: Displays the completed outfit suggestions to the user.

[2723] Step 6:

[2724] User: Review the proposed outfits and provide feedback.

[2725] 4. Suggestion and purchase of missing items

[2726] Step 1:

[2727] Server: Detects missing items based on the proposed outfit. For example, if the user does not have "leather shoes," it will detect this.

[2728] Step 2:

[2729] Server: Generates purchasing recommendation data (product images, brand, price, purchase link, etc.).

[2730] Step 3:

[2731] On the device: Display a list of missing items to the user with a link to purchase them.

[2732] Step 4:

[2733] User: Clicks on the purchase link if necessary and completes the purchase.

[2734] 5. Collaborative projects and promotions

[2735] Step 1:

[2736] Server: Regularly collects information on collaborations with popular fashion magazines and famous stylists.

[2737] Step 2:

[2738] Server: Sets collaboration information as a push notification and generates data to notify the user.

[2739] Step 3:

[2740] Device: Receives the push notification and displays it to the user.

[2741] Step 4:

[2742] User: Check push notifications to view collaborations and promotions that interest them.

[2743] 6. Recycling suggestions for unwanted items

[2744] Step 1:

[2745] Server: Analyzes the user's item database and detects items that have not been used for a long time.

[2746] Step 2:

[2747] Server: Generates notification data suggesting recycling or selling unwanted items at flea markets.

[2748] Step 3:

[2749] On the device: Show users a suggestion notification to guide them on how to recycle unwanted items.

[2750] Step 4:

[2751] User: Clicks on a link to recycle unwanted items.

[2752] 7. Suggestions for places to go

[2753] Step 1:

[2754] Server: Based on the suggested outfits, collects information on events, travel destinations, restaurants, etc. that the user may be interested in.

[2755] Step 2:

[2756] Server: Integrates the recommendation information and coordinated images to generate data to provide to users.

[2757] Step 3:

[2758] On your device: Show suggested places and events to the user.

[2759] Step 4:

[2760] Users: View events that interest them and learn more.

[2761] 8. Support for style and etiquette for ceremonial occasions

[2762] Step 1:

[2763] Server: Manages a database of information on style and general etiquette related to ceremonial occasions.

[2764] Step 2:

[2765] Server: Generates data to provide appropriate style and manner information in response to user requests.

[2766] Step 3:

[2767] On the device: Show suggested styles and mannerisms to the user and display the results on the user's avatar.

[2768] Step 4:

[2769] Users: Review suggested style and etiquette information to ease concerns.

[2770] This will enable us to provide comprehensive support to users regarding their fashion concerns.

[2771] Example 1

[2772] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2773] Today's consumers face daily fashion-related worries and difficulties in coordinating outfits. They particularly need a way to efficiently coordinate outfits for each season and occasion while making the most of the items they own. Furthermore, there is no comprehensive fashion management system that can suggest and purchase items they are running low on, provide information on collaborations with the fashion industry, or even recycle unwanted items. This often results in users wasting time and effort choosing their daily fashion, which can be stressful.

[2774] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2775] In this invention, the server includes means for photographing fashion items owned by the user and inputting item data, means for analyzing the item data and assigning classification information, means for accepting coordination conditions input from the user, means for searching the user's item data based on the coordination conditions and generating coordination suggestions, means for visually displaying the coordination suggestions, means for detecting missing items and generating purchase recommendation data, means for collecting collaboration information with specific fashion magazines and stylists and generating push notifications, means for analyzing the user's item database and detecting items that have not been used for a long time and suggesting recycling, and means for providing information on styles and general etiquette related to ceremonial occasions. This allows users to effectively utilize their own fashion items, reduces anxiety about coordinating outfits for everyday use and special events, and enables comprehensive fashion management.

[2776] "User" refers to any individual or organization that uses this system.

[2777] "Fashion items" refers to fashion-related items such as clothing, accessories, and shoes owned by a user.

[2778] "Item data" refers to data that includes information such as images, categories, colors, and materials of fashion items.

[2779] "Coordination conditions" refer to conditions such as season, scene, and style specified by the user.

[2780] "Coordination suggestions" refer to style suggestions generated based on the user's fashion items and coordination conditions.

[2781] "Purchase recommendation data" refers to data that identifies missing items and includes related purchasing information (such as product images, brand, price, and purchase links).

[2782] "Collaboration Information" refers to information provided in collaboration with specific fashion magazines or stylists.

[2783] "Push notification" refers to a notification sent from a server to a user.

[2784] "Recycling suggestions" refer to suggestions that encourage users to recycle or sell fashion items that they have not used for a long time.

[2785] "Style information" refers to information about fashion styles and general manners related to ceremonial occasions.

[2786] The specific processing and operation of the virtual stylist system of the present invention will be described below. The present invention manages the user's fashion items and suggests coordination suggestions by exchanging data between the user, the terminal, and the server.

[2787] Initial Setup and User Registration

[2788] User: First, the user downloads and installs a dedicated app onto their smartphone or other device. When the app is launched, an account creation screen appears. The user enters basic information such as name, email address, and password to create an account. They can then customize their profile information and avatar.

[2789] Terminal: This input information is sent from the terminal to the server.

[2790] Server: The server stores the received information in a database and creates a user account.

[2791] Fashion item registration

[2792] User: The user takes a photo of a fashion item they own, such as a sweater, shirt, or jeans, with their smartphone camera and uploads the image through the app's interface.

[2793] On your device: Uploaded images are sent to the server.

[2794] Server: The received image data is analyzed using services such as the Google Cloud Vision API. After analysis, classification information such as the item's category, color, and material is automatically tagged and stored in a database in association with the user's account information.

[2795] Coordination suggestions

[2796] User: The user inputs the conditions for which they would like to receive outfit suggestions (for example, season, occasion, style, etc.) into the app.

[2797] Terminal: The entered condition data is sent to the server.

[2798] Server: The server selects registered fashion items that match the input criteria and runs an algorithm to generate outfit suggestions. This algorithm also takes into account the user's past preferences and trend information to suggest optimal outfits. The generated outfit suggestions are prepared in a form that can be displayed visually, and image data to be applied to the user's avatar is generated.

[2799] Terminal: The completed outfit suggestions are displayed to the user, who can then review the suggestions and provide feedback if necessary.

[2800] Suggestion and purchase of missing items

[2801] Server: Detects missing items based on the suggested outfit. For example, if the user does not own "leather shoes," it detects this. To fill this gap, it generates purchasing recommendation data and provides detailed information such as product images, brand, price, and purchase links.

[2802] On the device: Display a list of missing items to the user with a link to purchase them.

[2803] User: Clicks on the purchase link if necessary and completes the purchase.

[2804] Collaborative projects and promotions

[2805] Server: Periodically collects information about collaborations with specific fashion magazines and famous stylists. The collected information is sent to users via push notifications.

[2806] Device: Receives the push notification and displays it to the user.

[2807] User: Check push notifications to view collaborations and promotions that interest them.

[2808] Recycling suggestions for unwanted items

[2809] Server: Analyzes the user's item database and detects items that have not been used for a long time. Based on this, it generates notification data recommending recycling or selling them at a flea market.

[2810] On the device: Show users a suggestion notification to guide them on how to recycle unwanted items.

[2811] User: Clicks on a link to recycle unwanted items.

[2812] Suggestions for places to go

[2813] Server: Based on the suggested outfits, the server collects information on events, travel destinations, restaurants, etc. that the user may be interested in. The server then integrates the suggested information with the outfit images to generate data to be provided to the user.

[2814] On your device: Show suggested places and events to the user.

[2815] Users: View events that interest them and learn more.

[2816] Style and etiquette support for ceremonial occasions

[2817] Server: Manages a database of information on style and general etiquette related to ceremonial occasions. Generates data to provide appropriate style and etiquette information in response to user requests.

[2818] On the device: Show suggested styles and mannerisms to the user, and show the styled screen on the user's avatar.

[2819] Users: Review suggested style and etiquette information to ease concerns.

[2820] Specific examples

[2821] As an example of a prompt, if the user sets the conditions "commute," "autumn," and "business casual" and requests outfit suggestions, the server will suggest an outfit of "gray sweater, white shirt, and dark blue denim." If the user provides feedback that "I need leather shoes," the server will detect the shortage and generate purchasing information, and the device will present the user with a purchase link.

[2822] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2823] Step 1:

[2824] User: Download and install the Virtual Stylist System app from the smartphone app store. Launch the app and proceed to the account creation screen. Here, enter information such as your name, email address, and password to create an account. The entered information is sent to your device.

[2825] Input: User's basic information (name, email address, password)

[2826] Output: User information is sent to the terminal

[2827] Step 2:

[2828] Terminal: Sends the account information entered by the user to the server.

[2829] Input: User information (name, email address, password)

[2830] Output: User information is sent to the server

[2831] Step 3:

[2832] Server: The received user information is saved in the database and a user account is created. This completes the user registration.

[2833] Input: User information sent from the device

[2834] Output: User information stored in database, account created

[2835] Step 4:

[2836] User: After creating an account, customize your profile information and avatar. Enter the customized information and send it to your device.

[2837] Input: Profile information, avatar setting information

[2838] Output: Profile information and avatar settings are sent to the device.

[2839] Step 5:

[2840] Device: Sends user-customized information to the server.

[2841] Input: Profile information, avatar setting information

[2842] Output: Customization information sent to the server

[2843] Step 6:

[2844] Server: Save the received customization information in the database. This completes the initial setup and user registration.

[2845] Input: Customization information sent from the device

[2846] Output: Customization information stored in the database

[2847] Step 7:

[2848] Users take photos of their fashion items with their smartphone camera and upload the images through the app's interface.

[2849] Input: Photographed fashion item images

[2850] Output: The image is captured to the device.

[2851] Step 8:

[2852] On the device: Sends the uploaded image to the server.

[2853] Input: Photographed fashion item images

[2854] Output: Image data sent to the server

[2855] Step 9:

[2856] Server: Analyzes the received image data using services such as Google Cloud Vision API to generate classification information such as item category, color, material, etc. The classification information is then tagged to the item data and stored in a database.

[2857] Input: Image data sent from the device

[2858] Output: Item data associated with classification information, item data stored in a database

[2859] Step 10:

[2860] User: Enter the conditions for which they would like to receive outfit suggestions (e.g., season, occasion, style, etc.) into the app and send the results to their device.

[2861] Input: Coordination conditions

[2862] Output: Coordinate conditions are captured in the terminal

[2863] Step 11:

[2864] Terminal: Sends the entered coordination condition data to the server.

[2865] Input: Coordination condition data

[2866] Output: Coordinate condition data sent to the server

[2867] Step 12:

[2868] Server: Runs an algorithm to select registered fashion items that match coordination criteria. Based on the selected items, it generates coordination suggestions and generates image data to be applied to the user's avatar.

[2869] Input: Registered fashion item data, coordination condition data

[2870] Output: Coordination suggestions and image data to be applied to the avatar

[2871] Step 13:

[2872] Device: Displays the completed outfit suggestions to the user.

[2873] Input: Coordination suggestions and image data sent from the server

[2874] Output: Outfit suggestions displayed on the user's device

[2875] Step 14:

[2876] User: Check the suggested outfit and provide feedback if necessary. For example, enter a comment such as "I need leather shoes" and send that information to the device.

[2877] Input: Feedback information (e.g., I need leather shoes)

[2878] Output: Feedback information captured in terminal

[2879] Step 15:

[2880] Device: Sends user feedback information to the server.

[2881] Input: Feedback information

[2882] Output: Feedback information sent to the server

[2883] Step 16:

[2884] Server: Based on the proposed outfit, detects missing items (e.g., leather shoes) and generates purchasing recommendations, including product images, brand, price, and purchase link details.

[2885] Input: User feedback information, suggested coordinate data

[2886] Output: Purchase recommendation data

[2887] Step 17:

[2888] Device: Display purchase recommendation data to the user and provide a purchase link.

[2889] Input: Purchase recommendation data sent from the server

[2890] Output: Purchase recommendation data displayed on the user's device

[2891] Step 18:

[2892] User: Clicks on the purchase link and completes the purchase.

[2893] Input: Purchase link click

[2894] Output: Completed checkout

[2895] Step 19:

[2896] Server: Periodically collects collaboration information with specific fashion magazines and stylists and generates push notifications.

[2897] Input: Information collected from fashion magazines and stylists

[2898] Output: Generates push notification data

[2899] Step 20:

[2900] Device: Receives the push notification and displays it to the user.

[2901] Input: Push notification data sent from the server

[2902] Output: A push notification that appears on the user's device

[2903] Step 21:

[2904] User: Check push notifications to view collaborations and promotions that interest them.

[2905] Input: Push notification click

[2906] Output: Collaboration and promotion information to be displayed

[2907] Step 22:

[2908] Server: Analyzes the user's item database and detects items that have not been used for a long time. Generates notification data suggesting recycling or selling items at a flea market.

[2909] Input: Item database

[2910] Output: Recycling suggestion notification data

[2911] Step 23:

[2912] Device: Show recycling suggestion notification to user.

[2913] Input: Recycle proposal notification data sent from the server

[2914] Output: A recycling suggestion notification displayed on the user's device.

[2915] Step 24:

[2916] User: Clicks on a link to recycle unwanted items.

[2917] Input: Click on recycle link

[2918] Output: Start of recycling procedure

[2919] Step 25:

[2920] Server: Based on the suggested outfits, collects information on events, travel destinations, and restaurants that the user may be interested in, and generates suggestion data.

[2921] Input: Proposed coordinate data

[2922] Output: Event and location suggestions

[2923] Step 26:

[2924] On your device: Show suggested places and events to the user.

[2925] Input: Proposal data sent from the server

[2926] Output: Event and location information displayed on the user's device

[2927] Step 27:

[2928] Users: View events that interest them and learn more.

[2929] Input: Click on event information

[2930] Output: Detailed information about the event displayed

[2931] Step 28:

[2932] Server: Manages a database of information on style and general etiquette related to ceremonial occasions, and provides data in response to user requests.

[2933] Input: User request

[2934] Output: Style and manner information provided

[2935] Step 29:

[2936] On the device: Show suggested styles and mannerisms to the user, and show the styled screen on the user's avatar.

[2937] Input: Style and manner information sent from the server

[2938] Output: Style and manner information displayed on the user's device

[2939] Step 30:

[2940] Users: Review suggested style and etiquette information to ease concerns.

[2941] Input: Check style and manner information

[2942] Output: User concerns resolved

[2943] In this way, users can effectively manage their fashion items through every step, eliminating the worry of choosing everyday fashion or coordinating outfits for special events.

[2944] (Application example 1)

[2945] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2946] Problems faced by users of conventional fashion coordination systems include the complicated process of suggesting outfits in real time and purchasing missing items, making it difficult to use on the go. Furthermore, there are limited ways for users to visually check the results of applying suggested outfits to their owned fashion items, limiting the user experience. The problem this invention aims to solve is to enable real-time suggestions and purchasing procedures, thereby improving user convenience and satisfaction.

[2947] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2948] In this invention, the server includes means for photographing fashion items owned by the user and inputting item data, means for analyzing the item data and assigning classification information, means for accepting coordination condition input from the user, means for searching the user's item data based on the coordination conditions and generating coordination suggestions, means for visually displaying the coordination suggestions, means for detecting missing items and generating purchase recommendation data, means for allowing the user to check the suggested coordination in real time and proceed with purchasing the missing items, and means for visually displaying the results via smart glasses or a head-mounted display, thereby enabling the user to check the coordination suggestions in real time and immediately proceed with purchasing the missing items.

[2949] "User" refers to an individual consumer who uses the system.

[2950] "Fashion items" refer to items that make up an individual's appearance, such as clothing, accessories, and shoes.

[2951] "Item data" refers to images of each fashion item owned by the user and their characteristic information.

[2952] "Classification information" refers to information that describes characteristics such as the category, color, material, and style of a fashion item.

[2953] "Coordination conditions" refer to the desired scene, season, style, and other conditions that the user enters into the app.

[2954] "Coordination suggestions" refer to fashion combinations generated based on the user's item data and coordination conditions.

[2955] "Visual display means" refers to the method by which visual information is presented to the user through an application or device.

[2956] "Missing items" refer to items that are necessary for the proposed outfit but that the user does not own.

[2957] "Purchase recommendation data" refers to product information and links provided to help users purchase items they are missing.

[2958] "Means for checking in real time" refers to a method in which suggested outfits are instantly presented to users, allowing them to provide feedback and complete the purchase process.

[2959] "Smart glasses" are a type of wearable device that displays information in the user's field of vision.

[2960] A "head-mounted display" is a display device that is worn on the user's head and has the function of providing visual information.

[2961] "On-the-go use" refers to a user using the system's functions outside of their home.

[2962] The following describes in detail the embodiments of the present invention: First, an overview of the entire system and specific examples of each processing step are shown.

[2963] System Configuration

[2964] This invention consists of a user, a terminal (smart glasses or a head-mounted display), and a server.

[2965] Hardware and Software

[2966] 1. Hardware:

[2967] Smart glasses (e.g., smart glasses, head-mounted displays)

[2968] Device (e.g. smartphone, tablet)

[2969] 2. Software:

[2970] Image analysis algorithms (e.g., OpenCV)

[2971] Deep learning models (e.g. TensorFlow, PyTorch)

[2972] Cloud services (e.g., Google Cloud Vision API, Amazon Rekognition)

[2973] Database (e.g. Firebase Cloud Firestore)

[2974] API (e.g., online shopping API)

[2975] Program processing description

[2976] The program for this system is designed to efficiently manage the fashion items owned by the user, provide real-time coordination suggestions, and assist in purchasing missing items. Each processing step and its operation are explained in detail below.

[2977] 1. Register user item data:

[2978] Users use their device's camera function to take photos of fashion items they own and upload the image data to the app.

[2979] The terminal transmits the image data received from the user to the server.

[2980] The server uses Google Cloud Vision API and Amazon Rekognition to analyze images and generate classification information such as item ca...

Claims

1. A means for users to take photos of fashion items they own and input item data; A means for analyzing item data and assigning classification information; A means for receiving coordinate condition input from a user; A means for searching for user item data based on coordination conditions and generating coordination suggestions; A means for visually displaying coordination suggestions; a means for detecting missing items and generating purchase recommendation data; A system that integrates these methods.

2. The system according to claim 1, further comprising means for suggesting places and event information for the user to go to based on the coordination suggestion.

3. 10. The system of claim 1, further comprising means for visually displaying the suggested outfit applied to the user's avatar.

4. 10. The system of claim 1, further comprising means for generating a notification of a suggestion for recycling an unwanted item.

5. 10. The system of claim 1, further comprising means for providing information regarding style and etiquette for ceremonial occasions.

Images