System

Abstract

A system is provided.SOLUTION: A system comprising: means for capturing an image of a user's clothes; means for analyzing the captured image to extract clothes attributes; means for storing the extracted clothes attributes in a database; means for generating coordinates based on a user's clothes list and external information based on a user's request; means for displaying the generated coordinates to a user's vision; means for the user to send feedback; and means for suggesting suitable clothes and generating a purchase link thereof based on the user's feedback.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention aims to effectively solve various problems that users face daily when choosing clothes. Specifically, the present invention aims to solve the following problems:

[0005] Choosing clothes every day is a hassle and takes time.

[0006] I don't know what clothes suit me, so I always end up wearing the same clothes.

[0007] I don't know how to coordinate appropriately for different ages and situations.

[0008] I'm so busy I don't have time to go clothes shopping.

[0009] When purchasing online, there are cases where the size, texture, or design is not what you expected. [Means for solving the problem]

[0010] The present invention is a "coordination suggestion system" for solving the above-mentioned problems, and is a system including the following means.

[0011] means for capturing images of a user's clothing;

[0012] A means for analyzing the captured image to extract attributes of the clothing;

[0013] A means for storing the extracted clothing attributes in a database;

[0014] A means for generating a coordinate based on a user's clothing list and external information in response to a user's request;

[0015] a means for visually displaying the generated coordinates to a user;

[0016] a means for users to submit feedback;

[0017] A means for suggesting suitable clothing items based on user feedback and generating a link to purchase the clothing items;

[0018] It is a system including:

[0019] "Capture" means taking an image of the user's clothes.

[0020] "Image analysis" is the process of extracting attributes from captured images of clothing.

[0021] "Attributes" refer to characteristic information such as clothing category, color, pattern, and material.

[0022] The "database" is a storage device for storing the extracted attribute information of clothes.

[0023] "Coordination" refers to combining multiple pieces of clothing owned by the user to create a single style.

[0024] "External information" refers to additional information necessary for creating a coordinated outfit, such as a user's request, weather information, and schedule information.

[0025] A "generative AI model" is an artificial intelligence algorithm that creates the optimal outfit from multiple options based on input data.

[0026] "Feedback" refers to the user's evaluation or opinion on the proposed outfit.

[0027] A "purchase link" is the URL or access means of an online store that allows a user to immediately purchase the suggested clothing.

[0028] "Suggestion" means showing the user the results of the coordination generated by the system. [Brief explanation of the drawings]

[0029] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0030] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0031] First, the terms used in the following description will be explained.

[0032] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0033] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0034] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0035] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0036] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0037] [First embodiment]

[0038] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0039] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0040] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0041] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0042] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0043] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0044] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0045] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0046] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0047] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0048] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0049] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0050] An embodiment of this invention is a coordination suggestion system that helps users solve various problems they may have when choosing clothes for everyday use. This system has the function of capturing images of clothes owned by the user, analyzing them to extract attributes, and suggesting optimal coordination based on the user's requests. In addition, the user can provide feedback through a virtual try-on session, and based on the results, more suitable clothes can be suggested and purchased.

[0051] Explain the program's processing in natural language.

[0052] 1. User clothing registration:

[0053] The user launches the terminal app and selects the option to register their clothes.

[0054] The device activates the camera function and prompts the user to take a photo of the clothes.

[0055] The user takes a photo of the clothes and the device sends the image to the server.

[0056] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[0057] The server stores the extracted attribute information in a database and returns the results to the terminal.

[0058] The terminal displays a notification to the user that registration is complete.

[0059] Examples:

[0060] A user takes a photo of a white shirt and sends it to the server via the device app. The server extracts the attributes "top," "white," and "shirt" and stores them in a database. The device then displays the message "White shirt registered."

[0061] 2. Coordination suggestions:

[0062] The user selects the outfit suggestion option on the terminal.

[0063] The device prompts the user to enter information such as today's weather and schedule.

[0064] The user enters the necessary information and the terminal sends it to the server.

[0065] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[0066] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[0067] Examples:

[0068] A user requests "Today's Work Outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the device.

[0069] 3. Virtual try-on through AR glasses:

[0070] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[0071] The device captures an image of the user's body and sends it to a server.

[0072] The server virtually overlays the suggested clothing onto an image of the user's body.

[0073] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[0074] The user checks the results of the virtual try-on and sends feedback to the terminal.

[0075] Examples:

[0076] When a user wears the AR glasses and looks into the mirror, the server displays a "white shirt," "blue jeans," or "black jacket" over the user's image. The user then sends feedback saying, "I don't like the color."

[0077] 4. Suitable clothing suggestions and purchases:

[0078] The server receives user feedback and fine-tunes the coordination as needed.

[0079] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[0080] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[0081] Examples:

[0082] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[0083] The detailed processing steps described above realize a system that effectively solves the user's worries about choosing clothes and suggests optimal outfits.

[0084] The processing flow will be explained below.

[0085] Step 1:

[0086] The user starts the terminal application and selects an option to register clothes.

[0087] Step 2:

[0088] The device activates the camera function and prompts the user to take a photo of the clothes.

[0089] Step 3:

[0090] The user takes a photo of the clothes and the device sends the image to the server.

[0091] Step 4:

[0092] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[0093] Step 5:

[0094] The server stores the extracted attribute information in a database.

[0095] Step 6:

[0096] The terminal displays a notification to the user that registration is complete.

[0097] Step 7:

[0098] The user selects the outfit suggestion option on the terminal.

[0099] Step 8:

[0100] The device prompts the user to enter information such as today's weather and schedule.

[0101] Step 9:

[0102] The user enters the necessary information, and the terminal sends it to the server.

[0103] Step 10:

[0104] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[0105] Step 11:

[0106] The server transmits the generated coordinates to the terminal.

[0107] Step 12:

[0108] The terminal displays the coordinated outfit suggestions to the user.

[0109] Step 13:

[0110] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[0111] Step 14:

[0112] The device captures an image of the user's body and sends it to a server.

[0113] Step 15:

[0114] The server virtually overlays the suggested clothing onto an image of the user's body.

[0115] Step 16:

[0116] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[0117] Step 17:

[0118] The user checks the results of the virtual try-on and sends feedback to the terminal.

[0119] Step 18:

[0120] The server receives user feedback and fine-tunes the coordination as needed.

[0121] Step 19:

[0122] The server generates information about the newly suggested clothing and a link to purchase it from the online store.

[0123] Step 20:

[0124] The device displays the generated purchase link to the user.

[0125] Step 21:

[0126] The user clicks on the purchase link and is taken to the online store to purchase the clothing.

[0127] Example 1

[0128] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0129] Conventional clothing selection and coordination suggestion systems have had problems such as the complicated process of registering and categorizing the clothes a user owns, and the quality of the suggested coordinations does not adequately reflect the user's requests or external information (weather, schedule, etc.). Furthermore, they lacked the functionality to actually try on the suggested coordinations, limiting the benefits of online purchases.

[0130] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0131] In this invention, the server includes means for acquiring images of the user's clothing, means for analyzing the acquired images to extract clothing features, means for storing the extracted clothing features in a database, means for generating outfits based on the user's clothing list and external information in response to the user's request, means for visually displaying the generated outfits to the user, means for the user to send feedback, and means for suggesting suitable clothing based on the user's feedback and generating a link to purchase the clothing. This allows users to easily register and categorize their clothing and receive high-quality outfit suggestions that take external information into account. Furthermore, the integration of the virtual try-on function and online purchase further improves convenience.

[0132] "User" refers to an individual who uses the system to register clothes and receive coordination suggestions.

[0133] "Clothing" refers to any clothing owned by a user and registered in the system.

[0134] "Means for acquiring images" refers to a function that allows a user to take a photo of clothing using a terminal and send the image data to a server.

[0135] "Means of analyzing images and extracting clothing characteristics" refers to the function of using analysis software on a server to identify characteristic information such as category, color, pattern, and material from acquired images of clothing.

[0136] "Means for storing in a database" refers to the function of recording the analyzed and extracted clothing characteristic information in storage within the system and managing it efficiently.

[0137] "User request" refers to the content of a request for a coordination proposal made by a user operating the system.

[0138] "External information" refers to additional information such as weather and the user's schedule that the system should take into consideration when proposing outfits.

[0139] "Means for generating outfits" refers to the function of using a generative AI model to create optimal outfits based on the user's clothing list and external information.

[0140] The "means for visually displaying" refers to a function for displaying the generated coordinate on the user's terminal screen and providing it to the user visually.

[0141] "Means for sending feedback" refers to the function that allows users to input their thoughts and requests for improvement regarding the proposed coordination and send them back to the system.

[0142] "Means for suggesting suitable clothing based on feedback and generating a link to purchase it" refers to a function that modifies outfits based on user feedback and provides optimal clothing and a link to an online store where it can be purchased.

[0143] "Augmented reality glasses" refers to a device that users can wear to visually view information from the system and virtual clothing.

[0144] "Means for displaying virtual clothing" refers to a function for displaying virtual clothing overlaid on the user's image.

[0145] "Online store" refers to an e-commerce site where clothing can be purchased via the Internet.

[0146] This invention is a coordination suggestion system that helps users solve various problems they may have when choosing clothes. The system captures images of the user's clothing, analyzes them to extract attributes, and suggests optimal coordinations based on the user's requests. The system also allows the user to provide feedback through a virtual try-on session, which allows further suggestions to be made based on the results, and allows the user to purchase clothing online.

[0147] The system configuration is as follows:

[0148] 1. User clothing registration:

[0149] The user launches the device app and selects the "Register Clothing" option. The device activates its camera function and prompts the user to take a photo of the clothing. The user takes a photo of the clothing, and the device sends the image to the server. The server uses image analysis software (e.g., OpenCV) to extract attributes such as clothing category, color, pattern, and material from the image and stores these attributes in a database. For example, if the user takes a photo of a white shirt and sends it, the server extracts the attributes "top," "white," and "shirt" and stores them in the database. The device then notifies the user that "your white shirt has been registered."

[0150] 2. Coordination suggestions:

[0151] The user selects the "Outfit Suggestion" option on the device app. The device displays a form prompting the user to "Enter today's weather and schedule." When the user enters and submits information such as "sunny" and "work schedule," the device sends this to the server. The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information. As a specific example, when the user requests "Today's work outfit," the server suggests "white shirt," "blue denim," and "black jacket" and sends them to the device. The device then displays the suggested outfit to the user. An example of a prompt sentence is as follows:

[0152] "The user requests registration of a white shirt and suggestions for outfits. Please suggest today's outfits that include a white shirt based on the current weather and schedule."

[0153] 3. Virtual try-on through AR glasses:

[0154] The user selects the virtual try-on mode and puts on the AR glasses. The device uses the AR glasses' camera to capture an image of the user's body and sends it to the server. The server then virtually overlays suggested clothing items based on the received image. For example, the server may overlay a "white shirt," "blue denim," or "black jacket" onto the user's image. The device visually displays the results of this virtual try-on to the user in real time through the AR glasses. The user reviews the image and provides feedback through the app. For example, they may provide feedback such as, "I don't like the color."

[0155] 4. Suitable clothing suggestions and purchases:

[0156] The server receives the user's feedback and fine-tunes the outfit as needed. It then generates new suggested clothing items and a link to the online store where they can be purchased, and sends them to the device. For example, if the user sends feedback that they "don't like the color," the server might suggest a gray shirt instead of a white one. The device then displays the new suggested items along with a link to purchase the gray shirt. The user clicks the link and purchases the gray shirt from the online store.

[0157] This allows users to easily register and categorize their clothing and receive high-quality outfit suggestions that take external information into account. Furthermore, the integration of the virtual try-on feature and online purchase further enhances convenience.

[0158] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0159] Step 1:

[0160] The user launches the device app and selects the "Clothing Registration" option. A screen prompting the user to perform an action is displayed on the device.

[0161] (Input) User operations (app launch and option selection)

[0162] (Output) App instruction screen

[0163] Step 2:

[0164] The device will activate its camera function and prompt the user to take a photo of the clothing. The message "Take a photo" will be displayed on the screen.

[0165] (Input) User operation (option selection)

[0166] (Output) Camera startup and instruction message

[0167] Step 3:

[0168] The user takes a photo of the garment and taps the "Send" button. An image file is generated.

[0169] (Input) User operation (photographing)

[0170] (Output) Image file

[0171] Step 4:

[0172] The device sends the captured image to the server, which prepares to analyze the received image.

[0173] (Input) Image file

[0174] (Output) Image data sent to server

[0175] Step 5:

[0176] The server uses image analysis software (e.g., OpenCV) to analyze the received images and extract attributes such as clothing category, color, pattern, and material.

[0177] (Input) Image data

[0178] (Output) Extracted attribute data (category, color, pattern, material)

[0179] Step 6:

[0180] The server saves the extracted attribute information in the database and returns a status to the terminal if the save is successful.

[0181] (Input) Extracted attribute data

[0182] (Output) Database save status

[0183] Step 7:

[0184] The device notifies the user that registration is complete. The message "White shirt successfully registered" appears on the screen.

[0185] (Input) Database save status

[0186] (Output) Registration completion notification

[0187] Step 8:

[0188] The user selects the "Outfit suggestions" option in the device app. A screen prompting the user to perform an action is displayed on the device.

[0189] (Input) User operation (option selection)

[0190] (Output) Coordination proposal input form

[0191] Step 9:

[0192] The device displays a form prompting the user to "Enter today's weather and schedule." The user enters the information and taps the "Submit" button.

[0193] (Input) User operation (option selection)

[0194] (Output) Input form

[0195] Step 10:

[0196] The user inputs "Sunny" and "Work schedule" and submits the information. The device then sends the information to the server.

[0197] (Input) Input form

[0198] (Output) Weather and schedule data

[0199] Step 11:

[0200] The server takes the user's clothing list and submitted information and uses a generative AI model to generate the optimal outfit.

[0201] (Input) Clothing list and weather / schedule data

[0202] (Output) Generated coordinates

[0203] Step 12:

[0204] The server sends the generated outfit to the device, which then displays the suggestions to the user. Suggested outfits include a "white shirt," "blue jeans," and a "black jacket."

[0205] (Input) Generated coordinates

[0206] (Output) Display of suggested outfits

[0207] Step 13:

[0208] The user selects the virtual try-on mode and puts on the AR glasses. The device then activates the camera in the AR glasses.

[0209] (Input) User operation (select virtual try-on mode)

[0210] (Output) Activate the camera on the AR glasses

[0211] Step 14:

[0212] The device captures an image of the user's body and sends it to the server, which receives the image data.

[0213] (Input) Image of the user's body

[0214] (Output) Video data sent to server

[0215] Step 15:

[0216] The server virtually overlays the suggested clothing based on the video received.

[0217] (Input) Video data and proposed coordinates

[0218] (Output) Virtual try-on results

[0219] Step 16:

[0220] The device visually displays the results of the virtual try-on to the user in real time through the AR glasses, making it appear as if the user is actually wearing the garment.

[0221] (Input) Virtual try-on results

[0222] (Output) Visual display

[0223] Step 17:

[0224] The user checks the video and sends feedback through the app, saying, "I don't like the colors." The device then sends the feedback to the server.

[0225] (Input) Virtual fitting results

[0226] (Output) Feedback Data

[0227] Step 18:

[0228] The server generates new outfits based on user feedback, suggesting, for example, a gray shirt instead of a white one.

[0229] (Input) Feedback data

[0230] (Output) New coordinates

[0231] Step 19:

[0232] The server sends the new outfit along with a link to purchase from the online store to the device, which then displays the new suggestions to the user.

[0233] (Input) New coordinates

[0234] (Output) New offer and purchase link

[0235] Step 20:

[0236] A user clicks on a purchase link to go to an online store and purchase clothing, for example, a gray shirt.

[0237] (Input) Purchase link

[0238] (Print) Online purchase complete

[0239] (Application example 1)

[0240] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0241] Conventional clothing selection systems make it difficult for users to easily visually check how their existing accessories coordinate with the new accessories they are purchasing. Additionally, the time and effort involved in trying on and purchasing items in a retail store is a significant burden for users. While systems exist that suggest suitable accessories based on user feedback, there are few systems that can reflect this feedback in real time and improve the user experience through virtual try-on.

[0242] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0243] In this invention, the server includes means for capturing images of a user's accessories, means for analyzing the captured images to extract characteristics of the accessories, means for storing the extracted characteristics in a database, means for generating a coordinated outfit based on the user's accessory list and external information in response to a user request, means for displaying the generated coordinated outfit to the user's visual display, means for the user to send feedback, means for suggesting suitable accessories based on the user's feedback and generating a purchase link for the accessories, means for the user to capture images of the accessories using a terminal in a retail store and purchase the accessories on the spot, and means for providing a virtual try-on experience in conjunction with a mirror in the store for the user to virtually try on the accessories. This allows users to easily coordinate and virtually try on accessories in a retail store without pressure, and to purchase the optimal accessories on the spot. Furthermore, by using a generative AI model to generate an optimal outfit based on the user's request and characteristics using prompt sentences and visually displaying the generated results to the user, the user can more easily and efficiently select accessories.

[0244] The "means for capturing images of user's accessories" refers to a device that has the function of recording the user's clothes and accessories as photographs using the camera function of the terminal.

[0245] The "means for analyzing the captured image and extracting the characteristics of the attached item" refers to algorithms and software that automatically identify the characteristics of the attached item, such as its color, material, pattern, and shape, from the captured image and treat them as data.

[0246] The "means for storing extracted characteristics of the attached item in a database" is a system that has the function of storing characteristic data of the attached item obtained by image analysis in a database so that it can be reused later.

[0247] "Means for generating an outfit based on the user's accessory list and external information in response to the user's request" refers to software that generates the optimal outfit based on the desired conditions and event information entered by the user, taking into account the accessories the user has and external factors such as weather information and fashion information.

[0248] The "means for visually displaying the generated coordination to the user" is a system having the function of displaying information about the generated coordination on the user's terminal screen, etc., so that the user can visually confirm it.

[0249] "Means for users to send feedback" refers to a mechanism or interface that allows users to input their ratings and comments on the proposed outfits and send that information to the system.

[0250] The "means for proposing suitable accessories based on user feedback and generating a link to purchase them" refers to software and a system that analyzes feedback information from users, and based on that, proposes new accessories and generates a link to purchase the accessories, which are then provided to the user.

[0251] "A means by which a user can capture an image of an accessory using a terminal in a retail store and purchase the accessory on the spot" is a system that allows a user to take an image of an accessory using a camera on a terminal in a store, obtain necessary information based on that data, and complete the purchase on the spot.

[0252] "Means for providing a virtual fitting experience in cooperation with mirrors in a store so that users can virtually try on clothes" refers to a system that allows users to use a specific device in a store to virtually superimpose items onto their own image to provide a fitting experience without actually trying on the items.

[0253] "Means for using a generative AI model to generate optimal outfits using prompt sentences based on the user's requests and characteristics, and visually displaying the generated results to the user" refers to a system that uses an artificial intelligence algorithm to automatically create outfits that take into account the user's individual information and requests, generates them as prompt sentences, and visually presents the results to the user.

[0254] An embodiment of this invention is a coordination suggestion system that solves various problems users may have when choosing clothes for everyday use. This system has a function that allows users to capture images of clothes using their smartphones in a retail store, analyzes the images to extract attributes, and suggests optimal coordination based on the user's requests. In addition, the user can provide feedback through a virtual try-on, and based on the results, more suitable clothes can be suggested and purchased.

[0255] The system uses the following hardware and software:

[0256] Smartphone: Uses the camera function to capture images of the user's clothes and accessories.

[0257] Server: Performs image analysis and coordinate generation.

[0258] Database: Stores the user's clothing attribute information and coordination information.

[0259] Generative AI model: Automatically generates optimal outfits based on user requests and characteristics.

[0260] Specifically, the following process is performed:

[0261] 1. Image capture and attribute extraction: When a user takes a picture of clothing in a retail store using their smartphone, the smartphone sends the image to a server. The server analyzes the image using an image processing library such as OpenCV and extracts the attributes of the clothing (color, material, pattern, shape, etc.). This attribute information is stored in a database.

[0262] 2. Coordination Generation: When a user requests a coordination suggestion, the server uses a generative AI model to generate the optimal coordination based on the user's clothing list and external information (weather forecast, event information, etc.). This coordination is generated using a prompt sentence and displayed on the smartphone.

[0263] 3. Virtual Try-On: The user selects the virtual try-on mode on their smartphone and tries on the clothes virtually using the in-store mirror. The server then virtually overlays the suggested clothes onto the captured image of the user's body and displays the results on the smartphone screen or in the in-store mirror.

[0264] 4. Feedback and Purchase: When the user submits feedback based on the results of the virtual try-on, the server analyzes the feedback and suggests more suitable items to wear. The suggestions include a purchase link, allowing the user to purchase the items from the online store immediately.

[0265] Examples:

[0266] A user requests a stylish outfit and uploads a photo of light blue jeans and a white shirt. The system then suggests outfits based on blue and white, performs a virtual try-on, and provides visual feedback to the user.

[0267] Example prompts to input to the generative AI model:

[0268] User input: Light blue jeans and a white shirt. Today's weather is sunny and I plan to go to work. Coordination suggestions and virtual try-on are performed.

[0269] AI model: Based on these criteria, it suggests the best outfit and allows the user to virtually try it on.

[0270] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0271] Step 1:

[0272] A user uses a terminal to capture images of clothes in a retail store. The user takes a photo of the clothes using the camera function of their smartphone and uploads the image to the system. The input is the image data of the clothes, and the output is the image data sent to the server.

[0273] Step 2:

[0274] The server analyzes the received image data and extracts the attributes of the worn item. The server uses an image processing library such as OpenCV to analyze the image and identify the attributes of the worn item, such as color, material, pattern, and shape. The input is image data, and the output is the extracted attribute data.

[0275] Step 3:

[0276] The extracted attributes of the attached items are saved in a database. The server stores the attribute information obtained by the analysis in a database so that it can be used later upon request. The input is the attribute data of the attached items, and the output is the attribute information saved in the database.

[0277] Step 4:

[0278] The user requests outfit suggestions using a device. The user selects an outfit suggestion option from the smartphone application and enters the necessary external information (weather, schedule, etc.). The input is the outfit suggestion request and the external information, and the output is the request data sent to the server.

[0279] Step 5:

[0280] The server generates an outfit based on the user's request. The server uses a generative AI model to generate the optimal outfit based on the attribute data of the items stored in the database and external information. Prompt statements are used in this process. The inputs are the attribute data of the items, external information, and the prompt statement, and the output is the generated outfit data.

[0281] Step 6:

[0282] The generated coordinated outfit is displayed visually to the user. The server sends the generated coordinated outfit information to the terminal, which then displays the information on the user's smartphone screen. The input is the generated coordinated outfit data, and the output is the displayed coordinated outfit.

[0283] Step 7:

[0284] The user performs a virtual try-on using a device. The user selects the virtual try-on mode on their smartphone and uses the in-store mirror to superimpose their image onto the suggested outfit. The input is an image of the user's body and outfit data, and the output is a video of the results of the virtual try-on.

[0285] Step 8:

[0286] The user submits feedback based on the results of the virtual try-on. The user enters ratings and comments on the proposed outfits through a smartphone application and sends them to the server. The input is feedback data, and the output is the feedback sent to the server.

[0287] Step 9:

[0288] The server proposes suitable accessories based on the user's feedback and generates a purchase link for them. The server analyzes the feedback data, proposes new suitable accessories, generates a purchase link for them, and sends it to the user. The input is the feedback data, and the output is the new proposal and a purchase link.

[0289] Step 10:

[0290] The user clicks on the purchase link to access the online store and purchase the accessory. The user uses their smartphone to access the online store through the suggested link and purchase the accessory. The input is the purchase link, and the output is a purchase completion notification.

[0291] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0292] This invention incorporates an emotion engine into a coordination suggestion system to solve the problems users face when choosing clothes. The system aims to improve user satisfaction by recognizing the emotions felt when choosing clothes and suggesting optimal coordination based on those emotions. This system includes the following specific means:

[0293] Explain the program's processing in natural language.

[0294] 1. User clothing registration:

[0295] The user launches the terminal app and selects the option to register their clothes.

[0296] The device activates the camera function and prompts the user to take a photo of the clothes.

[0297] The user takes a photo of the clothes and the device sends the image to the server.

[0298] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[0299] The server stores the extracted attribute information in a database and returns the results to the terminal.

[0300] The terminal displays a notification to the user that registration is complete.

[0301] Examples:

[0302] A user takes a photo of a white shirt and sends it to the server via the device app. The server extracts the attributes "top," "white," and "shirt" and stores them in a database. The device then displays the message "White shirt registered."

[0303] 2. Coordination suggestions:

[0304] The user selects the outfit suggestion option on the terminal.

[0305] The device prompts the user to enter information such as today's weather and schedule.

[0306] The user enters the necessary information and the terminal sends it to the server.

[0307] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[0308] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[0309] Examples:

[0310] A user requests "Today's Work Outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the device.

[0311] 3. Emotion recognition:

[0312] The emotion engine analyzes the user's facial expression, voice, and behavioral data to recognize the user's emotions.

[0313] The device captures the user's video and audio during the virtual try-on and sends them to the server.

[0314] The server analyzes the user's emotions using an emotion engine and modifies the coordination based on the emotions.

[0315] Examples:

[0316] If the user mutters "It's a bit awkward" during the virtual fitting, the device captures the voice and facial expression and sends them to the server. The emotion engine recognizes that the user is dissatisfied, and the server changes the outfit to a "casual shirt" and "jeans."

[0317] 4. Virtual try-on through AR glasses:

[0318] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[0319] The device captures an image of the user's body and sends it to a server.

[0320] The server virtually overlays the suggested clothing onto an image of the user's body.

[0321] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[0322] The user checks the results of the virtual try-on and sends feedback to the terminal.

[0323] Examples:

[0324] When the user wears the AR glasses and looks into the mirror, the server displays a "white shirt," "blue jeans," and "black jacket" over the user's image. The user again submits dissatisfied feedback.

[0325] 5. Suitable clothing suggestions and purchases:

[0326] The server receives user feedback and fine-tunes the coordination as needed.

[0327] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[0328] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[0329] Examples:

[0330] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[0331] In this way, we have realized a coordination suggestion system that combines an emotion engine to effectively solve the user's clothing selection problem.

[0332] The processing flow will be explained below.

[0333] Step 1:

[0334] The user starts the terminal application and selects an option to register clothes.

[0335] Step 2:

[0336] The device activates the camera function and prompts the user to take a photo of the clothes.

[0337] Step 3:

[0338] The user takes a photo of the clothes and the device sends the image to the server.

[0339] Step 4:

[0340] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[0341] Step 5:

[0342] The server stores the extracted attribute information in a database.

[0343] Step 6:

[0344] The terminal displays a notification to the user that registration is complete.

[0345] Step 7:

[0346] The user selects the outfit suggestion option on the terminal.

[0347] Step 8:

[0348] The device prompts the user to enter information such as today's weather and schedule.

[0349] Step 9:

[0350] The user enters the necessary information, and the terminal sends it to the server.

[0351] Step 10:

[0352] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[0353] Step 11:

[0354] The server transmits the generated coordinates to the terminal.

[0355] Step 12:

[0356] The terminal displays the coordinated outfit suggestions to the user.

[0357] Step 13:

[0358] The emotion engine analyzes the user's facial expression, voice, and behavioral data to recognize the user's emotions.

[0359] Step 14:

[0360] The user confirms the option to use the emotion engine, and the device captures the user's facial expressions and voice and sends them to the server.

[0361] Step 15:

[0362] The server analyzes the user's emotions using an emotion engine and modifies the coordination based on the emotions.

[0363] Step 16:

[0364] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[0365] Step 17:

[0366] The device captures an image of the user's body and sends it to a server.

[0367] Step 18:

[0368] The server virtually overlays the suggested clothing onto an image of the user's body.

[0369] Step 19:

[0370] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[0371] Step 20:

[0372] The user checks the results of the virtual try-on and sends feedback to the terminal.

[0373] Step 21:

[0374] The server receives user feedback and fine-tunes the coordination as needed.

[0375] Step 22:

[0376] The server generates information about the newly suggested clothing and a link to purchase it from the online store.

[0377] Step 23:

[0378] The device displays the generated purchase link to the user.

[0379] Step 24:

[0380] The user clicks on the purchase link and is taken to the online store to purchase the clothing.

[0381] Example 2

[0382] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0383] Conventional clothing selection and coordination suggestion systems often provide one-sided suggestions without considering the user's feelings. Furthermore, users must visit a physical store to check the actual fit and appearance of the suggested clothing, which is time-consuming and laborious. Furthermore, the purchasing process at online stores is cumbersome, which can discourage users from making purchases. A new system is needed to solve these issues and increase user satisfaction.

[0384] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0385] In this invention, the server includes a means for capturing images of the user's belongings, a means for analyzing the captured images to extract the characteristics of the belongings, and a means for storing the extracted characteristics in a database. This allows the server to generate suggestions using a generative AI model based on the user's belongings list and external information, and visually display the results. Furthermore, by suggesting suitable belongings based on user feedback and generating links to the suggestions in collaboration with an online store, the server can easily purchase the items on the spot. Furthermore, a virtual try-on function using augmented reality glasses can be provided, eliminating the stress of actually trying on the belongings.

[0386] A "user" is an individual or group that uses the system and performs operations such as registering belongings, suggesting outfits, providing feedback, and making purchases.

[0387] "Belongings" are clothing, accessories, and other related items owned by the user, and are registered in the system as image data.

[0388] The "capturing means" is a function for acquiring image data using a photographing device such as a camera mounted on the user's terminal.

[0389] "Means for analyzing images" refers to software and algorithms for extracting characteristics of belongings from captured image data.

[0390] "Means for extracting features" refers to the process of analyzing images to identify attribute information such as the category, color, pattern, and material of the belongings.

[0391] "Means for storing in a database" refers to a system or software for storing and managing the extracted characteristic information of belongings as digital data.

[0392] "Means for generating proposals based on requests" refers to a generative AI model that generates optimal coordination and proposals based on user input information and external data.

[0393] The "visual display means" is a function that displays the generated coordination and proposal content on the display of the user's terminal.

[0394] "Means for sending feedback" refers to an interface that allows a user to input evaluations and opinions on a proposal and send them to the system.

[0395] The "means for suggesting suitable items and generating links to purchase them" refers to a process for making new suggestions based on user feedback and generating links to promote purchases at an online store.

[0396] "Augmented reality glasses" are wearable devices that overlay virtual objects onto the real world.

[0397] "Virtual Try-On" is a feature that allows users to virtually try on items using augmented reality glasses.

[0398] "Online Store" refers to a website or platform that sells and purchases products over the Internet.

[0399] "Purchase Link" means a hyperlink that allows you to purchase the suggested item directly from the online store.

[0400] MODE FOR CARRYING OUT THE INVENTION

[0401] This invention incorporates an emotion engine into a coordination suggestion system that helps users solve their troubles when choosing clothes. The aim is to improve user satisfaction by recognizing the emotions of users when choosing clothes and suggesting optimal coordination based on those emotions.

[0402] A specific embodiment of the system is as follows.

[0403] 1. User clothing registration

[0404] When a user launches the device app, the app displays an option to register clothes. If the user selects this option, the device's camera function is activated and the user is prompted to take a photo of the clothes. When the user takes a photo and presses the send button, the device sends the image to the server.

[0405] The server uses image analysis software such as Label Studio to extract features from the received image, such as the clothing category (for example, "top" or "pants"), color, pattern, and material. The extracted feature information is stored in a database. The processing results are sent back to the terminal, which displays a message such as "A white shirt has been registered."

[0406] Examples:

[0407] A user takes a photo of a white shirt and sends the image to the server via a smartphone app. The server extracts the attributes "top," "white," and "shirt" and stores this information in a database. The device displays the message "White shirt registered."

[0408] 2. Coordination suggestions

[0409] The user selects an outfit suggestion option on the device. The device prompts the user to enter information such as today's weather and schedule. When the user enters and submits this information, the data is sent from the device to the server.

[0410] The server uses a generative AI model (e.g., ChatGPT) to generate the optimal outfit based on the user's clothing list and the submitted information. The generative AI model uses the following prompt:

[0411] "Please suggest an outfit for work today. The weather is sunny and I'm heading to work."

[0412] The generated coordinates are sent from the server to the terminal, which then visually displays them to the user.

[0413] Examples:

[0414] A user requests "Today's Work Outfit," inputs the weather forecast and the date they plan to go to work, and the server suggests "white shirt," "blue jeans," and "black jacket." The following is displayed on the device.

[0415] 3. Emotion recognition

[0416] The system incorporates an emotion engine that analyzes the user's facial expressions, voice, and behavioral data to recognize their emotions. The device captures video and audio of the user while trying on clothes and sends them to a server. The server then uses Microsoft Azure's emotion recognition API to analyze the user's emotions and adjusts the outfit based on the results.

[0417] Examples:

[0418] If the user mutters "It's a bit awkward" during the virtual fitting, the device captures the voice and facial expression and sends them to the server. The emotion engine recognizes the user's dissatisfaction, and the server changes the outfit to a "casual shirt" and "jeans."

[0419] 4. Virtual try-on with AR glasses

[0420] The user selects the virtual try-on mode, puts on AR glasses (e.g., Magic Leap), and looks in the mirror. The device captures a full-body image of the user's body and sends it to the server. The server uses Unity to generate an image in which virtual clothes are superimposed on the user's body in real time. The device visually displays the results of the virtual try-on to the user through the AR glasses. The user checks this and sends feedback to the device.

[0421] Examples:

[0422] When a user wears the AR glasses and looks in the mirror, the server superimposes a "white shirt," "blue jeans," or "black jacket" onto the user's image. The user then submits their dissatisfaction feedback.

[0423] 5. Buying clothes

[0424] The server receives the user's feedback, fine-tunes the outfit as needed, and sends the newly suggested outfit information and a link to purchase it from an online store to the user's device. When the user clicks the link, they are taken to an online store such as Amazon, where they can purchase the outfit on the spot.

[0425] Examples:

[0426] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[0427] In this way, we can realize a coordination suggestion system that combines an emotion engine and a generative AI model to effectively support users in choosing clothes.

[0428] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0429] Step 1:

[0430] The user starts the terminal application and selects the clothes registration option.

[0431] Input: User action (selecting the "Register Clothes" option).

[0432] Output: The device's camera function is activated.

[0433] Specific operation: The user taps or clicks the "Register Clothes" button on the app screen. The device's camera automatically starts up and an interface for taking a photo of the clothes is displayed.

[0434] Step 2:

[0435] The user takes a photo of the clothes and the device sends the image to the server.

[0436] Input: A user-taken photo of an outfit.

[0437] Output: The image data is sent to the server.

[0438] Specific operation: The user takes a photo of the clothes using the camera and presses the "Send" button to send the image to the server.

[0439] Step 3:

[0440] The server analyzes the received images and uses Label Studio to extract features of the belongings.

[0441] Input: Image data of clothes.

[0442] Output: Clothing attribute information (category, color, pattern, material, etc.).

[0443] Specific operation: The server uses Label Studio to analyze the image data and extract attributes such as "tops," "white," and "shirt."

[0444] Step 4:

[0445] The server stores the extracted feature information in a database and returns the results to the terminal.

[0446] Input: Extracted clothing attribute information.

[0447] Output: Attribute information stored in the database and a notification of registration completion.

[0448] Specific operation: The extracted attribute information is stored in the database, and the result is sent back to the device. The device displays "White shirt registered."

[0449] Step 5:

[0450] The user selects the outfit suggestion option on the terminal.

[0451] Input: User action (selection of the "Outfit suggestions" option).

[0452] Output: An input form for weather, schedule, etc. is displayed.

[0453] Specific operation: The user taps or clicks the "Outfit suggestions" button, and an input form appears on the device.

[0454] Step 6:

[0455] The user inputs information such as today's weather and schedule, and the terminal sends the information to the server.

[0456] Input: Weather and schedule information entered by the user.

[0457] Output: The transmitted information arrives at the server.

[0458] Specific operation: The user enters the weather and schedule into the form and presses the "Submit" button.

[0459] Step 7:

[0460] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[0461] Input: User's clothing list and weather and schedule information.

[0462] Output: The generated coordinate plan.

[0463] Specific operation: Based on the user's belongings list and the submitted information, the server sends the following prompt to the generative AI model: "Please suggest an outfit for work today. The weather is sunny and I plan to go to work." The generative AI model then generates the optimal outfit.

[0464] Step 8:

[0465] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[0466] Input: Generated coordinate plan.

[0467] Output: Coordination suggestions displayed on the device.

[0468] Specific operation: The server sends the generated coordination plan to the terminal, and the terminal notifies the user of it.

[0469] Step 9:

[0470] The device captures the user's facial, voice, and behavioral data and transmits it to a server.

[0471] Input: User facial, voice, and behavioral data.

[0472] Output: The captured data is sent to the server.

[0473] Specific operation: The device camera and microphone record the user's facial expressions and voice, and send the data to the server.

[0474] Step 10:

[0475] The server uses an emotion engine to analyze the user's emotions and corrects the coordinates based on the emotions.

[0476] Input: User facial, voice, and behavioral data.

[0477] Output: Revised coordinate plan.

[0478] How it works: The server uses Microsoft Azure's emotion recognition API to analyze the user's emotions and generate new coordination suggestions based on the results.

[0479] Step 11:

[0480] The user selects the virtual try-on mode, puts on the AR glasses, and looks in the mirror.

[0481] Input: User operation (selecting virtual try-on mode), wearing AR glasses.

[0482] Output: Virtual try-on screen.

[0483] Specific operation: The user selects "virtual try-on mode" and puts on the AR glasses.

[0484] Step 12:

[0485] The device captures an image of the user's body and sends it to a server.

[0486] Input: Video data of the user's body.

[0487] Output: Transmitted video data.

[0488] Specific operation: The device camera captures the user's full-body image and sends the image data to the server.

[0489] Step 13:

[0490] The server uses Unity to virtually overlay the suggested clothing onto an image of the user's body.

[0491] Input: User's body image data, suggested clothing data.

[0492] Output: Virtually superimposed video data.

[0493] How it works: The server uses Unity to virtually overlay the suggested clothing onto an image of the user's body.

[0494] Step 14:

[0495] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[0496] Input: Virtual and superimposed video data.

[0497] Output: Virtual try-on results displayed on the AR glasses.

[0498] Specific operation: The AR glasses display an image of the clothes being virtually tried on to the user.

[0499] Step 15:

[0500] The server receives user feedback and fine-tunes the coordination as needed.

[0501] Input: User feedback data.

[0502] Output: A fine-tuned outfit plan.

[0503] Specific operation: The server analyzes the feedback from the user and generates new suggestions.

[0504] Step 16:

[0505] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[0506] Input: Proposal information and purchase link from the server.

[0507] Output: Proposal information and purchase link displayed on the device.

[0508] Specific behavior: A link such as "Buy the gray shirt" will appear on the device screen.

[0509] Step 17:

[0510] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[0511] Input: User action (click on purchase link).

[0512] Output: Proceed to the online store purchase page and complete the purchase.

[0513] Specific behavior: The user clicks on the link, goes to the online store's purchase page, and completes the purchase process.

[0514] (Application example 2)

[0515] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0516] Conventional fashion coordination systems make clothing recommendations without considering the user's emotions, making it difficult to fully improve user satisfaction. They also lack advanced features such as a virtual try-on environment and real-time revision of recommendations based on user feedback. As a result, users may be dissatisfied with the suggested outfits and lose their motivation to purchase.

[0517] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0518] In this invention, the server includes means for capturing images of a user's clothes, means for analyzing the captured images to extract clothing attributes, means for saving the extracted clothing attributes in a database, means for generating outfits based on the user's clothes list and external information in response to a user request, means for visually displaying the generated outfit to the user, means for recognizing emotions by analyzing the user's facial expressions and voice data using an emotion engine, means for modifying the outfit based on the recognized emotions, means for the user to send feedback, means for suggesting suitable clothes based on the user's feedback and generating a purchase link for the clothes, and means for providing a virtual try-on mode. This makes it possible to flexibly adjust outfit suggestions based on the user's emotions and feedback.

[0519] "Means for capturing images of the user's clothes" refers to a device or software that takes a photo of the clothes owned by the user and acquires the image data.

[0520] "Means for analyzing captured images and extracting clothing attributes" refers to technology that analyzes acquired image data and automatically identifies and extracts characteristics such as clothing category, color, pattern, and material.

[0521] "Means for storing extracted clothing attributes in a database" refers to a data storage or database system for permanently storing clothing characteristic information obtained through analysis.

[0522] "Means for generating outfits based on the user's clothing list and external information in response to the user's request" refers to an algorithm or system that automatically creates optimal fashion outfits in response to the user's requests and input information, taking into account pre-registered clothing and external data such as weather and schedules.

[0523] "Means for visually displaying the generated outfits to the user" refers to a digital display or device that displays the suggested outfits in a format that is easy for the user to view.

[0524] "Means of recognizing emotions by analyzing a user's facial expressions and voice data using an emotion engine" refers to an AI model or algorithm that analyzes a user's facial expressions and voice as input data and recognizes their emotions.

[0525] "Means for modifying outfits based on recognized emotions" refers to a system or method for improving or changing suggested outfits based on emotional data determined by the emotion engine.

[0526] "Means for users to submit feedback" refers to an interface or function that allows users to input and submit their opinions and thoughts about the provided coordination or the operation of the system.

[0527] "Means for suggesting suitable clothes based on user feedback and generating a link to purchase them" refers to an algorithm or system that re-suggests optimal clothes based on user feedback and generates a link for direct purchase.

[0528] "Means for providing a virtual try-on mode" refers to a system or software that provides a function that allows a user to virtually try on the clothes they have selected.

[0529] The present invention provides a coordination suggestion system incorporating an emotion engine to solve the problems users face when choosing clothes. This system allows users to register clothes using a smartphone or other device, virtually try on suggested coordinations, and then suggests optimal coordinations based on emotional feedback.

[0530] 1. User clothing registration

[0531] The user activates their smartphone or other device and selects the option to register their clothes. The smartphone activates the camera function and instructs the user to take a photo of the clothes. The user takes a photo of the clothes, and the smartphone sends the image to the server. The server analyzes the received image and extracts attributes such as the clothing category, color, pattern, and material. The extracted attribute information is stored in a database, and the results are sent back to the smartphone.

[0532] Example: A user takes a photo of a blue shirt and sends the image to a server using a fashion coordination app. The server analyzes the image, extracts the attributes "tops," "blue," and "shirt," and stores them in a database. The smartphone then displays the message "Blue shirt registered."

[0533] 2. Coordination suggestions

[0534] The user selects an outfit suggestion option on their smartphone. The smartphone prompts the user to enter information such as today's weather and schedule. The user enters the necessary information, and the smartphone sends it to the server. The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information. The generated outfit is sent to the smartphone, and the suggestions are displayed to the user.

[0535] Example: A user requests "Today's work outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the smartphone.

[0536] 3. Emotion recognition

[0537] The emotion engine analyzes the user's facial expressions, voice, and behavioral data to recognize their emotions. The smartphone captures the user's video and audio during the virtual try-on and sends them to the server. The server then analyzes the user's emotions using the emotion engine and modifies the outfit based on those emotions.

[0538] Example: If a user mutters "It's a bit awkward" during a virtual try-on session, the smartphone captures the user's voice and facial expression and sends them to the server. The emotion engine recognizes that the user is dissatisfied, and the server changes the outfit to a "casual shirt" and "jeans."

[0539] 4. Virtual try-on through AR glasses

[0540] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror. The smartphone captures an image of the user's body and sends it to the server. The server then virtually overlays the suggested clothing on the image of the user's body. The smartphone visually displays the results of the virtual try-on to the user through the AR glasses. The user checks the results of the virtual try-on and sends feedback to the smartphone.

[0541] Example: When a user wears AR glasses and looks into a mirror, the server displays "white shirt," "blue jeans," and "black jacket" over the user's image. The user again sends dissatisfied feedback.

[0542] 5. Proposal and purchase of suitable clothing

[0543] The server receives the user's feedback and fine-tunes the outfit as necessary. The smartphone receives the newly suggested outfit information and a link to purchase it from the online store from the server and displays it to the user. The user clicks the link to access the online store and purchase the outfit.

[0544] Example: If the user gives feedback that they "don't like the color," the server suggests a gray shirt instead of a white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[0545] This system uses devices such as smartphones and AR glasses to provide real-time coordination suggestions based on the user's emotions and feedback. The hardware used includes standard smartphones (iPhones, Android devices) and AR glasses (commonly known as head-mounted displays), while the software includes a server (cloud computing service), generative AI model (language model), emotion engine (emotion recognition tool), and database (data storage system). This system allows users to receive high-quality fashion coordination suggestions from the comfort of their own homes.

[0546] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0547] Step 1:

[0548] A user takes a photo of clothing with their smartphone and sends the image to a server via their device. The server receives the image and uses an image analysis algorithm to extract attributes such as clothing category, color, pattern, and material. The input is the image of the clothing, and the output is attribute information. This attribute information is stored in a database.

[0549] Step 2:

[0550] The user selects an outfit suggestion option on the smartphone app and enters the necessary information (such as weather and schedule). The device sends this to the server. The server uses a generative AI model based on the user's clothing list and the submitted information to generate the optimal outfit. The input is the user's clothing list and external information (weather, schedule), and the output is a suggested outfit. The suggested outfit is displayed to the user on the device.

[0551] Step 3:

[0552] The user selects the virtual try-on option. The device activates the camera, captures an image of the user's body, and sends it to the server. The server then uses a virtual try-on algorithm to virtually overlay the suggested outfit on the image of the user's body. The input is the image of the user's body and the suggested outfit, and the output is the virtual try-on result. This result is visually displayed to the user through the device.

[0553] Step 4:

[0554] The emotion engine analyzes the user's facial and voice data to recognize their emotions. The device captures the user's video and audio and sends them to the server. The server uses the emotion engine to analyze the user's emotions and modifies the outfit based on those emotions. The input is the user's facial and voice data, and the output is the modified outfit. The modified outfit is displayed to the user via the device.

[0555] Step 5:

[0556] The user inputs feedback and sends it to the server via the terminal. The server receives the feedback and re-proposes outfits as needed. The input is the user's feedback, and the output is the re-proposed outfits. The re-proposed outfits are displayed to the user via the terminal.

[0557] Step 6:

[0558] The server finally suggests suitable outfits and generates a purchase link to send to the device. The input is the final coordinated outfit and the user's willingness to purchase, and the output is the purchase link. When the user clicks the purchase link, they can access the online store and purchase the outfit right away.

[0559] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0560] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0561] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0562] [Second embodiment]

[0563] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0564] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0565] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0566] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0567] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0568] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0569] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0570] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0571] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0572] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0573] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0574] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0575] An embodiment of this invention is a coordination suggestion system that helps users solve various problems they may have when choosing clothes for everyday use. This system has the function of capturing images of clothes owned by the user, analyzing them to extract attributes, and suggesting optimal coordination based on the user's requests. In addition, the user can provide feedback through a virtual try-on session, and based on the results, more suitable clothes can be suggested and purchased.

[0576] Explain the program's processing in natural language.

[0577] 1. User clothing registration:

[0578] The user launches the terminal app and selects the option to register their clothes.

[0579] The device activates the camera function and prompts the user to take a photo of the clothes.

[0580] The user takes a photo of the clothes and the device sends the image to the server.

[0581] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[0582] The server stores the extracted attribute information in a database and returns the results to the terminal.

[0583] The terminal displays a notification to the user that registration is complete.

[0584] Examples:

[0585] A user takes a photo of a white shirt and sends it to the server via the device app. The server extracts the attributes "top," "white," and "shirt" and stores them in a database. The device then displays the message "White shirt registered."

[0586] 2. Coordination suggestions:

[0587] The user selects the outfit suggestion option on the terminal.

[0588] The device prompts the user to enter information such as today's weather and schedule.

[0589] The user enters the necessary information and the terminal sends it to the server.

[0590] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[0591] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[0592] Examples:

[0593] A user requests "Today's Work Outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the device.

[0594] 3. Virtual try-on through AR glasses:

[0595] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[0596] The device captures an image of the user's body and sends it to a server.

[0597] The server virtually overlays the suggested clothing onto an image of the user's body.

[0598] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[0599] The user checks the results of the virtual try-on and sends feedback to the terminal.

[0600] Examples:

[0601] When a user wears the AR glasses and looks into the mirror, the server displays a "white shirt," "blue jeans," or "black jacket" over the user's image. The user then sends feedback saying, "I don't like the color."

[0602] 4. Suitable clothing suggestions and purchases:

[0603] The server receives user feedback and fine-tunes the coordination as needed.

[0604] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[0605] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[0606] Examples:

[0607] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[0608] The detailed processing steps described above realize a system that effectively solves the user's worries about choosing clothes and suggests optimal outfits.

[0609] The processing flow will be explained below.

[0610] Step 1:

[0611] The user starts the terminal application and selects an option to register clothes.

[0612] Step 2:

[0613] The device activates the camera function and prompts the user to take a photo of the clothes.

[0614] Step 3:

[0615] The user takes a photo of the clothes and the device sends the image to the server.

[0616] Step 4:

[0617] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[0618] Step 5:

[0619] The server stores the extracted attribute information in a database.

[0620] Step 6:

[0621] The terminal displays a notification to the user that registration is complete.

[0622] Step 7:

[0623] The user selects the outfit suggestion option on the terminal.

[0624] Step 8:

[0625] The device prompts the user to enter information such as today's weather and schedule.

[0626] Step 9:

[0627] The user enters the necessary information, and the terminal sends it to the server.

[0628] Step 10:

[0629] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[0630] Step 11:

[0631] The server transmits the generated coordinates to the terminal.

[0632] Step 12:

[0633] The terminal displays the coordinated outfit suggestions to the user.

[0634] Step 13:

[0635] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[0636] Step 14:

[0637] The device captures an image of the user's body and sends it to a server.

[0638] Step 15:

[0639] The server virtually overlays the suggested clothing onto an image of the user's body.

[0640] Step 16:

[0641] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[0642] Step 17:

[0643] The user checks the results of the virtual try-on and sends feedback to the terminal.

[0644] Step 18:

[0645] The server receives user feedback and fine-tunes the coordination as needed.

[0646] Step 19:

[0647] The server generates information about the newly suggested clothing and a link to purchase it from the online store.

[0648] Step 20:

[0649] The device displays the generated purchase link to the user.

[0650] Step 21:

[0651] The user clicks on the purchase link and is taken to the online store to purchase the clothing.

[0652] Example 1

[0653] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0654] Conventional clothing selection and coordination suggestion systems have had problems such as the complicated process of registering and categorizing the clothes a user owns, and the quality of the suggested coordinations does not adequately reflect the user's requests or external information (weather, schedule, etc.). Furthermore, they lacked the functionality to actually try on the suggested coordinations, limiting the benefits of online purchases.

[0655] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0656] In this invention, the server includes means for acquiring images of the user's clothing, means for analyzing the acquired images to extract clothing features, means for storing the extracted clothing features in a database, means for generating outfits based on the user's clothing list and external information in response to the user's request, means for visually displaying the generated outfits to the user, means for the user to send feedback, and means for suggesting suitable clothing based on the user's feedback and generating a link to purchase the clothing. This allows users to easily register and categorize their clothing and receive high-quality outfit suggestions that take external information into account. Furthermore, the integration of the virtual try-on function and online purchase further improves convenience.

[0657] "User" refers to an individual who uses the system to register clothes and receive coordination suggestions.

[0658] "Clothing" refers to any clothing owned by a user and registered in the system.

[0659] "Means for acquiring images" refers to a function that allows a user to take a photo of clothing using a terminal and send the image data to a server.

[0660] "Means of analyzing images and extracting clothing characteristics" refers to the function of using analysis software on a server to identify characteristic information such as category, color, pattern, and material from acquired images of clothing.

[0661] "Means for storing in a database" refers to the function of recording the analyzed and extracted clothing characteristic information in storage within the system and managing it efficiently.

[0662] "User request" refers to the content of a request for a coordination proposal made by a user operating the system.

[0663] "External information" refers to additional information such as weather and the user's schedule that the system should take into consideration when proposing outfits.

[0664] "Means for generating outfits" refers to the function of using a generative AI model to create optimal outfits based on the user's clothing list and external information.

[0665] The "means for visually displaying" refers to a function for displaying the generated coordinate on the user's terminal screen and providing it to the user visually.

[0666] "Means for sending feedback" refers to the function that allows users to input their thoughts and requests for improvement regarding the proposed coordination and send them back to the system.

[0667] "Means for suggesting suitable clothing based on feedback and generating a link to purchase it" refers to a function that modifies outfits based on user feedback and provides optimal clothing and a link to an online store where it can be purchased.

[0668] "Augmented reality glasses" refers to a device that users can wear to visually view information from the system and virtual clothing.

[0669] "Means for displaying virtual clothing" refers to a function for displaying virtual clothing overlaid on the user's image.

[0670] "Online store" refers to an e-commerce site where clothing can be purchased via the Internet.

[0671] This invention is a coordination suggestion system that helps users solve various problems they may have when choosing clothes. The system captures images of the user's clothing, analyzes them to extract attributes, and suggests optimal coordinations based on the user's requests. The system also allows the user to provide feedback through a virtual try-on session, which allows further suggestions to be made based on the results, and allows the user to purchase clothing online.

[0672] The system configuration is as follows:

[0673] 1. User clothing registration:

[0674] The user launches the device app and selects the "Register Clothing" option. The device activates its camera function and prompts the user to take a photo of the clothing. The user takes a photo of the clothing, and the device sends the image to the server. The server uses image analysis software (e.g., OpenCV) to extract attributes such as clothing category, color, pattern, and material from the image and stores these attributes in a database. For example, if the user takes a photo of a white shirt and sends it, the server extracts the attributes "top," "white," and "shirt" and stores them in the database. The device then notifies the user that "your white shirt has been registered."

[0675] 2. Coordination suggestions:

[0676] The user selects the "Outfit Suggestion" option on the device app. The device displays a form prompting the user to "Enter today's weather and schedule." When the user enters and submits information such as "sunny" and "work schedule," the device sends this to the server. The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information. As a specific example, when the user requests "Today's work outfit," the server suggests "white shirt," "blue denim," and "black jacket" and sends them to the device. The device then displays the suggested outfit to the user. An example of a prompt sentence is as follows:

[0677] "The user requests registration of a white shirt and suggestions for outfits. Please suggest today's outfits that include a white shirt based on the current weather and schedule."

[0678] 3. Virtual try-on through AR glasses:

[0679] The user selects the virtual try-on mode and puts on the AR glasses. The device uses the AR glasses' camera to capture an image of the user's body and sends it to the server. The server then virtually overlays suggested clothing items based on the received image. For example, the server may overlay a "white shirt," "blue denim," or "black jacket" onto the user's image. The device visually displays the results of this virtual try-on to the user in real time through the AR glasses. The user reviews the image and provides feedback through the app. For example, they may provide feedback such as, "I don't like the color."

[0680] 4. Suitable clothing suggestions and purchases:

[0681] The server receives the user's feedback and fine-tunes the outfit as needed. It then generates new suggested clothing items and a link to the online store where they can be purchased, and sends them to the device. For example, if the user sends feedback that they "don't like the color," the server might suggest a gray shirt instead of a white one. The device then displays the new suggested items along with a link to purchase the gray shirt. The user clicks the link and purchases the gray shirt from the online store.

[0682] This allows users to easily register and categorize their clothing and receive high-quality outfit suggestions that take external information into account. Furthermore, the integration of the virtual try-on feature and online purchase further enhances convenience.

[0683] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0684] Step 1:

[0685] The user launches the device app and selects the "Clothing Registration" option. A screen prompting the user to perform an action is displayed on the device.

[0686] (Input) User operations (app launch and option selection)

[0687] (Output) App instruction screen

[0688] Step 2:

[0689] The device will activate its camera function and prompt the user to take a photo of the clothing. The message "Take a photo" will be displayed on the screen.

[0690] (Input) User operation (option selection)

[0691] (Output) Camera startup and instruction message

[0692] Step 3:

[0693] The user takes a photo of the garment and taps the "Send" button. An image file is generated.

[0694] (Input) User operation (photographing)

[0695] (Output) Image file

[0696] Step 4:

[0697] The device sends the captured image to the server, which prepares to analyze the received image.

[0698] (Input) Image file

[0699] (Output) Image data sent to server

[0700] Step 5:

[0701] The server uses image analysis software (e.g., OpenCV) to analyze the received images and extract attributes such as clothing category, color, pattern, and material.

[0702] (Input) Image data

[0703] (Output) Extracted attribute data (category, color, pattern, material)

[0704] Step 6:

[0705] The server saves the extracted attribute information in the database and returns a status to the terminal if the save is successful.

[0706] (Input) Extracted attribute data

[0707] (Output) Database save status

[0708] Step 7:

[0709] The device notifies the user that registration is complete. The message "White shirt successfully registered" appears on the screen.

[0710] (Input) Database save status

[0711] (Output) Registration completion notification

[0712] Step 8:

[0713] The user selects the "Outfit suggestions" option in the device app. A screen prompting the user to perform an action is displayed on the device.

[0714] (Input) User operation (option selection)

[0715] (Output) Coordination proposal input form

[0716] Step 9:

[0717] The device displays a form prompting the user to "Enter today's weather and schedule." The user enters the information and taps the "Submit" button.

[0718] (Input) User operation (option selection)

[0719] (Output) Input form

[0720] Step 10:

[0721] The user inputs "Sunny" and "Work schedule" and submits the information. The device then sends the information to the server.

[0722] (Input) Input form

[0723] (Output) Weather and schedule data

[0724] Step 11:

[0725] The server takes the user's clothing list and submitted information and uses a generative AI model to generate the optimal outfit.

[0726] (Input) Clothing list and weather / schedule data

[0727] (Output) Generated coordinates

[0728] Step 12:

[0729] The server sends the generated outfit to the device, which then displays the suggestions to the user. Suggested outfits include a "white shirt," "blue jeans," and a "black jacket."

[0730] (Input) Generated coordinates

[0731] (Output) Display of suggested outfits

[0732] Step 13:

[0733] The user selects the virtual try-on mode and puts on the AR glasses. The device then activates the camera in the AR glasses.

[0734] (Input) User operation (select virtual try-on mode)

[0735] (Output) Activate the camera on the AR glasses

[0736] Step 14:

[0737] The device captures an image of the user's body and sends it to the server, which receives the image data.

[0738] (Input) Image of the user's body

[0739] (Output) Video data sent to server

[0740] Step 15:

[0741] The server virtually overlays the suggested clothing based on the video received.

[0742] (Input) Video data and proposed coordinates

[0743] (Output) Virtual try-on results

[0744] Step 16:

[0745] The device visually displays the results of the virtual try-on to the user in real time through the AR glasses, making it appear as if the user is actually wearing the garment.

[0746] (Input) Virtual try-on results

[0747] (Output) Visual display

[0748] Step 17:

[0749] The user checks the video and sends feedback through the app, saying, "I don't like the colors." The device then sends the feedback to the server.

[0750] (Input) Virtual fitting results

[0751] (Output) Feedback Data

[0752] Step 18:

[0753] The server generates new outfits based on user feedback, suggesting, for example, a gray shirt instead of a white one.

[0754] (Input) Feedback data

[0755] (Output) New coordinates

[0756] Step 19:

[0757] The server sends the new outfit along with a link to purchase from the online store to the device, which then displays the new suggestions to the user.

[0758] (Input) New coordinates

[0759] (Output) New offer and purchase link

[0760] Step 20:

[0761] A user clicks on a purchase link to go to an online store and purchase clothing, for example, a gray shirt.

[0762] (Input) Purchase link

[0763] (Print) Online purchase complete

[0764] (Application example 1)

[0765] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0766] Conventional clothing selection systems make it difficult for users to easily visually check how their existing accessories coordinate with the new accessories they are purchasing. Additionally, the time and effort involved in trying on and purchasing items in a retail store is a significant burden for users. While systems exist that suggest suitable accessories based on user feedback, there are few systems that can reflect this feedback in real time and improve the user experience through virtual try-on.

[0767] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0768] In this invention, the server includes means for capturing images of a user's accessories, means for analyzing the captured images to extract characteristics of the accessories, means for storing the extracted characteristics in a database, means for generating a coordinated outfit based on the user's accessory list and external information in response to a user request, means for displaying the generated coordinated outfit to the user's visual display, means for the user to send feedback, means for suggesting suitable accessories based on the user's feedback and generating a purchase link for the accessories, means for the user to capture images of the accessories using a terminal in a retail store and purchase the accessories on the spot, and means for providing a virtual try-on experience in conjunction with a mirror in the store for the user to virtually try on the accessories. This allows users to easily coordinate and virtually try on accessories in a retail store without pressure, and to purchase the optimal accessories on the spot. Furthermore, by using a generative AI model to generate an optimal outfit based on the user's request and characteristics using prompt sentences and visually displaying the generated results to the user, the user can more easily and efficiently select accessories.

[0769] The "means for capturing images of user's accessories" refers to a device that has the function of recording the user's clothes and accessories as photographs using the camera function of the terminal.

[0770] The "means for analyzing the captured image and extracting the characteristics of the attached item" refers to algorithms and software that automatically identify the characteristics of the attached item, such as its color, material, pattern, and shape, from the captured image and treat them as data.

[0771] The "means for storing extracted characteristics of the attached item in a database" is a system that has the function of storing characteristic data of the attached item obtained by image analysis in a database so that it can be reused later.

[0772] "Means for generating an outfit based on the user's accessory list and external information in response to the user's request" refers to software that generates the optimal outfit based on the desired conditions and event information entered by the user, taking into account the accessories the user has and external factors such as weather information and fashion information.

[0773] The "means for visually displaying the generated coordination to the user" is a system having the function of displaying information about the generated coordination on the user's terminal screen, etc., so that the user can visually confirm it.

[0774] "Means for users to send feedback" refers to a mechanism or interface that allows users to input their ratings and comments on the proposed outfits and send that information to the system.

[0775] The "means for proposing suitable accessories based on user feedback and generating a link to purchase them" refers to software and a system that analyzes feedback information from users, and based on that, proposes new accessories and generates a link to purchase the accessories, which are then provided to the user.

[0776] "A means by which a user can capture an image of an accessory using a terminal in a retail store and purchase the accessory on the spot" is a system that allows a user to take an image of an accessory using a camera on a terminal in a store, obtain necessary information based on that data, and complete the purchase on the spot.

[0777] "Means for providing a virtual fitting experience in cooperation with mirrors in a store so that users can virtually try on clothes" refers to a system that allows users to use a specific device in a store to virtually superimpose items onto their own image to provide a fitting experience without actually trying on the items.

[0778] "Means for using a generative AI model to generate optimal outfits using prompt sentences based on the user's requests and characteristics, and visually displaying the generated results to the user" refers to a system that uses an artificial intelligence algorithm to automatically create outfits that take into account the user's individual information and requests, generates them as prompt sentences, and visually presents the results to the user.

[0779] An embodiment of this invention is a coordination suggestion system that solves various problems users may have when choosing clothes for everyday use. This system has a function that allows users to capture images of clothes using their smartphones in a retail store, analyzes the images to extract attributes, and suggests optimal coordination based on the user's requests. In addition, the user can provide feedback through a virtual try-on, and based on the results, more suitable clothes can be suggested and purchased.

[0780] The system uses the following hardware and software:

[0781] Smartphone: Uses the camera function to capture images of the user's clothes and accessories.

[0782] Server: Performs image analysis and coordinate generation.

[0783] Database: Stores the user's clothing attribute information and coordination information.

[0784] Generative AI model: Automatically generates optimal outfits based on user requests and characteristics.

[0785] Specifically, the following process is performed:

[0786] 1. Image capture and attribute extraction: When a user takes a picture of clothing in a retail store using their smartphone, the smartphone sends the image to a server. The server analyzes the image using an image processing library such as OpenCV and extracts the attributes of the clothing (color, material, pattern, shape, etc.). This attribute information is stored in a database.

[0787] 2. Coordination Generation: When a user requests a coordination suggestion, the server uses a generative AI model to generate the optimal coordination based on the user's clothing list and external information (weather forecast, event information, etc.). This coordination is generated using a prompt sentence and displayed on the smartphone.

[0788] 3. Virtual Try-On: The user selects the virtual try-on mode on their smartphone and tries on the clothes virtually using the in-store mirror. The server then virtually overlays the suggested clothes onto the captured image of the user's body and displays the results on the smartphone screen or in the in-store mirror.

[0789] 4. Feedback and Purchase: When the user submits feedback based on the results of the virtual try-on, the server analyzes the feedback and suggests more suitable items to wear. The suggestions include a purchase link, allowing the user to purchase the items from the online store immediately.

[0790] Examples:

[0791] A user requests a stylish outfit and uploads a photo of light blue jeans and a white shirt. The system then suggests outfits based on blue and white, performs a virtual try-on, and provides visual feedback to the user.

[0792] Example prompts to input to the generative AI model:

[0793] User input: Light blue jeans and a white shirt. Today's weather is sunny and I plan to go to work. Coordination suggestions and virtual try-on are performed.

[0794] AI model: Based on these criteria, it suggests the best outfit and allows the user to virtually try it on.

[0795] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0796] Step 1:

[0797] A user uses a terminal to capture images of clothes in a retail store. The user takes a photo of the clothes using the camera function of their smartphone and uploads the image to the system. The input is the image data of the clothes, and the output is the image data sent to the server.

[0798] Step 2:

[0799] The server analyzes the received image data and extracts the attributes of the worn item. The server uses an image processing library such as OpenCV to analyze the image and identify the attributes of the worn item, such as color, material, pattern, and shape. The input is image data, and the output is the extracted attribute data.

[0800] Step 3:

[0801] The extracted attributes of the attached items are saved in a database. The server stores the attribute information obtained by the analysis in a database so that it can be used later upon request. The input is the attribute data of the attached items, and the output is the attribute information saved in the database.

[0802] Step 4:

[0803] The user requests outfit suggestions using a device. The user selects an outfit suggestion option from the smartphone application and enters the necessary external information (weather, schedule, etc.). The input is the outfit suggestion request and the external information, and the output is the request data sent to the server.

[0804] Step 5:

[0805] The server generates an outfit based on the user's request. The server uses a generative AI model to generate the optimal outfit based on the attribute data of the items stored in the database and external information. Prompt statements are used in this process. The inputs are the attribute data of the items, external information, and the prompt statement, and the output is the generated outfit data.

[0806] Step 6:

[0807] The generated coordinated outfit is displayed visually to the user. The server sends the generated coordinated outfit information to the terminal, which then displays the information on the user's smartphone screen. The input is the generated coordinated outfit data, and the output is the displayed coordinated outfit.

[0808] Step 7:

[0809] The user performs a virtual try-on using a device. The user selects the virtual try-on mode on their smartphone and uses the in-store mirror to superimpose their image onto the suggested outfit. The input is an image of the user's body and outfit data, and the output is a video of the results of the virtual try-on.

[0810] Step 8:

[0811] The user submits feedback based on the results of the virtual try-on. The user enters ratings and comments on the proposed outfits through a smartphone application and sends them to the server. The input is feedback data, and the output is the feedback sent to the server.

[0812] Step 9:

[0813] The server proposes suitable accessories based on the user's feedback and generates a purchase link for them. The server analyzes the feedback data, proposes new suitable accessories, generates a purchase link for them, and sends it to the user. The input is the feedback data, and the output is the new proposal and a purchase link.

[0814] Step 10:

[0815] The user clicks on the purchase link to access the online store and purchase the accessory. The user uses their smartphone to access the online store through the suggested link and purchase the accessory. The input is the purchase link, and the output is a purchase completion notification.

[0816] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0817] This invention incorporates an emotion engine into a coordination suggestion system to solve the problems users face when choosing clothes. The system aims to improve user satisfaction by recognizing the emotions felt when choosing clothes and suggesting optimal coordination based on those emotions. This system includes the following specific means:

[0818] Explain the program's processing in natural language.

[0819] 1. User clothing registration:

[0820] The user launches the terminal app and selects the option to register their clothes.

[0821] The device activates the camera function and prompts the user to take a photo of the clothes.

[0822] The user takes a photo of the clothes and the device sends the image to the server.

[0823] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[0824] The server stores the extracted attribute information in a database and returns the results to the terminal.

[0825] The terminal displays a notification to the user that registration is complete.

[0826] Examples:

[0827] A user takes a photo of a white shirt and sends it to the server via the device app. The server extracts the attributes "top," "white," and "shirt" and stores them in a database. The device then displays the message "White shirt registered."

[0828] 2. Coordination suggestions:

[0829] The user selects the outfit suggestion option on the terminal.

[0830] The device prompts the user to enter information such as today's weather and schedule.

[0831] The user enters the necessary information and the terminal sends it to the server.

[0832] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[0833] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[0834] Examples:

[0835] A user requests "Today's Work Outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the device.

[0836] 3. Emotion recognition:

[0837] The emotion engine analyzes the user's facial expression, voice, and behavioral data to recognize the user's emotions.

[0838] The device captures the user's video and audio during the virtual try-on and sends them to the server.

[0839] The server analyzes the user's emotions using an emotion engine and modifies the coordination based on the emotions.

[0840] Examples:

[0841] If the user mutters "It's a bit awkward" during the virtual fitting, the device captures the voice and facial expression and sends them to the server. The emotion engine recognizes that the user is dissatisfied, and the server changes the outfit to a "casual shirt" and "jeans."

[0842] 4. Virtual try-on through AR glasses:

[0843] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[0844] The device captures an image of the user's body and sends it to a server.

[0845] The server virtually overlays the suggested clothing onto an image of the user's body.

[0846] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[0847] The user checks the results of the virtual try-on and sends feedback to the terminal.

[0848] Examples:

[0849] When the user wears the AR glasses and looks into the mirror, the server displays a "white shirt," "blue jeans," and "black jacket" over the user's image. The user again submits dissatisfied feedback.

[0850] 5. Suitable clothing suggestions and purchases:

[0851] The server receives user feedback and fine-tunes the coordination as needed.

[0852] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[0853] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[0854] Examples:

[0855] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[0856] In this way, we have realized a coordination suggestion system that combines an emotion engine to effectively solve the user's clothing selection problem.

[0857] The processing flow will be explained below.

[0858] Step 1:

[0859] The user starts the terminal application and selects an option to register clothes.

[0860] Step 2:

[0861] The device activates the camera function and prompts the user to take a photo of the clothes.

[0862] Step 3:

[0863] The user takes a photo of the clothes and the device sends the image to the server.

[0864] Step 4:

[0865] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[0866] Step 5:

[0867] The server stores the extracted attribute information in a database.

[0868] Step 6:

[0869] The terminal displays a notification to the user that registration is complete.

[0870] Step 7:

[0871] The user selects the outfit suggestion option on the terminal.

[0872] Step 8:

[0873] The device prompts the user to enter information such as today's weather and schedule.

[0874] Step 9:

[0875] The user enters the necessary information, and the terminal sends it to the server.

[0876] Step 10:

[0877] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[0878] Step 11:

[0879] The server transmits the generated coordinates to the terminal.

[0880] Step 12:

[0881] The terminal displays the coordinated outfit suggestions to the user.

[0882] Step 13:

[0883] The emotion engine analyzes the user's facial expression, voice, and behavioral data to recognize the user's emotions.

[0884] Step 14:

[0885] The user confirms the option to use the emotion engine, and the device captures the user's facial expressions and voice and sends them to the server.

[0886] Step 15:

[0887] The server analyzes the user's emotions using an emotion engine and modifies the coordination based on the emotions.

[0888] Step 16:

[0889] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[0890] Step 17:

[0891] The device captures an image of the user's body and sends it to a server.

[0892] Step 18:

[0893] The server virtually overlays the suggested clothing onto an image of the user's body.

[0894] Step 19:

[0895] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[0896] Step 20:

[0897] The user checks the results of the virtual try-on and sends feedback to the terminal.

[0898] Step 21:

[0899] The server receives user feedback and fine-tunes the coordination as needed.

[0900] Step 22:

[0901] The server generates information about the newly suggested clothing and a link to purchase it from the online store.

[0902] Step 23:

[0903] The device displays the generated purchase link to the user.

[0904] Step 24:

[0905] The user clicks on the purchase link and is taken to the online store to purchase the clothing.

[0906] Example 2

[0907] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0908] Conventional clothing selection and coordination suggestion systems often provide one-sided suggestions without considering the user's feelings. Furthermore, users must visit a physical store to check the actual fit and appearance of the suggested clothing, which is time-consuming and laborious. Furthermore, the purchasing process at online stores is cumbersome, which can discourage users from making purchases. A new system is needed to solve these issues and increase user satisfaction.

[0909] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0910] In this invention, the server includes a means for capturing images of the user's belongings, a means for analyzing the captured images to extract the characteristics of the belongings, and a means for storing the extracted characteristics in a database. This allows the server to generate suggestions using a generative AI model based on the user's belongings list and external information, and visually display the results. Furthermore, by suggesting suitable belongings based on user feedback and generating links to the suggestions in collaboration with an online store, the server can easily purchase the items on the spot. Furthermore, a virtual try-on function using augmented reality glasses can be provided, eliminating the stress of actually trying on the belongings.

[0911] A "user" is an individual or group that uses the system and performs operations such as registering belongings, suggesting outfits, providing feedback, and making purchases.

[0912] "Belongings" are clothing, accessories, and other related items owned by the user, and are registered in the system as image data.

[0913] The "capturing means" is a function for acquiring image data using a photographing device such as a camera mounted on the user's terminal.

[0914] "Means for analyzing images" refers to software and algorithms for extracting characteristics of belongings from captured image data.

[0915] "Means for extracting features" refers to the process of analyzing images to identify attribute information such as the category, color, pattern, and material of the belongings.

[0916] "Means for storing in a database" refers to a system or software for storing and managing the extracted characteristic information of belongings as digital data.

[0917] "Means for generating proposals based on requests" refers to a generative AI model that generates optimal coordination and proposals based on user input information and external data.

[0918] The "visual display means" is a function that displays the generated coordination and proposal content on the display of the user's terminal.

[0919] "Means for sending feedback" refers to an interface that allows a user to input evaluations and opinions on a proposal and send them to the system.

[0920] The "means for suggesting suitable items and generating links to purchase them" refers to a process for making new suggestions based on user feedback and generating links to promote purchases at an online store.

[0921] "Augmented reality glasses" are wearable devices that overlay virtual objects onto the real world.

[0922] "Virtual Try-On" is a feature that allows users to virtually try on items using augmented reality glasses.

[0923] "Online Store" refers to a website or platform that sells and purchases products over the Internet.

[0924] "Purchase Link" means a hyperlink that allows you to purchase the suggested item directly from the online store.

[0925] MODE FOR CARRYING OUT THE INVENTION

[0926] This invention incorporates an emotion engine into a coordination suggestion system that helps users solve their troubles when choosing clothes. The aim is to improve user satisfaction by recognizing the emotions of users when choosing clothes and suggesting optimal coordination based on those emotions.

[0927] A specific embodiment of the system is as follows.

[0928] 1. User clothing registration

[0929] When a user launches the device app, the app displays an option to register clothes. If the user selects this option, the device's camera function is activated and the user is prompted to take a photo of the clothes. When the user takes a photo and presses the send button, the device sends the image to the server.

[0930] The server uses image analysis software such as Label Studio to extract features from the received image, such as the clothing category (for example, "top" or "pants"), color, pattern, and material. The extracted feature information is stored in a database. The processing results are sent back to the terminal, which displays a message such as "A white shirt has been registered."

[0931] Examples:

[0932] A user takes a photo of a white shirt and sends the image to the server via a smartphone app. The server extracts the attributes "top," "white," and "shirt" and stores this information in a database. The device displays the message "White shirt registered."

[0933] 2. Coordination suggestions

[0934] The user selects an outfit suggestion option on the device. The device prompts the user to enter information such as today's weather and schedule. When the user enters and submits this information, the data is sent from the device to the server.

[0935] The server uses a generative AI model (e.g., ChatGPT) to generate the optimal outfit based on the user's clothing list and the submitted information. The generative AI model uses the following prompt:

[0936] "Please suggest an outfit for work today. The weather is sunny and I'm heading to work."

[0937] The generated coordinates are sent from the server to the terminal, which then visually displays them to the user.

[0938] Examples:

[0939] A user requests "Today's Work Outfit," inputs the weather forecast and the date they plan to go to work, and the server suggests "white shirt," "blue jeans," and "black jacket." The following is displayed on the device.

[0940] 3. Emotion recognition

[0941] The system incorporates an emotion engine that analyzes the user's facial expressions, voice, and behavioral data to recognize their emotions. The device captures video and audio of the user while trying on clothes and sends them to a server. The server then uses Microsoft Azure's emotion recognition API to analyze the user's emotions and adjusts the outfit based on the results.

[0942] Examples:

[0943] If the user mutters "It's a bit awkward" during the virtual fitting, the device captures the voice and facial expression and sends them to the server. The emotion engine recognizes the user's dissatisfaction, and the server changes the outfit to a "casual shirt" and "jeans."

[0944] 4. Virtual try-on with AR glasses

[0945] The user selects the virtual try-on mode, puts on AR glasses (e.g., Magic Leap), and looks in the mirror. The device captures a full-body image of the user's body and sends it to the server. The server uses Unity to generate an image in which virtual clothes are superimposed on the user's body in real time. The device visually displays the results of the virtual try-on to the user through the AR glasses. The user checks this and sends feedback to the device.

[0946] Examples:

[0947] When a user wears the AR glasses and looks in the mirror, the server superimposes a "white shirt," "blue jeans," or "black jacket" onto the user's image. The user then submits their dissatisfaction feedback.

[0948] 5. Buying clothes

[0949] The server receives the user's feedback, fine-tunes the outfit as needed, and sends the newly suggested outfit information and a link to purchase it from an online store to the user's device. When the user clicks the link, they are taken to an online store such as Amazon, where they can purchase the outfit on the spot.

[0950] Examples:

[0951] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[0952] In this way, we can realize a coordination suggestion system that combines an emotion engine and a generative AI model to effectively support users in choosing clothes.

[0953] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0954] Step 1:

[0955] The user starts the terminal application and selects the clothes registration option.

[0956] Input: User action (selecting the "Register Clothes" option).

[0957] Output: The device's camera function is activated.

[0958] Specific operation: The user taps or clicks the "Register Clothes" button on the app screen. The device's camera automatically starts up and an interface for taking a photo of the clothes is displayed.

[0959] Step 2:

[0960] The user takes a photo of the clothes and the device sends the image to the server.

[0961] Input: A user-taken photo of an outfit.

[0962] Output: The image data is sent to the server.

[0963] Specific operation: The user takes a photo of the clothes using the camera and presses the "Send" button to send the image to the server.

[0964] Step 3:

[0965] The server analyzes the received images and uses Label Studio to extract features of the belongings.

[0966] Input: Image data of clothes.

[0967] Output: Clothing attribute information (category, color, pattern, material, etc.).

[0968] Specific operation: The server uses Label Studio to analyze the image data and extract attributes such as "tops," "white," and "shirt."

[0969] Step 4:

[0970] The server stores the extracted feature information in a database and returns the results to the terminal.

[0971] Input: Extracted clothing attribute information.

[0972] Output: Attribute information stored in the database and a notification of registration completion.

[0973] Specific operation: The extracted attribute information is stored in the database, and the result is sent back to the device. The device displays "White shirt registered."

[0974] Step 5:

[0975] The user selects the outfit suggestion option on the terminal.

[0976] Input: User action (selection of the "Outfit suggestions" option).

[0977] Output: An input form for weather, schedule, etc. is displayed.

[0978] Specific operation: The user taps or clicks the "Outfit suggestions" button, and an input form appears on the device.

[0979] Step 6:

[0980] The user inputs information such as today's weather and schedule, and the terminal sends the information to the server.

[0981] Input: Weather and schedule information entered by the user.

[0982] Output: The transmitted information arrives at the server.

[0983] Specific operation: The user enters the weather and schedule into the form and presses the "Submit" button.

[0984] Step 7:

[0985] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[0986] Input: User's clothing list and weather and schedule information.

[0987] Output: The generated coordinate plan.

[0988] Specific operation: Based on the user's belongings list and the submitted information, the server sends the following prompt to the generative AI model: "Please suggest an outfit for work today. The weather is sunny and I plan to go to work." The generative AI model then generates the optimal outfit.

[0989] Step 8:

[0990] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[0991] Input: Generated coordinate plan.

[0992] Output: Coordination suggestions displayed on the device.

[0993] Specific operation: The server sends the generated coordination plan to the terminal, and the terminal notifies the user of it.

[0994] Step 9:

[0995] The device captures the user's facial, voice, and behavioral data and transmits it to a server.

[0996] Input: User facial, voice, and behavioral data.

[0997] Output: The captured data is sent to the server.

[0998] Specific operation: The device camera and microphone record the user's facial expressions and voice, and send the data to the server.

[0999] Step 10:

[1000] The server uses an emotion engine to analyze the user's emotions and corrects the coordinates based on the emotions.

[1001] Input: User facial, voice, and behavioral data.

[1002] Output: Revised coordinate plan.

[1003] How it works: The server uses Microsoft Azure's emotion recognition API to analyze the user's emotions and generate new coordination suggestions based on the results.

[1004] Step 11:

[1005] The user selects the virtual try-on mode, puts on the AR glasses, and looks in the mirror.

[1006] Input: User operation (selecting virtual try-on mode), wearing AR glasses.

[1007] Output: Virtual try-on screen.

[1008] Specific operation: The user selects "virtual try-on mode" and puts on the AR glasses.

[1009] Step 12:

[1010] The device captures an image of the user's body and sends it to a server.

[1011] Input: Video data of the user's body.

[1012] Output: Transmitted video data.

[1013] Specific operation: The device camera captures the user's full-body image and sends the image data to the server.

[1014] Step 13:

[1015] The server uses Unity to virtually overlay the suggested clothing onto an image of the user's body.

[1016] Input: User's body image data, suggested clothing data.

[1017] Output: Virtually superimposed video data.

[1018] How it works: The server uses Unity to virtually overlay the suggested clothing onto an image of the user's body.

[1019] Step 14:

[1020] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[1021] Input: Virtual and superimposed video data.

[1022] Output: Virtual try-on results displayed on the AR glasses.

[1023] Specific operation: The AR glasses display an image of the clothes being virtually tried on to the user.

[1024] Step 15:

[1025] The server receives user feedback and fine-tunes the coordination as needed.

[1026] Input: User feedback data.

[1027] Output: A fine-tuned outfit plan.

[1028] Specific operation: The server analyzes the feedback from the user and generates new suggestions.

[1029] Step 16:

[1030] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[1031] Input: Proposal information and purchase link from the server.

[1032] Output: Proposal information and purchase link displayed on the device.

[1033] Specific behavior: A link such as "Buy the gray shirt" will appear on the device screen.

[1034] Step 17:

[1035] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[1036] Input: User action (click on purchase link).

[1037] Output: Proceed to the online store purchase page and complete the purchase.

[1038] Specific behavior: The user clicks on the link, goes to the online store's purchase page, and completes the purchase process.

[1039] (Application example 2)

[1040] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1041] Conventional fashion coordination systems make clothing recommendations without considering the user's emotions, making it difficult to fully improve user satisfaction. They also lack advanced features such as a virtual try-on environment and real-time revision of recommendations based on user feedback. As a result, users may be dissatisfied with the suggested outfits and lose their motivation to purchase.

[1042] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1043] In this invention, the server includes means for capturing images of a user's clothes, means for analyzing the captured images to extract clothing attributes, means for saving the extracted clothing attributes in a database, means for generating outfits based on the user's clothes list and external information in response to a user request, means for visually displaying the generated outfit to the user, means for recognizing emotions by analyzing the user's facial expressions and voice data using an emotion engine, means for modifying the outfit based on the recognized emotions, means for the user to send feedback, means for suggesting suitable clothes based on the user's feedback and generating a purchase link for the clothes, and means for providing a virtual try-on mode. This makes it possible to flexibly adjust outfit suggestions based on the user's emotions and feedback.

[1044] "Means for capturing images of the user's clothes" refers to a device or software that takes a photo of the clothes owned by the user and acquires the image data.

[1045] "Means for analyzing captured images and extracting clothing attributes" refers to technology that analyzes acquired image data and automatically identifies and extracts characteristics such as clothing category, color, pattern, and material.

[1046] "Means for storing extracted clothing attributes in a database" refers to a data storage or database system for permanently storing clothing characteristic information obtained through analysis.

[1047] "Means for generating outfits based on the user's clothing list and external information in response to the user's request" refers to an algorithm or system that automatically creates optimal fashion outfits in response to the user's requests and input information, taking into account pre-registered clothing and external data such as weather and schedules.

[1048] "Means for visually displaying the generated outfits to the user" refers to a digital display or device that displays the suggested outfits in a format that is easy for the user to view.

[1049] "Means of recognizing emotions by analyzing a user's facial expressions and voice data using an emotion engine" refers to an AI model or algorithm that analyzes a user's facial expressions and voice as input data and recognizes their emotions.

[1050] "Means for modifying outfits based on recognized emotions" refers to a system or method for improving or changing suggested outfits based on emotional data determined by the emotion engine.

[1051] "Means for users to submit feedback" refers to an interface or function that allows users to input and submit their opinions and thoughts about the provided coordination or the operation of the system.

[1052] "Means for suggesting suitable clothes based on user feedback and generating a link to purchase them" refers to an algorithm or system that re-suggests optimal clothes based on user feedback and generates a link for direct purchase.

[1053] "Means for providing a virtual try-on mode" refers to a system or software that provides a function that allows a user to virtually try on the clothes they have selected.

[1054] The present invention provides a coordination suggestion system incorporating an emotion engine to solve the problems users face when choosing clothes. This system allows users to register clothes using a smartphone or other device, virtually try on suggested coordinations, and then suggests optimal coordinations based on emotional feedback.

[1055] 1. User clothing registration

[1056] The user activates their smartphone or other device and selects the option to register their clothes. The smartphone activates the camera function and instructs the user to take a photo of the clothes. The user takes a photo of the clothes, and the smartphone sends the image to the server. The server analyzes the received image and extracts attributes such as the clothing category, color, pattern, and material. The extracted attribute information is stored in a database, and the results are sent back to the smartphone.

[1057] Example: A user takes a photo of a blue shirt and sends the image to a server using a fashion coordination app. The server analyzes the image, extracts the attributes "tops," "blue," and "shirt," and stores them in a database. The smartphone then displays the message "Blue shirt registered."

[1058] 2. Coordination suggestions

[1059] The user selects an outfit suggestion option on their smartphone. The smartphone prompts the user to enter information such as today's weather and schedule. The user enters the necessary information, and the smartphone sends it to the server. The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information. The generated outfit is sent to the smartphone, and the suggestions are displayed to the user.

[1060] Example: A user requests "Today's work outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the smartphone.

[1061] 3. Emotion recognition

[1062] The emotion engine analyzes the user's facial expressions, voice, and behavioral data to recognize their emotions. The smartphone captures the user's video and audio during the virtual try-on and sends them to the server. The server then analyzes the user's emotions using the emotion engine and modifies the outfit based on those emotions.

[1063] Example: If a user mutters "It's a bit awkward" during a virtual try-on session, the smartphone captures the user's voice and facial expression and sends them to the server. The emotion engine recognizes that the user is dissatisfied, and the server changes the outfit to a "casual shirt" and "jeans."

[1064] 4. Virtual try-on through AR glasses

[1065] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror. The smartphone captures an image of the user's body and sends it to the server. The server then virtually overlays the suggested clothing on the image of the user's body. The smartphone visually displays the results of the virtual try-on to the user through the AR glasses. The user checks the results of the virtual try-on and sends feedback to the smartphone.

[1066] Example: When a user wears AR glasses and looks into a mirror, the server displays "white shirt," "blue jeans," and "black jacket" over the user's image. The user again sends dissatisfied feedback.

[1067] 5. Proposal and purchase of suitable clothing

[1068] The server receives the user's feedback and fine-tunes the outfit as necessary. The smartphone receives the newly suggested outfit information and a link to purchase it from the online store from the server and displays it to the user. The user clicks the link to access the online store and purchase the outfit.

[1069] Example: If the user gives feedback that they "don't like the color," the server suggests a gray shirt instead of a white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[1070] This system uses devices such as smartphones and AR glasses to provide real-time coordination suggestions based on the user's emotions and feedback. The hardware used includes standard smartphones (iPhones, Android devices) and AR glasses (commonly known as head-mounted displays), while the software includes a server (cloud computing service), generative AI model (language model), emotion engine (emotion recognition tool), and database (data storage system). This system allows users to receive high-quality fashion coordination suggestions from the comfort of their own homes.

[1071] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1072] Step 1:

[1073] A user takes a photo of clothing with their smartphone and sends the image to a server via their device. The server receives the image and uses an image analysis algorithm to extract attributes such as clothing category, color, pattern, and material. The input is the image of the clothing, and the output is attribute information. This attribute information is stored in a database.

[1074] Step 2:

[1075] The user selects an outfit suggestion option on the smartphone app and enters the necessary information (such as weather and schedule). The device sends this to the server. The server uses a generative AI model based on the user's clothing list and the submitted information to generate the optimal outfit. The input is the user's clothing list and external information (weather, schedule), and the output is a suggested outfit. The suggested outfit is displayed to the user on the device.

[1076] Step 3:

[1077] The user selects the virtual try-on option. The device activates the camera, captures an image of the user's body, and sends it to the server. The server then uses a virtual try-on algorithm to virtually overlay the suggested outfit on the image of the user's body. The input is the image of the user's body and the suggested outfit, and the output is the virtual try-on result. This result is visually displayed to the user through the device.

[1078] Step 4:

[1079] The emotion engine analyzes the user's facial and voice data to recognize their emotions. The device captures the user's video and audio and sends them to the server. The server uses the emotion engine to analyze the user's emotions and modifies the outfit based on those emotions. The input is the user's facial and voice data, and the output is the modified outfit. The modified outfit is displayed to the user via the device.

[1080] Step 5:

[1081] The user inputs feedback and sends it to the server via the terminal. The server receives the feedback and re-proposes outfits as needed. The input is the user's feedback, and the output is the re-proposed outfits. The re-proposed outfits are displayed to the user via the terminal.

[1082] Step 6:

[1083] The server finally suggests suitable outfits and generates a purchase link to send to the device. The input is the final coordinated outfit and the user's willingness to purchase, and the output is the purchase link. When the user clicks the purchase link, they can access the online store and purchase the outfit right away.

[1084] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1085] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1086] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1087] [Third embodiment]

[1088] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1089] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1090] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1091] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1092] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1093] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1094] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1095] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1096] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1097] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1098] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1099] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1100] An embodiment of this invention is a coordination suggestion system that helps users solve various problems they may have when choosing clothes for everyday use. This system has the function of capturing images of clothes owned by the user, analyzing them to extract attributes, and suggesting optimal coordination based on the user's requests. In addition, the user can provide feedback through a virtual try-on session, and based on the results, more suitable clothes can be suggested and purchased.

[1101] Explain the program's processing in natural language.

[1102] 1. User clothing registration:

[1103] The user launches the terminal app and selects the option to register their clothes.

[1104] The device activates the camera function and prompts the user to take a photo of the clothes.

[1105] The user takes a photo of the clothes and the device sends the image to the server.

[1106] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[1107] The server stores the extracted attribute information in a database and returns the results to the terminal.

[1108] The terminal displays a notification to the user that registration is complete.

[1109] Examples:

[1110] A user takes a photo of a white shirt and sends it to the server via the device app. The server extracts the attributes "top," "white," and "shirt" and stores them in a database. The device then displays the message "White shirt registered."

[1111] 2. Coordination suggestions:

[1112] The user selects the outfit suggestion option on the terminal.

[1113] The device prompts the user to enter information such as today's weather and schedule.

[1114] The user enters the necessary information and the terminal sends it to the server.

[1115] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[1116] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[1117] Examples:

[1118] A user requests "Today's Work Outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the device.

[1119] 3. Virtual try-on through AR glasses:

[1120] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[1121] The device captures an image of the user's body and sends it to a server.

[1122] The server virtually overlays the suggested clothing onto an image of the user's body.

[1123] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[1124] The user checks the results of the virtual try-on and sends feedback to the terminal.

[1125] Examples:

[1126] When a user wears the AR glasses and looks into the mirror, the server displays a "white shirt," "blue jeans," or "black jacket" over the user's image. The user then sends feedback saying, "I don't like the color."

[1127] 4. Suitable clothing suggestions and purchases:

[1128] The server receives user feedback and fine-tunes the coordination as needed.

[1129] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[1130] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[1131] Examples:

[1132] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[1133] The detailed processing steps described above realize a system that effectively solves the user's worries about choosing clothes and suggests optimal outfits.

[1134] The processing flow will be explained below.

[1135] Step 1:

[1136] The user starts the terminal application and selects an option to register clothes.

[1137] Step 2:

[1138] The device activates the camera function and prompts the user to take a photo of the clothes.

[1139] Step 3:

[1140] The user takes a photo of the clothes and the device sends the image to the server.

[1141] Step 4:

[1142] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[1143] Step 5:

[1144] The server stores the extracted attribute information in a database.

[1145] Step 6:

[1146] The terminal displays a notification to the user that registration is complete.

[1147] Step 7:

[1148] The user selects the outfit suggestion option on the terminal.

[1149] Step 8:

[1150] The device prompts the user to enter information such as today's weather and schedule.

[1151] Step 9:

[1152] The user enters the necessary information, and the terminal sends it to the server.

[1153] Step 10:

[1154] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[1155] Step 11:

[1156] The server transmits the generated coordinates to the terminal.

[1157] Step 12:

[1158] The terminal displays the coordinated outfit suggestions to the user.

[1159] Step 13:

[1160] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[1161] Step 14:

[1162] The device captures an image of the user's body and sends it to a server.

[1163] Step 15:

[1164] The server virtually overlays the suggested clothing onto an image of the user's body.

[1165] Step 16:

[1166] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[1167] Step 17:

[1168] The user checks the results of the virtual try-on and sends feedback to the terminal.

[1169] Step 18:

[1170] The server receives user feedback and fine-tunes the coordination as needed.

[1171] Step 19:

[1172] The server generates information about the newly suggested clothing and a link to purchase it from the online store.

[1173] Step 20:

[1174] The device displays the generated purchase link to the user.

[1175] Step 21:

[1176] The user clicks on the purchase link and is taken to the online store to purchase the clothing.

[1177] Example 1

[1178] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1179] Conventional clothing selection and coordination suggestion systems have had problems such as the complicated process of registering and categorizing the clothes a user owns, and the quality of the suggested coordinations does not adequately reflect the user's requests or external information (weather, schedule, etc.). Furthermore, they lacked the functionality to actually try on the suggested coordinations, limiting the benefits of online purchases.

[1180] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1181] In this invention, the server includes means for acquiring images of the user's clothing, means for analyzing the acquired images to extract clothing features, means for storing the extracted clothing features in a database, means for generating outfits based on the user's clothing list and external information in response to the user's request, means for visually displaying the generated outfits to the user, means for the user to send feedback, and means for suggesting suitable clothing based on the user's feedback and generating a link to purchase the clothing. This allows users to easily register and categorize their clothing and receive high-quality outfit suggestions that take external information into account. Furthermore, the integration of the virtual try-on function and online purchase further improves convenience.

[1182] "User" refers to an individual who uses the system to register clothes and receive coordination suggestions.

[1183] "Clothing" refers to any clothing owned by a user and registered in the system.

[1184] "Means for acquiring images" refers to a function that allows a user to take a photo of clothing using a terminal and send the image data to a server.

[1185] "Means of analyzing images and extracting clothing characteristics" refers to the function of using analysis software on a server to identify characteristic information such as category, color, pattern, and material from acquired images of clothing.

[1186] "Means for storing in a database" refers to the function of recording the analyzed and extracted clothing characteristic information in storage within the system and managing it efficiently.

[1187] "User request" refers to the content of a request for a coordination proposal made by a user operating the system.

[1188] "External information" refers to additional information such as weather and the user's schedule that the system should take into consideration when proposing outfits.

[1189] "Means for generating outfits" refers to the function of using a generative AI model to create optimal outfits based on the user's clothing list and external information.

[1190] The "means for visually displaying" refers to a function for displaying the generated coordinate on the user's terminal screen and providing it to the user visually.

[1191] "Means for sending feedback" refers to the function that allows users to input their thoughts and requests for improvement regarding the proposed coordination and send them back to the system.

[1192] "Means for suggesting suitable clothing based on feedback and generating a link to purchase it" refers to a function that modifies outfits based on user feedback and provides optimal clothing and a link to an online store where it can be purchased.

[1193] "Augmented reality glasses" refers to a device that users can wear to visually view information from the system and virtual clothing.

[1194] "Means for displaying virtual clothing" refers to a function for displaying virtual clothing overlaid on the user's image.

[1195] "Online store" refers to an e-commerce site where clothing can be purchased via the Internet.

[1196] This invention is a coordination suggestion system that helps users solve various problems they may have when choosing clothes. The system captures images of the user's clothing, analyzes them to extract attributes, and suggests optimal coordinations based on the user's requests. The system also allows the user to provide feedback through a virtual try-on session, which allows further suggestions to be made based on the results, and allows the user to purchase clothing online.

[1197] The system configuration is as follows:

[1198] 1. User clothing registration:

[1199] The user launches the device app and selects the "Register Clothing" option. The device activates its camera function and prompts the user to take a photo of the clothing. The user takes a photo of the clothing, and the device sends the image to the server. The server uses image analysis software (e.g., OpenCV) to extract attributes such as clothing category, color, pattern, and material from the image and stores these attributes in a database. For example, if the user takes a photo of a white shirt and sends it, the server extracts the attributes "top," "white," and "shirt" and stores them in the database. The device then notifies the user that "your white shirt has been registered."

[1200] 2. Coordination suggestions:

[1201] The user selects the "Outfit Suggestion" option on the device app. The device displays a form prompting the user to "Enter today's weather and schedule." When the user enters and submits information such as "sunny" and "work schedule," the device sends this to the server. The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information. As a specific example, when the user requests "Today's work outfit," the server suggests "white shirt," "blue denim," and "black jacket" and sends them to the device. The device then displays the suggested outfit to the user. An example of a prompt sentence is as follows:

[1202] "The user requests registration of a white shirt and suggestions for outfits. Please suggest today's outfits that include a white shirt based on the current weather and schedule."

[1203] 3. Virtual try-on through AR glasses:

[1204] The user selects the virtual try-on mode and puts on the AR glasses. The device uses the AR glasses' camera to capture an image of the user's body and sends it to the server. The server then virtually overlays suggested clothing items based on the received image. For example, the server may overlay a "white shirt," "blue denim," or "black jacket" onto the user's image. The device visually displays the results of this virtual try-on to the user in real time through the AR glasses. The user reviews the image and provides feedback through the app. For example, they may provide feedback such as, "I don't like the color."

[1205] 4. Suitable clothing suggestions and purchases:

[1206] The server receives the user's feedback and fine-tunes the outfit as needed. It then generates new suggested clothing items and a link to the online store where they can be purchased, and sends them to the device. For example, if the user sends feedback that they "don't like the color," the server might suggest a gray shirt instead of a white one. The device then displays the new suggested items along with a link to purchase the gray shirt. The user clicks the link and purchases the gray shirt from the online store.

[1207] This allows users to easily register and categorize their clothing and receive high-quality outfit suggestions that take external information into account. Furthermore, the integration of the virtual try-on feature and online purchase further enhances convenience.

[1208] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1209] Step 1:

[1210] The user launches the device app and selects the "Clothing Registration" option. A screen prompting the user to perform an action is displayed on the device.

[1211] (Input) User operations (app launch and option selection)

[1212] (Output) App instruction screen

[1213] Step 2:

[1214] The device will activate its camera function and prompt the user to take a photo of the clothing. The message "Take a photo" will be displayed on the screen.

[1215] (Input) User operation (option selection)

[1216] (Output) Camera startup and instruction message

[1217] Step 3:

[1218] The user takes a photo of the garment and taps the "Send" button. An image file is generated.

[1219] (Input) User operation (photographing)

[1220] (Output) Image file

[1221] Step 4:

[1222] The device sends the captured image to the server, which prepares to analyze the received image.

[1223] (Input) Image file

[1224] (Output) Image data sent to server

[1225] Step 5:

[1226] The server uses image analysis software (e.g., OpenCV) to analyze the received images and extract attributes such as clothing category, color, pattern, and material.

[1227] (Input) Image data

[1228] (Output) Extracted attribute data (category, color, pattern, material)

[1229] Step 6:

[1230] The server saves the extracted attribute information in the database and returns a status to the terminal if the save is successful.

[1231] (Input) Extracted attribute data

[1232] (Output) Database save status

[1233] Step 7:

[1234] The device notifies the user that registration is complete. The message "White shirt successfully registered" appears on the screen.

[1235] (Input) Database save status

[1236] (Output) Registration completion notification

[1237] Step 8:

[1238] The user selects the "Outfit suggestions" option in the device app. A screen prompting the user to perform an action is displayed on the device.

[1239] (Input) User operation (option selection)

[1240] (Output) Coordination proposal input form

[1241] Step 9:

[1242] The device displays a form prompting the user to "Enter today's weather and schedule." The user enters the information and taps the "Submit" button.

[1243] (Input) User operation (option selection)

[1244] (Output) Input form

[1245] Step 10:

[1246] The user inputs "Sunny" and "Work schedule" and submits the information. The device then sends the information to the server.

[1247] (Input) Input form

[1248] (Output) Weather and schedule data

[1249] Step 11:

[1250] The server takes the user's clothing list and submitted information and uses a generative AI model to generate the optimal outfit.

[1251] (Input) Clothing list and weather / schedule data

[1252] (Output) Generated coordinates

[1253] Step 12:

[1254] The server sends the generated outfit to the device, which then displays the suggestions to the user. Suggested outfits include a "white shirt," "blue jeans," and a "black jacket."

[1255] (Input) Generated coordinates

[1256] (Output) Display of suggested outfits

[1257] Step 13:

[1258] The user selects the virtual try-on mode and puts on the AR glasses. The device then activates the camera in the AR glasses.

[1259] (Input) User operation (select virtual try-on mode)

[1260] (Output) Activate the camera on the AR glasses

[1261] Step 14:

[1262] The device captures an image of the user's body and sends it to the server, which receives the image data.

[1263] (Input) Image of the user's body

[1264] (Output) Video data sent to server

[1265] Step 15:

[1266] The server virtually overlays the suggested clothing based on the video received.

[1267] (Input) Video data and proposed coordinates

[1268] (Output) Virtual try-on results

[1269] Step 16:

[1270] The device visually displays the results of the virtual try-on to the user in real time through the AR glasses, making it appear as if the user is actually wearing the garment.

[1271] (Input) Virtual try-on results

[1272] (Output) Visual display

[1273] Step 17:

[1274] The user checks the video and sends feedback through the app, saying, "I don't like the colors." The device then sends the feedback to the server.

[1275] (Input) Virtual fitting results

[1276] (Output) Feedback Data

[1277] Step 18:

[1278] The server generates new outfits based on user feedback, suggesting, for example, a gray shirt instead of a white one.

[1279] (Input) Feedback data

[1280] (Output) New coordinates

[1281] Step 19:

[1282] The server sends the new outfit along with a link to purchase from the online store to the device, which then displays the new suggestions to the user.

[1283] (Input) New coordinates

[1284] (Output) New offer and purchase link

[1285] Step 20:

[1286] A user clicks on a purchase link to go to an online store and purchase clothing, for example, a gray shirt.

[1287] (Input) Purchase link

[1288] (Print) Online purchase complete

[1289] (Application example 1)

[1290] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1291] Conventional clothing selection systems make it difficult for users to easily visually check how their existing accessories coordinate with the new accessories they are purchasing. Additionally, the time and effort involved in trying on and purchasing items in a retail store is a significant burden for users. While systems exist that suggest suitable accessories based on user feedback, there are few systems that can reflect this feedback in real time and improve the user experience through virtual try-on.

[1292] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1293] In this invention, the server includes means for capturing images of a user's accessories, means for analyzing the captured images to extract characteristics of the accessories, means for storing the extracted characteristics in a database, means for generating a coordinated outfit based on the user's accessory list and external information in response to a user request, means for displaying the generated coordinated outfit to the user's visual display, means for the user to send feedback, means for suggesting suitable accessories based on the user's feedback and generating a purchase link for the accessories, means for the user to capture images of the accessories using a terminal in a retail store and purchase the accessories on the spot, and means for providing a virtual try-on experience in conjunction with a mirror in the store for the user to virtually try on the accessories. This allows users to easily coordinate and virtually try on accessories in a retail store without pressure, and to purchase the optimal accessories on the spot. Furthermore, by using a generative AI model to generate an optimal outfit based on the user's request and characteristics using prompt sentences and visually displaying the generated results to the user, the user can more easily and efficiently select accessories.

[1294] The "means for capturing images of user's accessories" refers to a device that has the function of recording the user's clothes and accessories as photographs using the camera function of the terminal.

[1295] The "means for analyzing the captured image and extracting the characteristics of the attached item" refers to algorithms and software that automatically identify the characteristics of the attached item, such as its color, material, pattern, and shape, from the captured image and treat them as data.

[1296] The "means for storing extracted characteristics of the attached item in a database" is a system that has the function of storing characteristic data of the attached item obtained by image analysis in a database so that it can be reused later.

[1297] "Means for generating an outfit based on the user's accessory list and external information in response to the user's request" refers to software that generates the optimal outfit based on the desired conditions and event information entered by the user, taking into account the accessories the user has and external factors such as weather information and fashion information.

[1298] The "means for visually displaying the generated coordination to the user" is a system having the function of displaying information about the generated coordination on the user's terminal screen, etc., so that the user can visually confirm it.

[1299] "Means for users to send feedback" refers to a mechanism or interface that allows users to input their ratings and comments on the proposed outfits and send that information to the system.

[1300] The "means for proposing suitable accessories based on user feedback and generating a link to purchase them" refers to software and a system that analyzes feedback information from users, and based on that, proposes new accessories and generates a link to purchase the accessories, which are then provided to the user.

[1301] "A means by which a user can capture an image of an accessory using a terminal in a retail store and purchase the accessory on the spot" is a system that allows a user to take an image of an accessory using a camera on a terminal in a store, obtain necessary information based on that data, and complete the purchase on the spot.

[1302] "Means for providing a virtual fitting experience in cooperation with mirrors in a store so that users can virtually try on clothes" refers to a system that allows users to use a specific device in a store to virtually superimpose items onto their own image to provide a fitting experience without actually trying on the items.

[1303] "Means for using a generative AI model to generate optimal outfits using prompt sentences based on the user's requests and characteristics, and visually displaying the generated results to the user" refers to a system that uses an artificial intelligence algorithm to automatically create outfits that take into account the user's individual information and requests, generates them as prompt sentences, and visually presents the results to the user.

[1304] An embodiment of this invention is a coordination suggestion system that solves various problems users may have when choosing clothes for everyday use. This system has a function that allows users to capture images of clothes using their smartphones in a retail store, analyzes the images to extract attributes, and suggests optimal coordination based on the user's requests. In addition, the user can provide feedback through a virtual try-on, and based on the results, more suitable clothes can be suggested and purchased.

[1305] The system uses the following hardware and software:

[1306] Smartphone: Uses the camera function to capture images of the user's clothes and accessories.

[1307] Server: Performs image analysis and coordinate generation.

[1308] Database: Stores the user's clothing attribute information and coordination information.

[1309] Generative AI model: Automatically generates optimal outfits based on user requests and characteristics.

[1310] Specifically, the following process is performed:

[1311] 1. Image capture and attribute extraction: When a user takes a picture of clothing in a retail store using their smartphone, the smartphone sends the image to a server. The server analyzes the image using an image processing library such as OpenCV and extracts the attributes of the clothing (color, material, pattern, shape, etc.). This attribute information is stored in a database.

[1312] 2. Coordination Generation: When a user requests a coordination suggestion, the server uses a generative AI model to generate the optimal coordination based on the user's clothing list and external information (weather forecast, event information, etc.). This coordination is generated using a prompt sentence and displayed on the smartphone.

[1313] 3. Virtual Try-On: The user selects the virtual try-on mode on their smartphone and tries on the clothes virtually using the in-store mirror. The server then virtually overlays the suggested clothes onto the captured image of the user's body and displays the results on the smartphone screen or in the in-store mirror.

[1314] 4. Feedback and Purchase: When the user submits feedback based on the results of the virtual try-on, the server analyzes the feedback and suggests more suitable items to wear. The suggestions include a purchase link, allowing the user to purchase the items from the online store immediately.

[1315] Examples:

[1316] A user requests a stylish outfit and uploads a photo of light blue jeans and a white shirt. The system then suggests outfits based on blue and white, performs a virtual try-on, and provides visual feedback to the user.

[1317] Example prompts to input to the generative AI model:

[1318] User input: Light blue jeans and a white shirt. Today's weather is sunny and I plan to go to work. Coordination suggestions and virtual try-on are performed.

[1319] AI model: Based on these criteria, it suggests the best outfit and allows the user to virtually try it on.

[1320] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1321] Step 1:

[1322] A user uses a terminal to capture images of clothes in a retail store. The user takes a photo of the clothes using the camera function of their smartphone and uploads the image to the system. The input is the image data of the clothes, and the output is the image data sent to the server.

[1323] Step 2:

[1324] The server analyzes the received image data and extracts the attributes of the worn item. The server uses an image processing library such as OpenCV to analyze the image and identify the attributes of the worn item, such as color, material, pattern, and shape. The input is image data, and the output is the extracted attribute data.

[1325] Step 3:

[1326] The extracted attributes of the attached items are saved in a database. The server stores the attribute information obtained by the analysis in a database so that it can be used later upon request. The input is the attribute data of the attached items, and the output is the attribute information saved in the database.

[1327] Step 4:

[1328] The user requests outfit suggestions using a device. The user selects an outfit suggestion option from the smartphone application and enters the necessary external information (weather, schedule, etc.). The input is the outfit suggestion request and the external information, and the output is the request data sent to the server.

[1329] Step 5:

[1330] The server generates an outfit based on the user's request. The server uses a generative AI model to generate the optimal outfit based on the attribute data of the items stored in the database and external information. Prompt statements are used in this process. The inputs are the attribute data of the items, external information, and the prompt statement, and the output is the generated outfit data.

[1331] Step 6:

[1332] The generated coordinated outfit is displayed visually to the user. The server sends the generated coordinated outfit information to the terminal, which then displays the information on the user's smartphone screen. The input is the generated coordinated outfit data, and the output is the displayed coordinated outfit.

[1333] Step 7:

[1334] The user performs a virtual try-on using a device. The user selects the virtual try-on mode on their smartphone and uses the in-store mirror to superimpose their image onto the suggested outfit. The input is an image of the user's body and outfit data, and the output is a video of the results of the virtual try-on.

[1335] Step 8:

[1336] The user submits feedback based on the results of the virtual try-on. The user enters ratings and comments on the proposed outfits through a smartphone application and sends them to the server. The input is feedback data, and the output is the feedback sent to the server.

[1337] Step 9:

[1338] The server proposes suitable accessories based on the user's feedback and generates a purchase link for them. The server analyzes the feedback data, proposes new suitable accessories, generates a purchase link for them, and sends it to the user. The input is the feedback data, and the output is the new proposal and a purchase link.

[1339] Step 10:

[1340] The user clicks on the purchase link to access the online store and purchase the accessory. The user uses their smartphone to access the online store through the suggested link and purchase the accessory. The input is the purchase link, and the output is a purchase completion notification.

[1341] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1342] This invention incorporates an emotion engine into a coordination suggestion system to solve the problems users face when choosing clothes. The system aims to improve user satisfaction by recognizing the emotions felt when choosing clothes and suggesting optimal coordination based on those emotions. This system includes the following specific means:

[1343] Explain the program's processing in natural language.

[1344] 1. User clothing registration:

[1345] The user launches the terminal app and selects the option to register their clothes.

[1346] The device activates the camera function and prompts the user to take a photo of the clothes.

[1347] The user takes a photo of the clothes and the device sends the image to the server.

[1348] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[1349] The server stores the extracted attribute information in a database and returns the results to the terminal.

[1350] The terminal displays a notification to the user that registration is complete.

[1351] Examples:

[1352] A user takes a photo of a white shirt and sends it to the server via the device app. The server extracts the attributes "top," "white," and "shirt" and stores them in a database. The device then displays the message "White shirt registered."

[1353] 2. Coordination suggestions:

[1354] The user selects the outfit suggestion option on the terminal.

[1355] The device prompts the user to enter information such as today's weather and schedule.

[1356] The user enters the necessary information and the terminal sends it to the server.

[1357] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[1358] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[1359] Examples:

[1360] A user requests "Today's Work Outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the device.

[1361] 3. Emotion recognition:

[1362] The emotion engine analyzes the user's facial expression, voice, and behavioral data to recognize the user's emotions.

[1363] The device captures the user's video and audio during the virtual try-on and sends them to the server.

[1364] The server analyzes the user's emotions using an emotion engine and modifies the coordination based on the emotions.

[1365] Examples:

[1366] If the user mutters "It's a bit awkward" during the virtual fitting, the device captures the voice and facial expression and sends them to the server. The emotion engine recognizes that the user is dissatisfied, and the server changes the outfit to a "casual shirt" and "jeans."

[1367] 4. Virtual try-on through AR glasses:

[1368] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[1369] The device captures an image of the user's body and sends it to a server.

[1370] The server virtually overlays the suggested clothing onto an image of the user's body.

[1371] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[1372] The user checks the results of the virtual try-on and sends feedback to the terminal.

[1373] Examples:

[1374] When the user wears the AR glasses and looks into the mirror, the server displays a "white shirt," "blue jeans," and "black jacket" over the user's image. The user again submits dissatisfied feedback.

[1375] 5. Suitable clothing suggestions and purchases:

[1376] The server receives user feedback and fine-tunes the coordination as needed.

[1377] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[1378] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[1379] Examples:

[1380] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[1381] In this way, we have realized a coordination suggestion system that combines an emotion engine to effectively solve the user's clothing selection problem.

[1382] The processing flow will be explained below.

[1383] Step 1:

[1384] The user starts the terminal application and selects an option to register clothes.

[1385] Step 2:

[1386] The device activates the camera function and prompts the user to take a photo of the clothes.

[1387] Step 3:

[1388] The user takes a photo of the clothes and the device sends the image to the server.

[1389] Step 4:

[1390] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[1391] Step 5:

[1392] The server stores the extracted attribute information in a database.

[1393] Step 6:

[1394] The terminal displays a notification to the user that registration is complete.

[1395] Step 7:

[1396] The user selects the outfit suggestion option on the terminal.

[1397] Step 8:

[1398] The device prompts the user to enter information such as today's weather and schedule.

[1399] Step 9:

[1400] The user enters the necessary information, and the terminal sends it to the server.

[1401] Step 10:

[1402] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[1403] Step 11:

[1404] The server transmits the generated coordinates to the terminal.

[1405] Step 12:

[1406] The terminal displays the coordinated outfit suggestions to the user.

[1407] Step 13:

[1408] The emotion engine analyzes the user's facial expression, voice, and behavioral data to recognize the user's emotions.

[1409] Step 14:

[1410] The user confirms the option to use the emotion engine, and the device captures the user's facial expressions and voice and sends them to the server.

[1411] Step 15:

[1412] The server analyzes the user's emotions using an emotion engine and modifies the coordination based on the emotions.

[1413] Step 16:

[1414] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[1415] Step 17:

[1416] The device captures an image of the user's body and sends it to a server.

[1417] Step 18:

[1418] The server virtually overlays the suggested clothing onto an image of the user's body.

[1419] Step 19:

[1420] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[1421] Step 20:

[1422] The user checks the results of the virtual try-on and sends feedback to the terminal.

[1423] Step 21:

[1424] The server receives user feedback and fine-tunes the coordination as needed.

[1425] Step 22:

[1426] The server generates information about the newly suggested clothing and a link to purchase it from the online store.

[1427] Step 23:

[1428] The device displays the generated purchase link to the user.

[1429] Step 24:

[1430] The user clicks on the purchase link and is taken to the online store to purchase the clothing.

[1431] Example 2

[1432] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1433] Conventional clothing selection and coordination suggestion systems often provide one-sided suggestions without considering the user's feelings. Furthermore, users must visit a physical store to check the actual fit and appearance of the suggested clothing, which is time-consuming and laborious. Furthermore, the purchasing process at online stores is cumbersome, which can discourage users from making purchases. A new system is needed to solve these issues and increase user satisfaction.

[1434] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1435] In this invention, the server includes a means for capturing images of the user's belongings, a means for analyzing the captured images to extract the characteristics of the belongings, and a means for storing the extracted characteristics in a database. This allows the server to generate suggestions using a generative AI model based on the user's belongings list and external information, and visually display the results. Furthermore, by suggesting suitable belongings based on user feedback and generating links to the suggestions in collaboration with an online store, the server can easily purchase the items on the spot. Furthermore, a virtual try-on function using augmented reality glasses can be provided, eliminating the stress of actually trying on the belongings.

[1436] A "user" is an individual or group that uses the system and performs operations such as registering belongings, suggesting outfits, providing feedback, and making purchases.

[1437] "Belongings" are clothing, accessories, and other related items owned by the user, and are registered in the system as image data.

[1438] The "capturing means" is a function for acquiring image data using a photographing device such as a camera mounted on the user's terminal.

[1439] "Means for analyzing images" refers to software and algorithms for extracting characteristics of belongings from captured image data.

[1440] "Means for extracting features" refers to the process of analyzing images to identify attribute information such as the category, color, pattern, and material of the belongings.

[1441] "Means for storing in a database" refers to a system or software for storing and managing the extracted characteristic information of belongings as digital data.

[1442] "Means for generating proposals based on requests" refers to a generative AI model that generates optimal coordination and proposals based on user input information and external data.

[1443] The "visual display means" is a function that displays the generated coordination and proposal content on the display of the user's terminal.

[1444] "Means for sending feedback" refers to an interface that allows a user to input evaluations and opinions on a proposal and send them to the system.

[1445] The "means for suggesting suitable items and generating links to purchase them" refers to a process for making new suggestions based on user feedback and generating links to promote purchases at an online store.

[1446] "Augmented reality glasses" are wearable devices that overlay virtual objects onto the real world.

[1447] "Virtual Try-On" is a feature that allows users to virtually try on items using augmented reality glasses.

[1448] "Online Store" refers to a website or platform that sells and purchases products over the Internet.

[1449] "Purchase Link" means a hyperlink that allows you to purchase the suggested item directly from the online store.

[1450] MODE FOR CARRYING OUT THE INVENTION

[1451] This invention incorporates an emotion engine into a coordination suggestion system that helps users solve their troubles when choosing clothes. The aim is to improve user satisfaction by recognizing the emotions of users when choosing clothes and suggesting optimal coordination based on those emotions.

[1452] A specific embodiment of the system is as follows.

[1453] 1. User clothing registration

[1454] When a user launches the device app, the app displays an option to register clothes. If the user selects this option, the device's camera function is activated and the user is prompted to take a photo of the clothes. When the user takes a photo and presses the send button, the device sends the image to the server.

[1455] The server uses image analysis software such as Label Studio to extract features from the received image, such as the clothing category (for example, "top" or "pants"), color, pattern, and material. The extracted feature information is stored in a database. The processing results are sent back to the terminal, which displays a message such as "A white shirt has been registered."

[1456] Examples:

[1457] A user takes a photo of a white shirt and sends the image to the server via a smartphone app. The server extracts the attributes "top," "white," and "shirt" and stores this information in a database. The device displays the message "White shirt registered."

[1458] 2. Coordination suggestions

[1459] The user selects an outfit suggestion option on the device. The device prompts the user to enter information such as today's weather and schedule. When the user enters and submits this information, the data is sent from the device to the server.

[1460] The server uses a generative AI model (e.g., ChatGPT) to generate the optimal outfit based on the user's clothing list and the submitted information. The generative AI model uses the following prompt:

[1461] "Please suggest an outfit for work today. The weather is sunny and I'm heading to work."

[1462] The generated coordinates are sent from the server to the terminal, which then visually displays them to the user.

[1463] Examples:

[1464] A user requests "Today's Work Outfit," inputs the weather forecast and the date they plan to go to work, and the server suggests "white shirt," "blue jeans," and "black jacket." The following is displayed on the device.

[1465] 3. Emotion recognition

[1466] The system incorporates an emotion engine that analyzes the user's facial expressions, voice, and behavioral data to recognize their emotions. The device captures video and audio of the user while trying on clothes and sends them to a server. The server then uses Microsoft Azure's emotion recognition API to analyze the user's emotions and adjusts the outfit based on the results.

[1467] Examples:

[1468] If the user mutters "It's a bit awkward" during the virtual fitting, the device captures the voice and facial expression and sends them to the server. The emotion engine recognizes the user's dissatisfaction, and the server changes the outfit to a "casual shirt" and "jeans."

[1469] 4. Virtual try-on with AR glasses

[1470] The user selects the virtual try-on mode, puts on AR glasses (e.g., Magic Leap), and looks in the mirror. The device captures a full-body image of the user's body and sends it to the server. The server uses Unity to generate an image in which virtual clothes are superimposed on the user's body in real time. The device visually displays the results of the virtual try-on to the user through the AR glasses. The user checks this and sends feedback to the device.

[1471] Examples:

[1472] When a user wears the AR glasses and looks in the mirror, the server superimposes a "white shirt," "blue jeans," or "black jacket" onto the user's image. The user then submits their dissatisfaction feedback.

[1473] 5. Buying clothes

[1474] The server receives the user's feedback, fine-tunes the outfit as needed, and sends the newly suggested outfit information and a link to purchase it from an online store to the user's device. When the user clicks the link, they are taken to an online store such as Amazon, where they can purchase the outfit on the spot.

[1475] Examples:

[1476] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[1477] In this way, we can realize a coordination suggestion system that combines an emotion engine and a generative AI model to effectively support users in choosing clothes.

[1478] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1479] Step 1:

[1480] The user starts the terminal application and selects the clothes registration option.

[1481] Input: User action (selecting the "Register Clothes" option).

[1482] Output: The device's camera function is activated.

[1483] Specific operation: The user taps or clicks the "Register Clothes" button on the app screen. The device's camera automatically starts up and an interface for taking a photo of the clothes is displayed.

[1484] Step 2:

[1485] The user takes a photo of the clothes and the device sends the image to the server.

[1486] Input: A user-taken photo of an outfit.

[1487] Output: The image data is sent to the server.

[1488] Specific operation: The user takes a photo of the clothes using the camera and presses the "Send" button to send the image to the server.

[1489] Step 3:

[1490] The server analyzes the received images and uses Label Studio to extract features of the belongings.

[1491] Input: Image data of clothes.

[1492] Output: Clothing attribute information (category, color, pattern, material, etc.).

[1493] Specific operation: The server uses Label Studio to analyze the image data and extract attributes such as "tops," "white," and "shirt."

[1494] Step 4:

[1495] The server stores the extracted feature information in a database and returns the results to the terminal.

[1496] Input: Extracted clothing attribute information.

[1497] Output: Attribute information stored in the database and a notification of registration completion.

[1498] Specific operation: The extracted attribute information is stored in the database, and the result is sent back to the device. The device displays "White shirt registered."

[1499] Step 5:

[1500] The user selects the outfit suggestion option on the terminal.

[1501] Input: User action (selection of the "Outfit suggestions" option).

[1502] Output: An input form for weather, schedule, etc. is displayed.

[1503] Specific operation: The user taps or clicks the "Outfit suggestions" button, and an input form appears on the device.

[1504] Step 6:

[1505] The user inputs information such as today's weather and schedule, and the terminal sends the information to the server.

[1506] Input: Weather and schedule information entered by the user.

[1507] Output: The transmitted information arrives at the server.

[1508] Specific operation: The user enters the weather and schedule into the form and presses the "Submit" button.

[1509] Step 7:

[1510] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[1511] Input: User's clothing list and weather and schedule information.

[1512] Output: The generated coordinate plan.

[1513] Specific operation: Based on the user's belongings list and the submitted information, the server sends the following prompt to the generative AI model: "Please suggest an outfit for work today. The weather is sunny and I plan to go to work." The generative AI model then generates the optimal outfit.

[1514] Step 8:

[1515] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[1516] Input: Generated coordinate plan.

[1517] Output: Coordination suggestions displayed on the device.

[1518] Specific operation: The server sends the generated coordination plan to the terminal, and the terminal notifies the user of it.

[1519] Step 9:

[1520] The device captures the user's facial, voice, and behavioral data and transmits it to a server.

[1521] Input: User facial, voice, and behavioral data.

[1522] Output: The captured data is sent to the server.

[1523] Specific operation: The device camera and microphone record the user's facial expressions and voice, and send the data to the server.

[1524] Step 10:

[1525] The server uses an emotion engine to analyze the user's emotions and corrects the coordinates based on the emotions.

[1526] Input: User facial, voice, and behavioral data.

[1527] Output: Revised coordinate plan.

[1528] How it works: The server uses Microsoft Azure's emotion recognition API to analyze the user's emotions and generate new coordination suggestions based on the results.

[1529] Step 11:

[1530] The user selects the virtual try-on mode, puts on the AR glasses, and looks in the mirror.

[1531] Input: User operation (selecting virtual try-on mode), wearing AR glasses.

[1532] Output: Virtual try-on screen.

[1533] Specific operation: The user selects "virtual try-on mode" and puts on the AR glasses.

[1534] Step 12:

[1535] The device captures an image of the user's body and sends it to a server.

[1536] Input: Video data of the user's body.

[1537] Output: Transmitted video data.

[1538] Specific operation: The device camera captures the user's full-body image and sends the image data to the server.

[1539] Step 13:

[1540] The server uses Unity to virtually overlay the suggested clothing onto an image of the user's body.

[1541] Input: User's body image data, suggested clothing data.

[1542] Output: Virtually superimposed video data.

[1543] How it works: The server uses Unity to virtually overlay the suggested clothing onto an image of the user's body.

[1544] Step 14:

[1545] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[1546] Input: Virtual and superimposed video data.

[1547] Output: Virtual try-on results displayed on the AR glasses.

[1548] Specific operation: The AR glasses display an image of the clothes being virtually tried on to the user.

[1549] Step 15:

[1550] The server receives user feedback and fine-tunes the coordination as needed.

[1551] Input: User feedback data.

[1552] Output: A fine-tuned outfit plan.

[1553] Specific operation: The server analyzes the feedback from the user and generates new suggestions.

[1554] Step 16:

[1555] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[1556] Input: Proposal information and purchase link from the server.

[1557] Output: Proposal information and purchase link displayed on the device.

[1558] Specific behavior: A link such as "Buy the gray shirt" will appear on the device screen.

[1559] Step 17:

[1560] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[1561] Input: User action (click on purchase link).

[1562] Output: Proceed to the online store purchase page and complete the purchase.

[1563] Specific behavior: The user clicks on the link, goes to the online store's purchase page, and completes the purchase process.

[1564] (Application example 2)

[1565] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1566] Conventional fashion coordination systems make clothing recommendations without considering the user's emotions, making it difficult to fully improve user satisfaction. They also lack advanced features such as a virtual try-on environment and real-time revision of recommendations based on user feedback. As a result, users may be dissatisfied with the suggested outfits and lose their motivation to purchase.

[1567] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1568] In this invention, the server includes means for capturing images of a user's clothes, means for analyzing the captured images to extract clothing attributes, means for saving the extracted clothing attributes in a database, means for generating outfits based on the user's clothes list and external information in response to a user request, means for visually displaying the generated outfit to the user, means for recognizing emotions by analyzing the user's facial expressions and voice data using an emotion engine, means for modifying the outfit based on the recognized emotions, means for the user to send feedback, means for suggesting suitable clothes based on the user's feedback and generating a purchase link for the clothes, and means for providing a virtual try-on mode. This makes it possible to flexibly adjust outfit suggestions based on the user's emotions and feedback.

[1569] "Means for capturing images of the user's clothes" refers to a device or software that takes a photo of the clothes owned by the user and acquires the image data.

[1570] "Means for analyzing captured images and extracting clothing attributes" refers to technology that analyzes acquired image data and automatically identifies and extracts characteristics such as clothing category, color, pattern, and material.

[1571] "Means for storing extracted clothing attributes in a database" refers to a data storage or database system for permanently storing clothing characteristic information obtained through analysis.

[1572] "Means for generating outfits based on the user's clothing list and external information in response to the user's request" refers to an algorithm or system that automatically creates optimal fashion outfits in response to the user's requests and input information, taking into account pre-registered clothing and external data such as weather and schedules.

[1573] "Means for visually displaying the generated outfits to the user" refers to a digital display or device that displays the suggested outfits in a format that is easy for the user to view.

[1574] "Means of recognizing emotions by analyzing a user's facial expressions and voice data using an emotion engine" refers to an AI model or algorithm that analyzes a user's facial expressions and voice as input data and recognizes their emotions.

[1575] "Means for modifying outfits based on recognized emotions" refers to a system or method for improving or changing suggested outfits based on emotional data determined by the emotion engine.

[1576] "Means for users to submit feedback" refers to an interface or function that allows users to input and submit their opinions and thoughts about the provided coordination or the operation of the system.

[1577] "Means for suggesting suitable clothes based on user feedback and generating a link to purchase them" refers to an algorithm or system that re-suggests optimal clothes based on user feedback and generates a link for direct purchase.

[1578] "Means for providing a virtual try-on mode" refers to a system or software that provides a function that allows a user to virtually try on the clothes they have selected.

[1579] The present invention provides a coordination suggestion system incorporating an emotion engine to solve the problems users face when choosing clothes. This system allows users to register clothes using a smartphone or other device, virtually try on suggested coordinations, and then suggests optimal coordinations based on emotional feedback.

[1580] 1. User clothing registration

[1581] The user activates their smartphone or other device and selects the option to register their clothes. The smartphone activates the camera function and instructs the user to take a photo of the clothes. The user takes a photo of the clothes, and the smartphone sends the image to the server. The server analyzes the received image and extracts attributes such as the clothing category, color, pattern, and material. The extracted attribute information is stored in a database, and the results are sent back to the smartphone.

[1582] Example: A user takes a photo of a blue shirt and sends the image to a server using a fashion coordination app. The server analyzes the image, extracts the attributes "tops," "blue," and "shirt," and stores them in a database. The smartphone then displays the message "Blue shirt registered."

[1583] 2. Coordination suggestions

[1584] The user selects an outfit suggestion option on their smartphone. The smartphone prompts the user to enter information such as today's weather and schedule. The user enters the necessary information, and the smartphone sends it to the server. The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information. The generated outfit is sent to the smartphone, and the suggestions are displayed to the user.

[1585] Example: A user requests "Today's work outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the smartphone.

[1586] 3. Emotion recognition

[1587] The emotion engine analyzes the user's facial expressions, voice, and behavioral data to recognize their emotions. The smartphone captures the user's video and audio during the virtual try-on and sends them to the server. The server then analyzes the user's emotions using the emotion engine and modifies the outfit based on those emotions.

[1588] Example: If a user mutters "It's a bit awkward" during a virtual try-on session, the smartphone captures the user's voice and facial expression and sends them to the server. The emotion engine recognizes that the user is dissatisfied, and the server changes the outfit to a "casual shirt" and "jeans."

[1589] 4. Virtual try-on through AR glasses

[1590] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror. The smartphone captures an image of the user's body and sends it to the server. The server then virtually overlays the suggested clothing on the image of the user's body. The smartphone visually displays the results of the virtual try-on to the user through the AR glasses. The user checks the results of the virtual try-on and sends feedback to the smartphone.

[1591] Example: When a user wears AR glasses and looks into a mirror, the server displays "white shirt," "blue jeans," and "black jacket" over the user's image. The user again sends dissatisfied feedback.

[1592] 5. Proposal and purchase of suitable clothing

[1593] The server receives the user's feedback and fine-tunes the outfit as necessary. The smartphone receives the newly suggested outfit information and a link to purchase it from the online store from the server and displays it to the user. The user clicks the link to access the online store and purchase the outfit.

[1594] Example: If the user gives feedback that they "don't like the color," the server suggests a gray shirt instead of a white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[1595] This system uses devices such as smartphones and AR glasses to provide real-time coordination suggestions based on the user's emotions and feedback. The hardware used includes standard smartphones (iPhones, Android devices) and AR glasses (commonly known as head-mounted displays), while the software includes a server (cloud computing service), generative AI model (language model), emotion engine (emotion recognition tool), and database (data storage system). This system allows users to receive high-quality fashion coordination suggestions from the comfort of their own homes.

[1596] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1597] Step 1:

[1598] A user takes a photo of clothing with their smartphone and sends the image to a server via their device. The server receives the image and uses an image analysis algorithm to extract attributes such as clothing category, color, pattern, and material. The input is the image of the clothing, and the output is attribute information. This attribute information is stored in a database.

[1599] Step 2:

[1600] The user selects an outfit suggestion option on the smartphone app and enters the necessary information (such as weather and schedule). The device sends this to the server. The server uses a generative AI model based on the user's clothing list and the submitted information to generate the optimal outfit. The input is the user's clothing list and external information (weather, schedule), and the output is a suggested outfit. The suggested outfit is displayed to the user on the device.

[1601] Step 3:

[1602] The user selects the virtual try-on option. The device activates the camera, captures an image of the user's body, and sends it to the server. The server then uses a virtual try-on algorithm to virtually overlay the suggested outfit on the image of the user's body. The input is the image of the user's body and the suggested outfit, and the output is the virtual try-on result. This result is visually displayed to the user through the device.

[1603] Step 4:

[1604] The emotion engine analyzes the user's facial and voice data to recognize their emotions. The device captures the user's video and audio and sends them to the server. The server uses the emotion engine to analyze the user's emotions and modifies the outfit based on those emotions. The input is the user's facial and voice data, and the output is the modified outfit. The modified outfit is displayed to the user via the device.

[1605] Step 5:

[1606] The user inputs feedback and sends it to the server via the terminal. The server receives the feedback and re-proposes outfits as needed. The input is the user's feedback, and the output is the re-proposed outfits. The re-proposed outfits are displayed to the user via the terminal.

[1607] Step 6:

[1608] The server finally suggests suitable outfits and generates a purchase link to send to the device. The input is the final coordinated outfit and the user's willingness to purchase, and the output is the purchase link. When the user clicks the purchase link, they can access the online store and purchase the outfit right away.

[1609] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1610] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1611] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1612] [Fourth embodiment]

[1613] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1614] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1615] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1616] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1617] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1618] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1619] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1620] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1621] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1622] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1623] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1624] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1625] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1626] An embodiment of this invention is a coordination suggestion system that helps users solve various problems they may have when choosing clothes for everyday use. This system has the function of capturing images of clothes owned by the user, analyzing them to extract attributes, and suggesting optimal coordination based on the user's requests. In addition, the user can provide feedback through a virtual try-on session, and based on the results, more suitable clothes can be suggested and purchased.

[1627] Explain the program's processing in natural language.

[1628] 1. User clothing registration:

[1629] The user launches the terminal app and selects the option to register their clothes.

[1630] The device activates the camera function and prompts the user to take a photo of the clothes.

[1631] The user takes a photo of the clothes and the device sends the image to the server.

[1632] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[1633] The server stores the extracted attribute information in a database and returns the results to the terminal.

[1634] The terminal displays a notification to the user that registration is complete.

[1635] Examples:

[1636] A user takes a photo of a white shirt and sends it to the server via the device app. The server extracts the attributes "top," "white," and "shirt" and stores them in a database. The device then displays the message "White shirt registered."

[1637] 2. Coordination suggestions:

[1638] The user selects the outfit suggestion option on the terminal.

[1639] The device prompts the user to enter information such as today's weather and schedule.

[1640] The user enters the necessary information and the terminal sends it to the server.

[1641] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[1642] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[1643] Examples:

[1644] A user requests "Today's Work Outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the device.

[1645] 3. Virtual try-on through AR glasses:

[1646] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[1647] The device captures an image of the user's body and sends it to a server.

[1648] The server virtually overlays the suggested clothing onto an image of the user's body.

[1649] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[1650] The user checks the results of the virtual try-on and sends feedback to the terminal.

[1651] Examples:

[1652] When a user wears the AR glasses and looks into the mirror, the server displays a "white shirt," "blue jeans," or "black jacket" over the user's image. The user then sends feedback saying, "I don't like the color."

[1653] 4. Suitable clothing suggestions and purchases:

[1654] The server receives user feedback and fine-tunes the coordination as needed.

[1655] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[1656] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[1657] Examples:

[1658] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[1659] The detailed processing steps described above realize a system that effectively solves the user's worries about choosing clothes and suggests optimal outfits.

[1660] The processing flow will be explained below.

[1661] Step 1:

[1662] The user starts the terminal application and selects an option to register clothes.

[1663] Step 2:

[1664] The device activates the camera function and prompts the user to take a photo of the clothes.

[1665] Step 3:

[1666] The user takes a photo of the clothes and the device sends the image to the server.

[1667] Step 4:

[1668] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[1669] Step 5:

[1670] The server stores the extracted attribute information in a database.

[1671] Step 6:

[1672] The terminal displays a notification to the user that registration is complete.

[1673] Step 7:

[1674] The user selects the outfit suggestion option on the terminal.

[1675] Step 8:

[1676] The device prompts the user to enter information such as today's weather and schedule.

[1677] Step 9:

[1678] The user enters the necessary information, and the terminal sends it to the server.

[1679] Step 10:

[1680] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[1681] Step 11:

[1682] The server transmits the generated coordinates to the terminal.

[1683] Step 12:

[1684] The terminal displays the coordinated outfit suggestions to the user.

[1685] Step 13:

[1686] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[1687] Step 14:

[1688] The device captures an image of the user's body and sends it to a server.

[1689] Step 15:

[1690] The server virtually overlays the suggested clothing onto an image of the user's body.

[1691] Step 16:

[1692] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[1693] Step 17:

[1694] The user checks the results of the virtual try-on and sends feedback to the terminal.

[1695] Step 18:

[1696] The server receives user feedback and fine-tunes the coordination as needed.

[1697] Step 19:

[1698] The server generates information about the newly suggested clothing and a link to purchase it from the online store.

[1699] Step 20:

[1700] The device displays the generated purchase link to the user.

[1701] Step 21:

[1702] The user clicks on the purchase link and is taken to the online store to purchase the clothing.

[1703] Example 1

[1704] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1705] Conventional clothing selection and coordination suggestion systems have had problems such as the complicated process of registering and categorizing the clothes a user owns, and the quality of the suggested coordinations does not adequately reflect the user's requests or external information (weather, schedule, etc.). Furthermore, they lacked the functionality to actually try on the suggested coordinations, limiting the benefits of online purchases.

[1706] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1707] In this invention, the server includes means for acquiring images of the user's clothing, means for analyzing the acquired images to extract clothing features, means for storing the extracted clothing features in a database, means for generating outfits based on the user's clothing list and external information in response to the user's request, means for visually displaying the generated outfits to the user, means for the user to send feedback, and means for suggesting suitable clothing based on the user's feedback and generating a link to purchase the clothing. This allows users to easily register and categorize their clothing and receive high-quality outfit suggestions that take external information into account. Furthermore, the integration of the virtual try-on function and online purchase further improves convenience.

[1708] "User" refers to an individual who uses the system to register clothes and receive coordination suggestions.

[1709] "Clothing" refers to any clothing owned by a user and registered in the system.

[1710] "Means for acquiring images" refers to a function that allows a user to take a photo of clothing using a terminal and send the image data to a server.

[1711] "Means of analyzing images and extracting clothing characteristics" refers to the function of using analysis software on a server to identify characteristic information such as category, color, pattern, and material from acquired images of clothing.

[1712] "Means for storing in a database" refers to the function of recording the analyzed and extracted clothing characteristic information in storage within the system and managing it efficiently.

[1713] "User request" refers to the content of a request for a coordination proposal made by a user operating the system.

[1714] "External information" refers to additional information such as weather and the user's schedule that the system should take into consideration when proposing outfits.

[1715] "Means for generating outfits" refers to the function of using a generative AI model to create optimal outfits based on the user's clothing list and external information.

[1716] The "means for visually displaying" refers to a function for displaying the generated coordinate on the user's terminal screen and providing it to the user visually.

[1717] "Means for sending feedback" refers to the function that allows users to input their thoughts and requests for improvement regarding the proposed coordination and send them back to the system.

[1718] "Means for suggesting suitable clothing based on feedback and generating a link to purchase it" refers to a function that modifies outfits based on user feedback and provides optimal clothing and a link to an online store where it can be purchased.

[1719] "Augmented reality glasses" refers to a device that users can wear to visually view information from the system and virtual clothing.

[1720] "Means for displaying virtual clothing" refers to a function for displaying virtual clothing overlaid on the user's image.

[1721] "Online store" refers to an e-commerce site where clothing can be purchased via the Internet.

[1722] This invention is a coordination suggestion system that helps users solve various problems they may have when choosing clothes. The system captures images of the user's clothing, analyzes them to extract attributes, and suggests optimal coordinations based on the user's requests. The system also allows the user to provide feedback through a virtual try-on session, which allows further suggestions to be made based on the results, and allows the user to purchase clothing online.

[1723] The system configuration is as follows:

[1724] 1. User clothing registration:

[1725] The user launches the device app and selects the "Register Clothing" option. The device activates its camera function and prompts the user to take a photo of the clothing. The user takes a photo of the clothing, and the device sends the image to the server. The server uses image analysis software (e.g., OpenCV) to extract attributes such as clothing category, color, pattern, and material from the image and stores these attributes in a database. For example, if the user takes a photo of a white shirt and sends it, the server extracts the attributes "top," "white," and "shirt" and stores them in the database. The device then notifies the user that "your white shirt has been registered."

[1726] 2. Coordination suggestions:

[1727] The user selects the "Outfit Suggestion" option on the device app. The device displays a form prompting the user to "Enter today's weather and schedule." When the user enters and submits information such as "sunny" and "work schedule," the device sends this to the server. The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information. As a specific example, when the user requests "Today's work outfit," the server suggests "white shirt," "blue denim," and "black jacket" and sends them to the device. The device then displays the suggested outfit to the user. An example of a prompt sentence is as follows:

[1728] "The user requests registration of a white shirt and suggestions for outfits. Please suggest today's outfits that include a white shirt based on the current weather and schedule."

[1729] 3. Virtual try-on through AR glasses:

[1730] The user selects the virtual try-on mode and puts on the AR glasses. The device uses the AR glasses' camera to capture an image of the user's body and sends it to the server. The server then virtually overlays suggested clothing items based on the received image. For example, the server may overlay a "white shirt," "blue denim," or "black jacket" onto the user's image. The device visually displays the results of this virtual try-on to the user in real time through the AR glasses. The user reviews the image and provides feedback through the app. For example, they may provide feedback such as, "I don't like the color."

[1731] 4. Suitable clothing suggestions and purchases:

[1732] The server receives the user's feedback and fine-tunes the outfit as needed. It then generates new suggested clothing items and a link to the online store where they can be purchased, and sends them to the device. For example, if the user sends feedback that they "don't like the color," the server might suggest a gray shirt instead of a white one. The device then displays the new suggested items along with a link to purchase the gray shirt. The user clicks the link and purchases the gray shirt from the online store.

[1733] This allows users to easily register and categorize their clothing and receive high-quality outfit suggestions that take external information into account. Furthermore, the integration of the virtual try-on feature and online purchase further enhances convenience.

[1734] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1735] Step 1:

[1736] The user launches the device app and selects the "Clothing Registration" option. A screen prompting the user to perform an action is displayed on the device.

[1737] (Input) User operations (app launch and option selection)

[1738] (Output) App instruction screen

[1739] Step 2:

[1740] The device will activate its camera function and prompt the user to take a photo of the clothing. The message "Take a photo" will be displayed on the screen.

[1741] (Input) User operation (option selection)

[1742] (Output) Camera startup and instruction message

[1743] Step 3:

[1744] The user takes a photo of the garment and taps the "Send" button. An image file is generated.

[1745] (Input) User operation (photographing)

[1746] (Output) Image file

[1747] Step 4:

[1748] The device sends the captured image to the server, which prepares to analyze the received image.

[1749] (Input) Image file

[1750] (Output) Image data sent to server

[1751] Step 5:

[1752] The server uses image analysis software (e.g., OpenCV) to analyze the received images and extract attributes such as clothing category, color, pattern, and material.

[1753] (Input) Image data

[1754] (Output) Extracted attribute data (category, color, pattern, material)

[1755] Step 6:

[1756] The server saves the extracted attribute information in the database and returns a status to the terminal if the save is successful.

[1757] (Input) Extracted attribute data

[1758] (Output) Database save status

[1759] Step 7:

[1760] The device notifies the user that registration is complete. The message "White shirt successfully registered" appears on the screen.

[1761] (Input) Database save status

[1762] (Output) Registration completion notification

[1763] Step 8:

[1764] The user selects the "Outfit suggestions" option in the device app. A screen prompting the user to perform an action is displayed on the device.

[1765] (Input) User operation (option selection)

[1766] (Output) Coordination proposal input form

[1767] Step 9:

[1768] The device displays a form prompting the user to "Enter today's weather and schedule." The user enters the information and taps the "Submit" button.

[1769] (Input) User operation (option selection)

[1770] (Output) Input form

[1771] Step 10:

[1772] The user inputs "Sunny" and "Work schedule" and submits the information. The device then sends the information to the server.

[1773] (Input) Input form

[1774] (Output) Weather and schedule data

[1775] Step 11:

[1776] The server takes the user's clothing list and submitted information and uses a generative AI model to generate the optimal outfit.

[1777] (Input) Clothing list and weather / schedule data

[1778] (Output) Generated coordinates

[1779] Step 12:

[1780] The server sends the generated outfit to the device, which then displays the suggestions to the user. Suggested outfits include a "white shirt," "blue jeans," and a "black jacket."

[1781] (Input) Generated coordinates

[1782] (Output) Display of suggested outfits

[1783] Step 13:

[1784] The user selects the virtual try-on mode and puts on the AR glasses. The device then activates the camera in the AR glasses.

[1785] (Input) User operation (select virtual try-on mode)

[1786] (Output) Activate the camera on the AR glasses

[1787] Step 14:

[1788] The device captures an image of the user's body and sends it to the server, which receives the image data.

[1789] (Input) Image of the user's body

[1790] (Output) Video data sent to server

[1791] Step 15:

[1792] The server virtually overlays the suggested clothing based on the video received.

[1793] (Input) Video data and proposed coordinates

[1794] (Output) Virtual try-on results

[1795] Step 16:

[1796] The device visually displays the results of the virtual try-on to the user in real time through the AR glasses, making it appear as if the user is actually wearing the garment.

[1797] (Input) Virtual try-on results

[1798] (Output) Visual display

[1799] Step 17:

[1800] The user checks the video and sends feedback through the app, saying, "I don't like the colors." The device then sends the feedback to the server.

[1801] (Input) Virtual fitting results

[1802] (Output) Feedback Data

[1803] Step 18:

[1804] The server generates new outfits based on user feedback, suggesting, for example, a gray shirt instead of a white one.

[1805] (Input) Feedback data

[1806] (Output) New coordinates

[1807] Step 19:

[1808] The server sends the new outfit along with a link to purchase from the online store to the device, which then displays the new suggestions to the user.

[1809] (Input) New coordinates

[1810] (Output) New offer and purchase link

[1811] Step 20:

[1812] A user clicks on a purchase link to go to an online store and purchase clothing, for example, a gray shirt.

[1813] (Input) Purchase link

[1814] (Print) Online purchase complete

[1815] (Application example 1)

[1816] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1817] Conventional clothing selection systems make it difficult for users to easily visually check how their existing accessories coordinate with the new accessories they are purchasing. Additionally, the time and effort involved in trying on and purchasing items in a retail store is a significant burden for users. While systems exist that suggest suitable accessories based on user feedback, there are few systems that can reflect this feedback in real time and improve the user experience through virtual try-on.

[1818] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1819] In this invention, the server includes means for capturing images of a user's accessories, means for analyzing the captured images to extract characteristics of the accessories, means for storing the extracted characteristics in a database, means for generating a coordinated outfit based on the user's accessory list and external information in response to a user request, means for displaying the generated coordinated outfit to the user's visual display, means for the user to send feedback, means for suggesting suitable accessories based on the user's feedback and generating a purchase link for the accessories, means for the user to capture images of the accessories using a terminal in a retail store and purchase the accessories on the spot, and means for providing a virtual try-on experience in conjunction with a mirror in the store for the user to virtually try on the accessories. This allows users to easily coordinate and virtually try on accessories in a retail store without pressure, and to purchase the optimal accessories on the spot. Furthermore, by using a generative AI model to generate an optimal outfit based on the user's request and characteristics using prompt sentences and visually displaying the generated results to the user, the user can more easily and efficiently select accessories.

[1820] The "means for capturing images of user's accessories" refers to a device that has the function of recording the user's clothes and accessories as photographs using the camera function of the terminal.

[1821] The "means for analyzing the captured image and extracting the characteristics of the attached item" refers to algorithms and software that automatically identify the characteristics of the attached item, such as its color, material, pattern, and shape, from the captured image and treat them as data.

[1822] The "means for storing extracted characteristics of the attached item in a database" is a system that has the function of storing characteristic data of the attached item obtained by image analysis in a database so that it can be reused later.

[1823] "Means for generating an outfit based on the user's accessory list and external information in response to the user's request" refers to software that generates the optimal outfit based on the desired conditions and event information entered by the user, taking into account the accessories the user has and external factors such as weather information and fashion information.

[1824] The "means for visually displaying the generated coordination to the user" is a system having the function of displaying information about the generated coordination on the user's terminal screen, etc., so that the user can visually confirm it.

[1825] "Means for users to send feedback" refers to a mechanism or interface that allows users to input their ratings and comments on the proposed outfits and send that information to the system.

[1826] The "means for proposing suitable accessories based on user feedback and generating a link to purchase them" refers to software and a system that analyzes feedback information from users, and based on that, proposes new accessories and generates a link to purchase the accessories, which are then provided to the user.

[1827] "A means by which a user can capture an image of an accessory using a terminal in a retail store and purchase the accessory on the spot" is a system that allows a user to take an image of an accessory using a camera on a terminal in a store, obtain necessary information based on that data, and complete the purchase on the spot.

[1828] "Means for providing a virtual fitting experience in cooperation with mirrors in a store so that users can virtually try on clothes" refers to a system that allows users to use a specific device in a store to virtually superimpose items onto their own image to provide a fitting experience without actually trying on the items.

[1829] "Means for using a generative AI model to generate optimal outfits using prompt sentences based on the user's requests and characteristics, and visually displaying the generated results to the user" refers to a system that uses an artificial intelligence algorithm to automatically create outfits that take into account the user's individual information and requests, generates them as prompt sentences, and visually presents the results to the user.

[1830] An embodiment of this invention is a coordination suggestion system that solves various problems users may have when choosing clothes for everyday use. This system has a function that allows users to capture images of clothes using their smartphones in a retail store, analyzes the images to extract attributes, and suggests optimal coordination based on the user's requests. In addition, the user can provide feedback through a virtual try-on, and based on the results, more suitable clothes can be suggested and purchased.

[1831] The system uses the following hardware and software:

[1832] Smartphone: Uses the camera function to capture images of the user's clothes and accessories.

[1833] Server: Performs image analysis and coordinate generation.

[1834] Database: Stores the user's clothing attribute information and coordination information.

[1835] Generative AI model: Automatically generates optimal outfits based on user requests and characteristics.

[1836] Specifically, the following process is performed:

[1837] 1. Image capture and attribute extraction: When a user takes a picture of clothing in a retail store using their smartphone, the smartphone sends the image to a server. The server analyzes the image using an image processing library such as OpenCV and extracts the attributes of the clothing (color, material, pattern, shape, etc.). This attribute information is stored in a database.

[1838] 2. Coordination Generation: When a user requests a coordination suggestion, the server uses a generative AI model to generate the optimal coordination based on the user's clothing list and external information (weather forecast, event information, etc.). This coordination is generated using a prompt sentence and displayed on the smartphone.

[1839] 3. Virtual Try-On: The user selects the virtual try-on mode on their smartphone and tries on the clothes virtually using the in-store mirror. The server then virtually overlays the suggested clothes onto the captured image of the user's body and displays the results on the smartphone screen or in the in-store mirror.

[1840] 4. Feedback and Purchase: When the user submits feedback based on the results of the virtual try-on, the server analyzes the feedback and suggests more suitable items to wear. The suggestions include a purchase link, allowing the user to purchase the items from the online store immediately.

[1841] Examples:

[1842] A user requests a stylish outfit and uploads a photo of light blue jeans and a white shirt. The system then suggests outfits based on blue and white, performs a virtual try-on, and provides visual feedback to the user.

[1843] Example prompts to input to the generative AI model:

[1844] User input: Light blue jeans and a white shirt. Today's weather is sunny and I plan to go to work. Coordination suggestions and virtual try-on are performed.

[1845] AI model: Based on these criteria, it suggests the best outfit and allows the user to virtually try it on.

[1846] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1847] Step 1:

[1848] A user uses a terminal to capture images of clothes in a retail store. The user takes a photo of the clothes using the camera function of their smartphone and uploads the image to the system. The input is the image data of the clothes, and the output is the image data sent to the server.

[1849] Step 2:

[1850] The server analyzes the received image data and extracts the attributes of the worn item. The server uses an image processing library such as OpenCV to analyze the image and identify the attributes of the worn item, such as color, material, pattern, and shape. The input is image data, and the output is the extracted attribute data.

[1851] Step 3:

[1852] The extracted attributes of the attached items are saved in a database. The server stores the attribute information obtained by the analysis in a database so that it can be used later upon request. The input is the attribute data of the attached items, and the output is the attribute information saved in the database.

[1853] Step 4:

[1854] The user requests outfit suggestions using a device. The user selects an outfit suggestion option from the smartphone application and enters the necessary external information (weather, schedule, etc.). The input is the outfit suggestion request and the external information, and the output is the request data sent to the server.

[1855] Step 5:

[1856] The server generates an outfit based on the user's request. The server uses a generative AI model to generate the optimal outfit based on the attribute data of the items stored in the database and external information. Prompt statements are used in this process. The inputs are the attribute data of the items, external information, and the prompt statement, and the output is the generated outfit data.

[1857] Step 6:

[1858] The generated coordinated outfit is displayed visually to the user. The server sends the generated coordinated outfit information to the terminal, which then displays the information on the user's smartphone screen. The input is the generated coordinated outfit data, and the output is the displayed coordinated outfit.

[1859] Step 7:

[1860] The user performs a virtual try-on using a device. The user selects the virtual try-on mode on their smartphone and uses the in-store mirror to superimpose their image onto the suggested outfit. The input is an image of the user's body and outfit data, and the output is a video of the results of the virtual try-on.

[1861] Step 8:

[1862] The user submits feedback based on the results of the virtual try-on. The user enters ratings and comments on the proposed outfits through a smartphone application and sends them to the server. The input is feedback data, and the output is the feedback sent to the server.

[1863] Step 9:

[1864] The server proposes suitable accessories based on the user's feedback and generates a purchase link for them. The server analyzes the feedback data, proposes new suitable accessories, generates a purchase link for them, and sends it to the user. The input is the feedback data, and the output is the new proposal and a purchase link.

[1865] Step 10:

[1866] The user clicks on the purchase link to access the online store and purchase the accessory. The user uses their smartphone to access the online store through the suggested link and purchase the accessory. The input is the purchase link, and the output is a purchase completion notification.

[1867] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1868] This invention incorporates an emotion engine into a coordination suggestion system to solve the problems users face when choosing clothes. The system aims to improve user satisfaction by recognizing the emotions felt when choosing clothes and suggesting optimal coordination based on those emotions. This system includes the following specific means:

[1869] Explain the program's processing in natural language.

[1870] 1. User clothing registration:

[1871] The user launches the terminal app and selects the option to register their clothes.

[1872] The device activates the camera function and prompts the user to take a photo of the clothes.

[1873] The user takes a photo of the clothes and the device sends the image to the server.

[1874] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[1875] The server stores the extracted attribute information in a database and returns the results to the terminal.

[1876] The terminal displays a notification to the user that registration is complete.

[1877] Examples:

[1878] A user takes a photo of a white shirt and sends it to the server via the device app. The server extracts the attributes "top," "white," and "shirt" and stores them in a database. The device then displays the message "White shirt registered."

[1879] 2. Coordination suggestions:

[1880] The user selects the outfit suggestion option on the terminal.

[1881] The device prompts the user to enter information such as today's weather and schedule.

[1882] The user enters the necessary information and the terminal sends it to the server.

[1883] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[1884] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[1885] Examples:

[1886] A user requests "Today's Work Outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the device.

[1887] 3. Emotion recognition:

[1888] The emotion engine analyzes the user's facial expression, voice, and behavioral data to recognize the user's emotions.

[1889] The device captures the user's video and audio during the virtual try-on and sends them to the server.

[1890] The server analyzes the user's emotions using an emotion engine and modifies the coordination based on the emotions.

[1891] Examples:

[1892] If the user mutters "It's a bit awkward" during the virtual fitting, the device captures the voice and facial expression and sends them to the server. The emotion engine recognizes that the user is dissatisfied, and the server changes the outfit to a "casual shirt" and "jeans."

[1893] 4. Virtual try-on through AR glasses:

[1894] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[1895] The device captures an image of the user's body and sends it to a server.

[1896] The server virtually overlays the suggested clothing onto an image of the user's body.

[1897] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[1898] The user checks the results of the virtual try-on and sends feedback to the terminal.

[1899] Examples:

[1900] When the user wears the AR glasses and looks into the mirror, the server displays a "white shirt," "blue jeans," and "black jacket" over the user's image. The user again submits dissatisfied feedback.

[1901] 5. Suitable clothing suggestions and purchases:

[1902] The server receives user feedback and fine-tunes the coordination as needed.

[1903] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[1904] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[1905] Examples:

[1906] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[1907] In this way, we have realized a coordination suggestion system that combines an emotion engine to effectively solve the user's clothing selection problem.

[1908] The processing flow will be explained below.

[1909] Step 1:

[1910] The user starts the terminal application and selects an option to register clothes.

[1911] Step 2:

[1912] The device activates the camera function and prompts the user to take a photo of the clothes.

[1913] Step 3:

[1914] The user takes a photo of the clothes and the device sends the image to the server.

[1915] Step 4:

[1916] The server analyzes the received images and extracts attributes such as clothing category, color, pattern, and material.

[1917] Step 5:

[1918] The server stores the extracted attribute information in a database.

[1919] Step 6:

[1920] The terminal displays a notification to the user that registration is complete.

[1921] Step 7:

[1922] The user selects the outfit suggestion option on the terminal.

[1923] Step 8:

[1924] The device prompts the user to enter information such as today's weather and schedule.

[1925] Step 9:

[1926] The user enters the necessary information, and the terminal sends it to the server.

[1927] Step 10:

[1928] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[1929] Step 11:

[1930] The server transmits the generated coordinates to the terminal.

[1931] Step 12:

[1932] The terminal displays the coordinated outfit suggestions to the user.

[1933] Step 13:

[1934] The emotion engine analyzes the user's facial expression, voice, and behavioral data to recognize the user's emotions.

[1935] Step 14:

[1936] The user confirms the option to use the emotion engine, and the device captures the user's facial expressions and voice and sends them to the server.

[1937] Step 15:

[1938] The server analyzes the user's emotions using an emotion engine and modifies the coordination based on the emotions.

[1939] Step 16:

[1940] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror.

[1941] Step 17:

[1942] The device captures an image of the user's body and sends it to a server.

[1943] Step 18:

[1944] The server virtually overlays the suggested clothing onto an image of the user's body.

[1945] Step 19:

[1946] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[1947] Step 20:

[1948] The user checks the results of the virtual try-on and sends feedback to the terminal.

[1949] Step 21:

[1950] The server receives user feedback and fine-tunes the coordination as needed.

[1951] Step 22:

[1952] The server generates information about the newly suggested clothing and a link to purchase it from the online store.

[1953] Step 23:

[1954] The device displays the generated purchase link to the user.

[1955] Step 24:

[1956] The user clicks on the purchase link and is taken to the online store to purchase the clothing.

[1957] Example 2

[1958] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1959] Conventional clothing selection and coordination suggestion systems often provide one-sided suggestions without considering the user's feelings. Furthermore, users must visit a physical store to check the actual fit and appearance of the suggested clothing, which is time-consuming and laborious. Furthermore, the purchasing process at online stores is cumbersome, which can discourage users from making purchases. A new system is needed to solve these issues and increase user satisfaction.

[1960] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1961] In this invention, the server includes a means for capturing images of the user's belongings, a means for analyzing the captured images to extract the characteristics of the belongings, and a means for storing the extracted characteristics in a database. This allows the server to generate suggestions using a generative AI model based on the user's belongings list and external information, and visually display the results. Furthermore, by suggesting suitable belongings based on user feedback and generating links to the suggestions in collaboration with an online store, the server can easily purchase the items on the spot. Furthermore, a virtual try-on function using augmented reality glasses can be provided, eliminating the stress of actually trying on the belongings.

[1962] A "user" is an individual or group that uses the system and performs operations such as registering belongings, suggesting outfits, providing feedback, and making purchases.

[1963] "Belongings" are clothing, accessories, and other related items owned by the user, and are registered in the system as image data.

[1964] The "capturing means" is a function for acquiring image data using a photographing device such as a camera mounted on the user's terminal.

[1965] "Means for analyzing images" refers to software and algorithms for extracting characteristics of belongings from captured image data.

[1966] "Means for extracting features" refers to the process of analyzing images to identify attribute information such as the category, color, pattern, and material of the belongings.

[1967] "Means for storing in a database" refers to a system or software for storing and managing the extracted characteristic information of belongings as digital data.

[1968] "Means for generating proposals based on requests" refers to a generative AI model that generates optimal coordination and proposals based on user input information and external data.

[1969] The "visual display means" is a function that displays the generated coordination and proposal content on the display of the user's terminal.

[1970] "Means for sending feedback" refers to an interface that allows a user to input evaluations and opinions on a proposal and send them to the system.

[1971] The "means for suggesting suitable items and generating links to purchase them" refers to a process for making new suggestions based on user feedback and generating links to promote purchases at an online store.

[1972] "Augmented reality glasses" are wearable devices that overlay virtual objects onto the real world.

[1973] "Virtual Try-On" is a feature that allows users to virtually try on items using augmented reality glasses.

[1974] "Online Store" refers to a website or platform that sells and purchases products over the Internet.

[1975] "Purchase Link" means a hyperlink that allows you to purchase the suggested item directly from the online store.

[1976] MODE FOR CARRYING OUT THE INVENTION

[1977] This invention incorporates an emotion engine into a coordination suggestion system that helps users solve their troubles when choosing clothes. The aim is to improve user satisfaction by recognizing the emotions of users when choosing clothes and suggesting optimal coordination based on those emotions.

[1978] A specific embodiment of the system is as follows.

[1979] 1. User clothing registration

[1980] When a user launches the device app, the app displays an option to register clothes. If the user selects this option, the device's camera function is activated and the user is prompted to take a photo of the clothes. When the user takes a photo and presses the send button, the device sends the image to the server.

[1981] The server uses image analysis software such as Label Studio to extract features from the received image, such as the clothing category (for example, "top" or "pants"), color, pattern, and material. The extracted feature information is stored in a database. The processing results are sent back to the terminal, which displays a message such as "A white shirt has been registered."

[1982] Examples:

[1983] A user takes a photo of a white shirt and sends the image to the server via a smartphone app. The server extracts the attributes "top," "white," and "shirt" and stores this information in a database. The device displays the message "White shirt registered."

[1984] 2. Coordination suggestions

[1985] The user selects an outfit suggestion option on the device. The device prompts the user to enter information such as today's weather and schedule. When the user enters and submits this information, the data is sent from the device to the server.

[1986] The server uses a generative AI model (e.g., ChatGPT) to generate the optimal outfit based on the user's clothing list and the submitted information. The generative AI model uses the following prompt:

[1987] "Please suggest an outfit for work today. The weather is sunny and I'm heading to work."

[1988] The generated coordinates are sent from the server to the terminal, which then visually displays them to the user.

[1989] Examples:

[1990] A user requests "Today's Work Outfit," inputs the weather forecast and the date they plan to go to work, and the server suggests "white shirt," "blue jeans," and "black jacket." The following is displayed on the device.

[1991] 3. Emotion recognition

[1992] The system incorporates an emotion engine that analyzes the user's facial expressions, voice, and behavioral data to recognize their emotions. The device captures video and audio of the user while trying on clothes and sends them to a server. The server then uses Microsoft Azure's emotion recognition API to analyze the user's emotions and adjusts the outfit based on the results.

[1993] Examples:

[1994] If the user mutters "It's a bit awkward" during the virtual fitting, the device captures the voice and facial expression and sends them to the server. The emotion engine recognizes the user's dissatisfaction, and the server changes the outfit to a "casual shirt" and "jeans."

[1995] 4. Virtual try-on with AR glasses

[1996] The user selects the virtual try-on mode, puts on AR glasses (e.g., Magic Leap), and looks in the mirror. The device captures a full-body image of the user's body and sends it to the server. The server uses Unity to generate an image in which virtual clothes are superimposed on the user's body in real time. The device visually displays the results of the virtual try-on to the user through the AR glasses. The user checks this and sends feedback to the device.

[1997] Examples:

[1998] When a user wears the AR glasses and looks in the mirror, the server superimposes a "white shirt," "blue jeans," or "black jacket" onto the user's image. The user then submits their dissatisfaction feedback.

[1999] 5. Buying clothes

[2000] The server receives the user's feedback, fine-tunes the outfit as needed, and sends the newly suggested outfit information and a link to purchase it from an online store to the user's device. When the user clicks the link, they are taken to an online store such as Amazon, where they can purchase the outfit on the spot.

[2001] Examples:

[2002] If the user provides feedback that they "don't like the color," the server suggests a gray shirt instead of the white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[2003] In this way, we can realize a coordination suggestion system that combines an emotion engine and a generative AI model to effectively support users in choosing clothes.

[2004] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2005] Step 1:

[2006] The user starts the terminal application and selects the clothes registration option.

[2007] Input: User action (selecting the "Register Clothes" option).

[2008] Output: The device's camera function is activated.

[2009] Specific operation: The user taps or clicks the "Register Clothes" button on the app screen. The device's camera automatically starts up and an interface for taking a photo of the clothes is displayed.

[2010] Step 2:

[2011] The user takes a photo of the clothes and the device sends the image to the server.

[2012] Input: A user-taken photo of an outfit.

[2013] Output: The image data is sent to the server.

[2014] Specific operation: The user takes a photo of the clothes using the camera and presses the "Send" button to send the image to the server.

[2015] Step 3:

[2016] The server analyzes the received images and uses Label Studio to extract features of the belongings.

[2017] Input: Image data of clothes.

[2018] Output: Clothing attribute information (category, color, pattern, material, etc.).

[2019] Specific operation: The server uses Label Studio to analyze the image data and extract attributes such as "tops," "white," and "shirt."

[2020] Step 4:

[2021] The server stores the extracted feature information in a database and returns the results to the terminal.

[2022] Input: Extracted clothing attribute information.

[2023] Output: Attribute information stored in the database and a notification of registration completion.

[2024] Specific operation: The extracted attribute information is stored in the database, and the result is sent back to the device. The device displays "White shirt registered."

[2025] Step 5:

[2026] The user selects the outfit suggestion option on the terminal.

[2027] Input: User action (selection of the "Outfit suggestions" option).

[2028] Output: An input form for weather, schedule, etc. is displayed.

[2029] Specific operation: The user taps or clicks the "Outfit suggestions" button, and an input form appears on the device.

[2030] Step 6:

[2031] The user inputs information such as today's weather and schedule, and the terminal sends the information to the server.

[2032] Input: Weather and schedule information entered by the user.

[2033] Output: The transmitted information arrives at the server.

[2034] Specific operation: The user enters the weather and schedule into the form and presses the "Submit" button.

[2035] Step 7:

[2036] The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information.

[2037] Input: User's clothing list and weather and schedule information.

[2038] Output: The generated coordinate plan.

[2039] Specific operation: Based on the user's belongings list and the submitted information, the server sends the following prompt to the generative AI model: "Please suggest an outfit for work today. The weather is sunny and I plan to go to work." The generative AI model then generates the optimal outfit.

[2040] Step 8:

[2041] The server transmits the generated coordinates to the terminal, which displays the proposed content to the user.

[2042] Input: Generated coordinate plan.

[2043] Output: Coordination suggestions displayed on the device.

[2044] Specific operation: The server sends the generated coordination plan to the terminal, and the terminal notifies the user of it.

[2045] Step 9:

[2046] The device captures the user's facial, voice, and behavioral data and transmits it to a server.

[2047] Input: User facial, voice, and behavioral data.

[2048] Output: The captured data is sent to the server.

[2049] Specific operation: The device camera and microphone record the user's facial expressions and voice, and send the data to the server.

[2050] Step 10:

[2051] The server uses an emotion engine to analyze the user's emotions and corrects the coordinates based on the emotions.

[2052] Input: User facial, voice, and behavioral data.

[2053] Output: Revised coordinate plan.

[2054] How it works: The server uses Microsoft Azure's emotion recognition API to analyze the user's emotions and generate new coordination suggestions based on the results.

[2055] Step 11:

[2056] The user selects the virtual try-on mode, puts on the AR glasses, and looks in the mirror.

[2057] Input: User operation (selecting virtual try-on mode), wearing AR glasses.

[2058] Output: Virtual try-on screen.

[2059] Specific operation: The user selects "virtual try-on mode" and puts on the AR glasses.

[2060] Step 12:

[2061] The device captures an image of the user's body and sends it to a server.

[2062] Input: Video data of the user's body.

[2063] Output: Transmitted video data.

[2064] Specific operation: The device camera captures the user's full-body image and sends the image data to the server.

[2065] Step 13:

[2066] The server uses Unity to virtually overlay the suggested clothing onto an image of the user's body.

[2067] Input: User's body image data, suggested clothing data.

[2068] Output: Virtually superimposed video data.

[2069] How it works: The server uses Unity to virtually overlay the suggested clothing onto an image of the user's body.

[2070] Step 14:

[2071] The device visually displays the results of the virtual try-on to the user through the AR glasses.

[2072] Input: Virtual and superimposed video data.

[2073] Output: Virtual try-on results displayed on the AR glasses.

[2074] Specific operation: The AR glasses display an image of the clothes being virtually tried on to the user.

[2075] Step 15:

[2076] The server receives user feedback and fine-tunes the coordination as needed.

[2077] Input: User feedback data.

[2078] Output: A fine-tuned outfit plan.

[2079] Specific operation: The server analyzes the feedback from the user and generates new suggestions.

[2080] Step 16:

[2081] The device receives information about the newly suggested clothing and a link to purchase it from the online store from the server and displays it to the user.

[2082] Input: Proposal information and purchase link from the server.

[2083] Output: Proposal information and purchase link displayed on the device.

[2084] Specific behavior: A link such as "Buy the gray shirt" will appear on the device screen.

[2085] Step 17:

[2086] The user clicks on the purchase link and is taken to the online store to purchase the clothes.

[2087] Input: User action (click on purchase link).

[2088] Output: Proceed to the online store purchase page and complete the purchase.

[2089] Specific behavior: The user clicks on the link, goes to the online store's purchase page, and completes the purchase process.

[2090] (Application example 2)

[2091] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2092] Conventional fashion coordination systems make clothing recommendations without considering the user's emotions, making it difficult to fully improve user satisfaction. They also lack advanced features such as a virtual try-on environment and real-time revision of recommendations based on user feedback. As a result, users may be dissatisfied with the suggested outfits and lose their motivation to purchase.

[2093] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2094] In this invention, the server includes means for capturing images of a user's clothes, means for analyzing the captured images to extract clothing attributes, means for saving the extracted clothing attributes in a database, means for generating outfits based on the user's clothes list and external information in response to a user request, means for visually displaying the generated outfit to the user, means for recognizing emotions by analyzing the user's facial expressions and voice data using an emotion engine, means for modifying the outfit based on the recognized emotions, means for the user to send feedback, means for suggesting suitable clothes based on the user's feedback and generating a purchase link for the clothes, and means for providing a virtual try-on mode. This makes it possible to flexibly adjust outfit suggestions based on the user's emotions and feedback.

[2095] "Means for capturing images of the user's clothes" refers to a device or software that takes a photo of the clothes owned by the user and acquires the image data.

[2096] "Means for analyzing captured images and extracting clothing attributes" refers to technology that analyzes acquired image data and automatically identifies and extracts characteristics such as clothing category, color, pattern, and material.

[2097] "Means for storing extracted clothing attributes in a database" refers to a data storage or database system for permanently storing clothing characteristic information obtained through analysis.

[2098] "Means for generating outfits based on the user's clothing list and external information in response to the user's request" refers to an algorithm or system that automatically creates optimal fashion outfits in response to the user's requests and input information, taking into account pre-registered clothing and external data such as weather and schedules.

[2099] "Means for visually displaying the generated outfits to the user" refers to a digital display or device that displays the suggested outfits in a format that is easy for the user to view.

[2100] "Means of recognizing emotions by analyzing a user's facial expressions and voice data using an emotion engine" refers to an AI model or algorithm that analyzes a user's facial expressions and voice as input data and recognizes their emotions.

[2101] "Means for modifying outfits based on recognized emotions" refers to a system or method for improving or changing suggested outfits based on emotional data determined by the emotion engine.

[2102] "Means for users to submit feedback" refers to an interface or function that allows users to input and submit their opinions and thoughts about the provided coordination or the operation of the system.

[2103] "Means for suggesting suitable clothes based on user feedback and generating a link to purchase them" refers to an algorithm or system that re-suggests optimal clothes based on user feedback and generates a link for direct purchase.

[2104] "Means for providing a virtual try-on mode" refers to a system or software that provides a function that allows a user to virtually try on the clothes they have selected.

[2105] The present invention provides a coordination suggestion system incorporating an emotion engine to solve the problems users face when choosing clothes. This system allows users to register clothes using a smartphone or other device, virtually try on suggested coordinations, and then suggests optimal coordinations based on emotional feedback.

[2106] 1. User clothing registration

[2107] The user activates their smartphone or other device and selects the option to register their clothes. The smartphone activates the camera function and instructs the user to take a photo of the clothes. The user takes a photo of the clothes, and the smartphone sends the image to the server. The server analyzes the received image and extracts attributes such as the clothing category, color, pattern, and material. The extracted attribute information is stored in a database, and the results are sent back to the smartphone.

[2108] Example: A user takes a photo of a blue shirt and sends the image to a server using a fashion coordination app. The server analyzes the image, extracts the attributes "tops," "blue," and "shirt," and stores them in a database. The smartphone then displays the message "Blue shirt registered."

[2109] 2. Coordination suggestions

[2110] The user selects an outfit suggestion option on their smartphone. The smartphone prompts the user to enter information such as today's weather and schedule. The user enters the necessary information, and the smartphone sends it to the server. The server uses a generative AI model to generate the optimal outfit based on the user's clothing list and the submitted information. The generated outfit is sent to the smartphone, and the suggestions are displayed to the user.

[2111] Example: A user requests "Today's work outfit" and inputs the weather forecast and the date of going to work. The server suggests "white shirt," "blue jeans," and "black jacket," and displays them on the smartphone.

[2112] 3. Emotion recognition

[2113] The emotion engine analyzes the user's facial expressions, voice, and behavioral data to recognize their emotions. The smartphone captures the user's video and audio during the virtual try-on and sends them to the server. The server then analyzes the user's emotions using the emotion engine and modifies the outfit based on those emotions.

[2114] Example: If a user mutters "It's a bit awkward" during a virtual try-on session, the smartphone captures the user's voice and facial expression and sends them to the server. The emotion engine recognizes that the user is dissatisfied, and the server changes the outfit to a "casual shirt" and "jeans."

[2115] 4. Virtual try-on through AR glasses

[2116] The user selects the virtual try-on mode, puts on the AR glasses, and looks into the mirror. The smartphone captures an image of the user's body and sends it to the server. The server then virtually overlays the suggested clothing on the image of the user's body. The smartphone visually displays the results of the virtual try-on to the user through the AR glasses. The user checks the results of the virtual try-on and sends feedback to the smartphone.

[2117] Example: When a user wears AR glasses and looks into a mirror, the server displays "white shirt," "blue jeans," and "black jacket" over the user's image. The user again sends dissatisfied feedback.

[2118] 5. Proposal and purchase of suitable clothing

[2119] The server receives the user's feedback and fine-tunes the outfit as necessary. The smartphone receives the newly suggested outfit information and a link to purchase it from the online store from the server and displays it to the user. The user clicks the link to access the online store and purchase the outfit.

[2120] Example: If the user gives feedback that they "don't like the color," the server suggests a gray shirt instead of a white one and generates a new purchase link, which the user clicks to purchase the gray shirt from the online store.

[2121] This system uses devices such as smartphones and AR glasses to provide real-time coordination suggestions based on the user's emotions and feedback. The hardware used includes standard smartphones (iPhones, Android ...

Claims

1. means for capturing images of a user's clothing; A means for analyzing the captured image to extract attributes of the clothing; A means for storing the extracted clothing attributes in a database; A means for generating a coordinate based on a user's clothing list and external information in response to a user's request; a means for visually displaying the generated coordinates to a user; a means for users to submit feedback; A means for suggesting suitable clothing items based on user feedback and generating a link to purchase the clothing items; A system including:

2. The system of claim 1 , further comprising: means for displaying the virtual clothing when the user wears the AR glasses and looks into a mirror.

3. The system according to claim 1 , further comprising means for enabling the suggested clothing to be purchased on the spot in cooperation with an online store.

Images