system

Abstract

We provide the system. [Solution] A means of analyzing a user's clothing using images and storing that information in a database, A method for suggesting clothing coordinates based on the user's mood and outing plans, A method for searching for and suggesting items that are missing from the proposed outfit, A means of acquiring and recording the user's body shape information, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, many people own a large number of clothes and feel difficulty in managing them. Also, they may forget where they stored them, and redundant shopping increases due to purchasing the same items repeatedly, which becomes an economic burden. Furthermore, the fashion options may be limited because appropriate coordination proposals cannot be made. There is a need to solve such problems.

Means for Solving the Problems

[0005] The present invention provides a system that includes means for image analysis of a user's clothing and storing the information in a database, means for suggesting clothing coordinates based on the user's mood and outing plans, means for searching for and suggesting items that are missing from the suggested coordinates, and means for acquiring and recording the user's body shape information. Furthermore, the system also includes means for automatically determining when clothing has deteriorated and generating a notification prompting replacement, and means for projecting coordinate suggestions onto a full-length mirror and presenting them to the user, thereby enabling the user to efficiently and effectively manage their clothing on a daily basis and easily select the optimal coordinate.

[0006] "User" refers to an individual or legal entity that uses the system.

[0007] "Clothing" refers to nonwoven fabric products, including but not limited to clothing such as shirts, pants, jackets, dresses, and skirts.

[0008] "Image analysis" refers to a technique that analyzes information about an object based on captured image data, and in this invention, it refers to extracting information such as the type, color, size, and material of clothing.

[0009] A "database" refers to a recording medium or system that systematically organizes and stores information about clothing, allowing for immediate access when needed.

[0010] "Mood" refers to the information that users input into the application regarding their feelings, emotions, or mood tendencies for the day.

[0011] "Outing plans" refers to the user's schedule for the day, such as meetings, dates, casual outings, and other planned activities.

[0012] "Coordination" refers to a style or fashion set suggested by combining the user's existing clothing.

[0013] "Suggesting" means presenting the user with information on selected outfits and items from the system, making that information available for the user to refer to.

[0014] A "full-length mirror" refers to a large mirror that allows the user to see their entire body, and in this context, it includes a display function that projects an image of the outfit.

[0015] "Body type information" refers to the user's physical data, such as height, weight, and measurements of various body parts.

[0016] "Deterioration" refers to a state in which the quality or function of clothing deteriorates due to use or the passage of time, and in this invention, it includes visual or physical damage.

[0017] "Notification" refers to a message sent by a system to a user to convey necessary information or warnings. [Brief explanation of the drawing]

[0018] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8]It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the language used in the following description will be explained.

[0021] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Further, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] This invention relates to an AI-powered closet and full-length mirror system that assists users with clothing management and outfit suggestions. This system is built using a cloud server and a terminal (e.g., a smartphone, tablet, or full-length mirror display).

[0040] Clothing image analysis and database management

[0041] 1. Register your clothes

[0042] The user buys new clothes, launches the app, and takes a picture of the clothes.

[0043] The device sends the captured image of the clothing to the server.

[0044] The server analyzes the received images using image analysis technology and extracts information such as type, color, size, and material.

[0045] The server saves the analyzed information to a database and sends a confirmation notification to the terminal.

[0046] The device displays a registration completion message to the user.

[0047] 2. Registration via QR code / tag

[0048] Users register new clothing items using QR codes or tag readers, hence the use of a tag reader.

[0049] The device reads the information from the QR code or tag and sends it to the server.

[0050] The server registers the received information in its database and sends the details to the terminal.

[0051] The device displays information and prompts the user for confirmation.

[0052] Coordination suggestions

[0053] 1. Enter your mood and schedule.

[0054] Users use the app to input their mood for the day and their plans for going out.

[0055] The terminal sends the entered information to the server.

[0056] 2. Coordination generation

[0057] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database.

[0058] The server generates clothing combinations as coordinated outfits and creates images based on them.

[0059] The server sends a coordinated image and an explanation of its key points to the terminal.

[0060] The device projects an outfit image onto a full-length mirror and presents it to the user.

[0061] Suggestions for additional items

[0062] 1. Suggestions for missing items

[0063] The server checks the generated outfit and searches the internet for any missing items.

[0064] The server adds information about the found items to a suggestion list and sends it to the terminal.

[0065] The device displays a list of suggestions to the user.

[0066] 2. Deterioration detection and notification

[0067] The server periodically checks the database and analyzes the condition of the clothing.

[0068] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[0069] The server suggests appropriate shop items and sends them to the terminal.

[0070] The device displays a notification to the user and introduces suggested items.

[0071] Size analysis and purchase procedure

[0072] 1. Body shape and size analysis

[0073] The user scans their body shape and size during the initial setup (for example, using a body scan app).

[0074] The device sends the acquired body shape data to the server.

[0075] The server analyzes body shape data and registers the user's size information in the database.

[0076] 2. Purchase Procedure

[0077] Users purchase items they like through suggestions and notifications.

[0078] The device will begin the purchase process and you will enter the required information.

[0079] The server suggests the optimal size based on body shape data and processes the purchase.

[0080] The server sends a purchase confirmation message to the device.

[0081] The device displays a purchase completion message to the user.

[0082] Specific example

[0083] 1. Registering and coordinating clothing items

[0084] User A buys a new shirt and scans the QR code with the app.

[0085] The server adds and organizes the shirt information in the database.

[0086] User B enters "I have plans for a casual outing today" into the app.

[0087] The server generates a casual outfit and sends it to the terminal.

[0088] The device projects the outfit onto a full-length mirror, and AI provides commentary.

[0089] 2. Suggestions and purchases of additional items

[0090] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[0091] The device displays a link, and the user purchases the shoes on the spot.

[0092] If user C wants to buy a winter coat, the server will suggest a coat of the appropriate size based on their body shape data.

[0093] The coat was successfully ordered through the purchase process.

[0094] This invention allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles.

[0095] The following describes the processing flow.

[0096] Step 1:

[0097] The user purchases new clothes, launches the app, and takes a picture of the clothes.

[0098] Step 2:

[0099] The device sends the image of the clothing it has photographed to the server.

[0100] Step 3:

[0101] The server analyzes the received images using image analysis technology to extract information such as type, color, size, and material.

[0102] Step 4:

[0103] The server saves the analyzed information to a database and sends a registration completion notification to the device.

[0104] Step 5:

[0105] The device displays a confirmation message to the user.

[0106] Step 6:

[0107] Users use the app to input their mood for the day and their plans for going out.

[0108] Step 7:

[0109] The terminal sends the entered information to the server.

[0110] Step 8:

[0111] Based on the information received by the server, the system selects the most suitable clothing items from the user's database and generates an outfit coordination.

[0112] Step 9:

[0113] The server generates a coordinated image and a point-by-point explanation, and sends it to the terminal.

[0114] Step 10:

[0115] The device projects an outfit image onto a full-length mirror and presents it to the user.

[0116] Step 11:

[0117] The server checks the outfit and searches the internet for any missing items.

[0118] Step 12:

[0119] The server adds information about the found items to a suggestion list and sends it to the terminal.

[0120] Step 13:

[0121] The device displays a list of suggestions to the user.

[0122] Step 14:

[0123] The server periodically checks the database and analyzes the condition of the clothing.

[0124] Step 15:

[0125] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[0126] Step 16:

[0127] The server suggests appropriate shop items and sends a notification to the device.

[0128] Step 17:

[0129] The device displays a degradation notification and suggested items to the user.

[0130] Step 18:

[0131] The user scans their body shape and size during the initial setup.

[0132] Step 19:

[0133] The device sends the acquired body shape data to the server.

[0134] Step 20:

[0135] The server analyzes body shape data and registers the user's size information in the database.

[0136] Step 21:

[0137] Users purchase items they like through suggestions and notifications.

[0138] Step 22:

[0139] The device initiates the purchase process and prompts the user to enter the required information.

[0140] Step 23:

[0141] The server suggests the optimal size based on body shape data and processes the purchase.

[0142] Step 24:

[0143] The server sends a purchase confirmation message to the device.

[0144] Step 25:

[0145] The device displays a purchase completion message to the user.

[0146] (Example 1)

[0147] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0148] In today's busy lifestyle, it is difficult for users to efficiently manage their clothing and choose the best outfits. Furthermore, there is a lack of assistance in registering newly purchased clothing, suggesting outfits based on the season and mood, checking and suggesting missing items, selecting the optimal size for their body type, and properly replacing worn-out clothing. As a result, users spend a lot of time managing their clothes, leading to problems such as unnecessary purchases and inappropriate outfit choices.

[0149] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0150] In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database; means for suggesting clothing coordinates based on the user's mood and outing plans; means for searching for and suggesting items that are missing from the suggested coordinates; means for acquiring and recording the user's body shape information; means for suggesting the optimal size of the suggested items based on the acquired body shape information; and means for registering the user's clothing using tags or QR codes. As a result, the user can enjoy efficient clothing management and coordinate suggestions, reduce unnecessary purchases, and improve their individual fashion lifestyle.

[0151] A "user" refers to an individual who uses the system to manage their clothing and receive styling suggestions.

[0152] "Image analysis" refers to the technology of extracting information such as type, color, size, and material from images of clothing that have been photographed.

[0153] A "database" refers to a collection of information where analyzed data is stored and searched or updated as needed.

[0154] "Mood and outing plans" refers to the user's daily emotions and activity plans, and is a fundamental data element for suggesting outfit combinations.

[0155] "Outfit suggestions" refers to generating and presenting the optimal clothing combinations based on the user's mood and planned outing.

[0156] "Missing items" refer to clothing or accessories that are missing or need to be added to the suggested outfit.

[0157] "Body shape information" refers to data that details the user's body shape and size, and is obtained through scanning or input.

[0158] "Suggesting the optimal size" means indicating the most suitable size among the suggested items based on the user's body type information.

[0159] "Registering using tags or QR codes" refers to a method of scanning tags or QR codes attached to clothing and registering that information in a database.

[0160] "Deterioration assessment" refers to the automatic determination of the decline in the quality of clothing based on factors such as the frequency of use and age.

[0161] A "notification prompting replacement" refers to a message sent to a user suggesting the purchase of new clothing items that have been deemed to be worn out.

[0162] "Projecting onto a full-length mirror" refers to displaying the suggested outfit on a screen or smart mirror for the user to see.

[0163] This invention relates to an AI-powered system that assists users with clothing management and outfit coordination suggestions. This system is built using a cloud server and the user's device (e.g., a smartphone, tablet, or smart mirror).

[0164] Clothing image analysis and database management

[0165] This system provides a function that allows users to take photos of new clothes they purchase through an app and send the image data to a cloud server. Smartphones and tablets can be used as the devices. The server uses image analysis technologies such as TensorFlow and OpenCV to extract information such as the type, color, size, and material of the clothing and store it in a database. This allows users to review the information later or use it for outfit suggestions.

[0166] Registration via QR code / tag

[0167] Users can easily register information about new clothing by scanning the QR code or tag attached to the garment. The device reads the QR code or tag information and sends it to the server. The server registers the received information in its database and sends the details to the device. The user can then review this information and enter additional details if necessary.

[0168] Coordination suggestions

[0169] The user uses the app to input their mood and outing plans for the day. This can be done using emojis, text input, or dropdown menus. The device sends the input information to a server, which then selects the most suitable clothing based on this information. The server uses a machine learning model (e.g., k-means clustering) to generate an outfit and sends the image and explanation to the device. The device then presents the outfit image to the user and projects it onto a display such as a smart mirror.

[0170] Suggestions for additional items

[0171] The server reviews the generated outfit and searches the internet for any missing items. For example, it searches online shops via API integration and adds appropriate items to the suggestion list. The suggestion list is in JSON format and sent to the device. The device displays the suggestion list to the user and provides purchase links as needed.

[0172] Degradation detection and notification

[0173] The server periodically checks the database and analyzes the deterioration status of clothing. For example, it uses a deterioration model to calculate the deterioration level, taking into account the frequency of use and the time elapsed. If deterioration is advanced, the server generates a notification and sends a message to the device prompting replacement. The device displays this notification to the user and introduces suggested new items.

[0174] Size analysis and purchase procedure

[0175] During the initial setup, the user scans their body shape and size. For example, they use a dedicated body scanning app to obtain 3D data of their body shape. The device sends this data to a server, which analyzes it and registers the user's size information in a database. When the user purchases an item they like through suggestions or notifications, the server suggests the optimal size based on the body shape information and initiates the purchase process. Once the purchase is complete, the server generates a confirmation message and sends it to the device, allowing the user to confirm the purchase.

[0176] Examples of specific cases and prompt statements

[0177] Specific example

[0178] 1. Registering and coordinating clothing items

[0179] A user purchases a new shirt and scans the QR code with the app.

[0180] The server adds and organizes the shirt information in the database.

[0181] Another user enters "I have plans for a casual outing today" into the app.

[0182] The server generates a casual outfit and sends it to the terminal.

[0183] The device projects the outfit onto a smart mirror, and AI provides commentary.

[0184] 2. Suggestions and purchases of additional items

[0185] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[0186] The device displays a link, and the user purchases the shoes on the spot.

[0187] Based on another user's body shape data, the server suggests a coat of the appropriate size.

[0188] The coat was successfully ordered through the purchase process.

[0189] Example of a prompt

[0190] "I have a casual outing planned. Please suggest the best outfit."

[0191] "I purchased a new shirt, so I'm registering my information. I scanned the QR code."

[0192] "I'd like to buy a new winter coat, so please suggest a size that fits my body type."

[0193] As described above, the system of the present invention provides various functions for efficiently and effectively managing the user's clothing and suggesting outfit combinations, thereby improving the user's fashion life.

[0194] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0195] Step 1: Take a photo of your clothes and send it to us.

[0196] The user takes a picture of new clothes. The input for this step is image data of the clothes taken by the user using the camera on their smartphone or tablet. The device receives the captured image and sends it to the server. Specifically, the device launches its camera app, temporarily saves the captured image, and then uploads it to the cloud server. The output is the image data of the clothes that arrives on the server.

[0197] Step 2: Image analysis and saving to database

[0198] The server analyzes the received image data. The input for this step is the image data of the clothing sent to the server. The server uses image analysis techniques (e.g., TensorFlow or OpenCV) to extract information such as type, color, size, and material. Specifically, it runs an image analysis model and extracts features. The output is the analyzed information data, which is stored in a database.

[0199] Step 3: Notification of analysis results

[0200] The server saves the analyzed information to a database and generates a registration completion notification. The input for this step is the image analysis result data. The server uses a REST API to generate the notification message and sends it to the terminal. The output is the registration completion notification message. The terminal receives this notification and displays it to the user.

[0201] Step 4: Read and send the QR code / tag.

[0202] The user enters information about new clothing using a QR code or tag reader. This input is the information from the QR code or tag attached to the clothing. The device scans the QR code or tag and sends the information to the server. Specifically, it uses a tag reader app to retrieve QR code data and uploads it to a cloud server. The output is the QR code or tag information data arriving at the server.

[0203] Step 5: Register QR code / tag information in the database

[0204] The server registers the information from the received QR code or tag into a database. The input for this step is the QR code or tag information sent to the server. The server saves the information in the database and sends the details to the terminal. Specifically, it extracts and transforms the data and registers it in the database. The output is the information saved in the database and the detailed information sent to the terminal.

[0205] Step 6: Enter your mood and schedule

[0206] The user uses the app to input their mood and outing plans for the day. This step involves inputting mood and plan data. The device then sends this information to the server. Specifically, the user records the information using an input form within the app and uploads it to the cloud server by pressing the submit button. The output is the mood and plan data arriving at the server.

[0207] Step 7: Creating the outfit

[0208] The server generates outfits based on the user's mood and schedule information. The input for this step is the user's mood and schedule information. The server uses a machine learning model (e.g., k-means clustering) to select the most suitable clothing from the database and generate the outfit. Specifically, it executes database queries and applies the selection algorithm. The output is the generated outfit image and its explanation.

[0209] Step 8: Send and present your outfit ideas

[0210] The server sends the generated coordinated image and explanation to the terminal. The input for this step is the generated coordinated image and explanation. The terminal receives this and projects it onto a smart mirror or display to show it to the user. Specifically, it receives the image and text data and projects it onto the display device. The output is the coordinated image presented to the user.

[0211] Step 9: Search for and suggest missing items

[0212] The server checks the outfit and searches the internet for any missing items. The input for this step is the generated outfit data. The server searches online shops via API integration and adds appropriate items to the suggestion list. Specifically, it uses an internet search API to retrieve item information and construct the list. The output is the information in the suggestion list.

[0213] Step 10: Notification and display of the suggestion list

[0214] The server sends the suggestion list to the device. The input for this step is the information in the suggestion list. The device receives this and displays it to the user. Specifically, the app displays the received list and provides a purchase link. The output is the suggestion list displayed to the user.

[0215] Step 11: Check and notify about deterioration status

[0216] The server periodically checks the database and analyzes the deterioration status of the clothing. The input for this step is the clothing information in the database. The server uses a deterioration model to analyze the data and generate deterioration reports and notification messages. Specifically, it executes a data analysis algorithm and determines the degree of deterioration. The output is a notification message.

[0217] Step 12: Sending Degradation Notifications and Proposals

[0218] The server sends the generated notification message to the terminal. The input for this step is the degraded notification message. The terminal receives this, displays a notification to the user, and introduces the suggested items. The specific action is to use the notification function to display the message and introduce the new items. The output is the notification message and suggested items that the user receives.

[0219] Step 13: Scan and submit body shape data

[0220] The user scans their body shape data during the initial setup. The input for this step is body shape data acquired by a scanning device (e.g., a body scanning app). The device receives this data and sends it to the server. Specifically, it uses the scanning device to acquire 3D body shape data and uploads it to a cloud server. The output is the body shape data sent to the server.

[0221] Step 14: Analysis and registration of body shape information

[0222] The server analyzes the received body shape data and registers the user's size information in the database. The input for this step is the received body shape data. The server performs data analysis, extracts size information for each body part, and registers it in the database. Specifically, it executes a body shape analysis algorithm to generate and save size data. The output is the size information stored in the database.

[0223] Step 15: Suggestion of the optimal size and purchase procedure

[0224] The user purchases an item through a suggestion or notification. The input for this step is information about the item to be purchased and the user's body shape data. The server suggests the optimal size based on the body shape data and processes the purchase. Specifically, it applies a size suggestion algorithm and handles purchase confirmation and payment. The output is a purchase completion confirmation message.

[0225] Step 16: Send and display purchase completion notification

[0226] The server sends a purchase completion confirmation message to the terminal. The input for this step is the purchase completion information. The terminal receives this and displays the purchase completion message to the user. Specifically, it uses a notification function to display the confirmation message and notify the user of its contents. The output is the purchase completion notification displayed to the user.

[0227] (Application Example 1)

[0228] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0229] Traditional clothing management systems simply stored users' clothing in a database, requiring users to physically try on clothes to actually see the outfits they wanted. Furthermore, there was a lack of systems that automatically suggested appropriate outfits based on the user's body type, mood, and outing plans, as well as suggesting replacements for worn-out clothing. There is also a need for a new virtual try-on experience, allowing users to try on clothes without actually going to a store. To address these challenges, a more efficient and user-friendly clothing management and outfit suggestion system is needed.

[0230] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0231] In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database; means for suggesting clothing coordinates based on the user's mood and outing plans; means for searching for and suggesting items missing from the suggested coordinates; means for acquiring and recording the user's body shape information; means for the user to experience trying on clothes in a virtual fitting room; and means for generating a corresponding virtual avatar and performing clothing fitting simulations. This allows users to check the suitability of clothing through virtual try-ons without having to go to a physical store, and further enables coordinate suggestions based on mood and body shape, as well as suggestions for replacing worn-out clothing.

[0232] A "user" is a person who uses the system and is the person who receives clothing management and coordination suggestions.

[0233] "Clothing" refers to the general term for clothes and accessories that a user owns and wears on a daily basis.

[0234] "Image analysis" is the process of extracting information such as the type, color, size, and material of clothing from a captured image.

[0235] A "database" is an information system for systematically organizing and storing information about analyzed clothing.

[0236] "Mood" refers to the emotions and mood that the user is feeling at any given time, and it is one of the criteria used for suggesting outfits.

[0237] "Outing plans" refer to the activities and schedules that a user has planned for a specific day.

[0238] "Outfit suggestions" refer to the system suggesting the optimal clothing combination based on the user's mood and planned outing.

[0239] "Suggested outfit" refers to the result of the clothing combinations generated by the system.

[0240] "Missing items" refer to clothing or accessories that are necessary for the suggested outfit but that the user does not currently own.

[0241] "Searching" is the process of using the internet or databases to find out whether or not specific information or products exist.

[0242] "Body shape information" refers to information about various body dimensions, such as the user's height, weight, shoulder width, and waist size.

[0243] A "virtual fitting room" is a simulation area where users can virtually try on clothes.

[0244] A "virtual avatar" is a virtual human model created based on the user's body shape information.

[0245] A "fitting simulation" is a process that allows users to visually check how actual clothing will look on a virtual avatar.

[0246] The present invention relates to a system that supports a user's clothing management and coordination suggestions. This system includes means for image analysis of the user's clothing and storing the information in a database, means for suggesting clothing coordinates based on the user's mood and outing plans, means for searching for and suggesting items that are missing from the suggested coordinates, means for acquiring and recording the user's body shape information, means for the user to experience trying on clothes in a virtual fitting room, and means for generating a corresponding virtual avatar and performing clothing fitting simulations.

[0247] System Configuration

[0248] Image analysis and database management

[0249] 1. Register your clothing:

[0250] The user takes a picture of new clothing using their smartphone camera and sends it to their device.

[0251] The server analyzes the received images using image analysis techniques to extract information such as type, color, size, and material. Specifically, it uses Python's image processing library (PIL) and request library (requests).

[0252] The server saves the analyzed information to a database (e.g., MySQL or PostgreSQL) and sends a completion notification to the terminal.

[0253] Coordination suggestions

[0254] 1. Enter your mood and schedule:

[0255] Users use a smartphone app to input their mood and outing plans for the day.

[0256] The terminal sends the entered information to the server.

[0257] 2. Coordination generation:

[0258] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database.

[0259] The server sends the generated outfit image and its key points to the terminal.

[0260] The device projects the coordinated look onto a virtual fitting room (for example, smart glasses or a head-mounted display) and presents it to the user.

[0261] Suggestions for missing items

[0262] 1. Search for and suggest missing items:

[0263] The server checks the generated outfit and searches the internet for any missing items.

[0264] The server adds information about the found items to a suggestion list and sends it to the terminal.

[0265] The device displays a list of suggestions to the user.

[0266] Virtual fitting

[0267] 1. Virtual try-on experience:

[0268] When a user enters a virtual fitting room, the server generates a virtual avatar based on the user's body shape information.

[0269] The generated outfit is applied to the virtual avatar, and a fitting simulation is performed.

[0270] The device (for example, smart glasses or a head-mounted display) visually displays the simulation results to the user.

[0271] Specific example

[0272] 1. Example of a prompt:

[0273] "Please suggest an outfit that's perfect for a casual day trip."

[0274] "I'm in a good mood today and have plans to go out. What would be the best outfit for me?"

[0275] Usage example

[0276] User A purchases a new shirt and scans the QR code with their smartphone camera. The server analyzes the shirt's information and adds it to the database.

[0277] User B enters "I have a casual outing planned for today" into the app. The server follows a series of processes, generates a casual outfit, and displays it in a virtual fitting room.

[0278] User C tries on clothes in a virtual fitting room, reviews the simulation results, and makes a final decision.

[0279] This invention allows users to efficiently and effectively manage their clothing and receive styling suggestions without having to go to a physical store, reducing unnecessary purchases and improving their fashion lifestyle.

[0280] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0281] Step 1:

[0282] Image analysis and registration of clothing

[0283] Input: Images of new clothing items taken by the user with their smartphone camera.

[0284] Specific action: The user launches the app and takes a picture of the clothing.

[0285] Data processing / Calculation: The terminal sends an image to the cloud server. The server analyzes the image using image analysis technologies (such as PIL, requests library, etc.) and extracts information on the type, color, size, and material.

[0286] Output: The analyzed clothing information is saved in the database, and a save completion notification is sent to the terminal.

[0287] Step 2:

[0288] Input of mood and schedule

[0289] Input: The user inputs their mood and going-out schedule for the day into the app.

[0290] Specific operation: The user inputs their mood and schedule in text into the form within the app.

[0291] Data processing / Calculation: The terminal sends the input information to the server. The server analyzes the information and matches it with the user's database.

[0292] Output: The mood and schedule information is saved on the server.

[0293] Step 3:

[0294] Generation of coordination

[0295] Input: Information on mood and going-out schedule, and the user's clothing database.

[0296] Specific operation: Based on the saved mood and schedule information, the server selects the most suitable clothing from the user's database.

[0297] Data processing / Calculation: The server uses a generated AI model to automatically generate a coordination, and creates a coordination image and its point explanation.

[0298] Output: The generated coordinate image and explanations are sent to the terminal.

[0299] Step 4:

[0300] Virtual fitting experience

[0301] Input: The generated coordinate image and the user's body type information.

[0302] Specific operation: The user enters the virtual fitting room. The terminal (e.g., smart glasses or head-mounted display) sends the user's body type information to the server.

[0303] Data processing / Calculation: The server generates a virtual avatar based on the body type information and reflects the coordinates on the avatar.

[0304] Output: The fitting simulation result is displayed on the terminal and visually provided to the user.

[0305] Step 5:

[0306] Search and proposal of missing items

[0307] Input: The generated coordinates and the user's database.

[0308] Specific operation: The server checks the coordinates and searches for missing items on shopping sites or databases on the Internet.

[0309] Data processing / Calculation: The server adds the found items to the proposal list and generates links and detailed information.

[0310] Output: The proposal list is sent to the terminal and displayed to the user.

[0311] Step 6:

[0312] Deterioration assessment and replacement notification

[0313] Input: User's clothing database.

[0314] Specific operation: The server periodically checks the database and analyzes the deterioration status of the clothing.

[0315] Data processing / calculation: The server uses a deterioration detection algorithm to identify deteriorated clothing and generates replacement suggestions.

[0316] Output: Notifications and suggestions are sent to the device and displayed to the user.

[0317] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0318] This invention relates to an AI-powered closet and mirror system that assists users with clothing management and outfit suggestions. The system is built by combining a cloud server, a terminal (e.g., a smartphone, tablet, or mirror display), and an emotion engine that recognizes the user's emotions.

[0319] Clothing image analysis and database management

[0320] 1. Register your clothes

[0321] The user buys new clothes, launches the app, and takes a picture of the clothes.

[0322] The device sends the captured image of the clothing to the server.

[0323] The server analyzes the received images using image analysis technology and extracts information such as type, color, size, and material.

[0324] The server saves the analyzed information to a database and sends a registration completion notification to the terminal.

[0325] The device displays a registration completion message to the user.

[0326] 2. Registration via QR code / tag

[0327] Users register new clothing items using QR codes or tag readers, hence the use of a tag reader.

[0328] The device reads the information from the QR code or tag and sends it to the server.

[0329] The server registers the received information in its database and sends the details to the terminal.

[0330] The device displays information and prompts the user for confirmation.

[0331] Coordination suggestions

[0332] 1. Enter your mood and schedule.

[0333] Users use the app to input their mood for the day and their plans for going out.

[0334] The terminal sends the entered information to the server.

[0335] 2. Coordination generation

[0336] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database and generates an outfit.

[0337] The server generates clothing combinations as coordinated outfits and creates images based on them.

[0338] The server sends a coordinated image and an explanation of its key points to the terminal.

[0339] The device projects an outfit image onto a full-length mirror and presents it to the user.

[0340] Suggestions for additional items

[0341] 1. Suggestions for missing items

[0342] The server checks the generated outfit and searches the internet for any missing items.

[0343] The server adds information about the found items to a suggestion list and sends it to the terminal.

[0344] The device displays a list of suggestions to the user.

[0345] 2. Deterioration detection and notification

[0346] The server periodically checks the database and analyzes the condition of the clothing.

[0347] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[0348] The server suggests appropriate shop items and sends a notification to the device.

[0349] The device displays a degradation notification and suggested items to the user.

[0350] Size analysis and purchase procedure

[0351] 1. Body shape and size analysis

[0352] The user scans their body shape and size during the initial setup (for example, using a body scan app).

[0353] The device sends the acquired body shape data to the server.

[0354] The server analyzes body shape data and registers the user's size information in the database.

[0355] 2. Purchase Procedure

[0356] Users purchase items they like through suggestions and notifications.

[0357] The device will begin the purchase process and you will enter the required information.

[0358] The server suggests the optimal size based on body shape data and processes the purchase.

[0359] The server sends a purchase confirmation message to the device.

[0360] The device displays a purchase completion message to the user.

[0361] Introducing an emotional engine

[0362] 1. Recognition of emotions

[0363] The device uses a built-in emotion engine to analyze the user's facial expressions and voice in real time.

[0364] The server recognizes the user's current emotions based on the data sent from the emotion engine.

[0365] 2. Adjusting coordination based on emotions

[0366] The server adjusts the outfit suggestions based on the recognized user's emotional data.

[0367] The server generates the optimal outfit based on the user's mood, schedule, and emotional data.

[0368] The device projects an optimized outfit onto a full-length mirror and presents it to the user.

[0369] Specific example

[0370] 1. Registering and coordinating clothing items

[0371] User A buys a new shirt and scans the QR code with the app.

[0372] The server adds and organizes the shirt information in the database.

[0373] User B enters "I have plans for a casual outing today" into the app.

[0374] The server generates a casual outfit and sends it to the terminal.

[0375] The device projects the outfit onto a full-length mirror, and AI provides commentary.

[0376] The emotion engine recognizes the user's facial expressions and detects the emotion of "looking happy."

[0377] The server adjusts the outfit based on emotional data, adding colors and styles.

[0378] 2. Suggestions and purchases of additional items

[0379] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[0380] The device displays a link, and the user purchases the shoes on the spot.

[0381] If user C wants to buy a winter coat, the server will suggest a coat of the appropriate size based on their body shape data.

[0382] The coat was successfully ordered through the purchase process.

[0383] This invention allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles. Furthermore, the introduction of an emotion engine enables more personalized suggestions, enhancing the user experience.

[0384] The following describes the processing flow.

[0385] Step 1:

[0386] The user purchases new clothes, launches the app, and takes a picture of the clothes.

[0387] Step 2:

[0388] The device sends the image of the clothing it has photographed to the server.

[0389] Step 3:

[0390] The server analyzes the received images using image analysis technology to extract information such as type, color, size, and material.

[0391] Step 4:

[0392] The server saves the analyzed information to a database and sends a registration completion notification to the device.

[0393] Step 5:

[0394] The device displays a confirmation message to the user.

[0395] Step 6:

[0396] Users use the app to input their mood for the day and their plans for going out.

[0397] Step 7:

[0398] The terminal sends the entered information to the server.

[0399] Step 8:

[0400] The device uses its built-in emotion engine to analyze the user's facial expressions and voice in real time.

[0401] Step 9:

[0402] The server recognizes the user's current emotions based on the data sent from the emotion engine.

[0403] Step 10:

[0404] Based on the mood, schedule, and emotional data received by the server, the system selects the most suitable clothing from the user's database.

[0405] Step 11:

[0406] The server generates clothing combinations as coordinated outfits and creates images based on them.

[0407] Step 12:

[0408] The server sends a coordinated image and an explanation of its key points to the terminal.

[0409] Step 13:

[0410] The device projects an outfit image onto a full-length mirror and presents it to the user.

[0411] Step 14:

[0412] The server checks the outfit and searches the internet for any missing items.

[0413] Step 15:

[0414] The server adds information about the found items to a suggestion list and sends it to the terminal.

[0415] Step 16:

[0416] The device displays a list of suggestions to the user.

[0417] Step 17:

[0418] The server periodically checks the database and analyzes the condition of the clothing.

[0419] Step 18:

[0420] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[0421] Step 19:

[0422] The server suggests appropriate shop items and sends a notification to the device.

[0423] Step 20:

[0424] The device displays a degradation notification and suggested items to the user.

[0425] Step 21:

[0426] The user scans their body shape and size during the initial setup (for example, by using a body scan app).

[0427] Step 22:

[0428] The device sends the acquired body shape data to the server.

[0429] Step 23:

[0430] The server analyzes body shape data and registers the user's size information in the database.

[0431] Step 24:

[0432] Users purchase items they like through suggestions and notifications.

[0433] Step 25:

[0434] The device initiates the purchase process and prompts the user to enter the required information.

[0435] Step 26:

[0436] The server suggests the optimal size based on body shape data and processes the purchase.

[0437] Step 27:

[0438] The server sends a purchase confirmation message to the device.

[0439] Step 28:

[0440] The device displays a purchase completion message to the user.

[0441] (Example 2)

[0442] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0443] Conventional clothing management systems lacked sufficient functionality to allow users to efficiently manage their clothing and receive optimal outfit suggestions. Furthermore, they lacked the ability to suggest outfits that considered the user's mood and feelings, and to provide replacement advice based on the condition of their clothing. As a result, users often made unnecessary purchases or struggled with coordinating outfits. This invention aims to solve these problems and provide a system that allows users to enjoy smarter, more personalized clothing management and outfit suggestions.

[0444] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0445] In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database; means for suggesting clothing coordinates based on the user's mood and outing plans; means for searching the internet for and suggesting items that are missing from the suggested coordinates; means for acquiring and recording the user's body shape information; means for analyzing the user's facial expressions and voice to recognize emotions; and means for adjusting the coordinate suggestions based on the recognized emotions. As a result, the user can receive personalized coordinate suggestions, reduce unnecessary purchases, and enjoy their daily fashion life more.

[0446] "Image analysis of clothing" involves analyzing images of clothing taken by the user using the app and extracting information such as the type, color, size, and material of the clothing.

[0447] "Saving to a database" means recording the analyzed clothing information in a cloud-based database so that it can be referenced and managed later.

[0448] "Suggesting clothing coordinates based on mood and outing plans" means suggesting the most suitable outfit combination based on the user's mood and plans for the day.

[0449] "Searching online and making suggestions" means finding items on the internet that are missing from the suggested outfit and recommending them to the user.

[0450] "Acquiring and recording body shape information" means analyzing body shape data obtained by the user through body scans, etc., and saving that information to a database.

[0451] "Recognizing emotions by analyzing facial expressions and voice" means determining what emotions the user is currently experiencing by analyzing their facial expressions and tone of voice.

[0452] "Adjusting outfit suggestions based on recognized emotions" means further personalizing outfits based on the user's emotional information and suggesting the most suitable clothing for the user's mental state.

[0453] This invention relates to an AI-powered closet and mirror system that assists users with clothing management and outfit suggestions. The system is built by combining a cloud server, a terminal (smartphone, tablet, or mirror display), and an emotion engine that recognizes the user's emotions.

[0454] Clothing image analysis and database management

[0455] Clothing registration

[0456] The user purchases new clothes, launches the app, and takes a picture of the clothes. The device sends the image of the clothes to the server. The server analyzes the received image using image analysis techniques (e.g., OpenCV and TensorFlow) and extracts information such as type, color, size, and material. The server saves the analyzed information to a database (e.g., MySQL) and sends a registration completion notification to the device. The device displays a registration completion message to the user.

[0457] Registration via QR code / tag

[0458] The user registers new clothing items using a QR code or tag reader, utilizing a tag reader. The device reads the information from the QR code or tag and sends it to the server. The server registers the received information in its database and sends the details to the device. The device uses the Zxing library to read the QR code information. The device displays the information and prompts the user for confirmation.

[0459] Coordination suggestions

[0460] Enter your mood and schedule.

[0461] The user uses the app to input their mood for the day and their outing plans. The device then sends the entered information to the server.

[0462] Coordination generation

[0463] Based on the mood and schedule information received by the server, the server selects the most suitable clothing from the user's database and generates an outfit. The server uses an AI model developed in Python to suggest outfits based on the user's past preference data and input information. The server generates clothing combinations as outfits and generates an image based on them. GANs (Generative Opposite Networks) are used for image generation. The server sends the outfit image and its key points to the terminal. The terminal projects the outfit image onto a full-length mirror and presents it to the user.

[0464] Suggestions for additional items

[0465] Suggestions for missing items

[0466] The server checks the generated outfit and searches the internet for any missing items. The server uses a Python web scraping library (such as Scrapy) to search for recommended items. The server adds information about the found items to a suggestion list and sends it to the terminal. The terminal displays the suggestion list to the user.

[0467] Degradation detection and notification

[0468] The server periodically checks the database and analyzes the deterioration status of clothing. The server creates deterioration prediction models using random forests and support vector machines. The server generates notifications for clothing that is deteriorating and creates messages encouraging replacement. The server suggests appropriate shop items and sends notifications to the device. The device displays the deterioration notification and suggested items to the user.

[0469] Size analysis and purchase procedure

[0470] Body shape and size analysis

[0471] The user scans their body shape and size during initial setup (for example, using a body scan app). The device sends the acquired body shape data to the server. The server processes the 3D scan data and analyzes the user's body shape. This is done using 3D modeling software such as MeshLab. The server stores the body shape data in a database.

[0472] Purchase procedure

[0473] The user purchases items they like through suggestions and notifications. The device initiates the purchase process and the user enters the necessary information. The server matches the user's body size data with product data, suggests the optimal size, and processes the purchase. The server sends a purchase completion confirmation message to the device. The device displays a purchase completion message to the user.

[0474] Introducing an emotional engine

[0475] Recognition of emotions

[0476] The device uses its built-in emotion engine to analyze the user's facial expressions and voice in real time. The device analyzes emotions using a facial recognition API (e.g., Google Cloud Vision API) or a speech recognition API. The server recognizes the user's current emotions based on the data sent from the emotion engine.

[0477] Adjusting coordination based on emotions

[0478] The server adjusts outfit suggestions based on the recognized user's emotional data. The server generates an optimal outfit that reflects the user's mood, schedule, and emotional data. The device projects the optimized outfit onto a full-length mirror and presents it to the user.

[0479] Specific example

[0480] Prompt message

[0481] The following is an example of a prompt message to use when inputting a specific command into a generating AI model.

[0482] "Please suggest a casual style for today."

[0483] "Please suggest outfit ideas using this shirt."

[0484] "Please recommend some clothes that suit my current mood."

[0485] This allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles. Furthermore, the introduction of an emotional engine enables more personalized suggestions, enhancing the user experience.

[0486] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0487] Processing steps

[0488] Clothing image analysis and database management

[0489] Step 1:

[0490] The user purchases new clothes, launches the app, and takes a picture of the clothes.

[0491] Input: New clothes, smartphone app

[0492] Output: Images of the photographed clothing

[0493] Step 2:

[0494] The device sends the image of the clothing it has photographed to the server.

[0495] Input: Images of clothing taken

[0496] Output: Image data transferred to the server

[0497] Step 3:

[0498] The server analyzes the received images using image analysis techniques (OpenCV and TensorFlow) to extract information such as type, color, size, and material.

[0499] Input: Transferred image data

[0500] Output: Image analysis results including type, color, size, and material.

[0501] Specific operation: Extract feature points from an image using OpenCV and classify them using TensorFlow.

[0502] Step 4:

[0503] The server analyzes the information, saves it to a database (MySQL), and sends a registration completion notification to the terminal.

[0504] Input: Analysis results

[0505] Output: Clothing information stored in the database, registration completion notification

[0506] Specific operation: Insert information into the database using SQL queries.

[0507] Step 5:

[0508] The device displays a registration completion message to the user.

[0509] Input: Registration completion notification

[0510] Output: Completion message displayed to the user

[0511] Registration via QR code / tag

[0512] Step 1:

[0513] A tag reader is used so that users can register new clothing items using QR codes or tag readers.

[0514] Input: New clothes, tag reader

[0515] Output: Information from the scanned QR code or tag

[0516] Step 2:

[0517] The device reads the information from the QR code or tag and sends it to the server. The Zxing library is used for reading.

[0518] Input: QR code or tag information

[0519] Output: Code information transferred to the server

[0520] Step 3:

[0521] The server registers the received information in the database and sends the details to the terminal.

[0522] Input: Transferred code information

[0523] Output: Clothing information stored in the database, notification of detailed information.

[0524] Step 4:

[0525] The device displays information and prompts the user for confirmation.

[0526] Input: Notification of detailed information

[0527] Output: Detailed information and confirmation message displayed to the user

[0528] Enter your mood and schedule.

[0529] Step 1:

[0530] Users use the app to input their mood for the day and their plans for going out.

[0531] Input: Mood, schedule information, smartphone app

[0532] Output: Information entered into the terminal

[0533] Step 2:

[0534] The terminal sends the entered information to the server.

[0535] Input: Entered information

[0536] Output: Mood and schedule information transferred to the server

[0537] Coordination generation

[0538] Step 1:

[0539] Based on the mood and schedule information received by the server, the system selects the most suitable clothing from the user's database and generates an outfit. An AI model developed in Python is used.

[0540] Input: Information about mood and schedule, clothing information in the database.

[0541] Output: Optimal coordination suggestion

[0542] Specific operation: Select based on past preference data and input information.

[0543] Step 2:

[0544] The server generates clothing combinations as coordinated outfits and then generates images based on these combinations. Image generation utilizes GANs (Generative Opposite-Guard Networks).

[0545] Input: Coordination suggestion

[0546] Output: Generated coordinated image

[0547] Specific operation: Create an image of the outfit using a GAN model.

[0548] Step 3:

[0549] The server sends a coordinated image and an explanation of its key points to the terminal.

[0550] Input: Generated coordinated image

[0551] Output: Coordinated image and explanation sent to the terminal

[0552] Step 4:

[0553] The device projects an outfit image onto a full-length mirror and presents it to the user.

[0554] Input: Submitted outfit image and description

[0555] Output: Coordinated outfit image and explanation displayed to the user.

[0556] Suggestions for missing items

[0557] Step 1:

[0558] The server checks the generated outfit and searches the internet for any missing items. A Python web scraping library (such as Scrapy) is used.

[0559] Input: Generated coordination information

[0560] Output: List of missing items

[0561] Specific operation: Perform web scraping and search for corresponding items.

[0562] Step 2:

[0563] The server adds information about the found items to a suggestion list and sends it to the terminal.

[0564] Input: List of missing items

[0565] Output: Information from the proposal list

[0566] Step 3:

[0567] The device displays a list of suggestions to the user.

[0568] Input: Information from the proposal list

[0569] Output: Suggestion list displayed to the user

[0570] Degradation detection and notification

[0571] Step 1:

[0572] The server periodically checks the database and analyzes the deterioration status of the clothing. A deterioration prediction model is created using random forests or support vector machines.

[0573] Input: Clothing data in the database

[0574] Output: Degradation judgment result

[0575] Specific operation: Predicting the deterioration status using a machine learning model.

[0576] Step 2:

[0577] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[0578] Input: Degradation judgment result

[0579] Output: Notification message

[0580] Step 3:

[0581] The server suggests appropriate shop items and sends a notification to the device.

[0582] Input: Notification message

[0583] Output: Proposed item information

[0584] Step 4:

[0585] The device displays a degradation notification and suggested items to the user.

[0586] Input: Suggested item information

[0587] Output: Deterioration notification and item information displayed to the user.

[0588] Body shape and size analysis

[0589] Step 1:

[0590] The user scans their body shape and size during the initial setup (for example, by using a body scan app).

[0591] Input: Body shape and size scan data

[0592] Output: Scan data

[0593] Step 2:

[0594] The device sends the acquired body shape data to the server.

[0595] Input: Scanned data

[0596] Output: Body shape data transferred to the server

[0597] Step 3:

[0598] The server processes the 3D scan data and analyzes the user's body shape. 3D modeling software such as MeshLab is used.

[0599] Input: Transferred body shape data

[0600] Output: Analyzed body shape information

[0601] Specific operation: Perform 3D modeling and analyze the user's body shape as digital data.

[0602] Step 4:

[0603] The server saves body shape data to a database.

[0604] Input: Analyzed body shape information

[0605] Output: Body shape information stored in the database

[0606] Purchase procedure

[0607] Step 1:

[0608] Users purchase items they like through suggestions and notifications.

[0609] Input: Suggestion and notification data

[0610] Output: Purchase Decision

[0611] Step 2:

[0612] The device will begin the purchase process and enter the required information.

[0613] Input: Purchase decision, required information

[0614] Output: Entered purchase information

[0615] Step 3:

[0616] The server matches the user's body shape data with product data, suggests the optimal size, and processes the purchase.

[0617] Input: Entered purchase information, body type data

[0618] Output: Optimal size suggestion, purchase process

[0619] Specific operation: Match body shape data with product size data to select the optimal size.

[0620] Step 4:

[0621] The server sends a purchase confirmation message to the device.

[0622] Input: Purchase process complete

[0623] Output: Purchase completion confirmation message

[0624] Step 5:

[0625] The device displays a purchase completion message to the user.

[0626] Input: Purchase completion confirmation message

[0627] Output: Purchase completion message displayed to the user

[0628] Introducing an emotional engine

[0629] Step 1:

[0630] The device uses its built-in emotion engine to analyze the user's facial expressions and voice in real time. It utilizes a facial recognition API (Google Cloud Vision API) and a speech recognition API.

[0631] Input: User's facial expressions, voice data

[0632] Output: Sentiment data as analysis results

[0633] Specific operation: The analysis terminal processes facial and voice data and extracts emotional information.

[0634] Step 2:

[0635] The server recognizes the user's current emotions based on the data sent from the emotion engine.

[0636] Input: Sentiment data

[0637] Output: User sentiment analysis results

[0638] Step 3:

[0639] The server adjusts outfit suggestions based on the recognized user's emotional data.

[0640] Input: Sentiment analysis results

[0641] Output: Adjusted coordination suggestions

[0642] Specific actions: Consider emotional data and optimize suggestions using an AI model.

[0643] Step 4:

[0644] The device projects an optimized outfit onto a full-length mirror and presents it to the user.

[0645] Input: Adjusted coordination suggestions

[0646] Output: Optimized coordinates displayed to the user

[0647] (Application Example 2)

[0648] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0649] In recent years, interest in fashion has increased, but it remains difficult for users to efficiently manage their clothing and receive outfit suggestions. In particular, there is no system that suggests outfits based on the user's mood or outing plans, or that provides optimal suggestions based on their feelings on a given day. Furthermore, the process of searching for and purchasing missing items in suggested outfits is also cumbersome. As a result of this situation, users often end up making unnecessary purchases, and there is a need for efficient means to improve the quality of individual fashion lives.

[0650] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database, means for suggesting clothing coordinates based on the user's mood and outing plans, and means for searching for and suggesting items that are missing from the suggested coordinates. As a result, the user can efficiently manage their clothing and receive appropriate coordinates according to their mood and plans for the day.

[0651] A "user" is an individual who uses the system to manage their own clothing and receive styling suggestions.

[0652] "Clothing" refers to fashion items owned by the user and registered in the system through image analysis.

[0653] "Image analysis" is a technology that extracts information such as type, color, size, and material from images of clothing that have been photographed.

[0654] A "database" is a system that organizes and stores information such as a user's clothing and the results of its analysis.

[0655] "Mood" refers to the user's emotions and psychological state on a given day, and is an element that is reflected in the outfit suggestions.

[0656] "Planned outings" refer to the activities and places the user has planned to visit that day, and are factors that influence outfit suggestions.

[0657] "Coordination" refers to suggestions for clothing combinations for the user, which are optimized based on their mood and planned outings.

[0658] "Searching" refers to the process of finding items online that are missing from a suggested outfit.

[0659] "Body shape information" refers to data about the user's body size and shape, and is used for suggesting outfits and selecting clothing sizes.

[0660] "Emotions" refer to the psychological state perceived from the user's facial expressions and voice, and are used to optimize outfit suggestions.

[0661] A "cloud server" is a server that performs data analysis, storage, and coordination proposals.

[0662] An "emotion recognition engine" is a technology that analyzes emotions based on the user's facial expressions and voice.

[0663] This invention relates to an AI-powered system that assists users with clothing management and outfit coordination suggestions. This system is built by combining a cloud server, a terminal (e.g., a smartphone), and an emotion recognition engine.

[0664] System Programs and Processing

[0665] 1. Register your clothes

[0666] Users use their smartphone camera to photograph new clothing items and upload the images to a cloud server. Alternatively, they can scan a QR code to automatically register clothing information. The cloud server uses image analysis technology (e.g., Google Cloud Vision API) to extract information such as clothing type, color, size, and material from the uploaded images. The extracted information is stored in a database (e.g., Amazon RDS).

[0667] 2. Enter your mood and schedule

[0668] Users use a smartphone app to input their mood and outing plans for the day. This information is sent to a cloud server, where it is cross-referenced with the user's database to generate appropriate outfit suggestions.

[0669] 3. Coordination generation

[0670] The cloud server generates the optimal outfit based on the user's mood and outing plans. An image and explanation of the generated outfit are created and sent to the smartphone.

[0671] 4. Recognition and regulation of emotions

[0672] The smartphone uses its built-in camera and microphone to analyze the user's facial expressions and voice in real time, and an emotion recognition engine (such as Microsoft Azure Emotion API) determines their emotions. The cloud server receives the data obtained from the emotion recognition engine and optimizes the outfit based on the user's emotional state.

[0673] 5. Suggestions for purchasing additional items

[0674] The cloud server searches the internet for items missing from the suggested outfit. Information on the appropriate items is generated, and the user can view the list on their smartphone and proceed with purchasing the items they like. This purchase process uses an electronic payment service (e.g., Stripe API).

[0675] Examples of specific cases and prompt statements

[0676] As a concrete example, a user purchases a new shirt and registers it by scanning a QR code with the app. The user enters their mood for the day (e.g., "Fresh") and their plans for going out (e.g., "Lunch with friends") into the app. Based on this information, the cloud server suggests a "bright shirt" and "white pants," and displays an image of the outfit on the smartphone screen. The emotion recognition engine analyzes the user's facial expression, and if it recognizes that the user looks "happy," additional suggestions such as "bright-colored accessories" are made. Furthermore, a list of suggested items that are missing (e.g., "summer sandals") is displayed on the smartphone, and the user completes the purchase process on the spot.

[0677] Examples of prompt statements are as follows:

[0678] "Users registered their new clothes using the app. After entering their mood and plans for the day, the server suggested the most suitable outfit. It recognized the user's emotions in real time and adjusted the outfit accordingly. Furthermore, it purchased any missing items based on the suggested list."

[0679] As described above, the present invention allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles. Furthermore, the introduction of an emotion recognition engine enables more personalized suggestions, improving the user experience.

[0680] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0681] Step 1:

[0682] Users register new clothing items.

[0683] Users take photos of new clothing items using their smartphone camera and upload the images to a cloud server via the app. Alternatively, they can scan a QR code to automatically register clothing information.

[0684] Input: Image or QR code information of the clothing item

[0685] Data processing: Image analysis technology (Google Cloud Vision API) is used to extract information on clothing type, color, size, and material.

[0686] Output: Extracted clothing information is saved to a database (Amazon RDS).

[0687] Step 2:

[0688] Users enter their mood and outing plans.

[0689] Users use a smartphone app to input their mood and outing plans for the day. This information is sent to a cloud server.

[0690] Input: User's mood and outing plans

[0691] Data processing: Matching entered information with the user database.

[0692] Output: Coordination suggestions based on matching results

[0693] Step 3:

[0694] The cloud server generates the coordinates.

[0695] The cloud server generates the optimal outfit based on the user's mood and outing plans. An image and explanation of the generated outfit are created and sent to the smartphone.

[0696] Input: User's mood and outing plans, clothing database

[0697] Data processing: Generates optimal outfits and creates outfit images and explanations.

[0698] Output: Send coordination image and explanation to your smartphone

[0699] Step 4:

[0700] Smartphones recognize user emotions.

[0701] The smartphone uses its built-in camera and microphone to analyze the user's facial expressions and voice in real time. The emotion recognition engine (Microsoft Azure Emotion API) determines the user's emotions.

[0702] Input: User's facial expressions and voice

[0703] Data processing: Emotions are determined using an emotion recognition engine.

[0704] Output: Send the determined emotion data to the cloud server.

[0705] Step 5:

[0706] Cloud servers adjust outfits based on emotions.

[0707] The cloud server optimizes the coordination based on the user's emotional state, using data obtained from the emotion recognition engine.

[0708] Input: User sentiment data, initial outfit suggestions

[0709] Data processing: Adjusting outfits according to emotions

[0710] Output: Optimized outfit resend to smartphone

[0711] Step 6:

[0712] The cloud server suggests additional items.

[0713] The cloud server searches the internet for items missing from the suggested outfit. It generates information on the appropriate items, and the user can view the list on their smartphone.

[0714] Input: Outfit suggestions, item database

[0715] Data processing: Search the internet for missing items and generate item information.

[0716] Output: Send item information to smartphone

[0717] Step 7:

[0718] Users purchase additional items

[0719] The user selects their preferred items from a suggested list on their smartphone and proceeds with the purchase. The purchase is completed using an electronic payment service.

[0720] Input: Suggested item list, purchase information

[0721] Data processing: Completion of the purchase process using the electronic payment service (Stripe API).

[0722] Output: Purchase confirmation and completion notification sent to your smartphone.

[0723] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0724] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0725] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0726] [Second Embodiment]

[0727] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0728] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0729] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0730] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0731] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0732] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0733] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0734] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0735] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0736] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0737] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0738] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0739] This invention relates to an AI-powered closet and full-length mirror system that assists users with clothing management and outfit suggestions. This system is built using a cloud server and a terminal (e.g., a smartphone, tablet, or full-length mirror display).

[0740] Clothing image analysis and database management

[0741] 1. Register your clothes

[0742] The user buys new clothes, launches the app, and takes a picture of the clothes.

[0743] The device sends the captured image of the clothing to the server.

[0744] The server analyzes the received images using image analysis technology and extracts information such as type, color, size, and material.

[0745] The server saves the analyzed information to a database and sends a confirmation notification to the terminal.

[0746] The device displays a registration completion message to the user.

[0747] 2. Registration via QR code / tag

[0748] Users register new clothing items using QR codes or tag readers, hence the use of a tag reader.

[0749] The device reads the information from the QR code or tag and sends it to the server.

[0750] The server registers the received information in its database and sends the details to the terminal.

[0751] The device displays information and prompts the user for confirmation.

[0752] Coordination suggestions

[0753] 1. Enter your mood and schedule.

[0754] Users use the app to input their mood for the day and their plans for going out.

[0755] The terminal sends the entered information to the server.

[0756] 2. Coordination generation

[0757] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database.

[0758] The server generates clothing combinations as coordinated outfits and creates images based on them.

[0759] The server sends a coordinated image and an explanation of its key points to the terminal.

[0760] The device projects an outfit image onto a full-length mirror and presents it to the user.

[0761] Suggestions for additional items

[0762] 1. Suggestions for missing items

[0763] The server checks the generated outfit and searches the internet for any missing items.

[0764] The server adds information about the found items to a suggestion list and sends it to the terminal.

[0765] The device displays a list of suggestions to the user.

[0766] 2. Deterioration detection and notification

[0767] The server periodically checks the database and analyzes the condition of the clothing.

[0768] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[0769] The server suggests appropriate shop items and sends them to the terminal.

[0770] The device displays a notification to the user and introduces suggested items.

[0771] Size analysis and purchase procedure

[0772] 1. Body shape and size analysis

[0773] The user scans their body shape and size during the initial setup (for example, using a body scan app).

[0774] The device sends the acquired body shape data to the server.

[0775] The server analyzes body shape data and registers the user's size information in the database.

[0776] 2. Purchase Procedure

[0777] Users purchase items they like through suggestions and notifications.

[0778] The device will begin the purchase process and you will enter the required information.

[0779] The server suggests the optimal size based on body shape data and processes the purchase.

[0780] The server sends a purchase confirmation message to the device.

[0781] The device displays a purchase completion message to the user.

[0782] Specific example

[0783] 1. Registering and coordinating clothing items

[0784] User A buys a new shirt and scans the QR code with the app.

[0785] The server adds and organizes the shirt information in the database.

[0786] User B enters "I have plans for a casual outing today" into the app.

[0787] The server generates a casual outfit and sends it to the terminal.

[0788] The device projects the outfit onto a full-length mirror, and AI provides commentary.

[0789] 2. Suggestions and purchases of additional items

[0790] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[0791] The device displays a link, and the user purchases the shoes on the spot.

[0792] If user C wants to buy a winter coat, the server will suggest a coat of the appropriate size based on their body shape data.

[0793] The coat was successfully ordered through the purchase process.

[0794] This invention allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles.

[0795] The following describes the processing flow.

[0796] Step 1:

[0797] The user purchases new clothes, launches the app, and takes a picture of the clothes.

[0798] Step 2:

[0799] The device sends the image of the clothing it has photographed to the server.

[0800] Step 3:

[0801] The server analyzes the received images using image analysis technology to extract information such as type, color, size, and material.

[0802] Step 4:

[0803] The server saves the analyzed information to a database and sends a registration completion notification to the device.

[0804] Step 5:

[0805] The device displays a confirmation message to the user.

[0806] Step 6:

[0807] Users use the app to input their mood for the day and their plans for going out.

[0808] Step 7:

[0809] The terminal sends the entered information to the server.

[0810] Step 8:

[0811] Based on the information received by the server, the system selects the most suitable clothing items from the user's database and generates an outfit coordination.

[0812] Step 9:

[0813] The server generates a coordinated image and a point-by-point explanation, and sends it to the terminal.

[0814] Step 10:

[0815] The device projects an outfit image onto a full-length mirror and presents it to the user.

[0816] Step 11:

[0817] The server checks the outfit and searches the internet for any missing items.

[0818] Step 12:

[0819] The server adds information about the found items to a suggestion list and sends it to the terminal.

[0820] Step 13:

[0821] The device displays a list of suggestions to the user.

[0822] Step 14:

[0823] The server periodically checks the database and analyzes the condition of the clothing.

[0824] Step 15:

[0825] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[0826] Step 16:

[0827] The server suggests appropriate shop items and sends a notification to the device.

[0828] Step 17:

[0829] The device displays a degradation notification and suggested items to the user.

[0830] Step 18:

[0831] The user scans their body shape and size during the initial setup.

[0832] Step 19:

[0833] The device sends the acquired body shape data to the server.

[0834] Step 20:

[0835] The server analyzes body shape data and registers the user's size information in the database.

[0836] Step 21:

[0837] Users purchase items they like through suggestions and notifications.

[0838] Step 22:

[0839] The device initiates the purchase process and prompts the user to enter the required information.

[0840] Step 23:

[0841] The server suggests the optimal size based on body shape data and processes the purchase.

[0842] Step 24:

[0843] The server sends a purchase confirmation message to the device.

[0844] Step 25:

[0845] The device displays a purchase completion message to the user.

[0846] (Example 1)

[0847] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0848] In today's busy lifestyle, it is difficult for users to efficiently manage their clothing and choose the best outfits. Furthermore, there is a lack of assistance in registering newly purchased clothing, suggesting outfits based on the season and mood, checking and suggesting missing items, selecting the optimal size for their body type, and properly replacing worn-out clothing. As a result, users spend a lot of time managing their clothes, leading to problems such as unnecessary purchases and inappropriate outfit choices.

[0849] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0850] In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database; means for suggesting clothing coordinates based on the user's mood and outing plans; means for searching for and suggesting items that are missing from the suggested coordinates; means for acquiring and recording the user's body shape information; means for suggesting the optimal size of the suggested items based on the acquired body shape information; and means for registering the user's clothing using tags or QR codes. As a result, the user can enjoy efficient clothing management and coordinate suggestions, reduce unnecessary purchases, and improve their individual fashion lifestyle.

[0851] A "user" refers to an individual who uses the system to manage their clothing and receive styling suggestions.

[0852] "Image analysis" refers to the technology of extracting information such as type, color, size, and material from images of clothing that have been photographed.

[0853] A "database" refers to a collection of information where analyzed data is stored and searched or updated as needed.

[0854] "Mood and outing plans" refers to the user's daily emotions and activity plans, and is a fundamental data element for suggesting outfit combinations.

[0855] "Outfit suggestions" refers to generating and presenting the optimal clothing combinations based on the user's mood and planned outing.

[0856] "Missing items" refer to clothing or accessories that are missing or need to be added to the suggested outfit.

[0857] "Body shape information" refers to data that details the user's body shape and size, and is obtained through scanning or input.

[0858] "Suggesting the optimal size" means indicating the most suitable size among the suggested items based on the user's body type information.

[0859] "Registering using tags or QR codes" refers to a method of scanning tags or QR codes attached to clothing and registering that information in a database.

[0860] "Deterioration assessment" refers to the automatic determination of the decline in the quality of clothing based on factors such as the frequency of use and age.

[0861] A "notification prompting replacement" refers to a message sent to a user suggesting the purchase of new clothing items that have been deemed to be worn out.

[0862] "Projecting onto a full-length mirror" refers to displaying the suggested outfit on a screen or smart mirror for the user to see.

[0863] This invention relates to an AI-powered system that assists users with clothing management and outfit coordination suggestions. This system is built using a cloud server and the user's device (e.g., a smartphone, tablet, or smart mirror).

[0864] Clothing image analysis and database management

[0865] This system provides a function that allows users to take photos of new clothes they purchase through an app and send the image data to a cloud server. Smartphones and tablets can be used as the devices. The server uses image analysis technologies such as TensorFlow and OpenCV to extract information such as the type, color, size, and material of the clothing and store it in a database. This allows users to review the information later or use it for outfit suggestions.

[0866] Registration via QR code / tag

[0867] Users can easily register information about new clothing by scanning the QR code or tag attached to the garment. The device reads the QR code or tag information and sends it to the server. The server registers the received information in its database and sends the details to the device. The user can then review this information and enter additional details if necessary.

[0868] Coordination suggestions

[0869] The user uses the app to input their mood and outing plans for the day. This can be done using emojis, text input, or dropdown menus. The device sends the input information to a server, which then selects the most suitable clothing based on this information. The server uses a machine learning model (e.g., k-means clustering) to generate an outfit and sends the image and explanation to the device. The device then presents the outfit image to the user and projects it onto a display such as a smart mirror.

[0870] Suggestions for additional items

[0871] The server reviews the generated outfit and searches the internet for any missing items. For example, it searches online shops via API integration and adds appropriate items to the suggestion list. The suggestion list is in JSON format and sent to the device. The device displays the suggestion list to the user and provides purchase links as needed.

[0872] Degradation detection and notification

[0873] The server periodically checks the database and analyzes the deterioration status of clothing. For example, it uses a deterioration model to calculate the deterioration level, taking into account the frequency of use and the time elapsed. If deterioration is advanced, the server generates a notification and sends a message to the device prompting replacement. The device displays this notification to the user and introduces suggested new items.

[0874] Size analysis and purchase procedure

[0875] During the initial setup, the user scans their body shape and size. For example, they use a dedicated body scanning app to obtain 3D data of their body shape. The device sends this data to a server, which analyzes it and registers the user's size information in a database. When the user purchases an item they like through suggestions or notifications, the server suggests the optimal size based on the body shape information and initiates the purchase process. Once the purchase is complete, the server generates a confirmation message and sends it to the device, allowing the user to confirm the purchase.

[0876] Examples of specific cases and prompt statements

[0877] Specific example

[0878] 1. Registering and coordinating clothing items

[0879] A user purchases a new shirt and scans the QR code with the app.

[0880] The server adds and organizes the shirt information in the database.

[0881] Another user enters "I have plans for a casual outing today" into the app.

[0882] The server generates a casual outfit and sends it to the terminal.

[0883] The device projects the outfit onto a smart mirror, and AI provides commentary.

[0884] 2. Suggestions and purchases of additional items

[0885] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[0886] The device displays a link, and the user purchases the shoes on the spot.

[0887] Based on another user's body shape data, the server suggests a coat of the appropriate size.

[0888] The coat was successfully ordered through the purchase process.

[0889] Example of a prompt

[0890] "I have a casual outing planned. Please suggest the best outfit."

[0891] "I purchased a new shirt, so I'm registering my information. I scanned the QR code."

[0892] "I'd like to buy a new winter coat, so please suggest a size that fits my body type."

[0893] As described above, the system of the present invention provides various functions for efficiently and effectively managing the user's clothing and suggesting outfit combinations, thereby improving the user's fashion life.

[0894] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0895] Step 1: Take a photo of your clothes and send it to us.

[0896] The user takes a picture of new clothes. The input for this step is image data of the clothes taken by the user using the camera on their smartphone or tablet. The device receives the captured image and sends it to the server. Specifically, the device launches its camera app, temporarily saves the captured image, and then uploads it to the cloud server. The output is the image data of the clothes that arrives on the server.

[0897] Step 2: Image analysis and saving to database

[0898] The server analyzes the received image data. The input for this step is the image data of the clothing sent to the server. The server uses image analysis techniques (e.g., TensorFlow or OpenCV) to extract information such as type, color, size, and material. Specifically, it runs an image analysis model and extracts features. The output is the analyzed information data, which is stored in a database.

[0899] Step 3: Notification of analysis results

[0900] The server saves the analyzed information to a database and generates a registration completion notification. The input for this step is the image analysis result data. The server uses a REST API to generate the notification message and sends it to the terminal. The output is the registration completion notification message. The terminal receives this notification and displays it to the user.

[0901] Step 4: Read and send the QR code / tag.

[0902] The user enters information about new clothing using a QR code or tag reader. This input is the information from the QR code or tag attached to the clothing. The device scans the QR code or tag and sends the information to the server. Specifically, it uses a tag reader app to retrieve QR code data and uploads it to a cloud server. The output is the QR code or tag information data arriving at the server.

[0903] Step 5: Register QR code / tag information in the database

[0904] The server registers the information from the received QR code or tag into a database. The input for this step is the QR code or tag information sent to the server. The server saves the information in the database and sends the details to the terminal. Specifically, it extracts and transforms the data and registers it in the database. The output is the information saved in the database and the detailed information sent to the terminal.

[0905] Step 6: Enter your mood and schedule

[0906] The user uses the app to input their mood and outing plans for the day. This step involves inputting mood and plan data. The device then sends this information to the server. Specifically, the user records the information using an input form within the app and uploads it to the cloud server by pressing the submit button. The output is the mood and plan data arriving at the server.

[0907] Step 7: Creating the outfit

[0908] The server generates outfits based on the user's mood and schedule information. The input for this step is the user's mood and schedule information. The server uses a machine learning model (e.g., k-means clustering) to select the most suitable clothing from the database and generate the outfit. Specifically, it executes database queries and applies the selection algorithm. The output is the generated outfit image and its explanation.

[0909] Step 8: Send and present your outfit ideas

[0910] The server sends the generated coordinated image and explanation to the terminal. The input for this step is the generated coordinated image and explanation. The terminal receives this and projects it onto a smart mirror or display to show it to the user. Specifically, it receives the image and text data and projects it onto the display device. The output is the coordinated image presented to the user.

[0911] Step 9: Search for and suggest missing items

[0912] The server checks the outfit and searches the internet for any missing items. The input for this step is the generated outfit data. The server searches online shops via API integration and adds appropriate items to the suggestion list. Specifically, it uses an internet search API to retrieve item information and construct the list. The output is the information in the suggestion list.

[0913] Step 10: Notification and display of the suggestion list

[0914] The server sends the suggestion list to the device. The input for this step is the information in the suggestion list. The device receives this and displays it to the user. Specifically, the app displays the received list and provides a purchase link. The output is the suggestion list displayed to the user.

[0915] Step 11: Check and notify about deterioration status

[0916] The server periodically checks the database and analyzes the deterioration status of the clothing. The input for this step is the clothing information in the database. The server uses a deterioration model to analyze the data and generate deterioration reports and notification messages. Specifically, it executes a data analysis algorithm and determines the degree of deterioration. The output is a notification message.

[0917] Step 12: Sending Degradation Notifications and Proposals

[0918] The server sends the generated notification message to the terminal. The input for this step is the degraded notification message. The terminal receives this, displays a notification to the user, and introduces the suggested items. The specific action is to use the notification function to display the message and introduce the new items. The output is the notification message and suggested items that the user receives.

[0919] Step 13: Scan and submit body shape data

[0920] The user scans their body shape data during the initial setup. The input for this step is body shape data acquired by a scanning device (e.g., a body scanning app). The device receives this data and sends it to the server. Specifically, it uses the scanning device to acquire 3D body shape data and uploads it to a cloud server. The output is the body shape data sent to the server.

[0921] Step 14: Analysis and registration of body shape information

[0922] The server analyzes the received body shape data and registers the user's size information in the database. The input for this step is the received body shape data. The server performs data analysis, extracts size information for each body part, and registers it in the database. Specifically, it executes a body shape analysis algorithm to generate and save size data. The output is the size information stored in the database.

[0923] Step 15: Suggestion of the optimal size and purchase procedure

[0924] The user purchases an item through a suggestion or notification. The input for this step is information about the item to be purchased and the user's body shape data. The server suggests the optimal size based on the body shape data and processes the purchase. Specifically, it applies a size suggestion algorithm and handles purchase confirmation and payment. The output is a purchase completion confirmation message.

[0925] Step 16: Send and display purchase completion notification

[0926] The server sends a purchase completion confirmation message to the terminal. The input for this step is the purchase completion information. The terminal receives this and displays the purchase completion message to the user. Specifically, it uses a notification function to display the confirmation message and notify the user of its contents. The output is the purchase completion notification displayed to the user.

[0927] (Application Example 1)

[0928] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0929] Traditional clothing management systems simply stored users' clothing in a database, requiring users to physically try on clothes to actually see the outfits they wanted. Furthermore, there was a lack of systems that automatically suggested appropriate outfits based on the user's body type, mood, and outing plans, as well as suggesting replacements for worn-out clothing. There is also a need for a new virtual try-on experience, allowing users to try on clothes without actually going to a store. To address these challenges, a more efficient and user-friendly clothing management and outfit suggestion system is needed.

[0930] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0931] In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database; means for suggesting clothing coordinates based on the user's mood and outing plans; means for searching for and suggesting items missing from the suggested coordinates; means for acquiring and recording the user's body shape information; means for the user to experience trying on clothes in a virtual fitting room; and means for generating a corresponding virtual avatar and performing clothing fitting simulations. This allows users to check the suitability of clothing through virtual try-ons without having to go to a physical store, and further enables coordinate suggestions based on mood and body shape, as well as suggestions for replacing worn-out clothing.

[0932] A "user" is a person who uses the system and is the person who receives clothing management and coordination suggestions.

[0933] "Clothing" refers to the general term for clothes and accessories that a user owns and wears on a daily basis.

[0934] "Image analysis" is the process of extracting information such as the type, color, size, and material of clothing from a captured image.

[0935] A "database" is an information system for systematically organizing and storing information about analyzed clothing.

[0936] "Mood" refers to the emotions and mood that the user is feeling at any given time, and it is one of the criteria used for suggesting outfits.

[0937] "Outing plans" refer to the activities and schedules that a user has planned for a specific day.

[0938] "Outfit suggestions" refer to the system suggesting the optimal clothing combination based on the user's mood and planned outing.

[0939] "Suggested outfit" refers to the result of the clothing combinations generated by the system.

[0940] "Missing items" refer to clothing or accessories that are necessary for the suggested outfit but that the user does not currently own.

[0941] "Searching" is the process of using the internet or databases to find out whether or not specific information or products exist.

[0942] "Body shape information" refers to information about various body dimensions, such as the user's height, weight, shoulder width, and waist size.

[0943] A "virtual fitting room" is a simulation area where users can virtually try on clothes.

[0944] A "virtual avatar" is a virtual human model created based on the user's body shape information.

[0945] A "fitting simulation" is a process that allows users to visually check how actual clothing will look on a virtual avatar.

[0946] The present invention relates to a system that supports a user's clothing management and coordination suggestions. This system includes means for image analysis of the user's clothing and storing the information in a database, means for suggesting clothing coordinates based on the user's mood and outing plans, means for searching for and suggesting items that are missing from the suggested coordinates, means for acquiring and recording the user's body shape information, means for the user to experience trying on clothes in a virtual fitting room, and means for generating a corresponding virtual avatar and performing clothing fitting simulations.

[0947] System Configuration

[0948] Image analysis and database management

[0949] 1. Register your clothing:

[0950] The user takes a picture of new clothing using their smartphone camera and sends it to their device.

[0951] The server analyzes the received images using image analysis techniques to extract information such as type, color, size, and material. Specifically, it uses Python's image processing library (PIL) and request library (requests).

[0952] The server saves the analyzed information to a database (e.g., MySQL or PostgreSQL) and sends a completion notification to the terminal.

[0953] Coordination suggestions

[0954] 1. Enter your mood and schedule:

[0955] Users use a smartphone app to input their mood and outing plans for the day.

[0956] The terminal sends the entered information to the server.

[0957] 2. Coordination generation:

[0958] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database.

[0959] The server sends the generated outfit image and its key points to the terminal.

[0960] The device projects the coordinated look onto a virtual fitting room (for example, smart glasses or a head-mounted display) and presents it to the user.

[0961] Suggestions for missing items

[0962] 1. Search for and suggest missing items:

[0963] The server checks the generated outfit and searches the internet for any missing items.

[0964] The server adds information about the found items to a suggestion list and sends it to the terminal.

[0965] The device displays a list of suggestions to the user.

[0966] Virtual fitting

[0967] 1. Virtual try-on experience:

[0968] When a user enters a virtual fitting room, the server generates a virtual avatar based on the user's body shape information.

[0969] The generated outfit is applied to the virtual avatar, and a fitting simulation is performed.

[0970] The device (for example, smart glasses or a head-mounted display) visually displays the simulation results to the user.

[0971] Specific example

[0972] 1. Example of a prompt:

[0973] "Please suggest an outfit that's perfect for a casual day trip."

[0974] "I'm in a good mood today and have plans to go out. What would be the best outfit for me?"

[0975] Usage example

[0976] User A purchases a new shirt and scans the QR code with their smartphone camera. The server analyzes the shirt's information and adds it to the database.

[0977] User B enters "I have a casual outing planned for today" into the app. The server follows a series of processes, generates a casual outfit, and displays it in a virtual fitting room.

[0978] User C tries on clothes in a virtual fitting room, reviews the simulation results, and makes a final decision.

[0979] This invention allows users to efficiently and effectively manage their clothing and receive styling suggestions without having to go to a physical store, reducing unnecessary purchases and improving their fashion lifestyle.

[0980] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0981] Step 1:

[0982] Image analysis and registration of clothing

[0983] Input: Images of new clothing items taken by the user with their smartphone camera.

[0984] Specific action: The user launches the app and takes a picture of the clothing.

[0985] Data processing / calculation: The device sends an image to the cloud server. The server analyzes the image using image analysis techniques (PIL, requests library, etc.) and extracts information about the type, color, size, and material.

[0986] Output: The analyzed clothing information is saved to the database, and a notification of completion is sent to the terminal.

[0987] Step 2:

[0988] Enter your mood and schedule.

[0989] Input: The user enters their mood and outing plans for the day into the app.

[0990] Specific action: The user enters their mood or plans as text into a form within the app.

[0991] Data processing / calculation: The terminal sends the input information to the server. The server analyzes the information and compares it with the user's database.

[0992] Output: Mood and schedule information is saved to the server.

[0993] Step 3:

[0994] Coordination generation

[0995] Input: Information about mood and outing plans, and the user's clothing database.

[0996] Specific operation: The server selects the most suitable clothing from the user's database based on the stored mood and schedule information.

[0997] Data processing / calculation: The server automatically generates outfits using a generation AI model, creating outfit images and explanations of their key points.

[0998] Output: The generated coordinated image and explanation are sent to the terminal.

[0999] Step 4:

[1000] Virtual Fitting Experience

[1001] Input: Generated outfit image and user body type information.

[1002] Specific operation: The user enters a virtual fitting room. The device (e.g., smart glasses or head-mounted display) sends the user's body shape information to the server.

[1003] Data processing / calculation: The server generates a virtual avatar based on body shape information and reflects the outfit on the avatar.

[1004] Output: The fitting simulation results are displayed on the terminal and provided to the user visually.

[1005] Step 5:

[1006] Search and suggestion of missing items

[1007] Input: A database of generated coordinates and users.

[1008] Specific operation: The server checks the outfit and searches for missing items on online shopping sites and databases.

[1009] Data processing / calculation: The server adds the found items to a suggestion list and generates links and detailed information.

[1010] Output: The suggestion list is sent to the terminal and displayed to the user.

[1011] Step 6:

[1012] Deterioration assessment and replacement notification

[1013] Input: User's clothing database.

[1014] Specific operation: The server periodically checks the database and analyzes the deterioration status of the clothing.

[1015] Data processing / calculation: The server uses a deterioration detection algorithm to identify deteriorated clothing and generates replacement suggestions.

[1016] Output: Notifications and suggestions are sent to the device and displayed to the user.

[1017] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1018] This invention relates to an AI-powered closet and mirror system that assists users with clothing management and outfit suggestions. The system is built by combining a cloud server, a terminal (e.g., a smartphone, tablet, or mirror display), and an emotion engine that recognizes the user's emotions.

[1019] Clothing image analysis and database management

[1020] 1. Register your clothes

[1021] The user buys new clothes, launches the app, and takes a picture of the clothes.

[1022] The device sends the captured image of the clothing to the server.

[1023] The server analyzes the received images using image analysis technology and extracts information such as type, color, size, and material.

[1024] The server saves the analyzed information to a database and sends a registration completion notification to the terminal.

[1025] The device displays a registration completion message to the user.

[1026] 2. Registration via QR code / tag

[1027] Users register new clothing items using QR codes or tag readers, hence the use of a tag reader.

[1028] The device reads the information from the QR code or tag and sends it to the server.

[1029] The server registers the received information in its database and sends the details to the terminal.

[1030] The device displays information and prompts the user for confirmation.

[1031] Coordination suggestions

[1032] 1. Enter your mood and schedule.

[1033] Users use the app to input their mood for the day and their plans for going out.

[1034] The terminal sends the entered information to the server.

[1035] 2. Coordination generation

[1036] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database and generates an outfit.

[1037] The server generates clothing combinations as coordinated outfits and creates images based on them.

[1038] The server sends a coordinated image and an explanation of its key points to the terminal.

[1039] The device projects an outfit image onto a full-length mirror and presents it to the user.

[1040] Suggestions for additional items

[1041] 1. Suggestions for missing items

[1042] The server checks the generated outfit and searches the internet for any missing items.

[1043] The server adds information about the found items to a suggestion list and sends it to the terminal.

[1044] The device displays a list of suggestions to the user.

[1045] 2. Deterioration detection and notification

[1046] The server periodically checks the database and analyzes the condition of the clothing.

[1047] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[1048] The server suggests appropriate shop items and sends a notification to the device.

[1049] The device displays a degradation notification and suggested items to the user.

[1050] Size analysis and purchase procedure

[1051] 1. Body shape and size analysis

[1052] The user scans their body shape and size during the initial setup (for example, using a body scan app).

[1053] The device sends the acquired body shape data to the server.

[1054] The server analyzes body shape data and registers the user's size information in the database.

[1055] 2. Purchase Procedure

[1056] Users purchase items they like through suggestions and notifications.

[1057] The device will begin the purchase process and you will enter the required information.

[1058] The server suggests the optimal size based on body shape data and processes the purchase.

[1059] The server sends a purchase confirmation message to the device.

[1060] The device displays a purchase completion message to the user.

[1061] Introducing an emotional engine

[1062] 1. Recognition of emotions

[1063] The device uses a built-in emotion engine to analyze the user's facial expressions and voice in real time.

[1064] The server recognizes the user's current emotions based on the data sent from the emotion engine.

[1065] 2. Adjusting coordination based on emotions

[1066] The server adjusts the outfit suggestions based on the recognized user's emotional data.

[1067] The server generates the optimal outfit based on the user's mood, schedule, and emotional data.

[1068] The device projects an optimized outfit onto a full-length mirror and presents it to the user.

[1069] Specific example

[1070] 1. Registering and coordinating clothing items

[1071] User A buys a new shirt and scans the QR code with the app.

[1072] The server adds and organizes the shirt information in the database.

[1073] User B enters "I have plans for a casual outing today" into the app.

[1074] The server generates a casual outfit and sends it to the terminal.

[1075] The device projects the outfit onto a full-length mirror, and AI provides commentary.

[1076] The emotion engine recognizes the user's facial expressions and detects the emotion of "looking happy."

[1077] The server adjusts the outfit based on emotional data, adding colors and styles.

[1078] 2. Suggestions and purchases of additional items

[1079] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[1080] The device displays a link, and the user purchases the shoes on the spot.

[1081] If user C wants to buy a winter coat, the server will suggest a coat of the appropriate size based on their body shape data.

[1082] The coat was successfully ordered through the purchase process.

[1083] This invention allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles. Furthermore, the introduction of an emotion engine enables more personalized suggestions, enhancing the user experience.

[1084] The following describes the processing flow.

[1085] Step 1:

[1086] The user purchases new clothes, launches the app, and takes a picture of the clothes.

[1087] Step 2:

[1088] The device sends the image of the clothing it has photographed to the server.

[1089] Step 3:

[1090] The server analyzes the received images using image analysis technology to extract information such as type, color, size, and material.

[1091] Step 4:

[1092] The server saves the analyzed information to a database and sends a registration completion notification to the device.

[1093] Step 5:

[1094] The device displays a confirmation message to the user.

[1095] Step 6:

[1096] Users use the app to input their mood for the day and their plans for going out.

[1097] Step 7:

[1098] The terminal sends the entered information to the server.

[1099] Step 8:

[1100] The device uses its built-in emotion engine to analyze the user's facial expressions and voice in real time.

[1101] Step 9:

[1102] The server recognizes the user's current emotions based on the data sent from the emotion engine.

[1103] Step 10:

[1104] Based on the mood, schedule, and emotional data received by the server, the system selects the most suitable clothing from the user's database.

[1105] Step 11:

[1106] The server generates clothing combinations as coordinated outfits and creates images based on them.

[1107] Step 12:

[1108] The server sends a coordinated image and an explanation of its key points to the terminal.

[1109] Step 13:

[1110] The device projects an outfit image onto a full-length mirror and presents it to the user.

[1111] Step 14:

[1112] The server checks the outfit and searches the internet for any missing items.

[1113] Step 15:

[1114] The server adds information about the found items to a suggestion list and sends it to the terminal.

[1115] Step 16:

[1116] The device displays a list of suggestions to the user.

[1117] Step 17:

[1118] The server periodically checks the database and analyzes the condition of the clothing.

[1119] Step 18:

[1120] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[1121] Step 19:

[1122] The server suggests appropriate shop items and sends a notification to the device.

[1123] Step 20:

[1124] The device displays a degradation notification and suggested items to the user.

[1125] Step 21:

[1126] The user scans their body shape and size during the initial setup (for example, by using a body scan app).

[1127] Step 22:

[1128] The device sends the acquired body shape data to the server.

[1129] Step 23:

[1130] The server analyzes body shape data and registers the user's size information in the database.

[1131] Step 24:

[1132] Users purchase items they like through suggestions and notifications.

[1133] Step 25:

[1134] The device initiates the purchase process and prompts the user to enter the required information.

[1135] Step 26:

[1136] The server suggests the optimal size based on body shape data and processes the purchase.

[1137] Step 27:

[1138] The server sends a purchase confirmation message to the device.

[1139] Step 28:

[1140] The device displays a purchase completion message to the user.

[1141] (Example 2)

[1142] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1143] Conventional clothing management systems lacked sufficient functionality to allow users to efficiently manage their clothing and receive optimal outfit suggestions. Furthermore, they lacked the ability to suggest outfits that considered the user's mood and feelings, and to provide replacement advice based on the condition of their clothing. As a result, users often made unnecessary purchases or struggled with coordinating outfits. This invention aims to solve these problems and provide a system that allows users to enjoy smarter, more personalized clothing management and outfit suggestions.

[1144] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1145] In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database; means for suggesting clothing coordinates based on the user's mood and outing plans; means for searching the internet for and suggesting items that are missing from the suggested coordinates; means for acquiring and recording the user's body shape information; means for analyzing the user's facial expressions and voice to recognize emotions; and means for adjusting the coordinate suggestions based on the recognized emotions. As a result, the user can receive personalized coordinate suggestions, reduce unnecessary purchases, and enjoy their daily fashion life more.

[1146] "Image analysis of clothing" involves analyzing images of clothing taken by the user using the app and extracting information such as the type, color, size, and material of the clothing.

[1147] "Saving to a database" means recording the analyzed clothing information in a cloud-based database so that it can be referenced and managed later.

[1148] "Suggesting clothing coordinates based on mood and outing plans" means suggesting the most suitable outfit combination based on the user's mood and plans for the day.

[1149] "Searching online and making suggestions" means finding items on the internet that are missing from the suggested outfit and recommending them to the user.

[1150] "Acquiring and recording body shape information" means analyzing body shape data obtained by the user through body scans, etc., and saving that information to a database.

[1151] "Recognizing emotions by analyzing facial expressions and voice" means determining what emotions the user is currently experiencing by analyzing their facial expressions and tone of voice.

[1152] "Adjusting outfit suggestions based on recognized emotions" means further personalizing outfits based on the user's emotional information and suggesting the most suitable clothing for the user's mental state.

[1153] This invention relates to an AI-powered closet and mirror system that assists users with clothing management and outfit suggestions. The system is built by combining a cloud server, a terminal (smartphone, tablet, or mirror display), and an emotion engine that recognizes the user's emotions.

[1154] Clothing image analysis and database management

[1155] Clothing registration

[1156] The user purchases new clothes, launches the app, and takes a picture of the clothes. The device sends the image of the clothes to the server. The server analyzes the received image using image analysis techniques (e.g., OpenCV and TensorFlow) and extracts information such as type, color, size, and material. The server saves the analyzed information to a database (e.g., MySQL) and sends a registration completion notification to the device. The device displays a registration completion message to the user.

[1157] Registration via QR code / tag

[1158] The user registers new clothing items using a QR code or tag reader, utilizing a tag reader. The device reads the information from the QR code or tag and sends it to the server. The server registers the received information in its database and sends the details to the device. The device uses the Zxing library to read the QR code information. The device displays the information and prompts the user for confirmation.

[1159] Coordination suggestions

[1160] Enter your mood and schedule.

[1161] The user uses the app to input their mood for the day and their outing plans. The device then sends the entered information to the server.

[1162] Coordination generation

[1163] Based on the mood and schedule information received by the server, the server selects the most suitable clothing from the user's database and generates an outfit. The server uses an AI model developed in Python to suggest outfits based on the user's past preference data and input information. The server generates clothing combinations as outfits and generates an image based on them. GANs (Generative Opposite Networks) are used for image generation. The server sends the outfit image and its key points to the terminal. The terminal projects the outfit image onto a full-length mirror and presents it to the user.

[1164] Suggestions for additional items

[1165] Suggestions for missing items

[1166] The server checks the generated outfit and searches the internet for any missing items. The server uses a Python web scraping library (such as Scrapy) to search for recommended items. The server adds information about the found items to a suggestion list and sends it to the terminal. The terminal displays the suggestion list to the user.

[1167] Degradation detection and notification

[1168] The server periodically checks the database and analyzes the deterioration status of clothing. The server creates deterioration prediction models using random forests and support vector machines. The server generates notifications for clothing that is deteriorating and creates messages encouraging replacement. The server suggests appropriate shop items and sends notifications to the device. The device displays the deterioration notification and suggested items to the user.

[1169] Size analysis and purchase procedure

[1170] Body shape and size analysis

[1171] The user scans their body shape and size during initial setup (for example, using a body scan app). The device sends the acquired body shape data to the server. The server processes the 3D scan data and analyzes the user's body shape. This is done using 3D modeling software such as MeshLab. The server stores the body shape data in a database.

[1172] Purchase procedure

[1173] The user purchases items they like through suggestions and notifications. The device initiates the purchase process and the user enters the necessary information. The server matches the user's body size data with product data, suggests the optimal size, and processes the purchase. The server sends a purchase completion confirmation message to the device. The device displays a purchase completion message to the user.

[1174] Introducing an emotional engine

[1175] Recognition of emotions

[1176] The device uses its built-in emotion engine to analyze the user's facial expressions and voice in real time. The device analyzes emotions using a facial recognition API (e.g., Google Cloud Vision API) or a speech recognition API. The server recognizes the user's current emotions based on the data sent from the emotion engine.

[1177] Adjusting coordination based on emotions

[1178] The server adjusts outfit suggestions based on the recognized user's emotional data. The server generates an optimal outfit that reflects the user's mood, schedule, and emotional data. The device projects the optimized outfit onto a full-length mirror and presents it to the user.

[1179] Specific example

[1180] Prompt message

[1181] The following is an example of a prompt message to use when inputting a specific command into a generating AI model.

[1182] "Please suggest a casual style for today."

[1183] "Please suggest outfit ideas using this shirt."

[1184] "Please recommend some clothes that suit my current mood."

[1185] This allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles. Furthermore, the introduction of an emotional engine enables more personalized suggestions, enhancing the user experience.

[1186] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1187] Processing steps

[1188] Clothing image analysis and database management

[1189] Step 1:

[1190] The user purchases new clothes, launches the app, and takes a picture of the clothes.

[1191] Input: New clothes, smartphone app

[1192] Output: Images of the photographed clothing

[1193] Step 2:

[1194] The device sends the image of the clothing it has photographed to the server.

[1195] Input: Images of clothing taken

[1196] Output: Image data transferred to the server

[1197] Step 3:

[1198] The server analyzes the received images using image analysis techniques (OpenCV and TensorFlow) to extract information such as type, color, size, and material.

[1199] Input: Transferred image data

[1200] Output: Image analysis results including type, color, size, and material.

[1201] Specific operation: Extract feature points from an image using OpenCV and classify them using TensorFlow.

[1202] Step 4:

[1203] The server analyzes the information, saves it to a database (MySQL), and sends a registration completion notification to the terminal.

[1204] Input: Analysis results

[1205] Output: Clothing information stored in the database, registration completion notification

[1206] Specific operation: Insert information into the database using SQL queries.

[1207] Step 5:

[1208] The device displays a registration completion message to the user.

[1209] Input: Registration completion notification

[1210] Output: Completion message displayed to the user

[1211] Registration via QR code / tag

[1212] Step 1:

[1213] A tag reader is used so that users can register new clothing items using QR codes or tag readers.

[1214] Input: New clothes, tag reader

[1215] Output: Information from the scanned QR code or tag

[1216] Step 2:

[1217] The device reads the information from the QR code or tag and sends it to the server. The Zxing library is used for reading.

[1218] Input: QR code or tag information

[1219] Output: Code information transferred to the server

[1220] Step 3:

[1221] The server registers the received information in the database and sends the details to the terminal.

[1222] Input: Transferred code information

[1223] Output: Clothing information stored in the database, notification of detailed information.

[1224] Step 4:

[1225] The device displays information and prompts the user for confirmation.

[1226] Input: Notification of detailed information

[1227] Output: Detailed information and confirmation message displayed to the user

[1228] Enter your mood and schedule.

[1229] Step 1:

[1230] Users use the app to input their mood for the day and their plans for going out.

[1231] Input: Mood, schedule information, smartphone app

[1232] Output: Information entered into the terminal

[1233] Step 2:

[1234] The terminal sends the entered information to the server.

[1235] Input: Entered information

[1236] Output: Mood and schedule information transferred to the server

[1237] Coordination generation

[1238] Step 1:

[1239] Based on the mood and schedule information received by the server, the system selects the most suitable clothing from the user's database and generates an outfit. An AI model developed in Python is used.

[1240] Input: Information about mood and schedule, clothing information in the database.

[1241] Output: Optimal coordination suggestion

[1242] Specific operation: Select based on past preference data and input information.

[1243] Step 2:

[1244] The server generates clothing combinations as coordinated outfits and then generates images based on these combinations. Image generation utilizes GANs (Generative Opposite-Guard Networks).

[1245] Input: Coordination suggestion

[1246] Output: Generated coordinated image

[1247] Specific operation: Create an image of the outfit using a GAN model.

[1248] Step 3:

[1249] The server sends a coordinated image and an explanation of its key points to the terminal.

[1250] Input: Generated coordinated image

[1251] Output: Coordinated image and explanation sent to the terminal

[1252] Step 4:

[1253] The device projects an outfit image onto a full-length mirror and presents it to the user.

[1254] Input: Submitted outfit image and description

[1255] Output: Coordinated outfit image and explanation displayed to the user.

[1256] Suggestions for missing items

[1257] Step 1:

[1258] The server checks the generated outfit and searches the internet for any missing items. A Python web scraping library (such as Scrapy) is used.

[1259] Input: Generated coordination information

[1260] Output: List of missing items

[1261] Specific operation: Perform web scraping and search for corresponding items.

[1262] Step 2:

[1263] The server adds information about the found items to a suggestion list and sends it to the terminal.

[1264] Input: List of missing items

[1265] Output: Information from the proposal list

[1266] Step 3:

[1267] The device displays a list of suggestions to the user.

[1268] Input: Information from the proposal list

[1269] Output: Suggestion list displayed to the user

[1270] Degradation detection and notification

[1271] Step 1:

[1272] The server periodically checks the database and analyzes the deterioration status of the clothing. A deterioration prediction model is created using random forests or support vector machines.

[1273] Input: Clothing data in the database

[1274] Output: Degradation judgment result

[1275] Specific operation: Predicting the deterioration status using a machine learning model.

[1276] Step 2:

[1277] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[1278] Input: Degradation judgment result

[1279] Output: Notification message

[1280] Step 3:

[1281] The server suggests appropriate shop items and sends a notification to the device.

[1282] Input: Notification message

[1283] Output: Proposed item information

[1284] Step 4:

[1285] The device displays a degradation notification and suggested items to the user.

[1286] Input: Suggested item information

[1287] Output: Deterioration notification and item information displayed to the user.

[1288] Body shape and size analysis

[1289] Step 1:

[1290] The user scans their body shape and size during the initial setup (for example, by using a body scan app).

[1291] Input: Body shape and size scan data

[1292] Output: Scan data

[1293] Step 2:

[1294] The device sends the acquired body shape data to the server.

[1295] Input: Scanned data

[1296] Output: Body shape data transferred to the server

[1297] Step 3:

[1298] The server processes the 3D scan data and analyzes the user's body shape. 3D modeling software such as MeshLab is used.

[1299] Input: Transferred body shape data

[1300] Output: Analyzed body shape information

[1301] Specific operation: Perform 3D modeling and analyze the user's body shape as digital data.

[1302] Step 4:

[1303] The server saves body shape data to a database.

[1304] Input: Analyzed body shape information

[1305] Output: Body shape information stored in the database

[1306] Purchase procedure

[1307] Step 1:

[1308] Users purchase items they like through suggestions and notifications.

[1309] Input: Suggestion and notification data

[1310] Output: Purchase Decision

[1311] Step 2:

[1312] The device will begin the purchase process and enter the required information.

[1313] Input: Purchase decision, required information

[1314] Output: Entered purchase information

[1315] Step 3:

[1316] The server matches the user's body shape data with product data, suggests the optimal size, and processes the purchase.

[1317] Input: Entered purchase information, body type data

[1318] Output: Optimal size suggestion, purchase process

[1319] Specific operation: Match body shape data with product size data to select the optimal size.

[1320] Step 4:

[1321] The server sends a purchase confirmation message to the device.

[1322] Input: Purchase process complete

[1323] Output: Purchase completion confirmation message

[1324] Step 5:

[1325] The device displays a purchase completion message to the user.

[1326] Input: Purchase completion confirmation message

[1327] Output: Purchase completion message displayed to the user

[1328] Introducing an emotional engine

[1329] Step 1:

[1330] The device uses its built-in emotion engine to analyze the user's facial expressions and voice in real time. It utilizes a facial recognition API (Google Cloud Vision API) and a speech recognition API.

[1331] Input: User's facial expressions, voice data

[1332] Output: Sentiment data as analysis results

[1333] Specific operation: The analysis terminal processes facial and voice data and extracts emotional information.

[1334] Step 2:

[1335] The server recognizes the user's current emotions based on the data sent from the emotion engine.

[1336] Input: Sentiment data

[1337] Output: User sentiment analysis results

[1338] Step 3:

[1339] The server adjusts outfit suggestions based on the recognized user's emotional data.

[1340] Input: Sentiment analysis results

[1341] Output: Adjusted coordination suggestions

[1342] Specific actions: Consider emotional data and optimize suggestions using an AI model.

[1343] Step 4:

[1344] The device projects an optimized outfit onto a full-length mirror and presents it to the user.

[1345] Input: Adjusted coordination suggestions

[1346] Output: Optimized coordinates displayed to the user

[1347] (Application Example 2)

[1348] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1349] In recent years, interest in fashion has increased, but it remains difficult for users to efficiently manage their clothing and receive outfit suggestions. In particular, there is no system that suggests outfits based on the user's mood or outing plans, or that provides optimal suggestions based on their feelings on a given day. Furthermore, the process of searching for and purchasing missing items in suggested outfits is also cumbersome. As a result of this situation, users often end up making unnecessary purchases, and there is a need for efficient means to improve the quality of individual fashion lives.

[1350] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database, means for suggesting clothing coordinates based on the user's mood and outing plans, and means for searching for and suggesting items that are missing from the suggested coordinates. As a result, the user can efficiently manage their clothing and receive appropriate coordinates according to their mood and plans for the day.

[1351] A "user" is an individual who uses the system to manage their own clothing and receive styling suggestions.

[1352] "Clothing" refers to fashion items owned by the user and registered in the system through image analysis.

[1353] "Image analysis" is a technology that extracts information such as type, color, size, and material from images of clothing that have been photographed.

[1354] A "database" is a system that organizes and stores information such as a user's clothing and the results of its analysis.

[1355] "Mood" refers to the user's emotions and psychological state on a given day, and is an element that is reflected in the outfit suggestions.

[1356] "Planned outings" refer to the activities and places the user has planned to visit that day, and are factors that influence outfit suggestions.

[1357] "Coordination" refers to suggestions for clothing combinations for the user, which are optimized based on their mood and planned outings.

[1358] "Searching" refers to the process of finding items online that are missing from a suggested outfit.

[1359] "Body shape information" refers to data about the user's body size and shape, and is used for suggesting outfits and selecting clothing sizes.

[1360] "Emotions" refer to the psychological state perceived from the user's facial expressions and voice, and are used to optimize outfit suggestions.

[1361] A "cloud server" is a server that performs data analysis, storage, and coordination proposals.

[1362] An "emotion recognition engine" is a technology that analyzes emotions based on the user's facial expressions and voice.

[1363] This invention relates to an AI-powered system that assists users with clothing management and outfit coordination suggestions. This system is built by combining a cloud server, a terminal (e.g., a smartphone), and an emotion recognition engine.

[1364] System Programs and Processing

[1365] 1. Register your clothes

[1366] Users use their smartphone camera to photograph new clothing items and upload the images to a cloud server. Alternatively, they can scan a QR code to automatically register clothing information. The cloud server uses image analysis technology (e.g., Google Cloud Vision API) to extract information such as clothing type, color, size, and material from the uploaded images. The extracted information is stored in a database (e.g., Amazon RDS).

[1367] 2. Enter your mood and schedule

[1368] Users use a smartphone app to input their mood and outing plans for the day. This information is sent to a cloud server, where it is cross-referenced with the user's database to generate appropriate outfit suggestions.

[1369] 3. Coordination generation

[1370] The cloud server generates the optimal outfit based on the user's mood and outing plans. An image and explanation of the generated outfit are created and sent to the smartphone.

[1371] 4. Recognition and regulation of emotions

[1372] The smartphone uses its built-in camera and microphone to analyze the user's facial expressions and voice in real time, and an emotion recognition engine (such as Microsoft Azure Emotion API) determines their emotions. The cloud server receives the data obtained from the emotion recognition engine and optimizes the outfit based on the user's emotional state.

[1373] 5. Suggestions for purchasing additional items

[1374] The cloud server searches the internet for items missing from the suggested outfit. Information on the appropriate items is generated, and the user can view the list on their smartphone and proceed with purchasing the items they like. This purchase process uses an electronic payment service (e.g., Stripe API).

[1375] Examples of specific cases and prompt statements

[1376] As a concrete example, a user purchases a new shirt and registers it by scanning a QR code with the app. The user enters their mood for the day (e.g., "Fresh") and their plans for going out (e.g., "Lunch with friends") into the app. Based on this information, the cloud server suggests a "bright shirt" and "white pants," and displays an image of the outfit on the smartphone screen. The emotion recognition engine analyzes the user's facial expression, and if it recognizes that the user looks "happy," additional suggestions such as "bright-colored accessories" are made. Furthermore, a list of suggested items that are missing (e.g., "summer sandals") is displayed on the smartphone, and the user completes the purchase process on the spot.

[1377] Examples of prompt statements are as follows:

[1378] "Users registered their new clothes using the app. After entering their mood and plans for the day, the server suggested the most suitable outfit. It recognized the user's emotions in real time and adjusted the outfit accordingly. Furthermore, it purchased any missing items based on the suggested list."

[1379] As described above, the present invention allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles. Furthermore, the introduction of an emotion recognition engine enables more personalized suggestions, improving the user experience.

[1380] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1381] Step 1:

[1382] Users register new clothing items.

[1383] Users take photos of new clothing items using their smartphone camera and upload the images to a cloud server via the app. Alternatively, they can scan a QR code to automatically register clothing information.

[1384] Input: Image or QR code information of the clothing item

[1385] Data processing: Image analysis technology (Google Cloud Vision API) is used to extract information on clothing type, color, size, and material.

[1386] Output: Extracted clothing information is saved to a database (Amazon RDS).

[1387] Step 2:

[1388] Users enter their mood and outing plans.

[1389] Users use a smartphone app to input their mood and outing plans for the day. This information is sent to a cloud server.

[1390] Input: User's mood and outing plans

[1391] Data processing: Matching entered information with the user database.

[1392] Output: Coordination suggestions based on matching results

[1393] Step 3:

[1394] The cloud server generates the coordinates.

[1395] The cloud server generates the optimal outfit based on the user's mood and outing plans. An image and explanation of the generated outfit are created and sent to the smartphone.

[1396] Input: User's mood and outing plans, clothing database

[1397] Data processing: Generates optimal outfits and creates outfit images and explanations.

[1398] Output: Send coordination image and explanation to your smartphone

[1399] Step 4:

[1400] Smartphones recognize user emotions.

[1401] The smartphone uses its built-in camera and microphone to analyze the user's facial expressions and voice in real time. The emotion recognition engine (Microsoft Azure Emotion API) determines the user's emotions.

[1402] Input: User's facial expressions and voice

[1403] Data processing: Emotions are determined using an emotion recognition engine.

[1404] Output: Send the determined emotion data to the cloud server.

[1405] Step 5:

[1406] Cloud servers adjust outfits based on emotions.

[1407] The cloud server optimizes the coordination based on the user's emotional state, using data obtained from the emotion recognition engine.

[1408] Input: User sentiment data, initial outfit suggestions

[1409] Data processing: Adjusting outfits according to emotions

[1410] Output: Optimized outfit resend to smartphone

[1411] Step 6:

[1412] The cloud server suggests additional items.

[1413] The cloud server searches the internet for items missing from the suggested outfit. It generates information on the appropriate items, and the user can view the list on their smartphone.

[1414] Input: Outfit suggestions, item database

[1415] Data processing: Search the internet for missing items and generate item information.

[1416] Output: Send item information to smartphone

[1417] Step 7:

[1418] Users purchase additional items

[1419] The user selects their preferred items from a suggested list on their smartphone and proceeds with the purchase. The purchase is completed using an electronic payment service.

[1420] Input: Suggested item list, purchase information

[1421] Data processing: Completion of the purchase process using the electronic payment service (Stripe API).

[1422] Output: Purchase confirmation and completion notification sent to your smartphone.

[1423] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1424] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1425] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1426] [Third Embodiment]

[1427] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1428] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1429] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1430] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1431] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1432] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1433] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1434] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1435] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1436] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1437] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1438] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1439] This invention relates to an AI-powered closet and full-length mirror system that assists users with clothing management and outfit suggestions. This system is built using a cloud server and a terminal (e.g., a smartphone, tablet, or full-length mirror display).

[1440] Clothing image analysis and database management

[1441] 1. Register your clothes

[1442] The user buys new clothes, launches the app, and takes a picture of the clothes.

[1443] The device sends the captured image of the clothing to the server.

[1444] The server analyzes the received images using image analysis technology and extracts information such as type, color, size, and material.

[1445] The server saves the analyzed information to a database and sends a confirmation notification to the terminal.

[1446] The device displays a registration completion message to the user.

[1447] 2. Registration via QR code / tag

[1448] Users register new clothing items using QR codes or tag readers, hence the use of a tag reader.

[1449] The device reads the information from the QR code or tag and sends it to the server.

[1450] The server registers the received information in its database and sends the details to the terminal.

[1451] The device displays information and prompts the user for confirmation.

[1452] Coordination suggestions

[1453] 1. Enter your mood and schedule.

[1454] Users use the app to input their mood for the day and their plans for going out.

[1455] The terminal sends the entered information to the server.

[1456] 2. Coordination generation

[1457] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database.

[1458] The server generates clothing combinations as coordinated outfits and creates images based on them.

[1459] The server sends a coordinated image and an explanation of its key points to the terminal.

[1460] The device projects an outfit image onto a full-length mirror and presents it to the user.

[1461] Suggestions for additional items

[1462] 1. Suggestions for missing items

[1463] The server checks the generated outfit and searches the internet for any missing items.

[1464] The server adds information about the found items to a suggestion list and sends it to the terminal.

[1465] The device displays a list of suggestions to the user.

[1466] 2. Deterioration detection and notification

[1467] The server periodically checks the database and analyzes the condition of the clothing.

[1468] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[1469] The server suggests appropriate shop items and sends them to the terminal.

[1470] The device displays a notification to the user and introduces suggested items.

[1471] Size analysis and purchase procedure

[1472] 1. Body shape and size analysis

[1473] The user scans their body shape and size during the initial setup (for example, using a body scan app).

[1474] The device sends the acquired body shape data to the server.

[1475] The server analyzes body shape data and registers the user's size information in the database.

[1476] 2. Purchase Procedure

[1477] Users purchase items they like through suggestions and notifications.

[1478] The device will begin the purchase process and you will enter the required information.

[1479] The server suggests the optimal size based on body shape data and processes the purchase.

[1480] The server sends a purchase confirmation message to the device.

[1481] The device displays a purchase completion message to the user.

[1482] Specific example

[1483] 1. Registering and coordinating clothing items

[1484] User A buys a new shirt and scans the QR code with the app.

[1485] The server adds and organizes the shirt information in the database.

[1486] User B enters "I have plans for a casual outing today" into the app.

[1487] The server generates a casual outfit and sends it to the terminal.

[1488] The device projects the outfit onto a full-length mirror, and AI provides commentary.

[1489] 2. Suggestions and purchases of additional items

[1490] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[1491] The device displays a link, and the user purchases the shoes on the spot.

[1492] If user C wants to buy a winter coat, the server will suggest a coat of the appropriate size based on their body shape data.

[1493] The coat was successfully ordered through the purchase process.

[1494] This invention allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles.

[1495] The following describes the processing flow.

[1496] Step 1:

[1497] The user purchases new clothes, launches the app, and takes a picture of the clothes.

[1498] Step 2:

[1499] The device sends the image of the clothing it has photographed to the server.

[1500] Step 3:

[1501] The server analyzes the received images using image analysis technology to extract information such as type, color, size, and material.

[1502] Step 4:

[1503] The server saves the analyzed information to a database and sends a registration completion notification to the device.

[1504] Step 5:

[1505] The device displays a confirmation message to the user.

[1506] Step 6:

[1507] Users use the app to input their mood for the day and their plans for going out.

[1508] Step 7:

[1509] The terminal sends the entered information to the server.

[1510] Step 8:

[1511] Based on the information received by the server, the system selects the most suitable clothing items from the user's database and generates an outfit coordination.

[1512] Step 9:

[1513] The server generates a coordinated image and a point-by-point explanation, and sends it to the terminal.

[1514] Step 10:

[1515] The device projects an outfit image onto a full-length mirror and presents it to the user.

[1516] Step 11:

[1517] The server checks the outfit and searches the internet for any missing items.

[1518] Step 12:

[1519] The server adds information about the found items to a suggestion list and sends it to the terminal.

[1520] Step 13:

[1521] The device displays a list of suggestions to the user.

[1522] Step 14:

[1523] The server periodically checks the database and analyzes the condition of the clothing.

[1524] Step 15:

[1525] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[1526] Step 16:

[1527] The server suggests appropriate shop items and sends a notification to the device.

[1528] Step 17:

[1529] The device displays a degradation notification and suggested items to the user.

[1530] Step 18:

[1531] The user scans their body shape and size during the initial setup.

[1532] Step 19:

[1533] The device sends the acquired body shape data to the server.

[1534] Step 20:

[1535] The server analyzes body shape data and registers the user's size information in the database.

[1536] Step 21:

[1537] Users purchase items they like through suggestions and notifications.

[1538] Step 22:

[1539] The device initiates the purchase process and prompts the user to enter the required information.

[1540] Step 23:

[1541] The server suggests the optimal size based on body shape data and processes the purchase.

[1542] Step 24:

[1543] The server sends a purchase confirmation message to the device.

[1544] Step 25:

[1545] The device displays a purchase completion message to the user.

[1546] (Example 1)

[1547] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1548] In today's busy lifestyle, it is difficult for users to efficiently manage their clothing and choose the best outfits. Furthermore, there is a lack of assistance in registering newly purchased clothing, suggesting outfits based on the season and mood, checking and suggesting missing items, selecting the optimal size for their body type, and properly replacing worn-out clothing. As a result, users spend a lot of time managing their clothes, leading to problems such as unnecessary purchases and inappropriate outfit choices.

[1549] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1550] In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database; means for suggesting clothing coordinates based on the user's mood and outing plans; means for searching for and suggesting items that are missing from the suggested coordinates; means for acquiring and recording the user's body shape information; means for suggesting the optimal size of the suggested items based on the acquired body shape information; and means for registering the user's clothing using tags or QR codes. As a result, the user can enjoy efficient clothing management and coordinate suggestions, reduce unnecessary purchases, and improve their individual fashion lifestyle.

[1551] A "user" refers to an individual who uses the system to manage their clothing and receive styling suggestions.

[1552] "Image analysis" refers to the technology of extracting information such as type, color, size, and material from images of clothing that have been photographed.

[1553] A "database" refers to a collection of information where analyzed data is stored and searched or updated as needed.

[1554] "Mood and outing plans" refers to the user's daily emotions and activity plans, and is a fundamental data element for suggesting outfit combinations.

[1555] "Outfit suggestions" refers to generating and presenting the optimal clothing combinations based on the user's mood and planned outing.

[1556] "Missing items" refer to clothing or accessories that are missing or need to be added to the suggested outfit.

[1557] "Body shape information" refers to data that details the user's body shape and size, and is obtained through scanning or input.

[1558] "Suggesting the optimal size" means indicating the most suitable size among the suggested items based on the user's body type information.

[1559] "Registering using tags or QR codes" refers to a method of scanning tags or QR codes attached to clothing and registering that information in a database.

[1560] "Deterioration assessment" refers to the automatic determination of the decline in the quality of clothing based on factors such as the frequency of use and age.

[1561] A "notification prompting replacement" refers to a message sent to a user suggesting the purchase of new clothing items that have been deemed to be worn out.

[1562] "Projecting onto a full-length mirror" refers to displaying the suggested outfit on a screen or smart mirror for the user to see.

[1563] This invention relates to an AI-powered system that assists users with clothing management and outfit coordination suggestions. This system is built using a cloud server and the user's device (e.g., a smartphone, tablet, or smart mirror).

[1564] Clothing image analysis and database management

[1565] This system provides a function that allows users to take photos of new clothes they purchase through an app and send the image data to a cloud server. Smartphones and tablets can be used as the devices. The server uses image analysis technologies such as TensorFlow and OpenCV to extract information such as the type, color, size, and material of the clothing and store it in a database. This allows users to review the information later or use it for outfit suggestions.

[1566] Registration via QR code / tag

[1567] Users can easily register information about new clothing by scanning the QR code or tag attached to the garment. The device reads the QR code or tag information and sends it to the server. The server registers the received information in its database and sends the details to the device. The user can then review this information and enter additional details if necessary.

[1568] Coordination suggestions

[1569] The user uses the app to input their mood and outing plans for the day. This can be done using emojis, text input, or dropdown menus. The device sends the input information to a server, which then selects the most suitable clothing based on this information. The server uses a machine learning model (e.g., k-means clustering) to generate an outfit and sends the image and explanation to the device. The device then presents the outfit image to the user and projects it onto a display such as a smart mirror.

[1570] Suggestions for additional items

[1571] The server reviews the generated outfit and searches the internet for any missing items. For example, it searches online shops via API integration and adds appropriate items to the suggestion list. The suggestion list is in JSON format and sent to the device. The device displays the suggestion list to the user and provides purchase links as needed.

[1572] Degradation detection and notification

[1573] The server periodically checks the database and analyzes the deterioration status of clothing. For example, it uses a deterioration model to calculate the deterioration level, taking into account the frequency of use and the time elapsed. If deterioration is advanced, the server generates a notification and sends a message to the device prompting replacement. The device displays this notification to the user and introduces suggested new items.

[1574] Size analysis and purchase procedure

[1575] During the initial setup, the user scans their body shape and size. For example, they use a dedicated body scanning app to obtain 3D data of their body shape. The device sends this data to a server, which analyzes it and registers the user's size information in a database. When the user purchases an item they like through suggestions or notifications, the server suggests the optimal size based on the body shape information and initiates the purchase process. Once the purchase is complete, the server generates a confirmation message and sends it to the device, allowing the user to confirm the purchase.

[1576] Examples of specific cases and prompt statements

[1577] Specific example

[1578] 1. Registering and coordinating clothing items

[1579] A user purchases a new shirt and scans the QR code with the app.

[1580] The server adds and organizes the shirt information in the database.

[1581] Another user enters "I have plans for a casual outing today" into the app.

[1582] The server generates a casual outfit and sends it to the terminal.

[1583] The device projects the outfit onto a smart mirror, and AI provides commentary.

[1584] 2. Suggestions and purchases of additional items

[1585] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[1586] The device displays a link, and the user purchases the shoes on the spot.

[1587] Based on another user's body shape data, the server suggests a coat of the appropriate size.

[1588] The coat was successfully ordered through the purchase process.

[1589] Example of a prompt

[1590] "I have a casual outing planned. Please suggest the best outfit."

[1591] "I purchased a new shirt, so I'm registering my information. I scanned the QR code."

[1592] "I'd like to buy a new winter coat, so please suggest a size that fits my body type."

[1593] As described above, the system of the present invention provides various functions for efficiently and effectively managing the user's clothing and suggesting outfit combinations, thereby improving the user's fashion life.

[1594] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1595] Step 1: Take a photo of your clothes and send it to us.

[1596] The user takes a picture of new clothes. The input for this step is image data of the clothes taken by the user using the camera on their smartphone or tablet. The device receives the captured image and sends it to the server. Specifically, the device launches its camera app, temporarily saves the captured image, and then uploads it to the cloud server. The output is the image data of the clothes that arrives on the server.

[1597] Step 2: Image analysis and saving to database

[1598] The server analyzes the received image data. The input for this step is the image data of the clothing sent to the server. The server uses image analysis techniques (e.g., TensorFlow or OpenCV) to extract information such as type, color, size, and material. Specifically, it runs an image analysis model and extracts features. The output is the analyzed information data, which is stored in a database.

[1599] Step 3: Notification of analysis results

[1600] The server saves the analyzed information to a database and generates a registration completion notification. The input for this step is the image analysis result data. The server uses a REST API to generate the notification message and sends it to the terminal. The output is the registration completion notification message. The terminal receives this notification and displays it to the user.

[1601] Step 4: Read and send the QR code / tag.

[1602] The user enters information about new clothing using a QR code or tag reader. This input is the information from the QR code or tag attached to the clothing. The device scans the QR code or tag and sends the information to the server. Specifically, it uses a tag reader app to retrieve QR code data and uploads it to a cloud server. The output is the QR code or tag information data arriving at the server.

[1603] Step 5: Register QR code / tag information in the database

[1604] The server registers the information from the received QR code or tag into a database. The input for this step is the QR code or tag information sent to the server. The server saves the information in the database and sends the details to the terminal. Specifically, it extracts and transforms the data and registers it in the database. The output is the information saved in the database and the detailed information sent to the terminal.

[1605] Step 6: Enter your mood and schedule

[1606] The user uses the app to input their mood and outing plans for the day. This step involves inputting mood and plan data. The device then sends this information to the server. Specifically, the user records the information using an input form within the app and uploads it to the cloud server by pressing the submit button. The output is the mood and plan data arriving at the server.

[1607] Step 7: Creating the outfit

[1608] The server generates outfits based on the user's mood and schedule information. The input for this step is the user's mood and schedule information. The server uses a machine learning model (e.g., k-means clustering) to select the most suitable clothing from the database and generate the outfit. Specifically, it executes database queries and applies the selection algorithm. The output is the generated outfit image and its explanation.

[1609] Step 8: Send and present your outfit ideas

[1610] The server sends the generated coordinated image and explanation to the terminal. The input for this step is the generated coordinated image and explanation. The terminal receives this and projects it onto a smart mirror or display to show it to the user. Specifically, it receives the image and text data and projects it onto the display device. The output is the coordinated image presented to the user.

[1611] Step 9: Search for and suggest missing items

[1612] The server checks the outfit and searches the internet for any missing items. The input for this step is the generated outfit data. The server searches online shops via API integration and adds appropriate items to the suggestion list. Specifically, it uses an internet search API to retrieve item information and construct the list. The output is the information in the suggestion list.

[1613] Step 10: Notification and display of the suggestion list

[1614] The server sends the suggestion list to the device. The input for this step is the information in the suggestion list. The device receives this and displays it to the user. Specifically, the app displays the received list and provides a purchase link. The output is the suggestion list displayed to the user.

[1615] Step 11: Check and notify about deterioration status

[1616] The server periodically checks the database and analyzes the deterioration status of the clothing. The input for this step is the clothing information in the database. The server uses a deterioration model to analyze the data and generate deterioration reports and notification messages. Specifically, it executes a data analysis algorithm and determines the degree of deterioration. The output is a notification message.

[1617] Step 12: Sending Degradation Notifications and Proposals

[1618] The server sends the generated notification message to the terminal. The input for this step is the degraded notification message. The terminal receives this, displays a notification to the user, and introduces the suggested items. The specific action is to use the notification function to display the message and introduce the new items. The output is the notification message and suggested items that the user receives.

[1619] Step 13: Scan and submit body shape data

[1620] The user scans their body shape data during the initial setup. The input for this step is body shape data acquired by a scanning device (e.g., a body scanning app). The device receives this data and sends it to the server. Specifically, it uses the scanning device to acquire 3D body shape data and uploads it to a cloud server. The output is the body shape data sent to the server.

[1621] Step 14: Analysis and registration of body shape information

[1622] The server analyzes the received body shape data and registers the user's size information in the database. The input for this step is the received body shape data. The server performs data analysis, extracts size information for each body part, and registers it in the database. Specifically, it executes a body shape analysis algorithm to generate and save size data. The output is the size information stored in the database.

[1623] Step 15: Suggestion of the optimal size and purchase procedure

[1624] The user purchases an item through a suggestion or notification. The input for this step is information about the item to be purchased and the user's body shape data. The server suggests the optimal size based on the body shape data and processes the purchase. Specifically, it applies a size suggestion algorithm and handles purchase confirmation and payment. The output is a purchase completion confirmation message.

[1625] Step 16: Send and display purchase completion notification

[1626] The server sends a purchase completion confirmation message to the terminal. The input for this step is the purchase completion information. The terminal receives this and displays the purchase completion message to the user. Specifically, it uses a notification function to display the confirmation message and notify the user of its contents. The output is the purchase completion notification displayed to the user.

[1627] (Application Example 1)

[1628] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1629] Traditional clothing management systems simply stored users' clothing in a database, requiring users to physically try on clothes to actually see the outfits they wanted. Furthermore, there was a lack of systems that automatically suggested appropriate outfits based on the user's body type, mood, and outing plans, as well as suggesting replacements for worn-out clothing. There is also a need for a new virtual try-on experience, allowing users to try on clothes without actually going to a store. To address these challenges, a more efficient and user-friendly clothing management and outfit suggestion system is needed.

[1630] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1631] In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database; means for suggesting clothing coordinates based on the user's mood and outing plans; means for searching for and suggesting items missing from the suggested coordinates; means for acquiring and recording the user's body shape information; means for the user to experience trying on clothes in a virtual fitting room; and means for generating a corresponding virtual avatar and performing clothing fitting simulations. This allows users to check the suitability of clothing through virtual try-ons without having to go to a physical store, and further enables coordinate suggestions based on mood and body shape, as well as suggestions for replacing worn-out clothing.

[1632] A "user" is a person who uses the system and is the person who receives clothing management and coordination suggestions.

[1633] "Clothing" refers to the general term for clothes and accessories that a user owns and wears on a daily basis.

[1634] "Image analysis" is the process of extracting information such as the type, color, size, and material of clothing from a captured image.

[1635] A "database" is an information system for systematically organizing and storing information about analyzed clothing.

[1636] "Mood" refers to the emotions and mood that the user is feeling at any given time, and it is one of the criteria used for suggesting outfits.

[1637] "Outing plans" refer to the activities and schedules that a user has planned for a specific day.

[1638] "Outfit suggestions" refer to the system suggesting the optimal clothing combination based on the user's mood and planned outing.

[1639] "Suggested outfit" refers to the result of the clothing combinations generated by the system.

[1640] "Missing items" refer to clothing or accessories that are necessary for the suggested outfit but that the user does not currently own.

[1641] "Searching" is the process of using the internet or databases to find out whether or not specific information or products exist.

[1642] "Body shape information" refers to information about various body dimensions, such as the user's height, weight, shoulder width, and waist size.

[1643] A "virtual fitting room" is a simulation area where users can virtually try on clothes.

[1644] A "virtual avatar" is a virtual human model created based on the user's body shape information.

[1645] A "fitting simulation" is a process that allows users to visually check how actual clothing will look on a virtual avatar.

[1646] The present invention relates to a system that supports a user's clothing management and coordination suggestions. This system includes means for image analysis of the user's clothing and storing the information in a database, means for suggesting clothing coordinates based on the user's mood and outing plans, means for searching for and suggesting items that are missing from the suggested coordinates, means for acquiring and recording the user's body shape information, means for the user to experience trying on clothes in a virtual fitting room, and means for generating a corresponding virtual avatar and performing clothing fitting simulations.

[1647] System Configuration

[1648] Image analysis and database management

[1649] 1. Register your clothing:

[1650] The user takes a picture of new clothing using their smartphone camera and sends it to their device.

[1651] The server analyzes the received images using image analysis techniques to extract information such as type, color, size, and material. Specifically, it uses Python's image processing library (PIL) and request library (requests).

[1652] The server saves the analyzed information to a database (e.g., MySQL or PostgreSQL) and sends a completion notification to the terminal.

[1653] Coordination suggestions

[1654] 1. Enter your mood and schedule:

[1655] Users use a smartphone app to input their mood and outing plans for the day.

[1656] The terminal sends the entered information to the server.

[1657] 2. Coordination generation:

[1658] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database.

[1659] The server sends the generated outfit image and its key points to the terminal.

[1660] The device projects the coordinated look onto a virtual fitting room (for example, smart glasses or a head-mounted display) and presents it to the user.

[1661] Suggestions for missing items

[1662] 1. Search for and suggest missing items:

[1663] The server checks the generated outfit and searches the internet for any missing items.

[1664] The server adds information about the found items to a suggestion list and sends it to the terminal.

[1665] The device displays a list of suggestions to the user.

[1666] Virtual fitting

[1667] 1. Virtual try-on experience:

[1668] When a user enters a virtual fitting room, the server generates a virtual avatar based on the user's body shape information.

[1669] The generated outfit is applied to the virtual avatar, and a fitting simulation is performed.

[1670] The device (for example, smart glasses or a head-mounted display) visually displays the simulation results to the user.

[1671] Specific example

[1672] 1. Example of a prompt:

[1673] "Please suggest an outfit that's perfect for a casual day trip."

[1674] "I'm in a good mood today and have plans to go out. What would be the best outfit for me?"

[1675] Usage example

[1676] User A purchases a new shirt and scans the QR code with their smartphone camera. The server analyzes the shirt's information and adds it to the database.

[1677] User B enters "I have a casual outing planned for today" into the app. The server follows a series of processes, generates a casual outfit, and displays it in a virtual fitting room.

[1678] User C tries on clothes in a virtual fitting room, reviews the simulation results, and makes a final decision.

[1679] This invention allows users to efficiently and effectively manage their clothing and receive styling suggestions without having to go to a physical store, reducing unnecessary purchases and improving their fashion lifestyle.

[1680] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1681] Step 1:

[1682] Image analysis and registration of clothing

[1683] Input: Images of new clothing items taken by the user with their smartphone camera.

[1684] Specific action: The user launches the app and takes a picture of the clothing.

[1685] Data processing / calculation: The device sends an image to the cloud server. The server analyzes the image using image analysis techniques (PIL, requests library, etc.) and extracts information about the type, color, size, and material.

[1686] Output: The analyzed clothing information is saved to the database, and a notification of completion is sent to the terminal.

[1687] Step 2:

[1688] Enter your mood and schedule.

[1689] Input: The user enters their mood and outing plans for the day into the app.

[1690] Specific action: The user enters their mood or plans as text into a form within the app.

[1691] Data processing / calculation: The terminal sends the input information to the server. The server analyzes the information and compares it with the user's database.

[1692] Output: Mood and schedule information is saved to the server.

[1693] Step 3:

[1694] Coordination generation

[1695] Input: Information about mood and outing plans, and the user's clothing database.

[1696] Specific operation: The server selects the most suitable clothing from the user's database based on the stored mood and schedule information.

[1697] Data processing / calculation: The server automatically generates outfits using a generation AI model, creating outfit images and explanations of their key points.

[1698] Output: The generated coordinated image and explanation are sent to the terminal.

[1699] Step 4:

[1700] Virtual Fitting Experience

[1701] Input: Generated outfit image and user body type information.

[1702] Specific operation: The user enters a virtual fitting room. The device (e.g., smart glasses or head-mounted display) sends the user's body shape information to the server.

[1703] Data processing / calculation: The server generates a virtual avatar based on body shape information and reflects the outfit on the avatar.

[1704] Output: The fitting simulation results are displayed on the terminal and provided to the user visually.

[1705] Step 5:

[1706] Search and suggestion of missing items

[1707] Input: A database of generated coordinates and users.

[1708] Specific operation: The server checks the outfit and searches for missing items on online shopping sites and databases.

[1709] Data processing / calculation: The server adds the found items to a suggestion list and generates links and detailed information.

[1710] Output: The suggestion list is sent to the terminal and displayed to the user.

[1711] Step 6:

[1712] Deterioration assessment and replacement notification

[1713] Input: User's clothing database.

[1714] Specific operation: The server periodically checks the database and analyzes the deterioration status of the clothing.

[1715] Data processing / calculation: The server uses a deterioration detection algorithm to identify deteriorated clothing and generates replacement suggestions.

[1716] Output: Notifications and suggestions are sent to the device and displayed to the user.

[1717] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1718] This invention relates to an AI-powered closet and mirror system that assists users with clothing management and outfit suggestions. The system is built by combining a cloud server, a terminal (e.g., a smartphone, tablet, or mirror display), and an emotion engine that recognizes the user's emotions.

[1719] Clothing image analysis and database management

[1720] 1. Register your clothes

[1721] The user buys new clothes, launches the app, and takes a picture of the clothes.

[1722] The device sends the captured image of the clothing to the server.

[1723] The server analyzes the received images using image analysis technology and extracts information such as type, color, size, and material.

[1724] The server saves the analyzed information to a database and sends a registration completion notification to the terminal.

[1725] The device displays a registration completion message to the user.

[1726] 2. Registration via QR code / tag

[1727] Users register new clothing items using QR codes or tag readers, hence the use of a tag reader.

[1728] The device reads the information from the QR code or tag and sends it to the server.

[1729] The server registers the received information in its database and sends the details to the terminal.

[1730] The device displays information and prompts the user for confirmation.

[1731] Coordination suggestions

[1732] 1. Enter your mood and schedule.

[1733] Users use the app to input their mood for the day and their plans for going out.

[1734] The terminal sends the entered information to the server.

[1735] 2. Coordination generation

[1736] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database and generates an outfit.

[1737] The server generates clothing combinations as coordinated outfits and creates images based on them.

[1738] The server sends a coordinated image and an explanation of its key points to the terminal.

[1739] The device projects an outfit image onto a full-length mirror and presents it to the user.

[1740] Suggestions for additional items

[1741] 1. Suggestions for missing items

[1742] The server checks the generated outfit and searches the internet for any missing items.

[1743] The server adds information about the found items to a suggestion list and sends it to the terminal.

[1744] The device displays a list of suggestions to the user.

[1745] 2. Deterioration detection and notification

[1746] The server periodically checks the database and analyzes the condition of the clothing.

[1747] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[1748] The server suggests appropriate shop items and sends a notification to the device.

[1749] The device displays a degradation notification and suggested items to the user.

[1750] Size analysis and purchase procedure

[1751] 1. Body shape and size analysis

[1752] The user scans their body shape and size during the initial setup (for example, using a body scan app).

[1753] The device sends the acquired body shape data to the server.

[1754] The server analyzes body shape data and registers the user's size information in the database.

[1755] 2. Purchase Procedure

[1756] Users purchase items they like through suggestions and notifications.

[1757] The device will begin the purchase process and you will enter the required information.

[1758] The server suggests the optimal size based on body shape data and processes the purchase.

[1759] The server sends a purchase confirmation message to the device.

[1760] The device displays a purchase completion message to the user.

[1761] Introducing an emotional engine

[1762] 1. Recognition of emotions

[1763] The device uses a built-in emotion engine to analyze the user's facial expressions and voice in real time.

[1764] The server recognizes the user's current emotions based on the data sent from the emotion engine.

[1765] 2. Adjusting coordination based on emotions

[1766] The server adjusts the outfit suggestions based on the recognized user's emotional data.

[1767] The server generates the optimal outfit based on the user's mood, schedule, and emotional data.

[1768] The device projects an optimized outfit onto a full-length mirror and presents it to the user.

[1769] Specific example

[1770] 1. Registering and coordinating clothing items

[1771] User A buys a new shirt and scans the QR code with the app.

[1772] The server adds and organizes the shirt information in the database.

[1773] User B enters "I have plans for a casual outing today" into the app.

[1774] The server generates a casual outfit and sends it to the terminal.

[1775] The device projects the outfit onto a full-length mirror, and AI provides commentary.

[1776] The emotion engine recognizes the user's facial expressions and detects the emotion of "looking happy."

[1777] The server adjusts the outfit based on emotional data, adding colors and styles.

[1778] 2. Suggestions and purchases of additional items

[1779] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[1780] The device displays a link, and the user purchases the shoes on the spot.

[1781] If user C wants to buy a winter coat, the server will suggest a coat of the appropriate size based on their body shape data.

[1782] The coat was successfully ordered through the purchase process.

[1783] This invention allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles. Furthermore, the introduction of an emotion engine enables more personalized suggestions, enhancing the user experience.

[1784] The following describes the processing flow.

[1785] Step 1:

[1786] The user purchases new clothes, launches the app, and takes a picture of the clothes.

[1787] Step 2:

[1788] The device sends the image of the clothing it has photographed to the server.

[1789] Step 3:

[1790] The server analyzes the received images using image analysis technology to extract information such as type, color, size, and material.

[1791] Step 4:

[1792] The server saves the analyzed information to a database and sends a registration completion notification to the device.

[1793] Step 5:

[1794] The device displays a confirmation message to the user.

[1795] Step 6:

[1796] Users use the app to input their mood for the day and their plans for going out.

[1797] Step 7:

[1798] The terminal sends the entered information to the server.

[1799] Step 8:

[1800] The device uses its built-in emotion engine to analyze the user's facial expressions and voice in real time.

[1801] Step 9:

[1802] The server recognizes the user's current emotions based on the data sent from the emotion engine.

[1803] Step 10:

[1804] Based on the mood, schedule, and emotional data received by the server, the system selects the most suitable clothing from the user's database.

[1805] Step 11:

[1806] The server generates clothing combinations as coordinated outfits and creates images based on them.

[1807] Step 12:

[1808] The server sends a coordinated image and an explanation of its key points to the terminal.

[1809] Step 13:

[1810] The device projects an outfit image onto a full-length mirror and presents it to the user.

[1811] Step 14:

[1812] The server checks the outfit and searches the internet for any missing items.

[1813] Step 15:

[1814] The server adds information about the found items to a suggestion list and sends it to the terminal.

[1815] Step 16:

[1816] The device displays a list of suggestions to the user.

[1817] Step 17:

[1818] The server periodically checks the database and analyzes the condition of the clothing.

[1819] Step 18:

[1820] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[1821] Step 19:

[1822] The server suggests appropriate shop items and sends a notification to the device.

[1823] Step 20:

[1824] The device displays a degradation notification and suggested items to the user.

[1825] Step 21:

[1826] The user scans their body shape and size during the initial setup (for example, by using a body scan app).

[1827] Step 22:

[1828] The device sends the acquired body shape data to the server.

[1829] Step 23:

[1830] The server analyzes body shape data and registers the user's size information in the database.

[1831] Step 24:

[1832] Users purchase items they like through suggestions and notifications.

[1833] Step 25:

[1834] The device initiates the purchase process and prompts the user to enter the required information.

[1835] Step 26:

[1836] The server suggests the optimal size based on body shape data and processes the purchase.

[1837] Step 27:

[1838] The server sends a purchase confirmation message to the device.

[1839] Step 28:

[1840] The device displays a purchase completion message to the user.

[1841] (Example 2)

[1842] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1843] Conventional clothing management systems lacked sufficient functionality to allow users to efficiently manage their clothing and receive optimal outfit suggestions. Furthermore, they lacked the ability to suggest outfits that considered the user's mood and feelings, and to provide replacement advice based on the condition of their clothing. As a result, users often made unnecessary purchases or struggled with coordinating outfits. This invention aims to solve these problems and provide a system that allows users to enjoy smarter, more personalized clothing management and outfit suggestions.

[1844] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1845] In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database; means for suggesting clothing coordinates based on the user's mood and outing plans; means for searching the internet for and suggesting items that are missing from the suggested coordinates; means for acquiring and recording the user's body shape information; means for analyzing the user's facial expressions and voice to recognize emotions; and means for adjusting the coordinate suggestions based on the recognized emotions. As a result, the user can receive personalized coordinate suggestions, reduce unnecessary purchases, and enjoy their daily fashion life more.

[1846] "Image analysis of clothing" involves analyzing images of clothing taken by the user using the app and extracting information such as the type, color, size, and material of the clothing.

[1847] "Saving to a database" means recording the analyzed clothing information in a cloud-based database so that it can be referenced and managed later.

[1848] "Suggesting clothing coordinates based on mood and outing plans" means suggesting the most suitable outfit combination based on the user's mood and plans for the day.

[1849] "Searching online and making suggestions" means finding items on the internet that are missing from the suggested outfit and recommending them to the user.

[1850] "Acquiring and recording body shape information" means analyzing body shape data obtained by the user through body scans, etc., and saving that information to a database.

[1851] "Recognizing emotions by analyzing facial expressions and voice" means determining what emotions the user is currently experiencing by analyzing their facial expressions and tone of voice.

[1852] "Adjusting outfit suggestions based on recognized emotions" means further personalizing outfits based on the user's emotional information and suggesting the most suitable clothing for the user's mental state.

[1853] This invention relates to an AI-powered closet and mirror system that assists users with clothing management and outfit suggestions. The system is built by combining a cloud server, a terminal (smartphone, tablet, or mirror display), and an emotion engine that recognizes the user's emotions.

[1854] Clothing image analysis and database management

[1855] Clothing registration

[1856] The user purchases new clothes, launches the app, and takes a picture of the clothes. The device sends the image of the clothes to the server. The server analyzes the received image using image analysis techniques (e.g., OpenCV and TensorFlow) and extracts information such as type, color, size, and material. The server saves the analyzed information to a database (e.g., MySQL) and sends a registration completion notification to the device. The device displays a registration completion message to the user.

[1857] Registration via QR code / tag

[1858] The user registers new clothing items using a QR code or tag reader, utilizing a tag reader. The device reads the information from the QR code or tag and sends it to the server. The server registers the received information in its database and sends the details to the device. The device uses the Zxing library to read the QR code information. The device displays the information and prompts the user for confirmation.

[1859] Coordination suggestions

[1860] Enter your mood and schedule.

[1861] The user uses the app to input their mood for the day and their outing plans. The device then sends the entered information to the server.

[1862] Coordination generation

[1863] Based on the mood and schedule information received by the server, the server selects the most suitable clothing from the user's database and generates an outfit. The server uses an AI model developed in Python to suggest outfits based on the user's past preference data and input information. The server generates clothing combinations as outfits and generates an image based on them. GANs (Generative Opposite Networks) are used for image generation. The server sends the outfit image and its key points to the terminal. The terminal projects the outfit image onto a full-length mirror and presents it to the user.

[1864] Suggestions for additional items

[1865] Suggestions for missing items

[1866] The server checks the generated outfit and searches the internet for any missing items. The server uses a Python web scraping library (such as Scrapy) to search for recommended items. The server adds information about the found items to a suggestion list and sends it to the terminal. The terminal displays the suggestion list to the user.

[1867] Degradation detection and notification

[1868] The server periodically checks the database and analyzes the deterioration status of clothing. The server creates deterioration prediction models using random forests and support vector machines. The server generates notifications for clothing that is deteriorating and creates messages encouraging replacement. The server suggests appropriate shop items and sends notifications to the device. The device displays the deterioration notification and suggested items to the user.

[1869] Size analysis and purchase procedure

[1870] Body shape and size analysis

[1871] The user scans their body shape and size during initial setup (for example, using a body scan app). The device sends the acquired body shape data to the server. The server processes the 3D scan data and analyzes the user's body shape. This is done using 3D modeling software such as MeshLab. The server stores the body shape data in a database.

[1872] Purchase procedure

[1873] The user purchases items they like through suggestions and notifications. The device initiates the purchase process and the user enters the necessary information. The server matches the user's body size data with product data, suggests the optimal size, and processes the purchase. The server sends a purchase completion confirmation message to the device. The device displays a purchase completion message to the user.

[1874] Introducing an emotional engine

[1875] Recognition of emotions

[1876] The device uses its built-in emotion engine to analyze the user's facial expressions and voice in real time. The device analyzes emotions using a facial recognition API (e.g., Google Cloud Vision API) or a speech recognition API. The server recognizes the user's current emotions based on the data sent from the emotion engine.

[1877] Adjusting coordination based on emotions

[1878] The server adjusts outfit suggestions based on the recognized user's emotional data. The server generates an optimal outfit that reflects the user's mood, schedule, and emotional data. The device projects the optimized outfit onto a full-length mirror and presents it to the user.

[1879] Specific example

[1880] Prompt message

[1881] The following is an example of a prompt message to use when inputting a specific command into a generating AI model.

[1882] "Please suggest a casual style for today."

[1883] "Please suggest outfit ideas using this shirt."

[1884] "Please recommend some clothes that suit my current mood."

[1885] This allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles. Furthermore, the introduction of an emotional engine enables more personalized suggestions, enhancing the user experience.

[1886] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1887] Processing steps

[1888] Clothing image analysis and database management

[1889] Step 1:

[1890] The user purchases new clothes, launches the app, and takes a picture of the clothes.

[1891] Input: New clothes, smartphone app

[1892] Output: Images of the photographed clothing

[1893] Step 2:

[1894] The device sends the image of the clothing it has photographed to the server.

[1895] Input: Images of clothing taken

[1896] Output: Image data transferred to the server

[1897] Step 3:

[1898] The server analyzes the received images using image analysis techniques (OpenCV and TensorFlow) to extract information such as type, color, size, and material.

[1899] Input: Transferred image data

[1900] Output: Image analysis results including type, color, size, and material.

[1901] Specific operation: Extract feature points from an image using OpenCV and classify them using TensorFlow.

[1902] Step 4:

[1903] The server analyzes the information, saves it to a database (MySQL), and sends a registration completion notification to the terminal.

[1904] Input: Analysis results

[1905] Output: Clothing information stored in the database, registration completion notification

[1906] Specific operation: Insert information into the database using SQL queries.

[1907] Step 5:

[1908] The device displays a registration completion message to the user.

[1909] Input: Registration completion notification

[1910] Output: Completion message displayed to the user

[1911] Registration via QR code / tag

[1912] Step 1:

[1913] A tag reader is used so that users can register new clothing items using QR codes or tag readers.

[1914] Input: New clothes, tag reader

[1915] Output: Information from the scanned QR code or tag

[1916] Step 2:

[1917] The device reads the information from the QR code or tag and sends it to the server. The Zxing library is used for reading.

[1918] Input: QR code or tag information

[1919] Output: Code information transferred to the server

[1920] Step 3:

[1921] The server registers the received information in the database and sends the details to the terminal.

[1922] Input: Transferred code information

[1923] Output: Clothing information stored in the database, notification of detailed information.

[1924] Step 4:

[1925] The device displays information and prompts the user for confirmation.

[1926] Input: Notification of detailed information

[1927] Output: Detailed information and confirmation message displayed to the user

[1928] Enter your mood and schedule.

[1929] Step 1:

[1930] Users use the app to input their mood for the day and their plans for going out.

[1931] Input: Mood, schedule information, smartphone app

[1932] Output: Information entered into the terminal

[1933] Step 2:

[1934] The terminal sends the entered information to the server.

[1935] Input: Entered information

[1936] Output: Mood and schedule information transferred to the server

[1937] Coordination generation

[1938] Step 1:

[1939] Based on the mood and schedule information received by the server, the system selects the most suitable clothing from the user's database and generates an outfit. An AI model developed in Python is used.

[1940] Input: Information about mood and schedule, clothing information in the database.

[1941] Output: Optimal coordination suggestion

[1942] Specific operation: Select based on past preference data and input information.

[1943] Step 2:

[1944] The server generates clothing combinations as coordinated outfits and then generates images based on these combinations. Image generation utilizes GANs (Generative Opposite-Guard Networks).

[1945] Input: Coordination suggestion

[1946] Output: Generated coordinated image

[1947] Specific operation: Create an image of the outfit using a GAN model.

[1948] Step 3:

[1949] The server sends a coordinated image and an explanation of its key points to the terminal.

[1950] Input: Generated coordinated image

[1951] Output: Coordinated image and explanation sent to the terminal

[1952] Step 4:

[1953] The device projects an outfit image onto a full-length mirror and presents it to the user.

[1954] Input: Submitted outfit image and description

[1955] Output: Coordinated outfit image and explanation displayed to the user.

[1956] Suggestions for missing items

[1957] Step 1:

[1958] The server checks the generated outfit and searches the internet for any missing items. A Python web scraping library (such as Scrapy) is used.

[1959] Input: Generated coordination information

[1960] Output: List of missing items

[1961] Specific operation: Perform web scraping and search for corresponding items.

[1962] Step 2:

[1963] The server adds information about the found items to a suggestion list and sends it to the terminal.

[1964] Input: List of missing items

[1965] Output: Information from the proposal list

[1966] Step 3:

[1967] The device displays a list of suggestions to the user.

[1968] Input: Information from the proposal list

[1969] Output: Suggestion list displayed to the user

[1970] Degradation detection and notification

[1971] Step 1:

[1972] The server periodically checks the database and analyzes the deterioration status of the clothing. A deterioration prediction model is created using random forests or support vector machines.

[1973] Input: Clothing data in the database

[1974] Output: Degradation judgment result

[1975] Specific operation: Predicting the deterioration status using a machine learning model.

[1976] Step 2:

[1977] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[1978] Input: Degradation judgment result

[1979] Output: Notification message

[1980] Step 3:

[1981] The server suggests appropriate shop items and sends a notification to the device.

[1982] Input: Notification message

[1983] Output: Proposed item information

[1984] Step 4:

[1985] The device displays a degradation notification and suggested items to the user.

[1986] Input: Suggested item information

[1987] Output: Deterioration notification and item information displayed to the user.

[1988] Body shape and size analysis

[1989] Step 1:

[1990] The user scans their body shape and size during the initial setup (for example, by using a body scan app).

[1991] Input: Body shape and size scan data

[1992] Output: Scan data

[1993] Step 2:

[1994] The device sends the acquired body shape data to the server.

[1995] Input: Scanned data

[1996] Output: Body shape data transferred to the server

[1997] Step 3:

[1998] The server processes the 3D scan data and analyzes the user's body shape. 3D modeling software such as MeshLab is used.

[1999] Input: Transferred body shape data

[2000] Output: Analyzed body shape information

[2001] Specific operation: Perform 3D modeling and analyze the user's body shape as digital data.

[2002] Step 4:

[2003] The server saves body shape data to a database.

[2004] Input: Analyzed body shape information

[2005] Output: Body shape information stored in the database

[2006] Purchase procedure

[2007] Step 1:

[2008] Users purchase items they like through suggestions and notifications.

[2009] Input: Suggestion and notification data

[2010] Output: Purchase Decision

[2011] Step 2:

[2012] The device will begin the purchase process and enter the required information.

[2013] Input: Purchase decision, required information

[2014] Output: Entered purchase information

[2015] Step 3:

[2016] The server matches the user's body shape data with product data, suggests the optimal size, and processes the purchase.

[2017] Input: Entered purchase information, body type data

[2018] Output: Optimal size suggestion, purchase process

[2019] Specific operation: Match body shape data with product size data to select the optimal size.

[2020] Step 4:

[2021] The server sends a purchase confirmation message to the device.

[2022] Input: Purchase process complete

[2023] Output: Purchase completion confirmation message

[2024] Step 5:

[2025] The device displays a purchase completion message to the user.

[2026] Input: Purchase completion confirmation message

[2027] Output: Purchase completion message displayed to the user

[2028] Introducing an emotional engine

[2029] Step 1:

[2030] The device uses its built-in emotion engine to analyze the user's facial expressions and voice in real time. It utilizes a facial recognition API (Google Cloud Vision API) and a speech recognition API.

[2031] Input: User's facial expressions, voice data

[2032] Output: Sentiment data as analysis results

[2033] Specific operation: The analysis terminal processes facial and voice data and extracts emotional information.

[2034] Step 2:

[2035] The server recognizes the user's current emotions based on the data sent from the emotion engine.

[2036] Input: Sentiment data

[2037] Output: User sentiment analysis results

[2038] Step 3:

[2039] The server adjusts outfit suggestions based on the recognized user's emotional data.

[2040] Input: Sentiment analysis results

[2041] Output: Adjusted coordination suggestions

[2042] Specific actions: Consider emotional data and optimize suggestions using an AI model.

[2043] Step 4:

[2044] The device projects an optimized outfit onto a full-length mirror and presents it to the user.

[2045] Input: Adjusted coordination suggestions

[2046] Output: Optimized coordinates displayed to the user

[2047] (Application Example 2)

[2048] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[2049] In recent years, interest in fashion has increased, but it remains difficult for users to efficiently manage their clothing and receive outfit suggestions. In particular, there is no system that suggests outfits based on the user's mood or outing plans, or that provides optimal suggestions based on their feelings on a given day. Furthermore, the process of searching for and purchasing missing items in suggested outfits is also cumbersome. As a result of this situation, users often end up making unnecessary purchases, and there is a need for efficient means to improve the quality of individual fashion lives.

[2050] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database, means for suggesting clothing coordinates based on the user's mood and outing plans, and means for searching for and suggesting items that are missing from the suggested coordinates. As a result, the user can efficiently manage their clothing and receive appropriate coordinates according to their mood and plans for the day.

[2051] A "user" is an individual who uses the system to manage their own clothing and receive styling suggestions.

[2052] "Clothing" refers to fashion items owned by the user and registered in the system through image analysis.

[2053] "Image analysis" is a technology that extracts information such as type, color, size, and material from images of clothing that have been photographed.

[2054] A "database" is a system that organizes and stores information such as a user's clothing and the results of its analysis.

[2055] "Mood" refers to the user's emotions and psychological state on a given day, and is an element that is reflected in the outfit suggestions.

[2056] "Planned outings" refer to the activities and places the user has planned to visit that day, and are factors that influence outfit suggestions.

[2057] "Coordination" refers to suggestions for clothing combinations for the user, which are optimized based on their mood and planned outings.

[2058] "Searching" refers to the process of finding items online that are missing from a suggested outfit.

[2059] "Body shape information" refers to data about the user's body size and shape, and is used for suggesting outfits and selecting clothing sizes.

[2060] "Emotions" refer to the psychological state perceived from the user's facial expressions and voice, and are used to optimize outfit suggestions.

[2061] A "cloud server" is a server that performs data analysis, storage, and coordination proposals.

[2062] An "emotion recognition engine" is a technology that analyzes emotions based on the user's facial expressions and voice.

[2063] This invention relates to an AI-powered system that assists users with clothing management and outfit coordination suggestions. This system is built by combining a cloud server, a terminal (e.g., a smartphone), and an emotion recognition engine.

[2064] System Programs and Processing

[2065] 1. Register your clothes

[2066] Users use their smartphone camera to photograph new clothing items and upload the images to a cloud server. Alternatively, they can scan a QR code to automatically register clothing information. The cloud server uses image analysis technology (e.g., Google Cloud Vision API) to extract information such as clothing type, color, size, and material from the uploaded images. The extracted information is stored in a database (e.g., Amazon RDS).

[2067] 2. Enter your mood and schedule

[2068] Users use a smartphone app to input their mood and outing plans for the day. This information is sent to a cloud server, where it is cross-referenced with the user's database to generate appropriate outfit suggestions.

[2069] 3. Coordination generation

[2070] The cloud server generates the optimal outfit based on the user's mood and outing plans. An image and explanation of the generated outfit are created and sent to the smartphone.

[2071] 4. Recognition and regulation of emotions

[2072] The smartphone uses its built-in camera and microphone to analyze the user's facial expressions and voice in real time, and an emotion recognition engine (such as Microsoft Azure Emotion API) determines their emotions. The cloud server receives the data obtained from the emotion recognition engine and optimizes the outfit based on the user's emotional state.

[2073] 5. Suggestions for purchasing additional items

[2074] The cloud server searches the internet for items missing from the suggested outfit. Information on the appropriate items is generated, and the user can view the list on their smartphone and proceed with purchasing the items they like. This purchase process uses an electronic payment service (e.g., Stripe API).

[2075] Examples of specific cases and prompt statements

[2076] As a concrete example, a user purchases a new shirt and registers it by scanning a QR code with the app. The user enters their mood for the day (e.g., "Fresh") and their plans for going out (e.g., "Lunch with friends") into the app. Based on this information, the cloud server suggests a "bright shirt" and "white pants," and displays an image of the outfit on the smartphone screen. The emotion recognition engine analyzes the user's facial expression, and if it recognizes that the user looks "happy," additional suggestions such as "bright-colored accessories" are made. Furthermore, a list of suggested items that are missing (e.g., "summer sandals") is displayed on the smartphone, and the user completes the purchase process on the spot.

[2077] Examples of prompt statements are as follows:

[2078] "Users registered their new clothes using the app. After entering their mood and plans for the day, the server suggested the most suitable outfit. It recognized the user's emotions in real time and adjusted the outfit accordingly. Furthermore, it purchased any missing items based on the suggested list."

[2079] As described above, the present invention allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles. Furthermore, the introduction of an emotion recognition engine enables more personalized suggestions, improving the user experience.

[2080] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2081] Step 1:

[2082] Users register new clothing items.

[2083] Users take photos of new clothing items using their smartphone camera and upload the images to a cloud server via the app. Alternatively, they can scan a QR code to automatically register clothing information.

[2084] Input: Image or QR code information of the clothing item

[2085] Data processing: Image analysis technology (Google Cloud Vision API) is used to extract information on clothing type, color, size, and material.

[2086] Output: Extracted clothing information is saved to a database (Amazon RDS).

[2087] Step 2:

[2088] Users enter their mood and outing plans.

[2089] Users use a smartphone app to input their mood and outing plans for the day. This information is sent to a cloud server.

[2090] Input: User's mood and outing plans

[2091] Data processing: Matching entered information with the user database.

[2092] Output: Coordination suggestions based on matching results

[2093] Step 3:

[2094] The cloud server generates the coordinates.

[2095] The cloud server generates the optimal outfit based on the user's mood and outing plans. An image and explanation of the generated outfit are created and sent to the smartphone.

[2096] Input: User's mood and outing plans, clothing database

[2097] Data processing: Generates optimal outfits and creates outfit images and explanations.

[2098] Output: Send coordination image and explanation to your smartphone

[2099] Step 4:

[2100] Smartphones recognize user emotions.

[2101] The smartphone uses its built-in camera and microphone to analyze the user's facial expressions and voice in real time. The emotion recognition engine (Microsoft Azure Emotion API) determines the user's emotions.

[2102] Input: User's facial expressions and voice

[2103] Data processing: Emotions are determined using an emotion recognition engine.

[2104] Output: Send the determined emotion data to the cloud server.

[2105] Step 5:

[2106] Cloud servers adjust outfits based on emotions.

[2107] The cloud server optimizes the coordination based on the user's emotional state, using data obtained from the emotion recognition engine.

[2108] Input: User sentiment data, initial outfit suggestions

[2109] Data processing: Adjusting outfits according to emotions

[2110] Output: Optimized outfit resend to smartphone

[2111] Step 6:

[2112] The cloud server suggests additional items.

[2113] The cloud server searches the internet for items missing from the suggested outfit. It generates information on the appropriate items, and the user can view the list on their smartphone.

[2114] Input: Outfit suggestions, item database

[2115] Data processing: Search the internet for missing items and generate item information.

[2116] Output: Send item information to smartphone

[2117] Step 7:

[2118] Users purchase additional items

[2119] The user selects their preferred items from a suggested list on their smartphone and proceeds with the purchase. The purchase is completed using an electronic payment service.

[2120] Input: Suggested item list, purchase information

[2121] Data processing: Completion of the purchase process using the electronic payment service (Stripe API).

[2122] Output: Purchase confirmation and completion notification sent to your smartphone.

[2123] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2124] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2125] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[2126] [Fourth Embodiment]

[2127] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[2128] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2129] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2130] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[2131] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[2132] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[2133] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[2134] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[2135] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[2136] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2137] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2138] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[2139] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2140] This invention relates to an AI-powered closet and full-length mirror system that assists users with clothing management and outfit suggestions. This system is built using a cloud server and a terminal (e.g., a smartphone, tablet, or full-length mirror display).

[2141] Clothing image analysis and database management

[2142] 1. Register your clothes

[2143] The user buys new clothes, launches the app, and takes a picture of the clothes.

[2144] The device sends the captured image of the clothing to the server.

[2145] The server analyzes the received images using image analysis technology and extracts information such as type, color, size, and material.

[2146] The server saves the analyzed information to a database and sends a confirmation notification to the terminal.

[2147] The device displays a registration completion message to the user.

[2148] 2. Registration via QR code / tag

[2149] Users register new clothing items using QR codes or tag readers, hence the use of a tag reader.

[2150] The device reads the information from the QR code or tag and sends it to the server.

[2151] The server registers the received information in its database and sends the details to the terminal.

[2152] The device displays information and prompts the user for confirmation.

[2153] Coordination suggestions

[2154] 1. Enter your mood and schedule.

[2155] Users use the app to input their mood for the day and their plans for going out.

[2156] The terminal sends the entered information to the server.

[2157] 2. Coordination generation

[2158] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database.

[2159] The server generates clothing combinations as coordinated outfits and creates images based on them.

[2160] The server sends a coordinated image and an explanation of its key points to the terminal.

[2161] The device projects an outfit image onto a full-length mirror and presents it to the user.

[2162] Suggestions for additional items

[2163] 1. Suggestions for missing items

[2164] The server checks the generated outfit and searches the internet for any missing items.

[2165] The server adds information about the found items to a suggestion list and sends it to the terminal.

[2166] The device displays a list of suggestions to the user.

[2167] 2. Deterioration detection and notification

[2168] The server periodically checks the database and analyzes the condition of the clothing.

[2169] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[2170] The server suggests appropriate shop items and sends them to the terminal.

[2171] The device displays a notification to the user and introduces suggested items.

[2172] Size analysis and purchase procedure

[2173] 1. Body shape and size analysis

[2174] The user scans their body shape and size during the initial setup (for example, using a body scan app).

[2175] The device sends the acquired body shape data to the server.

[2176] The server analyzes body shape data and registers the user's size information in the database.

[2177] 2. Purchase Procedure

[2178] Users purchase items they like through suggestions and notifications.

[2179] The device will begin the purchase process and you will enter the required information.

[2180] The server suggests the optimal size based on body shape data and processes the purchase.

[2181] The server sends a purchase confirmation message to the device.

[2182] The device displays a purchase completion message to the user.

[2183] Specific example

[2184] 1. Registering and coordinating clothing items

[2185] User A buys a new shirt and scans the QR code with the app.

[2186] The server adds and organizes the shirt information in the database.

[2187] User B enters "I have plans for a casual outing today" into the app.

[2188] The server generates a casual outfit and sends it to the terminal.

[2189] The device projects the outfit onto a full-length mirror, and AI provides commentary.

[2190] 2. Suggestions and purchases of additional items

[2191] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[2192] The device displays a link, and the user purchases the shoes on the spot.

[2193] If user C wants to buy a winter coat, the server will suggest a coat of the appropriate size based on their body shape data.

[2194] The coat was successfully ordered through the purchase process.

[2195] This invention allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles.

[2196] The following describes the processing flow.

[2197] Step 1:

[2198] The user purchases new clothes, launches the app, and takes a picture of the clothes.

[2199] Step 2:

[2200] The device sends the image of the clothing it has photographed to the server.

[2201] Step 3:

[2202] The server analyzes the received images using image analysis technology to extract information such as type, color, size, and material.

[2203] Step 4:

[2204] The server saves the analyzed information to a database and sends a registration completion notification to the device.

[2205] Step 5:

[2206] The device displays a confirmation message to the user.

[2207] Step 6:

[2208] Users use the app to input their mood for the day and their plans for going out.

[2209] Step 7:

[2210] The terminal sends the entered information to the server.

[2211] Step 8:

[2212] Based on the information received by the server, the system selects the most suitable clothing items from the user's database and generates an outfit coordination.

[2213] Step 9:

[2214] The server generates a coordinated image and a point-by-point explanation, and sends it to the terminal.

[2215] Step 10:

[2216] The device projects an outfit image onto a full-length mirror and presents it to the user.

[2217] Step 11:

[2218] The server checks the outfit and searches the internet for any missing items.

[2219] Step 12:

[2220] The server adds information about the found items to a suggestion list and sends it to the terminal.

[2221] Step 13:

[2222] The device displays a list of suggestions to the user.

[2223] Step 14:

[2224] The server periodically checks the database and analyzes the condition of the clothing.

[2225] Step 15:

[2226] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[2227] Step 16:

[2228] The server suggests appropriate shop items and sends a notification to the device.

[2229] Step 17:

[2230] The device displays a degradation notification and suggested items to the user.

[2231] Step 18:

[2232] The user scans their body shape and size during the initial setup.

[2233] Step 19:

[2234] The device sends the acquired body shape data to the server.

[2235] Step 20:

[2236] The server analyzes body shape data and registers the user's size information in the database.

[2237] Step 21:

[2238] Users purchase items they like through suggestions and notifications.

[2239] Step 22:

[2240] The device initiates the purchase process and prompts the user to enter the required information.

[2241] Step 23:

[2242] The server suggests the optimal size based on body shape data and processes the purchase.

[2243] Step 24:

[2244] The server sends a purchase confirmation message to the device.

[2245] Step 25:

[2246] The device displays a purchase completion message to the user.

[2247] (Example 1)

[2248] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2249] In today's busy lifestyle, it is difficult for users to efficiently manage their clothing and choose the best outfits. Furthermore, there is a lack of assistance in registering newly purchased clothing, suggesting outfits based on the season and mood, checking and suggesting missing items, selecting the optimal size for their body type, and properly replacing worn-out clothing. As a result, users spend a lot of time managing their clothes, leading to problems such as unnecessary purchases and inappropriate outfit choices.

[2250] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[2251] In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database; means for suggesting clothing coordinates based on the user's mood and outing plans; means for searching for and suggesting items that are missing from the suggested coordinates; means for acquiring and recording the user's body shape information; means for suggesting the optimal size of the suggested items based on the acquired body shape information; and means for registering the user's clothing using tags or QR codes. As a result, the user can enjoy efficient clothing management and coordinate suggestions, reduce unnecessary purchases, and improve their individual fashion lifestyle.

[2252] A "user" refers to an individual who uses the system to manage their clothing and receive styling suggestions.

[2253] "Image analysis" refers to the technology of extracting information such as type, color, size, and material from images of clothing that have been photographed.

[2254] A "database" refers to a collection of information where analyzed data is stored and searched or updated as needed.

[2255] "Mood and outing plans" refers to the user's daily emotions and activity plans, and is a fundamental data element for suggesting outfit combinations.

[2256] "Outfit suggestions" refers to generating and presenting the optimal clothing combinations based on the user's mood and planned outing.

[2257] "Missing items" refer to clothing or accessories that are missing or need to be added to the suggested outfit.

[2258] "Body shape information" refers to data that details the user's body shape and size, and is obtained through scanning or input.

[2259] "Suggesting the optimal size" means indicating the most suitable size among the suggested items based on the user's body type information.

[2260] "Registering using tags or QR codes" refers to a method of scanning tags or QR codes attached to clothing and registering that information in a database.

[2261] "Deterioration assessment" refers to the automatic determination of the decline in the quality of clothing based on factors such as the frequency of use and age.

[2262] A "notification prompting replacement" refers to a message sent to a user suggesting the purchase of new clothing items that have been deemed to be worn out.

[2263] "Projecting onto a full-length mirror" refers to displaying the suggested outfit on a screen or smart mirror for the user to see.

[2264] This invention relates to an AI-powered system that assists users with clothing management and outfit coordination suggestions. This system is built using a cloud server and the user's device (e.g., a smartphone, tablet, or smart mirror).

[2265] Clothing image analysis and database management

[2266] This system provides a function that allows users to take photos of new clothes they purchase through an app and send the image data to a cloud server. Smartphones and tablets can be used as the devices. The server uses image analysis technologies such as TensorFlow and OpenCV to extract information such as the type, color, size, and material of the clothing and store it in a database. This allows users to review the information later or use it for outfit suggestions.

[2267] Registration via QR code / tag

[2268] Users can easily register information about new clothing by scanning the QR code or tag attached to the garment. The device reads the QR code or tag information and sends it to the server. The server registers the received information in its database and sends the details to the device. The user can then review this information and enter additional details if necessary.

[2269] Coordination suggestions

[2270] The user uses the app to input their mood and outing plans for the day. This can be done using emojis, text input, or dropdown menus. The device sends the input information to a server, which then selects the most suitable clothing based on this information. The server uses a machine learning model (e.g., k-means clustering) to generate an outfit and sends the image and explanation to the device. The device then presents the outfit image to the user and projects it onto a display such as a smart mirror.

[2271] Suggestions for additional items

[2272] The server reviews the generated outfit and searches the internet for any missing items. For example, it searches online shops via API integration and adds appropriate items to the suggestion list. The suggestion list is in JSON format and sent to the device. The device displays the suggestion list to the user and provides purchase links as needed.

[2273] Degradation detection and notification

[2274] The server periodically checks the database and analyzes the deterioration status of clothing. For example, it uses a deterioration model to calculate the deterioration level, taking into account the frequency of use and the time elapsed. If deterioration is advanced, the server generates a notification and sends a message to the device prompting replacement. The device displays this notification to the user and introduces suggested new items.

[2275] Size analysis and purchase procedure

[2276] During the initial setup, the user scans their body shape and size. For example, they use a dedicated body scanning app to obtain 3D data of their body shape. The device sends this data to a server, which analyzes it and registers the user's size information in a database. When the user purchases an item they like through suggestions or notifications, the server suggests the optimal size based on the body shape information and initiates the purchase process. Once the purchase is complete, the server generates a confirmation message and sends it to the device, allowing the user to confirm the purchase.

[2277] Examples of specific cases and prompt statements

[2278] Specific example

[2279] 1. Registering and coordinating clothing items

[2280] A user purchases a new shirt and scans the QR code with the app.

[2281] The server adds and organizes the shirt information in the database.

[2282] Another user enters "I have plans for a casual outing today" into the app.

[2283] The server generates a casual outfit and sends it to the terminal.

[2284] The device projects the outfit onto a smart mirror, and AI provides commentary.

[2285] 2. Suggestions and purchases of additional items

[2286] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[2287] The device displays a link, and the user purchases the shoes on the spot.

[2288] Based on another user's body shape data, the server suggests a coat of the appropriate size.

[2289] The coat was successfully ordered through the purchase process.

[2290] Example of a prompt

[2291] "I have a casual outing planned. Please suggest the best outfit."

[2292] "I purchased a new shirt, so I'm registering my information. I scanned the QR code."

[2293] "I'd like to buy a new winter coat, so please suggest a size that fits my body type."

[2294] As described above, the system of the present invention provides various functions for efficiently and effectively managing the user's clothing and suggesting outfit combinations, thereby improving the user's fashion life.

[2295] The flow of the specific processing in Example 1 will be explained using Figure 11.

[2296] Step 1: Take a photo of your clothes and send it to us.

[2297] The user takes a picture of new clothes. The input for this step is image data of the clothes taken by the user using the camera on their smartphone or tablet. The device receives the captured image and sends it to the server. Specifically, the device launches its camera app, temporarily saves the captured image, and then uploads it to the cloud server. The output is the image data of the clothes that arrives on the server.

[2298] Step 2: Image analysis and saving to database

[2299] The server analyzes the received image data. The input for this step is the image data of the clothing sent to the server. The server uses image analysis techniques (e.g., TensorFlow or OpenCV) to extract information such as type, color, size, and material. Specifically, it runs an image analysis model and extracts features. The output is the analyzed information data, which is stored in a database.

[2300] Step 3: Notification of analysis results

[2301] The server saves the analyzed information to a database and generates a registration completion notification. The input for this step is the image analysis result data. The server uses a REST API to generate the notification message and sends it to the terminal. The output is the registration completion notification message. The terminal receives this notification and displays it to the user.

[2302] Step 4: Read and send the QR code / tag.

[2303] The user enters information about new clothing using a QR code or tag reader. This input is the information from the QR code or tag attached to the clothing. The device scans the QR code or tag and sends the information to the server. Specifically, it uses a tag reader app to retrieve QR code data and uploads it to a cloud server. The output is the QR code or tag information data arriving at the server.

[2304] Step 5: Register QR code / tag information in the database

[2305] The server registers the information from the received QR code or tag into a database. The input for this step is the QR code or tag information sent to the server. The server saves the information in the database and sends the details to the terminal. Specifically, it extracts and transforms the data and registers it in the database. The output is the information saved in the database and the detailed information sent to the terminal.

[2306] Step 6: Enter your mood and schedule

[2307] The user uses the app to input their mood and outing plans for the day. This step involves inputting mood and plan data. The device then sends this information to the server. Specifically, the user records the information using an input form within the app and uploads it to the cloud server by pressing the submit button. The output is the mood and plan data arriving at the server.

[2308] Step 7: Creating the outfit

[2309] The server generates outfits based on the user's mood and schedule information. The input for this step is the user's mood and schedule information. The server uses a machine learning model (e.g., k-means clustering) to select the most suitable clothing from the database and generate the outfit. Specifically, it executes database queries and applies the selection algorithm. The output is the generated outfit image and its explanation.

[2310] Step 8: Send and present your outfit ideas

[2311] The server sends the generated coordinated image and explanation to the terminal. The input for this step is the generated coordinated image and explanation. The terminal receives this and projects it onto a smart mirror or display to show it to the user. Specifically, it receives the image and text data and projects it onto the display device. The output is the coordinated image presented to the user.

[2312] Step 9: Search for and suggest missing items

[2313] The server checks the outfit and searches the internet for any missing items. The input for this step is the generated outfit data. The server searches online shops via API integration and adds appropriate items to the suggestion list. Specifically, it uses an internet search API to retrieve item information and construct the list. The output is the information in the suggestion list.

[2314] Step 10: Notification and display of the suggestion list

[2315] The server sends the suggestion list to the device. The input for this step is the information in the suggestion list. The device receives this and displays it to the user. Specifically, the app displays the received list and provides a purchase link. The output is the suggestion list displayed to the user.

[2316] Step 11: Check and notify about deterioration status

[2317] The server periodically checks the database and analyzes the deterioration status of the clothing. The input for this step is the clothing information in the database. The server uses a deterioration model to analyze the data and generate deterioration reports and notification messages. Specifically, it executes a data analysis algorithm and determines the degree of deterioration. The output is a notification message.

[2318] Step 12: Sending Degradation Notifications and Proposals

[2319] The server sends the generated notification message to the terminal. The input for this step is the degraded notification message. The terminal receives this, displays a notification to the user, and introduces the suggested items. The specific action is to use the notification function to display the message and introduce the new items. The output is the notification message and suggested items that the user receives.

[2320] Step 13: Scan and submit body shape data

[2321] The user scans their body shape data during the initial setup. The input for this step is body shape data acquired by a scanning device (e.g., a body scanning app). The device receives this data and sends it to the server. Specifically, it uses the scanning device to acquire 3D body shape data and uploads it to a cloud server. The output is the body shape data sent to the server.

[2322] Step 14: Analysis and registration of body shape information

[2323] The server analyzes the received body shape data and registers the user's size information in the database. The input for this step is the received body shape data. The server performs data analysis, extracts size information for each body part, and registers it in the database. Specifically, it executes a body shape analysis algorithm to generate and save size data. The output is the size information stored in the database.

[2324] Step 15: Suggestion of the optimal size and purchase procedure

[2325] The user purchases an item through a suggestion or notification. The input for this step is information about the item to be purchased and the user's body shape data. The server suggests the optimal size based on the body shape data and processes the purchase. Specifically, it applies a size suggestion algorithm and handles purchase confirmation and payment. The output is a purchase completion confirmation message.

[2326] Step 16: Send and display purchase completion notification

[2327] The server sends a purchase completion confirmation message to the terminal. The input for this step is the purchase completion information. The terminal receives this and displays the purchase completion message to the user. Specifically, it uses a notification function to display the confirmation message and notify the user of its contents. The output is the purchase completion notification displayed to the user.

[2328] (Application Example 1)

[2329] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2330] Traditional clothing management systems simply stored users' clothing in a database, requiring users to physically try on clothes to actually see the outfits they wanted. Furthermore, there was a lack of systems that automatically suggested appropriate outfits based on the user's body type, mood, and outing plans, as well as suggesting replacements for worn-out clothing. There is also a need for a new virtual try-on experience, allowing users to try on clothes without actually going to a store. To address these challenges, a more efficient and user-friendly clothing management and outfit suggestion system is needed.

[2331] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[2332] In this invention, the server includes means for image analysis of the user's clothing and storing the information in a database; means for suggesting clothing coordinates based on the user's mood and outing plans; means for searching for and suggesting items missing from the suggested coordinates; means for acquiring and recording the user's body shape information; means for the user to experience trying on clothes in a virtual fitting room; and means for generating a corresponding virtual avatar and performing clothing fitting simulations. This allows users to check the suitability of clothing through virtual try-ons without having to go to a physical store, and further enables coordinate suggestions based on mood and body shape, as well as suggestions for replacing worn-out clothing.

[2333] A "user" is a person who uses the system and is the person who receives clothing management and coordination suggestions.

[2334] "Clothing" refers to the general term for clothes and accessories that a user owns and wears on a daily basis.

[2335] "Image analysis" is the process of extracting information such as the type, color, size, and material of clothing from a captured image.

[2336] A "database" is an information system for systematically organizing and storing information about analyzed clothing.

[2337] "Mood" refers to the emotions and mood that the user is feeling at any given time, and it is one of the criteria used for suggesting outfits.

[2338] "Outing plans" refer to the activities and schedules that a user has planned for a specific day.

[2339] "Outfit suggestions" refer to the system suggesting the optimal clothing combination based on the user's mood and planned outing.

[2340] "Suggested outfit" refers to the result of the clothing combinations generated by the system.

[2341] "Missing items" refer to clothing or accessories that are necessary for the suggested outfit but that the user does not currently own.

[2342] "Searching" is the process of using the internet or databases to find out whether or not specific information or products exist.

[2343] "Body shape information" refers to information about various body dimensions, such as the user's height, weight, shoulder width, and waist size.

[2344] A "virtual fitting room" is a simulation area where users can virtually try on clothes.

[2345] A "virtual avatar" is a virtual human model created based on the user's body shape information.

[2346] A "fitting simulation" is a process that allows users to visually check how actual clothing will look on a virtual avatar.

[2347] The present invention relates to a system that supports a user's clothing management and coordination suggestions. This system includes means for image analysis of the user's clothing and storing the information in a database, means for suggesting clothing coordinates based on the user's mood and outing plans, means for searching for and suggesting items that are missing from the suggested coordinates, means for acquiring and recording the user's body shape information, means for the user to experience trying on clothes in a virtual fitting room, and means for generating a corresponding virtual avatar and performing clothing fitting simulations.

[2348] System Configuration

[2349] Image analysis and database management

[2350] 1. Register your clothing:

[2351] The user takes a picture of new clothing using their smartphone camera and sends it to their device.

[2352] The server analyzes the received images using image analysis techniques to extract information such as type, color, size, and material. Specifically, it uses Python's image processing library (PIL) and request library (requests).

[2353] The server saves the analyzed information to a database (e.g., MySQL or PostgreSQL) and sends a completion notification to the terminal.

[2354] Coordination suggestions

[2355] 1. Enter your mood and schedule:

[2356] Users use a smartphone app to input their mood and outing plans for the day.

[2357] The terminal sends the entered information to the server.

[2358] 2. Coordination generation:

[2359] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database.

[2360] The server sends the generated outfit image and its key points to the terminal.

[2361] The device projects the coordinated look onto a virtual fitting room (for example, smart glasses or a head-mounted display) and presents it to the user.

[2362] Suggestions for missing items

[2363] 1. Search for and suggest missing items:

[2364] The server checks the generated outfit and searches the internet for any missing items.

[2365] The server adds information about the found items to a suggestion list and sends it to the terminal.

[2366] The device displays a list of suggestions to the user.

[2367] Virtual fitting

[2368] 1. Virtual try-on experience:

[2369] When a user enters a virtual fitting room, the server generates a virtual avatar based on the user's body shape information.

[2370] The generated outfit is applied to the virtual avatar, and a fitting simulation is performed.

[2371] The device (for example, smart glasses or a head-mounted display) visually displays the simulation results to the user.

[2372] Specific example

[2373] 1. Example of a prompt:

[2374] "Please suggest an outfit that's perfect for a casual day trip."

[2375] "I'm in a good mood today and have plans to go out. What would be the best outfit for me?"

[2376] Usage example

[2377] User A purchases a new shirt and scans the QR code with their smartphone camera. The server analyzes the shirt's information and adds it to the database.

[2378] User B enters "I have a casual outing planned for today" into the app. The server follows a series of processes, generates a casual outfit, and displays it in a virtual fitting room.

[2379] User C tries on clothes in a virtual fitting room, reviews the simulation results, and makes a final decision.

[2380] This invention allows users to efficiently and effectively manage their clothing and receive styling suggestions without having to go to a physical store, reducing unnecessary purchases and improving their fashion lifestyle.

[2381] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[2382] Step 1:

[2383] Image analysis and registration of clothing

[2384] Input: Images of new clothing items taken by the user with their smartphone camera.

[2385] Specific action: The user launches the app and takes a picture of the clothing.

[2386] Data processing / calculation: The device sends an image to the cloud server. The server analyzes the image using image analysis techniques (PIL, requests library, etc.) and extracts information about the type, color, size, and material.

[2387] Output: The analyzed clothing information is saved to the database, and a notification of completion is sent to the terminal.

[2388] Step 2:

[2389] Enter your mood and schedule.

[2390] Input: The user enters their mood and outing plans for the day into the app.

[2391] Specific action: The user enters their mood or plans as text into a form within the app.

[2392] Data processing / calculation: The terminal sends the input information to the server. The server analyzes the information and compares it with the user's database.

[2393] Output: Mood and schedule information is saved to the server.

[2394] Step 3:

[2395] Coordination generation

[2396] Input: Information about mood and outing plans, and the user's clothing database.

[2397] Specific operation: The server selects the most suitable clothing from the user's database based on the stored mood and schedule information.

[2398] Data processing / calculation: The server automatically generates outfits using a generation AI model, creating outfit images and explanations of their key points.

[2399] Output: The generated coordinated image and explanation are sent to the terminal.

[2400] Step 4:

[2401] Virtual Fitting Experience

[2402] Input: Generated outfit image and user body type information.

[2403] Specific operation: The user enters a virtual fitting room. The device (e.g., smart glasses or head-mounted display) sends the user's body shape information to the server.

[2404] Data processing / calculation: The server generates a virtual avatar based on body shape information and reflects the outfit on the avatar.

[2405] Output: The fitting simulation results are displayed on the terminal and provided to the user visually.

[2406] Step 5:

[2407] Search and suggestion of missing items

[2408] Input: A database of generated coordinates and users.

[2409] Specific operation: The server checks the outfit and searches for missing items on online shopping sites and databases.

[2410] Data processing / calculation: The server adds the found items to a suggestion list and generates links and detailed information.

[2411] Output: The suggestion list is sent to the terminal and displayed to the user.

[2412] Step 6:

[2413] Deterioration assessment and replacement notification

[2414] Input: User's clothing database.

[2415] Specific operation: The server periodically checks the database and analyzes the deterioration status of the clothing.

[2416] Data processing / calculation: The server uses a deterioration detection algorithm to identify deteriorated clothing and generates replacement suggestions.

[2417] Output: Notifications and suggestions are sent to the device and displayed to the user.

[2418] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[2419] This invention relates to an AI-powered closet and mirror system that assists users with clothing management and outfit suggestions. The system is built by combining a cloud server, a terminal (e.g., a smartphone, tablet, or mirror display), and an emotion engine that recognizes the user's emotions.

[2420] Clothing image analysis and database management

[2421] 1. Register your clothes

[2422] The user buys new clothes, launches the app, and takes a picture of the clothes.

[2423] The device sends the captured image of the clothing to the server.

[2424] The server analyzes the received images using image analysis technology and extracts information such as type, color, size, and material.

[2425] The server saves the analyzed information to a database and sends a registration completion notification to the terminal.

[2426] The device displays a registration completion message to the user.

[2427] 2. Registration via QR code / tag

[2428] Users register new clothing items using QR codes or tag readers, hence the use of a tag reader.

[2429] The device reads the information from the QR code or tag and sends it to the server.

[2430] The server registers the received information in its database and sends the details to the terminal.

[2431] The device displays information and prompts the user for confirmation.

[2432] Coordination suggestions

[2433] 1. Enter your mood and schedule.

[2434] Users use the app to input their mood for the day and their plans for going out.

[2435] The terminal sends the entered information to the server.

[2436] 2. Coordination generation

[2437] Based on the received mood and schedule information, the server selects the most suitable clothing from the user's database and generates an outfit.

[2438] The server generates clothing combinations as coordinated outfits and creates images based on them.

[2439] The server sends a coordinated image and an explanation of its key points to the terminal.

[2440] The device projects an outfit image onto a full-length mirror and presents it to the user.

[2441] Suggestions for additional items

[2442] 1. Suggestions for missing items

[2443] The server checks the generated outfit and searches the internet for any missing items.

[2444] The server adds information about the found items to a suggestion list and sends it to the terminal.

[2445] The device displays a list of suggestions to the user.

[2446] 2. Deterioration detection and notification

[2447] The server periodically checks the database and analyzes the condition of the clothing.

[2448] The server generates notifications about clothing that is deteriorating and creates messages encouraging replacement.

[2449] The server suggests appropriate shop items and sends a notification to the device.

[2450] The device displays a degradation notification and suggested items to the user.

[2451] Size analysis and purchase procedure

[2452] 1. Body shape and size analysis

[2453] The user scans their body shape and size during the initial setup (for example, using a body scan app).

[2454] The device sends the acquired body shape data to the server.

[2455] The server analyzes body shape data and registers the user's size information in the database.

[2456] 2. Purchase Procedure

[2457] Users purchase items they like through suggestions and notifications.

[2458] The device will begin the purchase process and you will enter the required information.

[2459] The server suggests the optimal size based on body shape data and processes the purchase.

[2460] The server sends a purchase confirmation message to the device.

[2461] The device displays a purchase completion message to the user.

[2462] Introducing an emotional engine

[2463] 1. Recognition of emotions

[2464] The device uses a built-in emotion engine to analyze the user's facial expressions and voice in real time.

[2465] The server recognizes the user's current emotions based on the data sent from the emotion engine.

[2466] 2. Adjusting coordination based on emotions

[2467] The server adjusts the outfit suggestions based on the recognized user's emotional data.

[2468] The server generates the optimal outfit based on the user's mood, schedule, and emotional data.

[2469] The device projects an optimized outfit onto a full-length mirror and presents it to the user.

[2470] Specific example

[2471] 1. Registering and coordinating clothing items

[2472] User A buys a new shirt and scans the QR code with the app.

[2473] The server adds and organizes the shirt information in the database.

[2474] User B enters "I have plans for a casual outing today" into the app.

[2475] The server generates a casual outfit and sends it to the terminal.

[2476] The device projects the outfit onto a full-length mirror, and AI provides commentary.

[2477] The emotion engine recognizes the user's facial expressions and detects the emotion of "looking happy."

[2478] The server adjusts the outfit based on emotional data, adding colors and styles.

[2479] 2. Suggestions and purchases of additional items

[2480] The server will suggest any missing items (e.g., shoes) from the proposed outfit.

[2481] The device displays a link, and the user purchases the shoes on the spot.

[2482] If user C wants to buy a winter coat, the server will suggest a coat of the appropriate size based on their body shape data.

[2483] The coat was successfully ordered through the purchase process.

[2484] This invention allows users to efficiently and effectively manage and coordinate their clothing, reduce unnecessary purchases, and improve their individual fashion lifestyles. Furthermore, the introduction of an emotion engine enables more personalized suggestions, enhancing the user experience.

[2485] The following describes the processing flow.

[2486] Step 1:

[2487] The user purchases new clothes, launches the app, and takes a picture of the clothes.

[2488] Step 2:

[2489] The device sends the image of the clothing it has photographed to the server.

[2490] Step 3:

[2491] The server analyzes the received images using image analysis technology to extract information such as type, color, size, and material.

[2492] Step 4:

[2493] The server saves the analyzed information to a database and sends a registration completion notification to the device.

[2494] Step 5:

[2495] The device displays a confirmation message to the user.

[2496] Step 6:

[2497] Users use the app to input their mo...

Claims

1. A means of analyzing a user's clothing using images and storing that information in a database, A method for suggesting clothing coordinates based on the user's mood and outing plans, A method for searching for and suggesting items that are missing from the proposed outfit, A means of acquiring and recording the user's body shape information, A system that includes this.

2. The system according to claim 1, further comprising means for automatically determining that clothing has deteriorated and generating a notification prompting replacement.

3. The system according to claim 1, further comprising means for projecting coordination suggestions onto a full-length mirror and presenting them to the user.

Images