System

Abstract

A system is provided.SOLUTION: The system includes a data creation means, a going-out information input means, a weather forecast acquisition means, a coordination generation means, a feedback collection means, and a reuse / recycle proposal means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many people have trouble keeping track of the items in their closets and typically wear several different types of clothing on a daily basis. This often leads to unnecessary stress, especially when deciding what to wear before going out. Furthermore, many people have unused clothing items, which not only creates waste but also prevents appropriate reuse and recycling. These issues also affect the achievement of the Sustainable Development Goals (SDGs), which aim to realize a sustainable society. The present invention aims to solve these issues, enable users to efficiently utilize their closets, and contribute to a sustainable society. [Means for solving the problem]

[0005] The present invention provides a system including the following means.

[0006] The data generation means digitizes the items in the user's closet through image analysis and stores the data in a database.

[0007] The outing information input means allows the user to input information such as the purpose of going out, date and time, and location.

[0008] Weather forecast data is acquired from a weather forecast acquisition means based on the date, time and location of a planned outing.

[0009] A coordinate generating means generates a plurality of coordinate patterns based on the acquired weather forecast data and closet database, and proposes them to the user.

[0010] The feedback collection means records coordinates selected by the user and coordinates not selected by the user and stores the records in a database.

[0011] Reuse and recycling suggestion methods and infrequently used items in the closet database are identified, and suggestions for reuse or recycling are made.

[0012] This allows users to efficiently manage the items in their closet, reduce the stress of choosing an outfit before going out, and contribute to a more sustainable society.

[0013] The "data generation means" is a means for generating data from items in the user's closet through image analysis and storing the data in a database.

[0014] The "going out information input means" is a means for the user to input information such as the purpose, date and time of going out, and location into the terminal.

[0015] The "weather forecast acquisition means" is a means for acquiring weather forecast data based on the date, time and location of a planned outing.

[0016] The "coordination generating means" is a means for generating a plurality of coordination patterns based on the acquired weather forecast data and closet database, and proposing them to the user.

[0017] The "feedback collection means" is a means for recording coordinates selected and unselected by the user and storing them in a database.

[0018] The "reuse / recycle suggestion tool" is a tool that identifies infrequently used items in the closet database and makes suggestions for reuse or recycling.

[0019] A "closet database" is a database for storing and managing data about items in a user's closet.

[0020] "Items" refer to various clothing items such as clothes, accessories, shoes, bags, etc. that a user has in their closet. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention is a system that digitizes and visualizes Western and Japanese clothing stored in closets and dressers, and allows a stylist to suggest outfits for the entire body simply by inputting the purpose of going out. This system includes a data conversion means, a means for inputting outing information, a means for obtaining weather forecasts, a means for creating outfits, a means for collecting feedback, and a means for suggesting reuse and recycling.

[0043] 1. Data collection method

[0044] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads it to their device. The device then transfers the image to the server. The server analyzes the received image and identifies each item's color, material, and category (tops, bottoms, accessories, shoes, etc.). It also tags the items appropriately and stores them in the user's personal closet database.

[0045] 2. Means of inputting outing information

[0046] The user inputs information such as the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[0047] 3. How to get weather forecasts

[0048] The server retrieves weather forecast data based on the date, time, and location of the planned outing, for example, by retrieving data from a weather forecast service via an API.

[0049] 4. Coordination Generation Method

[0050] The server generates multiple outfit patterns based on the acquired weather forecast data and the user's closet database. The generated outfit patterns correspond to the user's specified purposes and conditions. For example, it suggests a combination of a white cotton shirt with blue denim pants, black leather boots, and a silver pendant. The server then sends the generated outfit patterns to the terminal.

[0051] 5. Feedback Collection Methods

[0052] The user selects one of several outfit patterns presented on the device screen. The device then sends information about the outfits selected and not selected back to the server. The server analyzes the feedback data and learns the user's preferences and style patterns. This information is then stored in a closet database.

[0053] 6. Reuse and recycling proposal methods

[0054] When a user wishes to organize their closet, they request reuse and recycling suggestions from their device. The device then sends the request to the server. The server checks the closet database and lists items that have not been used for a certain period of time (e.g., six months). The listed unwanted items are then suggested to the user via their device. The user selects candidate items for reuse and recycling, and the selection results are sent to the server. The server then provides a guide on how to reuse or recycle the selected unwanted items (e.g., contact information for recycling shops or donation destinations).

[0055] Specific examples

[0056] When a user requests a date outfit, the system suggests outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also suggested based on the weather forecast. When a user selects a specific outfit, that information is fed back to the server and reflected in future suggestions.

[0057] In the reuse and recycling scenario, the app identifies infrequently used items in the closet and suggests ways to reuse or recycle them. For example, it might suggest, "Your blue denim pants haven't been worn for six months." If the user selects that item, the app provides guidance on where to find a recycle shop or where to donate the items.

[0058] In this way, a system is realized that allows users to efficiently utilize their closets, reduce the stress of choosing outfits before going out, and contribute to a sustainable society.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] A user takes a photo of an item in their closet and uploads the image to their device.

[0062] Step 2:

[0063] The terminal receives the uploaded image and sends it to the server.

[0064] Step 3:

[0065] The server analyzes the image and identifies the item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.).

[0066] Step 4:

[0067] The server analyzes the item information, tags it, and stores it in the user's personal closet database.

[0068] Step 5:

[0069] The user inputs information about the purpose, date, time, and location of their outing via the terminal and requests outfit suggestions.

[0070] Step 6:

[0071] The device sends the request to the server.

[0072] Step 7:

[0073] The server retrieves weather forecast data based on the planned date, time, and location of the person going out, for example, using a weather forecast API.

[0074] Step 8:

[0075] The server generates multiple coordination patterns based on weather forecast data acquired and the user's closet database.

[0076] Step 9:

[0077] The server transmits the generated coordinate pattern to the terminal and presents it to the user.

[0078] Step 10:

[0079] The user selects a preferred coordination pattern from a plurality of presented patterns.

[0080] Step 11:

[0081] The terminal feeds back to the server information on the user's selection results and unselected coordinates.

[0082] Step 12:

[0083] The server analyzes the feedback data and updates a database to learn the user's preferences and coordination trends.

[0084] Step 13:

[0085] If a user wants to organize their closet, they can request reuse and recycling suggestions from their device.

[0086] Step 14:

[0087] The device sends a recycle request to the server.

[0088] Step 15:

[0089] The server references the closet database and lists items that are used less frequently.

[0090] Step 16:

[0091] The server generates a list of unwanted items and sends it to the terminal.

[0092] Step 17:

[0093] The user reviews the list and selects the items they want to reuse or recycle.

[0094] Step 18:

[0095] The terminal transmits the selection result to the server.

[0096] Step 19:

[0097] The server generates a guide on how to reuse and recycle the selected unwanted items (e.g., contact information for recycling shops or donation destinations) and sends it to the device.

[0098] Step 20:

[0099] The user follows the guide to dispose of unwanted items.

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] Currently, systems for managing users' owned items and suggesting appropriate outfits are not fully developed. Furthermore, there is a lack of methods for efficiently suggesting outfits appropriate for the weather when users are out, or for reusing and recycling unnecessary items. This often causes inconvenience to users when choosing their daily outfits. The present invention aims to solve these problems and provide a system that can suggest more efficient and appropriate outfits and manage owned items.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes a means for digitizing items in the closet, a means for inputting the purpose, date, time, and location of an outing, a means for acquiring a weather forecast for the planned outing, a means for generating outfits based on the acquired weather forecast and the digitized items, a means for collecting feedback on the generated outfits, and a means for suggesting reuse and recycling for items that have not been used for a certain period of time. This allows users to reduce the effort of choosing their daily outfits and find the perfect outfit for the weather and purpose. It also enables efficient management of infrequently used items and suggestions for reuse and recycling.

[0105] "A means of digitizing items in the closet" refers to a system that takes photos of the clothing and accessories owned by the user, analyzes the image data to identify attributes such as color, material, and category, and stores these in a database dedicated to the user.

[0106] "Means for inputting the purpose, date, time, and location of the outing" refers to a system that has an interface for users to input information such as the purpose of the outing (e.g., date, work, casual), planned date, and location.

[0107] "Means for obtaining weather forecasts for planned outings" refers to a system that has the function of obtaining forecast data from a weather forecast service based on a specified date, time and location, analyzing that information, and reflecting it in the closet system.

[0108] "Means for generating coordination" refers to a system that generates and suggests optimal clothing combinations based on the user's purpose and conditions for going out, based on acquired weather forecast data and digitized item information.

[0109] "Means for collecting feedback on the generated outfits" refers to a system that has the ability to record the outfits selected by the user from multiple suggested outfits, analyze the selection data, and learn about the user's preferences and style.

[0110] "A means of proposing reuse and recycling for items that have not been used for a certain period of time" refers to a system that has the function of identifying items that have not been used for a certain period of time based on the usage history of items stored in a closet database, and proposing recycling and reuse methods for these items.

[0111] This invention is a system that digitizes the clothes and accessories in a closet or dresser, and automatically suggests outfits for the entire body simply by the user inputting the purpose of going out. This system includes "means for digitizing items in the closet," "means for inputting the purpose, date, time, and location of the outing," "means for obtaining the weather forecast for the planned outing," "means for generating outfits," "means for collecting feedback on the generated outfits," and "means for suggesting reuse and recycling of items that have not been used for a certain period of time."

[0112] Hardware and software used

[0113] Devices are devices available to users, such as smartphones and tablets. Servers are often located in the cloud and have very powerful processors and large amounts of storage. Software includes:

[0114] Image analysis algorithms: TensorFlow, OpenCV

[0115] Weather APIs: OpenWeatherMap, Weather.com

[0116] Generative AI model: A machine learning model used to analyze user data and generate outfits

[0117] Specific processing of the system

[0118] 1. The user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) on their device, which then uploads the photo data to the server.

[0119] 2. The server uses image analysis algorithms to analyze the received photos and identify the color, material, and category of each item. Based on this, the items are tagged appropriately and stored in the user's personal closet database.

[0120] 3. The user enters information about the purpose of the outing (e.g., date, work, casual), date and time, location, etc. through the device interface. This information is sent from the device to the server.

[0121] 4. The server retrieves weather forecast data based on the planned date, time, and location of the person going out, for example, from a weather forecast service via the OpenWeatherMap API.

[0122] 5. The server uses the acquired weather forecast data and the generative AI model to generate multiple outfit patterns based on the user's closet database. The generated outfit patterns correspond to the user's specified purposes and conditions.

[0123] 6. The user selects one of several outfit patterns presented on the device screen. The device then sends information about the selected and unselected outfits back to the server. The server analyzes the feedback data and learns the user's preferences and style patterns. This information is then stored in a closet database.

[0124] 7. If a user wishes to organize their closet, they request reuse and recycling suggestions from their device. The device sends the request to the server. The server checks the closet database and lists items that have not been used in the past six months. The listed unwanted items are then suggested to the user via their device. The user selects candidate items for reuse and recycling, and the selection results are sent to the server. The server then provides a guide on how to reuse or recycle the selected unwanted items (e.g., contact information for recycle shops or donation destinations).

[0125] Examples of concrete examples and prompts

[0126] Specific examples

[0127] When a user requests a date outfit, the system suggests outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also suggested based on the weather forecast. When a user selects a specific outfit, that information is fed back to the server and reflected in future suggestions. In the reuse and recycling section, infrequently used items in the closet are identified and suggestions for reuse or recycling are made. For example, if the system suggests "Your blue denim pants haven't been used in six months," and the user selects that option, the system provides guidance on recycle shop contact details and donation locations.

[0128] Prompt Sentence Examples

[0129] "Please suggest an outfit for a date."

[0130] "Tell me what is appropriate clothing for work"

[0131] "Please suggest an outfit that would be suitable for a casual outing."

[0132] "Make a list of the least used items in your closet."

[0133] This allows users to efficiently manage their closets and reduce the effort required to choose outfits before going out. It also allows for efficient reuse and recycling, contributing to a sustainable society.

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1:

[0136] The user takes photos of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) on their device. The device then uploads these photo data to the server. The input is the photo of the item, and the output is the image data sent to the server. As soon as the server receives the photo data, it begins analyzing it using an image analysis algorithm (e.g., TensorFlow or OpenCV) to identify the color, material, and category of each item. Specifically, it analyzes the hue and texture pattern of each pixel to extract attributes. The output is the color, material, and category information resulting from the analysis.

[0137] Step 2:

[0138] The server generates appropriate tags (color, material, category) based on the results of image analysis and stores them in the user's personal closet database. The input is the analyzed item information, and the output is the data of the newly tagged item added to the closet database.

[0139] Step 3:

[0140] The user inputs information such as the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input is the purpose of the outing, date and time, and location, and the output is the data that is sent from the device to the server. The device receives the input information and sends it to the server as structured data. Specific operations include the use of text boxes and drop-down menus.

[0141] Step 4:

[0142] The server retrieves weather forecast data based on the received planned date, time, and location of the outing. The input is the planned date, time, and location of the outing, and the output is the retrieved weather forecast data. Specifically, it sends a request to a weather forecast service (e.g., OpenWeatherMap API) and retrieves the corresponding weather forecast data (e.g., wind speed, probability of precipitation, temperature, etc.).

[0143] Step 5:

[0144] The server uses a generative AI model to generate multiple outfit patterns based on weather forecast data and the user's closet database. The input is weather forecast data and the user's item data, and the output is the generated outfit patterns. Specifically, data such as frequency of use, category, color, material, and tag are input into the generative AI model to generate outfit patterns. The generated patterns are optimized for the user's purposes and conditions.

[0145] Step 6:

[0146] The server sends the generated coordinate pattern to the terminal. The input is the generated coordinate pattern, and the output is the coordinate pattern sent to the user's terminal. The terminal displays the received pattern on the user's screen.

[0147] Step 7:

[0148] The user selects one of several coordination patterns. The input is the selection information for the coordination pattern, and the output is the information for the selected coordination. The device sends the selected information to the server and saves it as feedback data, including data on coordinations that were not selected. Specific actions include selection operations by tapping or clicking.

[0149] Step 8:

[0150] The server analyzes the feedback data and learns the user's preferences and style patterns. The input is the feedback data, and the output is updated user style data. Specifically, the server uses a machine learning algorithm to analyze the feedback data and update the user's style information.

[0151] Step 9:

[0152] When a user wants to organize their closet, they request reuse and recycling suggestions from their terminal. The input is the request content, and the output is the transmission of the request data. The terminal receives the request content and sends it to the server.

[0153] Step 10:

[0154] The server checks the closet database and lists items that have not been used in the past six months. The input is the user's closet data, and the output is the list of unwanted items. The server sends the listed items to the terminal and suggests them to the user.

[0155] Step 11:

[0156] The user selects candidate items for reuse or recycling and sends the selection results to the server. The input is the selected item information, and the output is the selection results sent to the server.

[0157] Step 12:

[0158] The server provides the terminal with information on how to reuse or recycle the selected unwanted items (e.g., contact information for recycle shops or donation destinations). The input is the selection result data, and the output is information on reuse and recycling suggestions. Specifically, the server provides the user with contact information for recycle shops and details of donation destinations.

[0159] (Application example 1)

[0160] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0161] Conventional systems suggest outfits by digitizing the items a user owns and inputting information about outings. However, in physical stores, real-time customer service is required, so a system is needed that efficiently digitizes products in the physical store and allows users to instantly receive outfit suggestions. Furthermore, improving the customer's purchasing experience by providing personalized suggestions that take into account the purpose of the outing and weather forecast is a challenge.

[0162] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0163] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, a reuse / recycle suggestion means, a database construction means for converting products in the physical store into data, and an information presentation means using a display device that can be worn by a user. This makes it possible to quickly convert products in the physical store into data and provide optimal coordinates based on the weather forecast and outing information in real time.

[0164] "Data generation means" refers to the means of registering and managing items from users and physical stores in a database using image analysis, etc.

[0165] The "going out information input means" is a means by which the user inputs information such as the purpose, date and time of going out, and location.

[0166] The "weather forecast acquisition means" is a means for acquiring the weather forecast for the location and date / time when the user plans to go out.

[0167] The "coordination generating means" is a means for generating a coordination pattern based on weather forecast data, outing information, and product data in closets and physical stores.

[0168] The "feedback collection means" is a means for collecting feedback such as evaluations of the coordinated outfits selected by the user.

[0169] The "reuse and recycling suggestion tool" is a method for listing items that have not been used for a certain period of time and providing methods for reusing or recycling them.

[0170] "Means for building a database to digitize products in physical stores" refers to a means for registering and managing product information in physical stores in a database using image analysis, etc.

[0171] The "information presentation means using a display device that can be worn by a user" is a means for presenting information to a user through smart glasses or the like.

[0172] This invention is a system for improving the customer purchasing experience in a physical store, and includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordination generation means, a feedback collection means, a reuse / recycle suggestion means, a database construction means for converting products in the physical store into data, and an information presentation means using a display device that can be worn by the user.

[0173] The server executes a program that integrates these means. The specific processing is as follows:

[0174] 1. Data collection method

[0175] The server uses an image capture device to capture images of products in a physical store and uploads them to a cloud server (e.g., AWS). The uploaded images are analyzed using an image analysis library (e.g., OpenCV) or a machine learning model (e.g., TensorFlow). Information obtained from the analyzed images (e.g., color, material, category) is stored in a database (e.g., MySQL).

[0176] 2. Means of inputting outing information

[0177] The user wears a display device such as smart glasses and inputs the purpose, date, time, and location of their outing through the interface. The input information is then sent from the smart glasses to the server.

[0178] 3. How to get weather forecasts

[0179] Based on the outing information received from the user, the server uses a weather forecast acquisition API (for example, OpenWeatherMap API) to acquire weather forecast data for the location and date / time of the planned outing.

[0180] 4. Coordination Generation Method

[0181] The server generates optimal outfits from a database based on weather forecast data and the user's outing information, and the generated outfit patterns are displayed in real time on the user's smart glasses.

[0182] 5. Feedback Collection Methods

[0183] Users use the smart glasses to evaluate the suggested outfits, and the feedback information is sent to the server. The feedback data is stored in a database and reflected in future suggestions.

[0184] 6. Reuse and recycling proposal methods

[0185] Users can check information about recycled products and campaigns available at physical stores through the smart glasses, and the server will make reuse and recycling suggestions based on the user's preferences.

[0186] Specific examples

[0187] A user uses smart glasses and speaks: "Tomorrow is my date, so please dress casually and stylishly." The server checks the weather forecast, determines that it will be sunny, and suggests outfits such as a white cotton shirt, blue denim pants, casual sneakers, and a simple silver pendant.

[0188] Also, if a recycling campaign is being held in a physical store, the user can say to the smart glasses, "Tell me about the recycling campaign." The smart glasses will then display details of the campaign and eligible products.

[0189] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0190] Step 1:

[0191] The server takes pictures of products in a physical store using an image capture device and uploads the pictures to a cloud server (e.g., AWS). The input is the captured image, and the output is image data stored on the cloud server. Specifically, the captured product images are automatically transferred to the server.

[0192] Step 2:

[0193] The server uses an image analysis library (e.g., OpenCV) and a machine learning model (e.g., TensorFlow) to analyze the image data stored on the cloud server. The input is the stored image data, and the output is the analyzed information (color, material, category, etc.). Specifically, the image analysis algorithm extracts product features from the image and stores this information in a database.

[0194] Step 3:

[0195] The user puts on the smart glasses and inputs the purpose, date, time, and location of their outing by voice. The input is the user's voice information, and the output is the outing information sent to the server. Specifically, the user inputs a voice command such as "Tomorrow is a date, so please dress casually and stylishly" into the smart glasses.

[0196] Step 4:

[0197] The server uses a weather forecast acquisition API (for example, OpenWeatherMap API) to obtain weather forecast data for the planned outing location and date and time. The input is the user's outing information, and the output is the obtained weather forecast data. Specifically, the server sends an API request and analyzes and saves the returned weather forecast data.

[0198] Step 5:

[0199] The server generates optimal outfits from a database based on weather forecast data and outing information. The inputs are weather forecast data, outing information, and item information from the database, and the output is an outfit pattern. Specifically, it uses a generative AI model to suggest outfits based on the items in the database, the weather forecast, and outing information.

[0200] Step 6:

[0201] The server displays the generated coordination patterns on the user's smart glasses in real time. The input is the coordination pattern, and the output is the coordination information presented to the smart glasses. Specifically, the proposed coordination is displayed on the smart glasses display as text and images.

[0202] Step 7:

[0203] The user evaluates the outfits presented to them through the smart glasses and sends feedback information to the server. The input is the user's feedback, and the output is the feedback data sent to the server. Specifically, the user inputs their evaluation, such as "good" or "bad," through the interface of the smart glasses.

[0204] Step 8:

[0205] The server analyzes the feedback data and stores it in a database. The input is feedback data from users, and the output is a database that reflects the feedback. Specifically, the generative AI model is updated based on the feedback information and reflected in the next proposal.

[0206] Step 9:

[0207] This system notifies users of reuse and recycling campaign information at physical stores. The input is the store's campaign information, and the output is the campaign information displayed on the smart glasses. Specifically, the server pushes the campaign information to the smart glasses so that the user can check it.

[0208] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0209] The present invention is a system that digitizes items in a user's closet and suggests outfits based on the purpose of going out, and further combines an emotion engine that recognizes the user's emotions. This system includes a data conversion means, an outing information input means, a weather forecast acquisition means, an outfit creation means, a feedback collection means, a reuse / recycle suggestion means, and an emotion engine.

[0210] 1. Data collection method

[0211] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads the image to their device. The device then sends the image to the server. The server analyzes the image and identifies each item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), which are then stored in a database. Items are tagged and registered in the closet database.

[0212] 2. Means of inputting outing information

[0213] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[0214] 3. How to get weather forecasts

[0215] The server retrieves weather forecast data based on the input date, time, and location of the planned outing. For example, it retrieves the data using a weather forecast API.

[0216] 4. Coordination Generation Method

[0217] The server generates multiple outfit patterns based on the acquired weather forecast data and the user's closet database. The generated outfit patterns are adjusted to suit the user's purpose and conditions for going out. The server then transmits the generated outfit patterns to the terminal.

[0218] 5. Feedback Collection Methods

[0219] The user selects from multiple outfit patterns presented on the device screen. The device then sends the selection results and information on the outfits that were not selected back to the server. The server analyzes the feedback data, learns the user's preferences and style trends, and stores the information in a database.

[0220] 6. Emotion Engine

[0221] The device recognizes the user's emotions using facial expressions and voice data. This emotion data is sent from the device to a server, where it is analyzed. For example, the emotion engine determines whether the user is satisfied or dissatisfied with the outfit suggestions. The emotion data collected by the emotion engine is reflected in the outfit suggestions and learning process.

[0222] 7. Reuse and recycling proposal methods

[0223] When a user wants to organize their closet, they request reuse and recycling suggestions from their device. The request is sent from the device to the server. The server references the closet database and lists items that have not been used for a certain period of time. The listed items are sent to the device and suggested to the user. The user checks the suggested items and selects whether to reuse or recycle them. The selection results are sent from the device to the server, which provides a guide on how to reuse and recycle them.

[0224] Specific examples

[0225] If a user requests a full outfit for a date, the system will suggest outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also presented based on the weather forecast, and the user can choose from them. The emotion engine analyzes the user's facial expressions to determine whether they are satisfied. Based on the results, future suggestions are adjusted. If they are dissatisfied, the system will try a different outfit.

[0226] In the reuse and recycling scenario, the app identifies infrequently used items in the closet, such as "blue denim pants that haven't been worn for six months." When the user selects it, they are given instructions such as contact information for a recycle shop or a donation destination.

[0227] In this way, a system will be realized that allows users to efficiently utilize their closets and, by utilizing emotion recognition to receive more personalized outfit suggestions, will reduce the stress of choosing outfits before going out and contribute to a sustainable society.

[0228] The processing flow will be explained below.

[0229] Step 1:

[0230] A user takes a photo of an item in their closet and uploads the image to their device.

[0231] Step 2:

[0232] The device receives the uploaded image and sends it to the server.

[0233] Step 3:

[0234] The server analyzes the images, identifies the item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), and stores them in a database. Each item is given the appropriate tags.

[0235] Step 4:

[0236] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[0237] Step 5:

[0238] The server retrieves weather forecast data based on the date, time, and location of the planned outing. For example, it retrieves the data using a weather forecast API.

[0239] Step 6:

[0240] The server generates multiple outfit patterns based on weather forecast data and the user's closet database. The generated outfit patterns are adjusted to suit the user's needs and requirements.

[0241] Step 7:

[0242] The server sends the generated coordinate patterns to the terminal, which displays them on the screen and presents the user with multiple options.

[0243] Step 8:

[0244] The user selects a desired coordinate pattern from the presented patterns, and the terminal sends the selection result to the server.

[0245] Step 9:

[0246] As a means of collecting feedback, the terminal sends information about the coordinated outfits selected and not selected by the user to the server. The server analyzes this information, learns the user's preferences and style trends, and updates the database.

[0247] Step 10:

[0248] The device acts as an emotion engine, recognizing the user's emotions using facial expressions and voice data. This emotion data is sent from the device to a server, where it is analyzed.

[0249] Step 11:

[0250] Based on the emotional data analyzed by the emotion engine, the proposed outfits are evaluated and reflected in future suggestions. For example, if the user is smiling and satisfied, it will be easier to save that style.

[0251] Step 12:

[0252] When a user wants to organize their closet, they request reuse and recycling suggestions from their device, and the request is sent from the device to the server.

[0253] Step 13:

[0254] The server checks the closet database and lists items that have not been used for a certain period of time, for example, items that have not been used in the last six months.

[0255] Step 14:

[0256] The server generates a list of unwanted items and sends it to the terminal, which then presents the list to the user.

[0257] Step 15:

[0258] The user checks the list and selects the items they want to reuse or recycle, and the selection results are sent from the terminal to the server.

[0259] Step 16:

[0260] The server will provide a guide on how to reuse and recycle selected unwanted items (e.g., contact details for recycling shops and donation locations).

[0261] Step 17:

[0262] Users follow the provided guide to reuse and recycle unwanted items.

[0263] Example 2

[0264] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0265] Conventional coordination suggestion systems have had problems in that they are difficult to fully reflect the user's emotions and preferences, and lack the functionality to suggest reuse and recycling to promote sustainable fashion. In order to solve these problems, the present invention aims to provide a system that takes into account the user's emotions and can also suggest reuse and recycling.

[0266] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0267] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, an emotion recognition means, and a reuse / recycle suggestion means. This not only makes it possible to convert the items in the user's closet into data and suggest optimal coordinates based on the purpose of going out while utilizing weather forecast information, but also analyzes the user's emotions to increase satisfaction with the coordinates and make suggestions for reuse / recycle.

[0268] The "data generation means" is a means for registering items in the user's closet in a database through image analysis.

[0269] The "outing information input means" is a means by which a user can input information regarding the purpose, date, time, and location of going out.

[0270] The "weather forecast acquisition means" is a means for acquiring weather forecast information based on the date, time and location of a planned outing.

[0271] The "coordination generating means" is a means for generating an optimal coordination pattern based on the acquired weather forecast data and the user's closet database.

[0272] The "feedback collection means" is a means for allowing a user to select a proposed coordination pattern and collecting information on the selection results and non-selection results.

[0273] The "emotion recognition means" is a means for recognizing emotions by analyzing the user's facial expressions and voice data.

[0274] The "reuse / recycle suggestion tool" is a method for identifying items in the closet that have not been used for a certain period of time and making suggestions for reuse or recycling.

[0275] The present invention is a system that digitizes the items in a user's closet and suggests outfits based on the purpose of going out, and further combines it with an emotion engine that recognizes the user's emotions. This system includes a data generation means, an outing information input means, a weather forecast acquisition means, an outfit creation means, a feedback collection means, a reuse / recycle suggestion means, and an emotion recognition means.

[0276] System Overview

[0277] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads the image to a device, such as a smartphone or tablet. The device then sends the uploaded image to a server. The server analyzes the image and identifies each item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), which are then stored in a database. Image analysis algorithms such as OpenCV and TensorFlow are used for this image analysis.

[0278] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and location through the device interface, and this information is sent from the device to the server.

[0279] The server obtains weather forecast data based on the input date, time, and location of the planned outing. For example, it obtains data using a weather forecast API such as OpenWeatherMap or WeatherAPI. The server generates multiple outfit patterns based on the obtained weather forecast data and the user's closet database. This generation uses a generative AI model (e.g., GPT-3) and takes into account the user's preferences and style. The generated outfit patterns are sent from the server to the device, which then presents them to the user.

[0280] The user selects one of the presented outfit patterns. The device then sends the selection result and information about the outfits that were not selected back to the server. The server analyzes the feedback data, learns the user's preferences and style, and reflects this in its next outfit suggestions.

[0281] The device uses a built-in camera and microphone to collect the user's facial expressions and voice data. This data is analyzed using an emotion recognition algorithm (e.g., EmotionAPI). The emotion data is sent from the device to a server, where it is analyzed. This determines whether the user is satisfied or dissatisfied with the suggested outfit. This emotion data is also reflected in future outfit suggestions.

[0282] When a user wants to organize their closet, they request reuse and recycling suggestions from their device. The request is sent from the device to the server, which analyzes the closet database and lists items that have not been used for a certain period of time. The listed items are sent to the device and suggested to the user. The user reviews the suggested items and chooses whether to reuse or recycle them. The selection results are sent from the device to the server, which provides a guide on how to reuse and recycle them.

[0283] This system not only provides users with efficient and individually optimized outfit suggestions, but also offers highly satisfying suggestions based on emotion recognition.It also promotes the reuse and recycling of closet items, helping to realize a sustainable fashion lifestyle.

[0284] Specific examples

[0285] For example, if a user enters the prompt "I have a date in a week. Please suggest an outfit based on the weather forecast and the items in my closet," the system will suggest outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also presented based on the weather forecast, and the user can select one. The emotion engine analyzes the user's facial expressions to determine whether they are satisfied. Based on the results, future suggestions are adjusted. If they are dissatisfied, the system can try a different outfit.

[0286] In the reuse and recycling scenario, the app identifies infrequently used items in the closet, such as "blue denim pants that haven't been worn for six months." When the user selects it, they are given instructions such as contact information for a recycle shop or a donation destination.

[0287] As a result, users can make effective use of closet items and enjoy more personalized coordination.

[0288] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0289] Step 1:

[0290] A user takes a photo of an item in their closet and uploads the image to their device. This is the input, and the specific data is an image file. The device receives the image and sends it to the server. The device's processing is the transmission of image data.

[0291] Input: Image file

[0292] Output: Send image data to the server

[0293] Step 2:

[0294] The server analyzes the received images and identifies the color, material, and category of each item. This analysis uses image analysis algorithms (e.g., OpenCV, TensorFlow). The analysis results are stored in a database, and each item is assigned a tag. The data obtained from the image analysis process is the item's attribute information.

[0295] Input: Image data

[0296] Output: Item color, material, and category information

[0297] Step 3:

[0298] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the terminal interface. This information is the input data. The terminal sends the input information to the server. The terminal's processing is to send the outing information.

[0299] Input: Purpose of outing, date and time, location information

[0300] Output: Sending out-of-office information to the server

[0301] Step 4:

[0302] The server retrieves weather forecast data based on the input date, time, and location of the planned outing. It sends a request to a weather forecast API (e.g., OpenWeatherMap, WeatherAPI) and receives the forecast data. The retrieved data is the weather forecast. The server stores this data in a database.

[0303] Input: Date and time of outing, location information

[0304] Output: Weather forecast data

[0305] Step 5:

[0306] The server generates optimal outfit patterns based on the acquired weather forecast data and the user's closet database. Here, a generative AI model (e.g., GPT-3) is used, taking into account the user's preferences and style. The generated multiple outfit patterns are the output data.

[0307] Input: Weather forecast data, closet database

[0308] Output: Coordination pattern

[0309] Step 6:

[0310] The server sends the generated coordination patterns to the terminal, which then presents them to the user. The user selects one of the presented coordination patterns, and the selection information is the input data. The terminal then sends the selection result and information on the coordination patterns not selected to the server.

[0311] Input: Coordination pattern, user selection

[0312] Output: Send feedback data to the server

[0313] Step 7:

[0314] The server analyzes the feedback data and learns the user's preferences and style. The analysis results are stored in a database and reflected in the next outfit suggestions. The data obtained through the analysis process is the user's preferences and style trends.

[0315] Input: Feedback data

[0316] Output: User preferences and style trends data

[0317] Step 8:

[0318] The device uses the built-in camera and microphone to collect the user's facial expressions and voice data. An emotion recognition algorithm (e.g., EmotionAPI) is used to identify the user's emotions. This emotion data is the input data. The device sends the emotion data to the server, which then determines the user's satisfaction level based on the analysis results.

[0319] Input: facial expressions, voice data

[0320] Output: Emotion data, satisfaction rating

[0321] Step 9:

[0322] When a user wants to organize their closet, they request reuse and recycling suggestions through the device interface. The content of this request is the input data. The device sends the request to the server, which analyzes the closet database and lists items that have not been used for a certain period of time. The listed items are the output data.

[0323] Input: Reuse / Recycle Request

[0324] Output: List of items suggested for reuse and recycling

[0325] Step 10:

[0326] The terminal presents a list of suggested items to the user, who then selects whether to reuse or recycle. The selection result is the input data. The terminal then sends the selection result to the server, which provides a guide on how to reuse or recycle. The guide information is the output data.

[0327] Input: Reuse / recycle selection results

[0328] Output: Reuse and recycling guide

[0329] (Application example 2)

[0330] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0331] Conventional coordination suggestion systems do not collect enough information to make appropriate suggestions using the items in a user's closet, making it difficult to make personalized suggestions that take into account the user's emotions and external factors such as the weather. Furthermore, there is a lack of systems that effectively suggest ways to organize a user's closet and recycle and reuse items. Therefore, there is a need for improved user experience and environmental considerations.

[0332] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0333] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, a reuse / recycle suggestion means, an emotion analysis means, a means for analyzing a user's facial image to identify emotions, a means for creating multiple coordinate patterns based on the weather forecast data and facial image analysis data, and a means for accumulating the feedback emotion data and reflecting it in the next suggestion. This enables personalized coordinate suggestions that take the user's emotions and the weather forecast into consideration, and also enables efficient closet organization and recycling / reuse suggestions.

[0334] The "data generation means" is a means for generating data from items in the user's closet through image analysis.

[0335] The "going out information input means" is a means by which the user inputs information about the purpose, date and time of going out, and the location.

[0336] The "weather forecast acquisition means" is a means for acquiring weather forecast data based on the date, time and location of a planned outing.

[0337] The "coordination generating means" is a means for generating a plurality of coordination patterns based on the acquired weather forecast data and closet database.

[0338] The "feedback collection means" is a means for collecting the user's selection results and feedback on the presented coordination patterns.

[0339] The "reuse / recycle suggestion tool" is a tool that allows users who wish to organize their closets to list items that have not been used for a certain period of time and suggests reuse or recycling.

[0340] The "emotion analysis means" is a means for analyzing a facial image of a user to identify the emotion.

[0341] "Weather forecast data" refers to weather information for a specific date, time and location obtained using a weather forecast API or similar.

[0342] "Facial image analysis data" is emotional data obtained by analyzing the facial image of the user.

[0343] A "closet database" is a database in which items in a user's closet are stored as data.

[0344] The "emotion engine" is an engine that recognizes emotions by analyzing the user's facial expressions and voice.

[0345] A "server" is a computer system that analyzes data, stores data, calls APIs, and generates proposals.

[0346] This invention is implemented by the following means. A system in which a user uses a smartphone or other device to digitize the items in their closet through image analysis, and suggests optimal outfits to the user based on information such as the purpose of the outing and the weather forecast. It is also possible to identify emotions by analyzing the user's facial expressions and voice, and reflect this as feedback in the next suggestions.

[0347] Hardware and software used

[0348] Smartphone: Used by users to take pictures of items, input outing information, and check suggested outfits.

[0349] Server: Analyzes data, obtains weather forecasts, generates coordinates, and accumulates feedback data.

[0350] Keras: Used to run deep learning models for recognizing emotions from facial images.

[0351] OpenCV: Used to perform image analysis of items in the closet.

[0352] Weather forecast API: Obtain weather data based on the date, time, and location you plan to go out.

[0353] Data processing and calculation

[0354] 1. Data collection method: The user takes a photo of the items in their closet with their smartphone and uploads the image to the server. The server analyzes the image, extracts attribute information such as color, material, and category, and stores it in the closet database. This process is performed using OpenCV.

[0355] 2. Means for inputting outing information: The user inputs the purpose, date, time, and location of the outing through the smartphone application. This information is sent to the server.

[0356] 3. Weather forecast acquisition method: The server uses the weather forecast API to acquire the weather forecast for the date, time, and location specified by the user.

[0357] 4. Coordination Generation: The server generates multiple coordination patterns based on weather data and the closet database, taking into account the user's preferences and style trends. The generated coordination patterns are sent to the user's smartphone and displayed to them.

[0358] 5. Feedback collection method: The user selects from the proposed outfits and sends the results back to the server. This allows the server to collect information on outfits that were not selected, and learns the user's style tendencies.

[0359] 6. Emotion analysis: The smartphone captures the user's facial image and analyzes their facial expressions using Keras. The recognized emotion data is sent to the server and used to improve the accuracy of outfit suggestions.

[0360] 7. Reuse / Recycle Suggestion Method: When a user wants to organize their closet, the request is sent to the server. The server refers to the closet database and lists items that have not been used for a certain period of time. This allows the user to make suggestions for reuse or recycling.

[0361] Specific examples

[0362] 1. The user takes a photo of their own shirt or pants with their smartphone and registers these items in a database through the application.

[0363] 2. The user enters an upcoming "casual lunch" into the app, specifying the date, time, and location of the outing.

[0364] 3. The server uses the weather forecast API to get the weather for the specified date, time, and location. If the weather is "sunny," it generates a light outfit.

[0365] 4. The server sends the generated outfit to the user and displays it on their smartphone. The user selects the outfit, and the smartphone captures their face image and analyzes their emotions.

[0366] 5. The emotion engine recognizes "satisfaction" from the user's facial expression and feeds this data back to the server, which will then be reflected in the next suggestion.

[0367] Prompt Sentence Examples

[0368] Closet item image analysis:

[0369] image_path = 'user_shirt.jpg'

[0370] item_data = app.analyze_image(image_path)

[0371] Enter your travel information:

[0372] outing_type = "casual lunch"

[0373] outing_datetime = datetime(2023, 10, 24, 12, 30)

[0374] outing_location = "Tokyo"

[0375] outing_info = app.input_outing_info(outing_type, outing_datetime, outing_location)

[0376] Coordinate generation:

[0377] weather_data = app.get_weather_forecast(outing_location, outing_datetime)

[0378] coordinates = app.generate_coordinate(outing_info, weather_data)

[0379] Collecting emotional feedback:

[0380] user_choice = coordinates[0]

[0381] emotion_feedback = app.provide_feedback(user_choice)

[0382] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0383] Step 1:

[0384] A user takes a photo of an item in their closet with their smartphone and uploads the image to a device. The device then sends the image to the server, which uses OpenCV to analyze the image and extract attribute information such as the item's color, material, and category. This data is then stored in a closet database.

[0385] (Input) Item image

[0386] (Data processing) Image analysis, attribute information extraction

[0387] (Output) Saving attribute information to a database

[0388] Step 2:

[0389] The user enters information about the purpose, date, time, and location of their outing through a smartphone application, and this information is sent from the device to a server.

[0390] (Input) Purpose, date, time, and location information

[0391] (Data calculation) Information storage

[0392] (Output) Send data to the server, save information

[0393] Step 3:

[0394] The server uses the weather forecast API to retrieve weather forecast data for the date, time, and location specified by the user, which involves calling an external weather forecast service and retrieving the required data.

[0395] (Input) Date, time, and location information

[0396] (Data calculation) API call, weather data acquisition

[0397] (Output) Obtaining weather forecast data

[0398] Step 4:

[0399] The server generates multiple outfit patterns suitable for the user's outing based on weather data and a closet database. Past feedback data is also taken into consideration. The generated outfit patterns are sent to the device.

[0400] (Input) Weather data, closet database, usage information

[0401] (Data calculation) Coordination generation, reflecting preference trends

[0402] (Output) Sending coordinate patterns to the device

[0403] Step 5:

[0404] The user selects from multiple outfit patterns displayed on the smartphone screen. The selection result is fed back from the device to the server. The server accumulates this information and learns the user's preferences and style trends.

[0405] (Input) Selection result

[0406] (Data calculation) Feedback collection, trend learning

[0407] (Output) Saving to database

[0408] Step 6:

[0409] The device captures the user's facial image and performs emotion analysis. The emotion engine uses Keras to analyze the facial expressions and sends the recognized emotion data to the server. The server receives this emotion data as feedback and uses it to improve future suggestions.

[0410] (Input) Facial image

[0411] (Data calculation) Emotion analysis, data transmission

[0412] (Output) Acquisition and storage of emotion data

[0413] Step 7:

[0414] When a user wants to organize their closet, they send a request from their device to the server. The server then references the closet database, lists items that have not been used for a certain period of time, and suggests reuse or recycling to the user. The results and instructions are displayed on the device.

[0415] (Input) Arrangement request

[0416] (Data calculation) Data reference, list proposal

[0417] (Output) Reuse and recycling proposals

[0418] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0419] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0420] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0421] [Second embodiment]

[0422] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0423] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0424] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0425] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0426] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0427] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0428] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0429] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0430] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0431] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0432] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0433] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0434] This invention is a system that digitizes and visualizes Western and Japanese clothing stored in closets and dressers, and allows a stylist to suggest outfits for the entire body simply by inputting the purpose of going out. This system includes a data conversion means, a means for inputting outing information, a means for obtaining weather forecasts, a means for creating outfits, a means for collecting feedback, and a means for suggesting reuse and recycling.

[0435] 1. Data collection method

[0436] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads it to their device. The device then transfers the image to the server. The server analyzes the received image and identifies each item's color, material, and category (tops, bottoms, accessories, shoes, etc.). It also tags the items appropriately and stores them in the user's personal closet database.

[0437] 2. Means of inputting outing information

[0438] The user inputs information such as the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[0439] 3. How to get weather forecasts

[0440] The server retrieves weather forecast data based on the date, time, and location of the planned outing, for example, by retrieving data from a weather forecast service via an API.

[0441] 4. Coordination Generation Method

[0442] The server generates multiple outfit patterns based on the acquired weather forecast data and the user's closet database. The generated outfit patterns correspond to the user's specified purposes and conditions. For example, it suggests a combination of a white cotton shirt with blue denim pants, black leather boots, and a silver pendant. The server then sends the generated outfit patterns to the terminal.

[0443] 5. Feedback Collection Methods

[0444] The user selects one of several outfit patterns presented on the device screen. The device then sends information about the outfits selected and not selected back to the server. The server analyzes the feedback data and learns the user's preferences and style patterns. This information is then stored in a closet database.

[0445] 6. Reuse and recycling proposal methods

[0446] When a user wishes to organize their closet, they request reuse and recycling suggestions from their device. The device then sends the request to the server. The server checks the closet database and lists items that have not been used for a certain period of time (e.g., six months). The listed unwanted items are then suggested to the user via their device. The user selects candidate items for reuse and recycling, and the selection results are sent to the server. The server then provides a guide on how to reuse or recycle the selected unwanted items (e.g., contact information for recycling shops or donation destinations).

[0447] Specific examples

[0448] When a user requests a date outfit, the system suggests outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also suggested based on the weather forecast. When a user selects a specific outfit, that information is fed back to the server and reflected in future suggestions.

[0449] In the reuse and recycling scenario, the app identifies infrequently used items in the closet and suggests ways to reuse or recycle them. For example, it might suggest, "Your blue denim pants haven't been worn for six months." If the user selects that item, the app provides guidance on where to find a recycle shop or where to donate the items.

[0450] In this way, a system is realized that allows users to efficiently utilize their closets, reduce the stress of choosing outfits before going out, and contribute to a sustainable society.

[0451] The processing flow will be explained below.

[0452] Step 1:

[0453] A user takes a photo of an item in their closet and uploads the image to their device.

[0454] Step 2:

[0455] The terminal receives the uploaded image and sends it to the server.

[0456] Step 3:

[0457] The server analyzes the image and identifies the item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.).

[0458] Step 4:

[0459] The server analyzes the item information, tags it, and stores it in the user's personal closet database.

[0460] Step 5:

[0461] The user inputs information about the purpose, date, time, and location of their outing via the terminal and requests outfit suggestions.

[0462] Step 6:

[0463] The device sends the request to the server.

[0464] Step 7:

[0465] The server retrieves weather forecast data based on the planned date, time, and location of the person going out, for example, using a weather forecast API.

[0466] Step 8:

[0467] The server generates multiple coordination patterns based on weather forecast data acquired and the user's closet database.

[0468] Step 9:

[0469] The server transmits the generated coordinate pattern to the terminal and presents it to the user.

[0470] Step 10:

[0471] The user selects a preferred coordination pattern from a plurality of presented patterns.

[0472] Step 11:

[0473] The terminal feeds back to the server information on the user's selection results and unselected coordinates.

[0474] Step 12:

[0475] The server analyzes the feedback data and updates a database to learn the user's preferences and coordination trends.

[0476] Step 13:

[0477] If a user wants to organize their closet, they can request reuse and recycling suggestions from their device.

[0478] Step 14:

[0479] The device sends a recycle request to the server.

[0480] Step 15:

[0481] The server references the closet database and lists items that are used less frequently.

[0482] Step 16:

[0483] The server generates a list of unwanted items and sends it to the terminal.

[0484] Step 17:

[0485] The user reviews the list and selects the items they want to reuse or recycle.

[0486] Step 18:

[0487] The terminal transmits the selection result to the server.

[0488] Step 19:

[0489] The server generates a guide on how to reuse and recycle the selected unwanted items (e.g., contact information for recycling shops or donation destinations) and sends it to the device.

[0490] Step 20:

[0491] The user follows the guide to dispose of unwanted items.

[0492] Example 1

[0493] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0494] Currently, systems for managing users' owned items and suggesting appropriate outfits are not fully developed. Furthermore, there is a lack of methods for efficiently suggesting outfits appropriate for the weather when users are out, or for reusing and recycling unnecessary items. This often causes inconvenience to users when choosing their daily outfits. The present invention aims to solve these problems and provide a system that can suggest more efficient and appropriate outfits and manage owned items.

[0495] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0496] In this invention, the server includes a means for digitizing items in the closet, a means for inputting the purpose, date, time, and location of an outing, a means for acquiring a weather forecast for the planned outing, a means for generating outfits based on the acquired weather forecast and the digitized items, a means for collecting feedback on the generated outfits, and a means for suggesting reuse and recycling for items that have not been used for a certain period of time. This allows users to reduce the effort of choosing their daily outfits and find the perfect outfit for the weather and purpose. It also enables efficient management of infrequently used items and suggestions for reuse and recycling.

[0497] "A means of digitizing items in the closet" refers to a system that takes photos of the clothing and accessories owned by the user, analyzes the image data to identify attributes such as color, material, and category, and stores these in a database dedicated to the user.

[0498] "Means for inputting the purpose, date, time, and location of the outing" refers to a system that has an interface for users to input information such as the purpose of the outing (e.g., date, work, casual), planned date, and location.

[0499] "Means for obtaining weather forecasts for planned outings" refers to a system that has the function of obtaining forecast data from a weather forecast service based on a specified date, time and location, analyzing that information, and reflecting it in the closet system.

[0500] "Means for generating coordination" refers to a system that generates and suggests optimal clothing combinations based on the user's purpose and conditions for going out, based on acquired weather forecast data and digitized item information.

[0501] "Means for collecting feedback on the generated outfits" refers to a system that has the ability to record the outfits selected by the user from multiple suggested outfits, analyze the selection data, and learn about the user's preferences and style.

[0502] "A means of proposing reuse and recycling for items that have not been used for a certain period of time" refers to a system that has the function of identifying items that have not been used for a certain period of time based on the usage history of items stored in a closet database, and proposing recycling and reuse methods for these items.

[0503] This invention is a system that digitizes the clothes and accessories in a closet or dresser, and automatically suggests outfits for the entire body simply by the user inputting the purpose of going out. This system includes "means for digitizing items in the closet," "means for inputting the purpose, date, time, and location of the outing," "means for obtaining the weather forecast for the planned outing," "means for generating outfits," "means for collecting feedback on the generated outfits," and "means for suggesting reuse and recycling of items that have not been used for a certain period of time."

[0504] Hardware and software used

[0505] Devices are devices available to users, such as smartphones and tablets. Servers are often located in the cloud and have very powerful processors and large amounts of storage. Software includes:

[0506] Image analysis algorithms: TensorFlow, OpenCV

[0507] Weather APIs: OpenWeatherMap, Weather.com

[0508] Generative AI model: A machine learning model used to analyze user data and generate outfits

[0509] Specific processing of the system

[0510] 1. The user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) on their device, which then uploads the photo data to the server.

[0511] 2. The server uses image analysis algorithms to analyze the received photos and identify the color, material, and category of each item. Based on this, the items are tagged appropriately and stored in the user's personal closet database.

[0512] 3. The user enters information about the purpose of the outing (e.g., date, work, casual), date and time, location, etc. through the device interface. This information is sent from the device to the server.

[0513] 4. The server retrieves weather forecast data based on the planned date, time, and location of the person going out, for example, from a weather forecast service via the OpenWeatherMap API.

[0514] 5. The server uses the acquired weather forecast data and the generative AI model to generate multiple outfit patterns based on the user's closet database. The generated outfit patterns correspond to the user's specified purposes and conditions.

[0515] 6. The user selects one of several outfit patterns presented on the device screen. The device then sends information about the selected and unselected outfits back to the server. The server analyzes the feedback data and learns the user's preferences and style patterns. This information is then stored in a closet database.

[0516] 7. If a user wishes to organize their closet, they request reuse and recycling suggestions from their device. The device sends the request to the server. The server checks the closet database and lists items that have not been used in the past six months. The listed unwanted items are then suggested to the user via their device. The user selects candidate items for reuse and recycling, and the selection results are sent to the server. The server then provides a guide on how to reuse or recycle the selected unwanted items (e.g., contact information for recycle shops or donation destinations).

[0517] Examples of concrete examples and prompts

[0518] Specific examples

[0519] When a user requests a date outfit, the system suggests outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also suggested based on the weather forecast. When a user selects a specific outfit, that information is fed back to the server and reflected in future suggestions. In the reuse and recycling section, infrequently used items in the closet are identified and suggestions for reuse or recycling are made. For example, if the system suggests "Your blue denim pants haven't been used in six months," and the user selects that option, the system provides guidance on recycle shop contact details and donation locations.

[0520] Prompt Sentence Examples

[0521] "Please suggest an outfit for a date."

[0522] "Tell me what is appropriate clothing for work"

[0523] "Please suggest an outfit that would be suitable for a casual outing."

[0524] "Make a list of the least used items in your closet."

[0525] This allows users to efficiently manage their closets and reduce the effort required to choose outfits before going out. It also allows for efficient reuse and recycling, contributing to a sustainable society.

[0526] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0527] Step 1:

[0528] The user takes photos of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) on their device. The device then uploads these photo data to the server. The input is the photo of the item, and the output is the image data sent to the server. As soon as the server receives the photo data, it begins analyzing it using an image analysis algorithm (e.g., TensorFlow or OpenCV) to identify the color, material, and category of each item. Specifically, it analyzes the hue and texture pattern of each pixel to extract attributes. The output is the color, material, and category information resulting from the analysis.

[0529] Step 2:

[0530] The server generates appropriate tags (color, material, category) based on the results of image analysis and stores them in the user's personal closet database. The input is the analyzed item information, and the output is the data of the newly tagged item added to the closet database.

[0531] Step 3:

[0532] The user inputs information such as the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input is the purpose of the outing, date and time, and location, and the output is the data that is sent from the device to the server. The device receives the input information and sends it to the server as structured data. Specific operations include the use of text boxes and drop-down menus.

[0533] Step 4:

[0534] The server retrieves weather forecast data based on the received planned date, time, and location of the outing. The input is the planned date, time, and location of the outing, and the output is the retrieved weather forecast data. Specifically, it sends a request to a weather forecast service (e.g., OpenWeatherMap API) and retrieves the corresponding weather forecast data (e.g., wind speed, probability of precipitation, temperature, etc.).

[0535] Step 5:

[0536] The server uses a generative AI model to generate multiple outfit patterns based on weather forecast data and the user's closet database. The input is weather forecast data and the user's item data, and the output is the generated outfit patterns. Specifically, data such as frequency of use, category, color, material, and tag are input into the generative AI model to generate outfit patterns. The generated patterns are optimized for the user's purposes and conditions.

[0537] Step 6:

[0538] The server sends the generated coordinate pattern to the terminal. The input is the generated coordinate pattern, and the output is the coordinate pattern sent to the user's terminal. The terminal displays the received pattern on the user's screen.

[0539] Step 7:

[0540] The user selects one of several coordination patterns. The input is the selection information for the coordination pattern, and the output is the information for the selected coordination. The device sends the selected information to the server and saves it as feedback data, including data on coordinations that were not selected. Specific actions include selection operations by tapping or clicking.

[0541] Step 8:

[0542] The server analyzes the feedback data and learns the user's preferences and style patterns. The input is the feedback data, and the output is updated user style data. Specifically, the server uses a machine learning algorithm to analyze the feedback data and update the user's style information.

[0543] Step 9:

[0544] When a user wants to organize their closet, they request reuse and recycling suggestions from their terminal. The input is the request content, and the output is the transmission of the request data. The terminal receives the request content and sends it to the server.

[0545] Step 10:

[0546] The server checks the closet database and lists items that have not been used in the past six months. The input is the user's closet data, and the output is the list of unwanted items. The server sends the listed items to the terminal and suggests them to the user.

[0547] Step 11:

[0548] The user selects candidate items for reuse or recycling and sends the selection results to the server. The input is the selected item information, and the output is the selection results sent to the server.

[0549] Step 12:

[0550] The server provides the terminal with information on how to reuse or recycle the selected unwanted items (e.g., contact information for recycle shops or donation destinations). The input is the selection result data, and the output is information on reuse and recycling suggestions. Specifically, the server provides the user with contact information for recycle shops and details of donation destinations.

[0551] (Application example 1)

[0552] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0553] Conventional systems suggest outfits by digitizing the items a user owns and inputting information about outings. However, in physical stores, real-time customer service is required, so a system is needed that efficiently digitizes products in the physical store and allows users to instantly receive outfit suggestions. Furthermore, improving the customer's purchasing experience by providing personalized suggestions that take into account the purpose of the outing and weather forecast is a challenge.

[0554] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0555] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, a reuse / recycle suggestion means, a database construction means for converting products in the physical store into data, and an information presentation means using a display device that can be worn by a user. This makes it possible to quickly convert products in the physical store into data and provide optimal coordinates based on the weather forecast and outing information in real time.

[0556] "Data generation means" refers to the means of registering and managing items from users and physical stores in a database using image analysis, etc.

[0557] The "going out information input means" is a means by which the user inputs information such as the purpose, date and time of going out, and location.

[0558] The "weather forecast acquisition means" is a means for acquiring the weather forecast for the location and date / time when the user plans to go out.

[0559] The "coordination generating means" is a means for generating a coordination pattern based on weather forecast data, outing information, and product data in closets and physical stores.

[0560] The "feedback collection means" is a means for collecting feedback such as evaluations of the coordinated outfits selected by the user.

[0561] The "reuse and recycling suggestion tool" is a method for listing items that have not been used for a certain period of time and providing methods for reusing or recycling them.

[0562] "Means for building a database to digitize products in physical stores" refers to a means for registering and managing product information in physical stores in a database using image analysis, etc.

[0563] The "information presentation means using a display device that can be worn by a user" is a means for presenting information to a user through smart glasses or the like.

[0564] This invention is a system for improving the customer purchasing experience in a physical store, and includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordination generation means, a feedback collection means, a reuse / recycle suggestion means, a database construction means for converting products in the physical store into data, and an information presentation means using a display device that can be worn by the user.

[0565] The server executes a program that integrates these means. The specific processing is as follows:

[0566] 1. Data collection method

[0567] The server uses an image capture device to capture images of products in a physical store and uploads them to a cloud server (e.g., AWS). The uploaded images are analyzed using an image analysis library (e.g., OpenCV) or a machine learning model (e.g., TensorFlow). Information obtained from the analyzed images (e.g., color, material, category) is stored in a database (e.g., MySQL).

[0568] 2. Means of inputting outing information

[0569] The user wears a display device such as smart glasses and inputs the purpose, date, time, and location of their outing through the interface. The input information is then sent from the smart glasses to the server.

[0570] 3. How to get weather forecasts

[0571] Based on the outing information received from the user, the server uses a weather forecast acquisition API (for example, OpenWeatherMap API) to acquire weather forecast data for the location and date / time of the planned outing.

[0572] 4. Coordination Generation Method

[0573] The server generates optimal outfits from a database based on weather forecast data and the user's outing information, and the generated outfit patterns are displayed in real time on the user's smart glasses.

[0574] 5. Feedback Collection Methods

[0575] Users use the smart glasses to evaluate the suggested outfits, and the feedback information is sent to the server. The feedback data is stored in a database and reflected in future suggestions.

[0576] 6. Reuse and recycling proposal methods

[0577] Users can check information about recycled products and campaigns available at physical stores through the smart glasses, and the server will make reuse and recycling suggestions based on the user's preferences.

[0578] Specific examples

[0579] A user uses smart glasses and speaks: "Tomorrow is my date, so please dress casually and stylishly." The server checks the weather forecast, determines that it will be sunny, and suggests outfits such as a white cotton shirt, blue denim pants, casual sneakers, and a simple silver pendant.

[0580] Also, if a recycling campaign is being held in a physical store, the user can say to the smart glasses, "Tell me about the recycling campaign." The smart glasses will then display details of the campaign and eligible products.

[0581] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0582] Step 1:

[0583] The server takes pictures of products in a physical store using an image capture device and uploads the pictures to a cloud server (e.g., AWS). The input is the captured image, and the output is image data stored on the cloud server. Specifically, the captured product images are automatically transferred to the server.

[0584] Step 2:

[0585] The server uses an image analysis library (e.g., OpenCV) and a machine learning model (e.g., TensorFlow) to analyze the image data stored on the cloud server. The input is the stored image data, and the output is the analyzed information (color, material, category, etc.). Specifically, the image analysis algorithm extracts product features from the image and stores this information in a database.

[0586] Step 3:

[0587] The user puts on the smart glasses and inputs the purpose, date, time, and location of their outing by voice. The input is the user's voice information, and the output is the outing information sent to the server. Specifically, the user inputs a voice command such as "Tomorrow is a date, so please dress casually and stylishly" into the smart glasses.

[0588] Step 4:

[0589] The server uses a weather forecast acquisition API (for example, OpenWeatherMap API) to obtain weather forecast data for the planned outing location and date and time. The input is the user's outing information, and the output is the obtained weather forecast data. Specifically, the server sends an API request and analyzes and saves the returned weather forecast data.

[0590] Step 5:

[0591] The server generates optimal outfits from a database based on weather forecast data and outing information. The inputs are weather forecast data, outing information, and item information from the database, and the output is an outfit pattern. Specifically, it uses a generative AI model to suggest outfits based on the items in the database, the weather forecast, and outing information.

[0592] Step 6:

[0593] The server displays the generated coordination patterns on the user's smart glasses in real time. The input is the coordination pattern, and the output is the coordination information presented to the smart glasses. Specifically, the proposed coordination is displayed on the smart glasses display as text and images.

[0594] Step 7:

[0595] The user evaluates the outfits presented to them through the smart glasses and sends feedback information to the server. The input is the user's feedback, and the output is the feedback data sent to the server. Specifically, the user inputs their evaluation, such as "good" or "bad," through the interface of the smart glasses.

[0596] Step 8:

[0597] The server analyzes the feedback data and stores it in a database. The input is feedback data from users, and the output is a database that reflects the feedback. Specifically, the generative AI model is updated based on the feedback information and reflected in the next proposal.

[0598] Step 9:

[0599] This system notifies users of reuse and recycling campaign information at physical stores. The input is the store's campaign information, and the output is the campaign information displayed on the smart glasses. Specifically, the server pushes the campaign information to the smart glasses so that the user can check it.

[0600] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0601] The present invention is a system that digitizes items in a user's closet and suggests outfits based on the purpose of going out, and further combines an emotion engine that recognizes the user's emotions. This system includes a data conversion means, an outing information input means, a weather forecast acquisition means, an outfit creation means, a feedback collection means, a reuse / recycle suggestion means, and an emotion engine.

[0602] 1. Data collection method

[0603] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads the image to their device. The device then sends the image to the server. The server analyzes the image and identifies each item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), which are then stored in a database. Items are tagged and registered in the closet database.

[0604] 2. Means of inputting outing information

[0605] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[0606] 3. How to get weather forecasts

[0607] The server retrieves weather forecast data based on the input date, time, and location of the planned outing. For example, it retrieves the data using a weather forecast API.

[0608] 4. Coordination Generation Method

[0609] The server generates multiple outfit patterns based on the acquired weather forecast data and the user's closet database. The generated outfit patterns are adjusted to suit the user's purpose and conditions for going out. The server then transmits the generated outfit patterns to the terminal.

[0610] 5. Feedback Collection Methods

[0611] The user selects from multiple outfit patterns presented on the device screen. The device then sends the selection results and information on the outfits that were not selected back to the server. The server analyzes the feedback data, learns the user's preferences and style trends, and stores the information in a database.

[0612] 6. Emotion Engine

[0613] The device recognizes the user's emotions using facial expressions and voice data. This emotion data is sent from the device to a server, where it is analyzed. For example, the emotion engine determines whether the user is satisfied or dissatisfied with the outfit suggestions. The emotion data collected by the emotion engine is reflected in the outfit suggestions and learning process.

[0614] 7. Reuse and recycling proposal methods

[0615] When a user wants to organize their closet, they request reuse and recycling suggestions from their device. The request is sent from the device to the server. The server references the closet database and lists items that have not been used for a certain period of time. The listed items are sent to the device and suggested to the user. The user checks the suggested items and selects whether to reuse or recycle them. The selection results are sent from the device to the server, which provides a guide on how to reuse and recycle them.

[0616] Specific examples

[0617] If a user requests a full outfit for a date, the system will suggest outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also presented based on the weather forecast, and the user can choose from them. The emotion engine analyzes the user's facial expressions to determine whether they are satisfied. Based on the results, future suggestions are adjusted. If they are dissatisfied, the system will try a different outfit.

[0618] In the reuse and recycling scenario, the app identifies infrequently used items in the closet, such as "blue denim pants that haven't been worn for six months." When the user selects it, they are given instructions such as contact information for a recycle shop or a donation destination.

[0619] In this way, a system will be realized that allows users to efficiently utilize their closets and, by utilizing emotion recognition to receive more personalized outfit suggestions, will reduce the stress of choosing outfits before going out and contribute to a sustainable society.

[0620] The processing flow will be explained below.

[0621] Step 1:

[0622] A user takes a photo of an item in their closet and uploads the image to their device.

[0623] Step 2:

[0624] The device receives the uploaded image and sends it to the server.

[0625] Step 3:

[0626] The server analyzes the images, identifies the item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), and stores them in a database. Each item is given the appropriate tags.

[0627] Step 4:

[0628] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[0629] Step 5:

[0630] The server retrieves weather forecast data based on the date, time, and location of the planned outing. For example, it retrieves the data using a weather forecast API.

[0631] Step 6:

[0632] The server generates multiple outfit patterns based on weather forecast data and the user's closet database. The generated outfit patterns are adjusted to suit the user's needs and requirements.

[0633] Step 7:

[0634] The server sends the generated coordinate patterns to the terminal, which displays them on the screen and presents the user with multiple options.

[0635] Step 8:

[0636] The user selects a desired coordinate pattern from the presented patterns, and the terminal sends the selection result to the server.

[0637] Step 9:

[0638] As a means of collecting feedback, the terminal sends information about the coordinated outfits selected and not selected by the user to the server. The server analyzes this information, learns the user's preferences and style trends, and updates the database.

[0639] Step 10:

[0640] The device acts as an emotion engine, recognizing the user's emotions using facial expressions and voice data. This emotion data is sent from the device to a server, where it is analyzed.

[0641] Step 11:

[0642] Based on the emotional data analyzed by the emotion engine, the proposed outfits are evaluated and reflected in future suggestions. For example, if the user is smiling and satisfied, it will be easier to save that style.

[0643] Step 12:

[0644] When a user wants to organize their closet, they request reuse and recycling suggestions from their device, and the request is sent from the device to the server.

[0645] Step 13:

[0646] The server checks the closet database and lists items that have not been used for a certain period of time, for example, items that have not been used in the last six months.

[0647] Step 14:

[0648] The server generates a list of unwanted items and sends it to the terminal, which then presents the list to the user.

[0649] Step 15:

[0650] The user checks the list and selects the items they want to reuse or recycle, and the selection results are sent from the terminal to the server.

[0651] Step 16:

[0652] The server will provide a guide on how to reuse and recycle selected unwanted items (e.g., contact details for recycling shops and donation locations).

[0653] Step 17:

[0654] Users follow the provided guide to reuse and recycle unwanted items.

[0655] Example 2

[0656] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0657] Conventional coordination suggestion systems have had problems in that they are difficult to fully reflect the user's emotions and preferences, and lack the functionality to suggest reuse and recycling to promote sustainable fashion. In order to solve these problems, the present invention aims to provide a system that takes into account the user's emotions and can also suggest reuse and recycling.

[0658] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0659] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, an emotion recognition means, and a reuse / recycle suggestion means. This not only makes it possible to convert the items in the user's closet into data and suggest optimal coordinates based on the purpose of going out while utilizing weather forecast information, but also analyzes the user's emotions to increase satisfaction with the coordinates and make suggestions for reuse / recycle.

[0660] The "data generation means" is a means for registering items in the user's closet in a database through image analysis.

[0661] The "outing information input means" is a means by which a user can input information regarding the purpose, date, time, and location of going out.

[0662] The "weather forecast acquisition means" is a means for acquiring weather forecast information based on the date, time and location of a planned outing.

[0663] The "coordination generating means" is a means for generating an optimal coordination pattern based on the acquired weather forecast data and the user's closet database.

[0664] The "feedback collection means" is a means for allowing a user to select a proposed coordination pattern and collecting information on the selection results and non-selection results.

[0665] The "emotion recognition means" is a means for recognizing emotions by analyzing the user's facial expressions and voice data.

[0666] The "reuse / recycle suggestion tool" is a method for identifying items in the closet that have not been used for a certain period of time and making suggestions for reuse or recycling.

[0667] The present invention is a system that digitizes the items in a user's closet and suggests outfits based on the purpose of going out, and further combines it with an emotion engine that recognizes the user's emotions. This system includes a data generation means, an outing information input means, a weather forecast acquisition means, an outfit creation means, a feedback collection means, a reuse / recycle suggestion means, and an emotion recognition means.

[0668] System Overview

[0669] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads the image to a device, such as a smartphone or tablet. The device then sends the uploaded image to a server. The server analyzes the image and identifies each item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), which are then stored in a database. Image analysis algorithms such as OpenCV and TensorFlow are used for this image analysis.

[0670] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and location through the device interface, and this information is sent from the device to the server.

[0671] The server obtains weather forecast data based on the input date, time, and location of the planned outing. For example, it obtains data using a weather forecast API such as OpenWeatherMap or WeatherAPI. The server generates multiple outfit patterns based on the obtained weather forecast data and the user's closet database. This generation uses a generative AI model (e.g., GPT-3) and takes into account the user's preferences and style. The generated outfit patterns are sent from the server to the device, which then presents them to the user.

[0672] The user selects one of the presented outfit patterns. The device then sends the selection result and information about the outfits that were not selected back to the server. The server analyzes the feedback data, learns the user's preferences and style, and reflects this in its next outfit suggestions.

[0673] The device uses a built-in camera and microphone to collect the user's facial expressions and voice data. This data is analyzed using an emotion recognition algorithm (e.g., EmotionAPI). The emotion data is sent from the device to a server, where it is analyzed. This determines whether the user is satisfied or dissatisfied with the suggested outfit. This emotion data is also reflected in future outfit suggestions.

[0674] When a user wants to organize their closet, they request reuse and recycling suggestions from their device. The request is sent from the device to the server, which analyzes the closet database and lists items that have not been used for a certain period of time. The listed items are sent to the device and suggested to the user. The user reviews the suggested items and chooses whether to reuse or recycle them. The selection results are sent from the device to the server, which provides a guide on how to reuse and recycle them.

[0675] This system not only provides users with efficient and individually optimized outfit suggestions, but also offers highly satisfying suggestions based on emotion recognition.It also promotes the reuse and recycling of closet items, helping to realize a sustainable fashion lifestyle.

[0676] Specific examples

[0677] For example, if a user enters the prompt "I have a date in a week. Please suggest an outfit based on the weather forecast and the items in my closet," the system will suggest outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also presented based on the weather forecast, and the user can select one. The emotion engine analyzes the user's facial expressions to determine whether they are satisfied. Based on the results, future suggestions are adjusted. If they are dissatisfied, the system can try a different outfit.

[0678] In the reuse and recycling scenario, the app identifies infrequently used items in the closet, such as "blue denim pants that haven't been worn for six months." When the user selects it, they are given instructions such as contact information for a recycle shop or a donation destination.

[0679] As a result, users can make effective use of closet items and enjoy more personalized coordination.

[0680] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0681] Step 1:

[0682] A user takes a photo of an item in their closet and uploads the image to their device. This is the input, and the specific data is an image file. The device receives the image and sends it to the server. The device's processing is the transmission of image data.

[0683] Input: Image file

[0684] Output: Send image data to the server

[0685] Step 2:

[0686] The server analyzes the received images and identifies the color, material, and category of each item. This analysis uses image analysis algorithms (e.g., OpenCV, TensorFlow). The analysis results are stored in a database, and each item is assigned a tag. The data obtained from the image analysis process is the item's attribute information.

[0687] Input: Image data

[0688] Output: Item color, material, and category information

[0689] Step 3:

[0690] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the terminal interface. This information is the input data. The terminal sends the input information to the server. The terminal's processing is to send the outing information.

[0691] Input: Purpose of outing, date and time, location information

[0692] Output: Sending out-of-office information to the server

[0693] Step 4:

[0694] The server retrieves weather forecast data based on the input date, time, and location of the planned outing. It sends a request to a weather forecast API (e.g., OpenWeatherMap, WeatherAPI) and receives the forecast data. The retrieved data is the weather forecast. The server stores this data in a database.

[0695] Input: Date and time of outing, location information

[0696] Output: Weather forecast data

[0697] Step 5:

[0698] The server generates optimal outfit patterns based on the acquired weather forecast data and the user's closet database. Here, a generative AI model (e.g., GPT-3) is used, taking into account the user's preferences and style. The generated multiple outfit patterns are the output data.

[0699] Input: Weather forecast data, closet database

[0700] Output: Coordination pattern

[0701] Step 6:

[0702] The server sends the generated coordination patterns to the terminal, which then presents them to the user. The user selects one of the presented coordination patterns, and the selection information is the input data. The terminal then sends the selection result and information on the coordination patterns not selected to the server.

[0703] Input: Coordination pattern, user selection

[0704] Output: Send feedback data to the server

[0705] Step 7:

[0706] The server analyzes the feedback data and learns the user's preferences and style. The analysis results are stored in a database and reflected in the next outfit suggestions. The data obtained through the analysis process is the user's preferences and style trends.

[0707] Input: Feedback data

[0708] Output: User preferences and style trends data

[0709] Step 8:

[0710] The device uses the built-in camera and microphone to collect the user's facial expressions and voice data. An emotion recognition algorithm (e.g., EmotionAPI) is used to identify the user's emotions. This emotion data is the input data. The device sends the emotion data to the server, which then determines the user's satisfaction level based on the analysis results.

[0711] Input: facial expressions, voice data

[0712] Output: Emotion data, satisfaction rating

[0713] Step 9:

[0714] When a user wants to organize their closet, they request reuse and recycling suggestions through the device interface. The content of this request is the input data. The device sends the request to the server, which analyzes the closet database and lists items that have not been used for a certain period of time. The listed items are the output data.

[0715] Input: Reuse / Recycle Request

[0716] Output: List of items suggested for reuse and recycling

[0717] Step 10:

[0718] The terminal presents a list of suggested items to the user, who then selects whether to reuse or recycle. The selection result is the input data. The terminal then sends the selection result to the server, which provides a guide on how to reuse or recycle. The guide information is the output data.

[0719] Input: Reuse / recycle selection results

[0720] Output: Reuse and recycling guide

[0721] (Application example 2)

[0722] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0723] Conventional coordination suggestion systems do not collect enough information to make appropriate suggestions using the items in a user's closet, making it difficult to make personalized suggestions that take into account the user's emotions and external factors such as the weather. Furthermore, there is a lack of systems that effectively suggest ways to organize a user's closet and recycle and reuse items. Therefore, there is a need for improved user experience and environmental considerations.

[0724] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0725] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, a reuse / recycle suggestion means, an emotion analysis means, a means for analyzing a user's facial image to identify emotions, a means for creating multiple coordinate patterns based on the weather forecast data and facial image analysis data, and a means for accumulating the feedback emotion data and reflecting it in the next suggestion. This enables personalized coordinate suggestions that take the user's emotions and the weather forecast into consideration, and also enables efficient closet organization and recycling / reuse suggestions.

[0726] The "data generation means" is a means for generating data from items in the user's closet through image analysis.

[0727] The "going out information input means" is a means by which the user inputs information about the purpose, date and time of going out, and the location.

[0728] The "weather forecast acquisition means" is a means for acquiring weather forecast data based on the date, time and location of a planned outing.

[0729] The "coordination generating means" is a means for generating a plurality of coordination patterns based on the acquired weather forecast data and closet database.

[0730] The "feedback collection means" is a means for collecting the user's selection results and feedback on the presented coordination patterns.

[0731] The "reuse / recycle suggestion tool" is a tool that allows users who wish to organize their closets to list items that have not been used for a certain period of time and suggests reuse or recycling.

[0732] The "emotion analysis means" is a means for analyzing a facial image of a user to identify the emotion.

[0733] "Weather forecast data" refers to weather information for a specific date, time and location obtained using a weather forecast API or similar.

[0734] "Facial image analysis data" is emotional data obtained by analyzing the facial image of the user.

[0735] A "closet database" is a database in which items in a user's closet are stored as data.

[0736] The "emotion engine" is an engine that recognizes emotions by analyzing the user's facial expressions and voice.

[0737] A "server" is a computer system that analyzes data, stores data, calls APIs, and generates proposals.

[0738] This invention is implemented by the following means. A system in which a user uses a smartphone or other device to digitize the items in their closet through image analysis, and suggests optimal outfits to the user based on information such as the purpose of the outing and the weather forecast. It is also possible to identify emotions by analyzing the user's facial expressions and voice, and reflect this as feedback in the next suggestions.

[0739] Hardware and software used

[0740] Smartphone: Used by users to take pictures of items, input outing information, and check suggested outfits.

[0741] Server: Analyzes data, obtains weather forecasts, generates coordinates, and accumulates feedback data.

[0742] Keras: Used to run deep learning models for recognizing emotions from facial images.

[0743] OpenCV: Used to perform image analysis of items in the closet.

[0744] Weather forecast API: Obtain weather data based on the date, time, and location you plan to go out.

[0745] Data processing and calculation

[0746] 1. Data collection method: The user takes a photo of the items in their closet with their smartphone and uploads the image to the server. The server analyzes the image, extracts attribute information such as color, material, and category, and stores it in the closet database. This process is performed using OpenCV.

[0747] 2. Means for inputting outing information: The user inputs the purpose, date, time, and location of the outing through the smartphone application. This information is sent to the server.

[0748] 3. Weather forecast acquisition method: The server uses the weather forecast API to acquire the weather forecast for the date, time, and location specified by the user.

[0749] 4. Coordination Generation: The server generates multiple coordination patterns based on weather data and the closet database, taking into account the user's preferences and style trends. The generated coordination patterns are sent to the user's smartphone and displayed to them.

[0750] 5. Feedback collection method: The user selects from the proposed outfits and sends the results back to the server. This allows the server to collect information on outfits that were not selected, and learns the user's style tendencies.

[0751] 6. Emotion analysis: The smartphone captures the user's facial image and analyzes their facial expressions using Keras. The recognized emotion data is sent to the server and used to improve the accuracy of outfit suggestions.

[0752] 7. Reuse / Recycle Suggestion Method: When a user wants to organize their closet, the request is sent to the server. The server refers to the closet database and lists items that have not been used for a certain period of time. This allows the user to make suggestions for reuse or recycling.

[0753] Specific examples

[0754] 1. The user takes a photo of their own shirt or pants with their smartphone and registers these items in a database through the application.

[0755] 2. The user enters an upcoming "casual lunch" into the app, specifying the date, time, and location of the outing.

[0756] 3. The server uses the weather forecast API to get the weather for the specified date, time, and location. If the weather is "sunny," it generates a light outfit.

[0757] 4. The server sends the generated outfit to the user and displays it on their smartphone. The user selects the outfit, and the smartphone captures their face image and analyzes their emotions.

[0758] 5. The emotion engine recognizes "satisfaction" from the user's facial expression and feeds this data back to the server, which will then be reflected in the next suggestion.

[0759] Prompt Sentence Examples

[0760] Closet item image analysis:

[0761] image_path = 'user_shirt.jpg'

[0762] item_data = app.analyze_image(image_path)

[0763] Enter your travel information:

[0764] outing_type = "casual lunch"

[0765] outing_datetime = datetime(2023, 10, 24, 12, 30)

[0766] outing_location = "Tokyo"

[0767] outing_info = app.input_outing_info(outing_type, outing_datetime, outing_location)

[0768] Coordinate generation:

[0769] weather_data = app.get_weather_forecast(outing_location, outing_datetime)

[0770] coordinates = app.generate_coordinate(outing_info, weather_data)

[0771] Collecting emotional feedback:

[0772] user_choice = coordinates[0]

[0773] emotion_feedback = app.provide_feedback(user_choice)

[0774] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0775] Step 1:

[0776] A user takes a photo of an item in their closet with their smartphone and uploads the image to a device. The device then sends the image to the server, which uses OpenCV to analyze the image and extract attribute information such as the item's color, material, and category. This data is then stored in a closet database.

[0777] (Input) Item image

[0778] (Data processing) Image analysis, attribute information extraction

[0779] (Output) Saving attribute information to a database

[0780] Step 2:

[0781] The user enters information about the purpose, date, time, and location of their outing through a smartphone application, and this information is sent from the device to a server.

[0782] (Input) Purpose, date, time, and location information

[0783] (Data calculation) Information storage

[0784] (Output) Send data to the server, save information

[0785] Step 3:

[0786] The server uses the weather forecast API to retrieve weather forecast data for the date, time, and location specified by the user, which involves calling an external weather forecast service and retrieving the required data.

[0787] (Input) Date, time, and location information

[0788] (Data calculation) API call, weather data acquisition

[0789] (Output) Obtaining weather forecast data

[0790] Step 4:

[0791] The server generates multiple outfit patterns suitable for the user's outing based on weather data and a closet database. Past feedback data is also taken into consideration. The generated outfit patterns are sent to the device.

[0792] (Input) Weather data, closet database, usage information

[0793] (Data calculation) Coordination generation, reflecting preference trends

[0794] (Output) Sending coordinate patterns to the device

[0795] Step 5:

[0796] The user selects from multiple outfit patterns displayed on the smartphone screen. The selection result is fed back from the device to the server. The server accumulates this information and learns the user's preferences and style trends.

[0797] (Input) Selection result

[0798] (Data calculation) Feedback collection, trend learning

[0799] (Output) Saving to database

[0800] Step 6:

[0801] The device captures the user's facial image and performs emotion analysis. The emotion engine uses Keras to analyze the facial expressions and sends the recognized emotion data to the server. The server receives this emotion data as feedback and uses it to improve future suggestions.

[0802] (Input) Facial image

[0803] (Data calculation) Emotion analysis, data transmission

[0804] (Output) Acquisition and storage of emotion data

[0805] Step 7:

[0806] When a user wants to organize their closet, they send a request from their device to the server. The server then references the closet database, lists items that have not been used for a certain period of time, and suggests reuse or recycling to the user. The results and instructions are displayed on the device.

[0807] (Input) Arrangement request

[0808] (Data calculation) Data reference, list proposal

[0809] (Output) Reuse and recycling proposals

[0810] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0811] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0812] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0813] [Third embodiment]

[0814] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0815] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0816] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0817] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0818] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0819] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0820] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0821] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0822] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0823] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0824] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0825] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0826] This invention is a system that digitizes and visualizes Western and Japanese clothing stored in closets and dressers, and allows a stylist to suggest outfits for the entire body simply by inputting the purpose of going out. This system includes a data conversion means, a means for inputting outing information, a means for obtaining weather forecasts, a means for creating outfits, a means for collecting feedback, and a means for suggesting reuse and recycling.

[0827] 1. Data collection method

[0828] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads it to their device. The device then transfers the image to the server. The server analyzes the received image and identifies each item's color, material, and category (tops, bottoms, accessories, shoes, etc.). It also tags the items appropriately and stores them in the user's personal closet database.

[0829] 2. Means of inputting outing information

[0830] The user inputs information such as the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[0831] 3. How to get weather forecasts

[0832] The server retrieves weather forecast data based on the date, time, and location of the planned outing, for example, by retrieving data from a weather forecast service via an API.

[0833] 4. Coordination Generation Method

[0834] The server generates multiple outfit patterns based on the acquired weather forecast data and the user's closet database. The generated outfit patterns correspond to the user's specified purposes and conditions. For example, it suggests a combination of a white cotton shirt with blue denim pants, black leather boots, and a silver pendant. The server then sends the generated outfit patterns to the terminal.

[0835] 5. Feedback Collection Methods

[0836] The user selects one of several outfit patterns presented on the device screen. The device then sends information about the outfits selected and not selected back to the server. The server analyzes the feedback data and learns the user's preferences and style patterns. This information is then stored in a closet database.

[0837] 6. Reuse and recycling proposal methods

[0838] When a user wishes to organize their closet, they request reuse and recycling suggestions from their device. The device then sends the request to the server. The server checks the closet database and lists items that have not been used for a certain period of time (e.g., six months). The listed unwanted items are then suggested to the user via their device. The user selects candidate items for reuse and recycling, and the selection results are sent to the server. The server then provides a guide on how to reuse or recycle the selected unwanted items (e.g., contact information for recycling shops or donation destinations).

[0839] Specific examples

[0840] When a user requests a date outfit, the system suggests outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also suggested based on the weather forecast. When a user selects a specific outfit, that information is fed back to the server and reflected in future suggestions.

[0841] In the reuse and recycling scenario, the app identifies infrequently used items in the closet and suggests ways to reuse or recycle them. For example, it might suggest, "Your blue denim pants haven't been worn for six months." If the user selects that item, the app provides guidance on where to find a recycle shop or where to donate the items.

[0842] In this way, a system is realized that allows users to efficiently utilize their closets, reduce the stress of choosing outfits before going out, and contribute to a sustainable society.

[0843] The processing flow will be explained below.

[0844] Step 1:

[0845] A user takes a photo of an item in their closet and uploads the image to their device.

[0846] Step 2:

[0847] The terminal receives the uploaded image and sends it to the server.

[0848] Step 3:

[0849] The server analyzes the image and identifies the item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.).

[0850] Step 4:

[0851] The server analyzes the item information, tags it, and stores it in the user's personal closet database.

[0852] Step 5:

[0853] The user inputs information about the purpose, date, time, and location of their outing via the terminal and requests outfit suggestions.

[0854] Step 6:

[0855] The device sends the request to the server.

[0856] Step 7:

[0857] The server retrieves weather forecast data based on the planned date, time, and location of the person going out, for example, using a weather forecast API.

[0858] Step 8:

[0859] The server generates multiple coordination patterns based on weather forecast data acquired and the user's closet database.

[0860] Step 9:

[0861] The server transmits the generated coordinate pattern to the terminal and presents it to the user.

[0862] Step 10:

[0863] The user selects a preferred coordination pattern from a plurality of presented patterns.

[0864] Step 11:

[0865] The terminal feeds back to the server information on the user's selection results and unselected coordinates.

[0866] Step 12:

[0867] The server analyzes the feedback data and updates a database to learn the user's preferences and coordination trends.

[0868] Step 13:

[0869] If a user wants to organize their closet, they can request reuse and recycling suggestions from their device.

[0870] Step 14:

[0871] The device sends a recycle request to the server.

[0872] Step 15:

[0873] The server references the closet database and lists items that are used less frequently.

[0874] Step 16:

[0875] The server generates a list of unwanted items and sends it to the terminal.

[0876] Step 17:

[0877] The user reviews the list and selects the items they want to reuse or recycle.

[0878] Step 18:

[0879] The terminal transmits the selection result to the server.

[0880] Step 19:

[0881] The server generates a guide on how to reuse and recycle the selected unwanted items (e.g., contact information for recycling shops or donation destinations) and sends it to the device.

[0882] Step 20:

[0883] The user follows the guide to dispose of unwanted items.

[0884] Example 1

[0885] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0886] Currently, systems for managing users' owned items and suggesting appropriate outfits are not fully developed. Furthermore, there is a lack of methods for efficiently suggesting outfits appropriate for the weather when users are out, or for reusing and recycling unnecessary items. This often causes inconvenience to users when choosing their daily outfits. The present invention aims to solve these problems and provide a system that can suggest more efficient and appropriate outfits and manage owned items.

[0887] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0888] In this invention, the server includes a means for digitizing items in the closet, a means for inputting the purpose, date, time, and location of an outing, a means for acquiring a weather forecast for the planned outing, a means for generating outfits based on the acquired weather forecast and the digitized items, a means for collecting feedback on the generated outfits, and a means for suggesting reuse and recycling for items that have not been used for a certain period of time. This allows users to reduce the effort of choosing their daily outfits and find the perfect outfit for the weather and purpose. It also enables efficient management of infrequently used items and suggestions for reuse and recycling.

[0889] "A means of digitizing items in the closet" refers to a system that takes photos of the clothing and accessories owned by the user, analyzes the image data to identify attributes such as color, material, and category, and stores these in a database dedicated to the user.

[0890] "Means for inputting the purpose, date, time, and location of the outing" refers to a system that has an interface for users to input information such as the purpose of the outing (e.g., date, work, casual), planned date, and location.

[0891] "Means for obtaining weather forecasts for planned outings" refers to a system that has the function of obtaining forecast data from a weather forecast service based on a specified date, time and location, analyzing that information, and reflecting it in the closet system.

[0892] "Means for generating coordination" refers to a system that generates and suggests optimal clothing combinations based on the user's purpose and conditions for going out, based on acquired weather forecast data and digitized item information.

[0893] "Means for collecting feedback on the generated outfits" refers to a system that has the ability to record the outfits selected by the user from multiple suggested outfits, analyze the selection data, and learn about the user's preferences and style.

[0894] "A means of proposing reuse and recycling for items that have not been used for a certain period of time" refers to a system that has the function of identifying items that have not been used for a certain period of time based on the usage history of items stored in a closet database, and proposing recycling and reuse methods for these items.

[0895] This invention is a system that digitizes the clothes and accessories in a closet or dresser, and automatically suggests outfits for the entire body simply by the user inputting the purpose of going out. This system includes "means for digitizing items in the closet," "means for inputting the purpose, date, time, and location of the outing," "means for obtaining the weather forecast for the planned outing," "means for generating outfits," "means for collecting feedback on the generated outfits," and "means for suggesting reuse and recycling of items that have not been used for a certain period of time."

[0896] Hardware and software used

[0897] Devices are devices available to users, such as smartphones and tablets. Servers are often located in the cloud and have very powerful processors and large amounts of storage. Software includes:

[0898] Image analysis algorithms: TensorFlow, OpenCV

[0899] Weather APIs: OpenWeatherMap, Weather.com

[0900] Generative AI model: A machine learning model used to analyze user data and generate outfits

[0901] Specific processing of the system

[0902] 1. The user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) on their device, which then uploads the photo data to the server.

[0903] 2. The server uses image analysis algorithms to analyze the received photos and identify the color, material, and category of each item. Based on this, the items are tagged appropriately and stored in the user's personal closet database.

[0904] 3. The user enters information about the purpose of the outing (e.g., date, work, casual), date and time, location, etc. through the device interface. This information is sent from the device to the server.

[0905] 4. The server retrieves weather forecast data based on the planned date, time, and location of the person going out, for example, from a weather forecast service via the OpenWeatherMap API.

[0906] 5. The server uses the acquired weather forecast data and the generative AI model to generate multiple outfit patterns based on the user's closet database. The generated outfit patterns correspond to the user's specified purposes and conditions.

[0907] 6. The user selects one of several outfit patterns presented on the device screen. The device then sends information about the selected and unselected outfits back to the server. The server analyzes the feedback data and learns the user's preferences and style patterns. This information is then stored in a closet database.

[0908] 7. If a user wishes to organize their closet, they request reuse and recycling suggestions from their device. The device sends the request to the server. The server checks the closet database and lists items that have not been used in the past six months. The listed unwanted items are then suggested to the user via their device. The user selects candidate items for reuse and recycling, and the selection results are sent to the server. The server then provides a guide on how to reuse or recycle the selected unwanted items (e.g., contact information for recycle shops or donation destinations).

[0909] Examples of concrete examples and prompts

[0910] Specific examples

[0911] When a user requests a date outfit, the system suggests outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also suggested based on the weather forecast. When a user selects a specific outfit, that information is fed back to the server and reflected in future suggestions. In the reuse and recycling section, infrequently used items in the closet are identified and suggestions for reuse or recycling are made. For example, if the system suggests "Your blue denim pants haven't been used in six months," and the user selects that option, the system provides guidance on recycle shop contact details and donation locations.

[0912] Prompt Sentence Examples

[0913] "Please suggest an outfit for a date."

[0914] "Tell me what is appropriate clothing for work"

[0915] "Please suggest an outfit that would be suitable for a casual outing."

[0916] "Make a list of the least used items in your closet."

[0917] This allows users to efficiently manage their closets and reduce the effort required to choose outfits before going out. It also allows for efficient reuse and recycling, contributing to a sustainable society.

[0918] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0919] Step 1:

[0920] The user takes photos of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) on their device. The device then uploads these photo data to the server. The input is the photo of the item, and the output is the image data sent to the server. As soon as the server receives the photo data, it begins analyzing it using an image analysis algorithm (e.g., TensorFlow or OpenCV) to identify the color, material, and category of each item. Specifically, it analyzes the hue and texture pattern of each pixel to extract attributes. The output is the color, material, and category information resulting from the analysis.

[0921] Step 2:

[0922] The server generates appropriate tags (color, material, category) based on the results of image analysis and stores them in the user's personal closet database. The input is the analyzed item information, and the output is the data of the newly tagged item added to the closet database.

[0923] Step 3:

[0924] The user inputs information such as the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input is the purpose of the outing, date and time, and location, and the output is the data that is sent from the device to the server. The device receives the input information and sends it to the server as structured data. Specific operations include the use of text boxes and drop-down menus.

[0925] Step 4:

[0926] The server retrieves weather forecast data based on the received planned date, time, and location of the outing. The input is the planned date, time, and location of the outing, and the output is the retrieved weather forecast data. Specifically, it sends a request to a weather forecast service (e.g., OpenWeatherMap API) and retrieves the corresponding weather forecast data (e.g., wind speed, probability of precipitation, temperature, etc.).

[0927] Step 5:

[0928] The server uses a generative AI model to generate multiple outfit patterns based on weather forecast data and the user's closet database. The input is weather forecast data and the user's item data, and the output is the generated outfit patterns. Specifically, data such as frequency of use, category, color, material, and tag are input into the generative AI model to generate outfit patterns. The generated patterns are optimized for the user's purposes and conditions.

[0929] Step 6:

[0930] The server sends the generated coordinate pattern to the terminal. The input is the generated coordinate pattern, and the output is the coordinate pattern sent to the user's terminal. The terminal displays the received pattern on the user's screen.

[0931] Step 7:

[0932] The user selects one of several coordination patterns. The input is the selection information for the coordination pattern, and the output is the information for the selected coordination. The device sends the selected information to the server and saves it as feedback data, including data on coordinations that were not selected. Specific actions include selection operations by tapping or clicking.

[0933] Step 8:

[0934] The server analyzes the feedback data and learns the user's preferences and style patterns. The input is the feedback data, and the output is updated user style data. Specifically, the server uses a machine learning algorithm to analyze the feedback data and update the user's style information.

[0935] Step 9:

[0936] When a user wants to organize their closet, they request reuse and recycling suggestions from their terminal. The input is the request content, and the output is the transmission of the request data. The terminal receives the request content and sends it to the server.

[0937] Step 10:

[0938] The server checks the closet database and lists items that have not been used in the past six months. The input is the user's closet data, and the output is the list of unwanted items. The server sends the listed items to the terminal and suggests them to the user.

[0939] Step 11:

[0940] The user selects candidate items for reuse or recycling and sends the selection results to the server. The input is the selected item information, and the output is the selection results sent to the server.

[0941] Step 12:

[0942] The server provides the terminal with information on how to reuse or recycle the selected unwanted items (e.g., contact information for recycle shops or donation destinations). The input is the selection result data, and the output is information on reuse and recycling suggestions. Specifically, the server provides the user with contact information for recycle shops and details of donation destinations.

[0943] (Application example 1)

[0944] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0945] Conventional systems suggest outfits by digitizing the items a user owns and inputting information about outings. However, in physical stores, real-time customer service is required, so a system is needed that efficiently digitizes products in the physical store and allows users to instantly receive outfit suggestions. Furthermore, improving the customer's purchasing experience by providing personalized suggestions that take into account the purpose of the outing and weather forecast is a challenge.

[0946] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0947] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, a reuse / recycle suggestion means, a database construction means for converting products in the physical store into data, and an information presentation means using a display device that can be worn by a user. This makes it possible to quickly convert products in the physical store into data and provide optimal coordinates based on the weather forecast and outing information in real time.

[0948] "Data generation means" refers to the means of registering and managing items from users and physical stores in a database using image analysis, etc.

[0949] The "going out information input means" is a means by which the user inputs information such as the purpose, date and time of going out, and location.

[0950] The "weather forecast acquisition means" is a means for acquiring the weather forecast for the location and date / time when the user plans to go out.

[0951] The "coordination generating means" is a means for generating a coordination pattern based on weather forecast data, outing information, and product data in closets and physical stores.

[0952] The "feedback collection means" is a means for collecting feedback such as evaluations of the coordinated outfits selected by the user.

[0953] The "reuse and recycling suggestion tool" is a method for listing items that have not been used for a certain period of time and providing methods for reusing or recycling them.

[0954] "Means for building a database to digitize products in physical stores" refers to a means for registering and managing product information in physical stores in a database using image analysis, etc.

[0955] The "information presentation means using a display device that can be worn by a user" is a means for presenting information to a user through smart glasses or the like.

[0956] This invention is a system for improving the customer purchasing experience in a physical store, and includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordination generation means, a feedback collection means, a reuse / recycle suggestion means, a database construction means for converting products in the physical store into data, and an information presentation means using a display device that can be worn by the user.

[0957] The server executes a program that integrates these means. The specific processing is as follows:

[0958] 1. Data collection method

[0959] The server uses an image capture device to capture images of products in a physical store and uploads them to a cloud server (e.g., AWS). The uploaded images are analyzed using an image analysis library (e.g., OpenCV) or a machine learning model (e.g., TensorFlow). Information obtained from the analyzed images (e.g., color, material, category) is stored in a database (e.g., MySQL).

[0960] 2. Means of inputting outing information

[0961] The user wears a display device such as smart glasses and inputs the purpose, date, time, and location of their outing through the interface. The input information is then sent from the smart glasses to the server.

[0962] 3. How to get weather forecasts

[0963] Based on the outing information received from the user, the server uses a weather forecast acquisition API (for example, OpenWeatherMap API) to acquire weather forecast data for the location and date / time of the planned outing.

[0964] 4. Coordination Generation Method

[0965] The server generates optimal outfits from a database based on weather forecast data and the user's outing information, and the generated outfit patterns are displayed in real time on the user's smart glasses.

[0966] 5. Feedback Collection Methods

[0967] Users use the smart glasses to evaluate the suggested outfits, and the feedback information is sent to the server. The feedback data is stored in a database and reflected in future suggestions.

[0968] 6. Reuse and recycling proposal methods

[0969] Users can check information about recycled products and campaigns available at physical stores through the smart glasses, and the server will make reuse and recycling suggestions based on the user's preferences.

[0970] Specific examples

[0971] A user uses smart glasses and speaks: "Tomorrow is my date, so please dress casually and stylishly." The server checks the weather forecast, determines that it will be sunny, and suggests outfits such as a white cotton shirt, blue denim pants, casual sneakers, and a simple silver pendant.

[0972] Also, if a recycling campaign is being held in a physical store, the user can say to the smart glasses, "Tell me about the recycling campaign." The smart glasses will then display details of the campaign and eligible products.

[0973] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0974] Step 1:

[0975] The server takes pictures of products in a physical store using an image capture device and uploads the pictures to a cloud server (e.g., AWS). The input is the captured image, and the output is image data stored on the cloud server. Specifically, the captured product images are automatically transferred to the server.

[0976] Step 2:

[0977] The server uses an image analysis library (e.g., OpenCV) and a machine learning model (e.g., TensorFlow) to analyze the image data stored on the cloud server. The input is the stored image data, and the output is the analyzed information (color, material, category, etc.). Specifically, the image analysis algorithm extracts product features from the image and stores this information in a database.

[0978] Step 3:

[0979] The user puts on the smart glasses and inputs the purpose, date, time, and location of their outing by voice. The input is the user's voice information, and the output is the outing information sent to the server. Specifically, the user inputs a voice command such as "Tomorrow is a date, so please dress casually and stylishly" into the smart glasses.

[0980] Step 4:

[0981] The server uses a weather forecast acquisition API (for example, OpenWeatherMap API) to obtain weather forecast data for the planned outing location and date and time. The input is the user's outing information, and the output is the obtained weather forecast data. Specifically, the server sends an API request and analyzes and saves the returned weather forecast data.

[0982] Step 5:

[0983] The server generates optimal outfits from a database based on weather forecast data and outing information. The inputs are weather forecast data, outing information, and item information from the database, and the output is an outfit pattern. Specifically, it uses a generative AI model to suggest outfits based on the items in the database, the weather forecast, and outing information.

[0984] Step 6:

[0985] The server displays the generated coordination patterns on the user's smart glasses in real time. The input is the coordination pattern, and the output is the coordination information presented to the smart glasses. Specifically, the proposed coordination is displayed on the smart glasses display as text and images.

[0986] Step 7:

[0987] The user evaluates the outfits presented to them through the smart glasses and sends feedback information to the server. The input is the user's feedback, and the output is the feedback data sent to the server. Specifically, the user inputs their evaluation, such as "good" or "bad," through the interface of the smart glasses.

[0988] Step 8:

[0989] The server analyzes the feedback data and stores it in a database. The input is feedback data from users, and the output is a database that reflects the feedback. Specifically, the generative AI model is updated based on the feedback information and reflected in the next proposal.

[0990] Step 9:

[0991] This system notifies users of reuse and recycling campaign information at physical stores. The input is the store's campaign information, and the output is the campaign information displayed on the smart glasses. Specifically, the server pushes the campaign information to the smart glasses so that the user can check it.

[0992] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0993] The present invention is a system that digitizes items in a user's closet and suggests outfits based on the purpose of going out, and further combines an emotion engine that recognizes the user's emotions. This system includes a data conversion means, an outing information input means, a weather forecast acquisition means, an outfit creation means, a feedback collection means, a reuse / recycle suggestion means, and an emotion engine.

[0994] 1. Data collection method

[0995] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads the image to their device. The device then sends the image to the server. The server analyzes the image and identifies each item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), which are then stored in a database. Items are tagged and registered in the closet database.

[0996] 2. Means of inputting outing information

[0997] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[0998] 3. How to get weather forecasts

[0999] The server retrieves weather forecast data based on the input date, time, and location of the planned outing. For example, it retrieves the data using a weather forecast API.

[1000] 4. Coordination Generation Method

[1001] The server generates multiple outfit patterns based on the acquired weather forecast data and the user's closet database. The generated outfit patterns are adjusted to suit the user's purpose and conditions for going out. The server then transmits the generated outfit patterns to the terminal.

[1002] 5. Feedback Collection Methods

[1003] The user selects from multiple outfit patterns presented on the device screen. The device then sends the selection results and information on the outfits that were not selected back to the server. The server analyzes the feedback data, learns the user's preferences and style trends, and stores the information in a database.

[1004] 6. Emotion Engine

[1005] The device recognizes the user's emotions using facial expressions and voice data. This emotion data is sent from the device to a server, where it is analyzed. For example, the emotion engine determines whether the user is satisfied or dissatisfied with the outfit suggestions. The emotion data collected by the emotion engine is reflected in the outfit suggestions and learning process.

[1006] 7. Reuse and recycling proposal methods

[1007] When a user wants to organize their closet, they request reuse and recycling suggestions from their device. The request is sent from the device to the server. The server references the closet database and lists items that have not been used for a certain period of time. The listed items are sent to the device and suggested to the user. The user checks the suggested items and selects whether to reuse or recycle them. The selection results are sent from the device to the server, which provides a guide on how to reuse and recycle them.

[1008] Specific examples

[1009] If a user requests a full outfit for a date, the system will suggest outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also presented based on the weather forecast, and the user can choose from them. The emotion engine analyzes the user's facial expressions to determine whether they are satisfied. Based on the results, future suggestions are adjusted. If they are dissatisfied, the system will try a different outfit.

[1010] In the reuse and recycling scenario, the app identifies infrequently used items in the closet, such as "blue denim pants that haven't been worn for six months." When the user selects it, they are given instructions such as contact information for a recycle shop or a donation destination.

[1011] In this way, a system will be realized that allows users to efficiently utilize their closets and, by utilizing emotion recognition to receive more personalized outfit suggestions, will reduce the stress of choosing outfits before going out and contribute to a sustainable society.

[1012] The processing flow will be explained below.

[1013] Step 1:

[1014] A user takes a photo of an item in their closet and uploads the image to their device.

[1015] Step 2:

[1016] The device receives the uploaded image and sends it to the server.

[1017] Step 3:

[1018] The server analyzes the images, identifies the item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), and stores them in a database. Each item is given the appropriate tags.

[1019] Step 4:

[1020] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[1021] Step 5:

[1022] The server retrieves weather forecast data based on the date, time, and location of the planned outing. For example, it retrieves the data using a weather forecast API.

[1023] Step 6:

[1024] The server generates multiple outfit patterns based on weather forecast data and the user's closet database. The generated outfit patterns are adjusted to suit the user's needs and requirements.

[1025] Step 7:

[1026] The server sends the generated coordinate patterns to the terminal, which displays them on the screen and presents the user with multiple options.

[1027] Step 8:

[1028] The user selects a desired coordinate pattern from the presented patterns, and the terminal sends the selection result to the server.

[1029] Step 9:

[1030] As a means of collecting feedback, the terminal sends information about the coordinated outfits selected and not selected by the user to the server. The server analyzes this information, learns the user's preferences and style trends, and updates the database.

[1031] Step 10:

[1032] The device acts as an emotion engine, recognizing the user's emotions using facial expressions and voice data. This emotion data is sent from the device to a server, where it is analyzed.

[1033] Step 11:

[1034] Based on the emotional data analyzed by the emotion engine, the proposed outfits are evaluated and reflected in future suggestions. For example, if the user is smiling and satisfied, it will be easier to save that style.

[1035] Step 12:

[1036] When a user wants to organize their closet, they request reuse and recycling suggestions from their device, and the request is sent from the device to the server.

[1037] Step 13:

[1038] The server checks the closet database and lists items that have not been used for a certain period of time, for example, items that have not been used in the last six months.

[1039] Step 14:

[1040] The server generates a list of unwanted items and sends it to the terminal, which then presents the list to the user.

[1041] Step 15:

[1042] The user checks the list and selects the items they want to reuse or recycle, and the selection results are sent from the terminal to the server.

[1043] Step 16:

[1044] The server will provide a guide on how to reuse and recycle selected unwanted items (e.g., contact details for recycling shops and donation locations).

[1045] Step 17:

[1046] Users follow the provided guide to reuse and recycle unwanted items.

[1047] Example 2

[1048] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1049] Conventional coordination suggestion systems have had problems in that they are difficult to fully reflect the user's emotions and preferences, and lack the functionality to suggest reuse and recycling to promote sustainable fashion. In order to solve these problems, the present invention aims to provide a system that takes into account the user's emotions and can also suggest reuse and recycling.

[1050] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1051] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, an emotion recognition means, and a reuse / recycle suggestion means. This not only makes it possible to convert the items in the user's closet into data and suggest optimal coordinates based on the purpose of going out while utilizing weather forecast information, but also analyzes the user's emotions to increase satisfaction with the coordinates and make suggestions for reuse / recycle.

[1052] The "data generation means" is a means for registering items in the user's closet in a database through image analysis.

[1053] The "outing information input means" is a means by which a user can input information regarding the purpose, date, time, and location of going out.

[1054] The "weather forecast acquisition means" is a means for acquiring weather forecast information based on the date, time and location of a planned outing.

[1055] The "coordination generating means" is a means for generating an optimal coordination pattern based on the acquired weather forecast data and the user's closet database.

[1056] The "feedback collection means" is a means for allowing a user to select a proposed coordination pattern and collecting information on the selection results and non-selection results.

[1057] The "emotion recognition means" is a means for recognizing emotions by analyzing the user's facial expressions and voice data.

[1058] The "reuse / recycle suggestion tool" is a method for identifying items in the closet that have not been used for a certain period of time and making suggestions for reuse or recycling.

[1059] The present invention is a system that digitizes the items in a user's closet and suggests outfits based on the purpose of going out, and further combines it with an emotion engine that recognizes the user's emotions. This system includes a data generation means, an outing information input means, a weather forecast acquisition means, an outfit creation means, a feedback collection means, a reuse / recycle suggestion means, and an emotion recognition means.

[1060] System Overview

[1061] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads the image to a device, such as a smartphone or tablet. The device then sends the uploaded image to a server. The server analyzes the image and identifies each item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), which are then stored in a database. Image analysis algorithms such as OpenCV and TensorFlow are used for this image analysis.

[1062] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and location through the device interface, and this information is sent from the device to the server.

[1063] The server obtains weather forecast data based on the input date, time, and location of the planned outing. For example, it obtains data using a weather forecast API such as OpenWeatherMap or WeatherAPI. The server generates multiple outfit patterns based on the obtained weather forecast data and the user's closet database. This generation uses a generative AI model (e.g., GPT-3) and takes into account the user's preferences and style. The generated outfit patterns are sent from the server to the device, which then presents them to the user.

[1064] The user selects one of the presented outfit patterns. The device then sends the selection result and information about the outfits that were not selected back to the server. The server analyzes the feedback data, learns the user's preferences and style, and reflects this in its next outfit suggestions.

[1065] The device uses a built-in camera and microphone to collect the user's facial expressions and voice data. This data is analyzed using an emotion recognition algorithm (e.g., EmotionAPI). The emotion data is sent from the device to a server, where it is analyzed. This determines whether the user is satisfied or dissatisfied with the suggested outfit. This emotion data is also reflected in future outfit suggestions.

[1066] When a user wants to organize their closet, they request reuse and recycling suggestions from their device. The request is sent from the device to the server, which analyzes the closet database and lists items that have not been used for a certain period of time. The listed items are sent to the device and suggested to the user. The user reviews the suggested items and chooses whether to reuse or recycle them. The selection results are sent from the device to the server, which provides a guide on how to reuse and recycle them.

[1067] This system not only provides users with efficient and individually optimized outfit suggestions, but also offers highly satisfying suggestions based on emotion recognition.It also promotes the reuse and recycling of closet items, helping to realize a sustainable fashion lifestyle.

[1068] Specific examples

[1069] For example, if a user enters the prompt "I have a date in a week. Please suggest an outfit based on the weather forecast and the items in my closet," the system will suggest outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also presented based on the weather forecast, and the user can select one. The emotion engine analyzes the user's facial expressions to determine whether they are satisfied. Based on the results, future suggestions are adjusted. If they are dissatisfied, the system can try a different outfit.

[1070] In the reuse and recycling scenario, the app identifies infrequently used items in the closet, such as "blue denim pants that haven't been worn for six months." When the user selects it, they are given instructions such as contact information for a recycle shop or a donation destination.

[1071] As a result, users can make effective use of closet items and enjoy more personalized coordination.

[1072] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1073] Step 1:

[1074] A user takes a photo of an item in their closet and uploads the image to their device. This is the input, and the specific data is an image file. The device receives the image and sends it to the server. The device's processing is the transmission of image data.

[1075] Input: Image file

[1076] Output: Send image data to the server

[1077] Step 2:

[1078] The server analyzes the received images and identifies the color, material, and category of each item. This analysis uses image analysis algorithms (e.g., OpenCV, TensorFlow). The analysis results are stored in a database, and each item is assigned a tag. The data obtained from the image analysis process is the item's attribute information.

[1079] Input: Image data

[1080] Output: Item color, material, and category information

[1081] Step 3:

[1082] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the terminal interface. This information is the input data. The terminal sends the input information to the server. The terminal's processing is to send the outing information.

[1083] Input: Purpose of outing, date and time, location information

[1084] Output: Sending out-of-office information to the server

[1085] Step 4:

[1086] The server retrieves weather forecast data based on the input date, time, and location of the planned outing. It sends a request to a weather forecast API (e.g., OpenWeatherMap, WeatherAPI) and receives the forecast data. The retrieved data is the weather forecast. The server stores this data in a database.

[1087] Input: Date and time of outing, location information

[1088] Output: Weather forecast data

[1089] Step 5:

[1090] The server generates optimal outfit patterns based on the acquired weather forecast data and the user's closet database. Here, a generative AI model (e.g., GPT-3) is used, taking into account the user's preferences and style. The generated multiple outfit patterns are the output data.

[1091] Input: Weather forecast data, closet database

[1092] Output: Coordination pattern

[1093] Step 6:

[1094] The server sends the generated coordination patterns to the terminal, which then presents them to the user. The user selects one of the presented coordination patterns, and the selection information is the input data. The terminal then sends the selection result and information on the coordination patterns not selected to the server.

[1095] Input: Coordination pattern, user selection

[1096] Output: Send feedback data to the server

[1097] Step 7:

[1098] The server analyzes the feedback data and learns the user's preferences and style. The analysis results are stored in a database and reflected in the next outfit suggestions. The data obtained through the analysis process is the user's preferences and style trends.

[1099] Input: Feedback data

[1100] Output: User preferences and style trends data

[1101] Step 8:

[1102] The device uses the built-in camera and microphone to collect the user's facial expressions and voice data. An emotion recognition algorithm (e.g., EmotionAPI) is used to identify the user's emotions. This emotion data is the input data. The device sends the emotion data to the server, which then determines the user's satisfaction level based on the analysis results.

[1103] Input: facial expressions, voice data

[1104] Output: Emotion data, satisfaction rating

[1105] Step 9:

[1106] When a user wants to organize their closet, they request reuse and recycling suggestions through the device interface. The content of this request is the input data. The device sends the request to the server, which analyzes the closet database and lists items that have not been used for a certain period of time. The listed items are the output data.

[1107] Input: Reuse / Recycle Request

[1108] Output: List of items suggested for reuse and recycling

[1109] Step 10:

[1110] The terminal presents a list of suggested items to the user, who then selects whether to reuse or recycle. The selection result is the input data. The terminal then sends the selection result to the server, which provides a guide on how to reuse or recycle. The guide information is the output data.

[1111] Input: Reuse / recycle selection results

[1112] Output: Reuse and recycling guide

[1113] (Application example 2)

[1114] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1115] Conventional coordination suggestion systems do not collect enough information to make appropriate suggestions using the items in a user's closet, making it difficult to make personalized suggestions that take into account the user's emotions and external factors such as the weather. Furthermore, there is a lack of systems that effectively suggest ways to organize a user's closet and recycle and reuse items. Therefore, there is a need for improved user experience and environmental considerations.

[1116] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1117] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, a reuse / recycle suggestion means, an emotion analysis means, a means for analyzing a user's facial image to identify emotions, a means for creating multiple coordinate patterns based on the weather forecast data and facial image analysis data, and a means for accumulating the feedback emotion data and reflecting it in the next suggestion. This enables personalized coordinate suggestions that take the user's emotions and the weather forecast into consideration, and also enables efficient closet organization and recycling / reuse suggestions.

[1118] The "data generation means" is a means for generating data from items in the user's closet through image analysis.

[1119] The "going out information input means" is a means by which the user inputs information about the purpose, date and time of going out, and the location.

[1120] The "weather forecast acquisition means" is a means for acquiring weather forecast data based on the date, time and location of a planned outing.

[1121] The "coordination generating means" is a means for generating a plurality of coordination patterns based on the acquired weather forecast data and closet database.

[1122] The "feedback collection means" is a means for collecting the user's selection results and feedback on the presented coordination patterns.

[1123] The "reuse / recycle suggestion tool" is a tool that allows users who wish to organize their closets to list items that have not been used for a certain period of time and suggests reuse or recycling.

[1124] The "emotion analysis means" is a means for analyzing a facial image of a user to identify the emotion.

[1125] "Weather forecast data" refers to weather information for a specific date, time and location obtained using a weather forecast API or similar.

[1126] "Facial image analysis data" is emotional data obtained by analyzing the facial image of the user.

[1127] A "closet database" is a database in which items in a user's closet are stored as data.

[1128] The "emotion engine" is an engine that recognizes emotions by analyzing the user's facial expressions and voice.

[1129] A "server" is a computer system that analyzes data, stores data, calls APIs, and generates proposals.

[1130] This invention is implemented by the following means. A system in which a user uses a smartphone or other device to digitize the items in their closet through image analysis, and suggests optimal outfits to the user based on information such as the purpose of the outing and the weather forecast. It is also possible to identify emotions by analyzing the user's facial expressions and voice, and reflect this as feedback in the next suggestions.

[1131] Hardware and software used

[1132] Smartphone: Used by users to take pictures of items, input outing information, and check suggested outfits.

[1133] Server: Analyzes data, obtains weather forecasts, generates coordinates, and accumulates feedback data.

[1134] Keras: Used to run deep learning models for recognizing emotions from facial images.

[1135] OpenCV: Used to perform image analysis of items in the closet.

[1136] Weather forecast API: Obtain weather data based on the date, time, and location you plan to go out.

[1137] Data processing and calculation

[1138] 1. Data collection method: The user takes a photo of the items in their closet with their smartphone and uploads the image to the server. The server analyzes the image, extracts attribute information such as color, material, and category, and stores it in the closet database. This process is performed using OpenCV.

[1139] 2. Means for inputting outing information: The user inputs the purpose, date, time, and location of the outing through the smartphone application. This information is sent to the server.

[1140] 3. Weather forecast acquisition method: The server uses the weather forecast API to acquire the weather forecast for the date, time, and location specified by the user.

[1141] 4. Coordination Generation: The server generates multiple coordination patterns based on weather data and the closet database, taking into account the user's preferences and style trends. The generated coordination patterns are sent to the user's smartphone and displayed to them.

[1142] 5. Feedback collection method: The user selects from the proposed outfits and sends the results back to the server. This allows the server to collect information on outfits that were not selected, and learns the user's style tendencies.

[1143] 6. Emotion analysis: The smartphone captures the user's facial image and analyzes their facial expressions using Keras. The recognized emotion data is sent to the server and used to improve the accuracy of outfit suggestions.

[1144] 7. Reuse / Recycle Suggestion Method: When a user wants to organize their closet, the request is sent to the server. The server refers to the closet database and lists items that have not been used for a certain period of time. This allows the user to make suggestions for reuse or recycling.

[1145] Specific examples

[1146] 1. The user takes a photo of their own shirt or pants with their smartphone and registers these items in a database through the application.

[1147] 2. The user enters an upcoming "casual lunch" into the app, specifying the date, time, and location of the outing.

[1148] 3. The server uses the weather forecast API to get the weather for the specified date, time, and location. If the weather is "sunny," it generates a light outfit.

[1149] 4. The server sends the generated outfit to the user and displays it on their smartphone. The user selects the outfit, and the smartphone captures their face image and analyzes their emotions.

[1150] 5. The emotion engine recognizes "satisfaction" from the user's facial expression and feeds this data back to the server, which will then be reflected in the next suggestion.

[1151] Prompt Sentence Examples

[1152] Closet item image analysis:

[1153] image_path = 'user_shirt.jpg'

[1154] item_data = app.analyze_image(image_path)

[1155] Enter your travel information:

[1156] outing_type = "casual lunch"

[1157] outing_datetime = datetime(2023, 10, 24, 12, 30)

[1158] outing_location = "Tokyo"

[1159] outing_info = app.input_outing_info(outing_type, outing_datetime, outing_location)

[1160] Coordinate generation:

[1161] weather_data = app.get_weather_forecast(outing_location, outing_datetime)

[1162] coordinates = app.generate_coordinate(outing_info, weather_data)

[1163] Collecting emotional feedback:

[1164] user_choice = coordinates[0]

[1165] emotion_feedback = app.provide_feedback(user_choice)

[1166] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1167] Step 1:

[1168] A user takes a photo of an item in their closet with their smartphone and uploads the image to a device. The device then sends the image to the server, which uses OpenCV to analyze the image and extract attribute information such as the item's color, material, and category. This data is then stored in a closet database.

[1169] (Input) Item image

[1170] (Data processing) Image analysis, attribute information extraction

[1171] (Output) Saving attribute information to a database

[1172] Step 2:

[1173] The user enters information about the purpose, date, time, and location of their outing through a smartphone application, and this information is sent from the device to a server.

[1174] (Input) Purpose, date, time, and location information

[1175] (Data calculation) Information storage

[1176] (Output) Send data to the server, save information

[1177] Step 3:

[1178] The server uses the weather forecast API to retrieve weather forecast data for the date, time, and location specified by the user, which involves calling an external weather forecast service and retrieving the required data.

[1179] (Input) Date, time, and location information

[1180] (Data calculation) API call, weather data acquisition

[1181] (Output) Obtaining weather forecast data

[1182] Step 4:

[1183] The server generates multiple outfit patterns suitable for the user's outing based on weather data and a closet database. Past feedback data is also taken into consideration. The generated outfit patterns are sent to the device.

[1184] (Input) Weather data, closet database, usage information

[1185] (Data calculation) Coordination generation, reflecting preference trends

[1186] (Output) Sending coordinate patterns to the device

[1187] Step 5:

[1188] The user selects from multiple outfit patterns displayed on the smartphone screen. The selection result is fed back from the device to the server. The server accumulates this information and learns the user's preferences and style trends.

[1189] (Input) Selection result

[1190] (Data calculation) Feedback collection, trend learning

[1191] (Output) Saving to database

[1192] Step 6:

[1193] The device captures the user's facial image and performs emotion analysis. The emotion engine uses Keras to analyze the facial expressions and sends the recognized emotion data to the server. The server receives this emotion data as feedback and uses it to improve future suggestions.

[1194] (Input) Facial image

[1195] (Data calculation) Emotion analysis, data transmission

[1196] (Output) Acquisition and storage of emotion data

[1197] Step 7:

[1198] When a user wants to organize their closet, they send a request from their device to the server. The server then references the closet database, lists items that have not been used for a certain period of time, and suggests reuse or recycling to the user. The results and instructions are displayed on the device.

[1199] (Input) Arrangement request

[1200] (Data calculation) Data reference, list proposal

[1201] (Output) Reuse and recycling proposals

[1202] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1203] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1204] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1205] [Fourth embodiment]

[1206] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1207] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1208] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1209] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1210] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1211] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1212] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1213] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1214] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1215] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1216] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1217] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1218] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1219] This invention is a system that digitizes and visualizes Western and Japanese clothing stored in closets and dressers, and allows a stylist to suggest outfits for the entire body simply by inputting the purpose of going out. This system includes a data conversion means, a means for inputting outing information, a means for obtaining weather forecasts, a means for creating outfits, a means for collecting feedback, and a means for suggesting reuse and recycling.

[1220] 1. Data collection method

[1221] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads it to their device. The device then transfers the image to the server. The server analyzes the received image and identifies each item's color, material, and category (tops, bottoms, accessories, shoes, etc.). It also tags the items appropriately and stores them in the user's personal closet database.

[1222] 2. Means of inputting outing information

[1223] The user inputs information such as the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[1224] 3. How to get weather forecasts

[1225] The server retrieves weather forecast data based on the date, time, and location of the planned outing, for example, by retrieving data from a weather forecast service via an API.

[1226] 4. Coordination Generation Method

[1227] The server generates multiple outfit patterns based on the acquired weather forecast data and the user's closet database. The generated outfit patterns correspond to the user's specified purposes and conditions. For example, it suggests a combination of a white cotton shirt with blue denim pants, black leather boots, and a silver pendant. The server then sends the generated outfit patterns to the terminal.

[1228] 5. Feedback Collection Methods

[1229] The user selects one of several outfit patterns presented on the device screen. The device then sends information about the outfits selected and not selected back to the server. The server analyzes the feedback data and learns the user's preferences and style patterns. This information is then stored in a closet database.

[1230] 6. Reuse and recycling proposal methods

[1231] When a user wishes to organize their closet, they request reuse and recycling suggestions from their device. The device then sends the request to the server. The server checks the closet database and lists items that have not been used for a certain period of time (e.g., six months). The listed unwanted items are then suggested to the user via their device. The user selects candidate items for reuse and recycling, and the selection results are sent to the server. The server then provides a guide on how to reuse or recycle the selected unwanted items (e.g., contact information for recycling shops or donation destinations).

[1232] Specific examples

[1233] When a user requests a date outfit, the system suggests outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also suggested based on the weather forecast. When a user selects a specific outfit, that information is fed back to the server and reflected in future suggestions.

[1234] In the reuse and recycling scenario, the app identifies infrequently used items in the closet and suggests ways to reuse or recycle them. For example, it might suggest, "Your blue denim pants haven't been worn for six months." If the user selects that item, the app provides guidance on where to find a recycle shop or where to donate the items.

[1235] In this way, a system is realized that allows users to efficiently utilize their closets, reduce the stress of choosing outfits before going out, and contribute to a sustainable society.

[1236] The processing flow will be explained below.

[1237] Step 1:

[1238] A user takes a photo of an item in their closet and uploads the image to their device.

[1239] Step 2:

[1240] The terminal receives the uploaded image and sends it to the server.

[1241] Step 3:

[1242] The server analyzes the image and identifies the item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.).

[1243] Step 4:

[1244] The server analyzes the item information, tags it, and stores it in the user's personal closet database.

[1245] Step 5:

[1246] The user inputs information about the purpose, date, time, and location of their outing via the terminal and requests outfit suggestions.

[1247] Step 6:

[1248] The device sends the request to the server.

[1249] Step 7:

[1250] The server retrieves weather forecast data based on the planned date, time, and location of the person going out, for example, using a weather forecast API.

[1251] Step 8:

[1252] The server generates multiple coordination patterns based on weather forecast data acquired and the user's closet database.

[1253] Step 9:

[1254] The server transmits the generated coordinate pattern to the terminal and presents it to the user.

[1255] Step 10:

[1256] The user selects a preferred coordination pattern from a plurality of presented patterns.

[1257] Step 11:

[1258] The terminal feeds back to the server information on the user's selection results and unselected coordinates.

[1259] Step 12:

[1260] The server analyzes the feedback data and updates a database to learn the user's preferences and coordination trends.

[1261] Step 13:

[1262] If a user wants to organize their closet, they can request reuse and recycling suggestions from their device.

[1263] Step 14:

[1264] The device sends a recycle request to the server.

[1265] Step 15:

[1266] The server references the closet database and lists items that are used less frequently.

[1267] Step 16:

[1268] The server generates a list of unwanted items and sends it to the terminal.

[1269] Step 17:

[1270] The user reviews the list and selects the items they want to reuse or recycle.

[1271] Step 18:

[1272] The terminal transmits the selection result to the server.

[1273] Step 19:

[1274] The server generates a guide on how to reuse and recycle the selected unwanted items (e.g., contact information for recycling shops or donation destinations) and sends it to the device.

[1275] Step 20:

[1276] The user follows the guide to dispose of unwanted items.

[1277] Example 1

[1278] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1279] Currently, systems for managing users' owned items and suggesting appropriate outfits are not fully developed. Furthermore, there is a lack of methods for efficiently suggesting outfits appropriate for the weather when users are out, or for reusing and recycling unnecessary items. This often causes inconvenience to users when choosing their daily outfits. The present invention aims to solve these problems and provide a system that can suggest more efficient and appropriate outfits and manage owned items.

[1280] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1281] In this invention, the server includes a means for digitizing items in the closet, a means for inputting the purpose, date, time, and location of an outing, a means for acquiring a weather forecast for the planned outing, a means for generating outfits based on the acquired weather forecast and the digitized items, a means for collecting feedback on the generated outfits, and a means for suggesting reuse and recycling for items that have not been used for a certain period of time. This allows users to reduce the effort of choosing their daily outfits and find the perfect outfit for the weather and purpose. It also enables efficient management of infrequently used items and suggestions for reuse and recycling.

[1282] "A means of digitizing items in the closet" refers to a system that takes photos of the clothing and accessories owned by the user, analyzes the image data to identify attributes such as color, material, and category, and stores these in a database dedicated to the user.

[1283] "Means for inputting the purpose, date, time, and location of the outing" refers to a system that has an interface for users to input information such as the purpose of the outing (e.g., date, work, casual), planned date, and location.

[1284] "Means for obtaining weather forecasts for planned outings" refers to a system that has the function of obtaining forecast data from a weather forecast service based on a specified date, time and location, analyzing that information, and reflecting it in the closet system.

[1285] "Means for generating coordination" refers to a system that generates and suggests optimal clothing combinations based on the user's purpose and conditions for going out, based on acquired weather forecast data and digitized item information.

[1286] "Means for collecting feedback on the generated outfits" refers to a system that has the ability to record the outfits selected by the user from multiple suggested outfits, analyze the selection data, and learn about the user's preferences and style.

[1287] "A means of proposing reuse and recycling for items that have not been used for a certain period of time" refers to a system that has the function of identifying items that have not been used for a certain period of time based on the usage history of items stored in a closet database, and proposing recycling and reuse methods for these items.

[1288] This invention is a system that digitizes the clothes and accessories in a closet or dresser, and automatically suggests outfits for the entire body simply by the user inputting the purpose of going out. This system includes "means for digitizing items in the closet," "means for inputting the purpose, date, time, and location of the outing," "means for obtaining the weather forecast for the planned outing," "means for generating outfits," "means for collecting feedback on the generated outfits," and "means for suggesting reuse and recycling of items that have not been used for a certain period of time."

[1289] Hardware and software used

[1290] Devices are devices available to users, such as smartphones and tablets. Servers are often located in the cloud and have very powerful processors and large amounts of storage. Software includes:

[1291] Image analysis algorithms: TensorFlow, OpenCV

[1292] Weather APIs: OpenWeatherMap, Weather.com

[1293] Generative AI model: A machine learning model used to analyze user data and generate outfits

[1294] Specific processing of the system

[1295] 1. The user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) on their device, which then uploads the photo data to the server.

[1296] 2. The server uses image analysis algorithms to analyze the received photos and identify the color, material, and category of each item. Based on this, the items are tagged appropriately and stored in the user's personal closet database.

[1297] 3. The user enters information about the purpose of the outing (e.g., date, work, casual), date and time, location, etc. through the device interface. This information is sent from the device to the server.

[1298] 4. The server retrieves weather forecast data based on the planned date, time, and location of the person going out, for example, from a weather forecast service via the OpenWeatherMap API.

[1299] 5. The server uses the acquired weather forecast data and the generative AI model to generate multiple outfit patterns based on the user's closet database. The generated outfit patterns correspond to the user's specified purposes and conditions.

[1300] 6. The user selects one of several outfit patterns presented on the device screen. The device then sends information about the selected and unselected outfits back to the server. The server analyzes the feedback data and learns the user's preferences and style patterns. This information is then stored in a closet database.

[1301] 7. If a user wishes to organize their closet, they request reuse and recycling suggestions from their device. The device sends the request to the server. The server checks the closet database and lists items that have not been used in the past six months. The listed unwanted items are then suggested to the user via their device. The user selects candidate items for reuse and recycling, and the selection results are sent to the server. The server then provides a guide on how to reuse or recycle the selected unwanted items (e.g., contact information for recycle shops or donation destinations).

[1302] Examples of concrete examples and prompts

[1303] Specific examples

[1304] When a user requests a date outfit, the system suggests outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also suggested based on the weather forecast. When a user selects a specific outfit, that information is fed back to the server and reflected in future suggestions. In the reuse and recycling section, infrequently used items in the closet are identified and suggestions for reuse or recycling are made. For example, if the system suggests "Your blue denim pants haven't been used in six months," and the user selects that option, the system provides guidance on recycle shop contact details and donation locations.

[1305] Prompt Sentence Examples

[1306] "Please suggest an outfit for a date."

[1307] "Tell me what is appropriate clothing for work"

[1308] "Please suggest an outfit that would be suitable for a casual outing."

[1309] "Make a list of the least used items in your closet."

[1310] This allows users to efficiently manage their closets and reduce the effort required to choose outfits before going out. It also allows for efficient reuse and recycling, contributing to a sustainable society.

[1311] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1312] Step 1:

[1313] The user takes photos of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) on their device. The device then uploads these photo data to the server. The input is the photo of the item, and the output is the image data sent to the server. As soon as the server receives the photo data, it begins analyzing it using an image analysis algorithm (e.g., TensorFlow or OpenCV) to identify the color, material, and category of each item. Specifically, it analyzes the hue and texture pattern of each pixel to extract attributes. The output is the color, material, and category information resulting from the analysis.

[1314] Step 2:

[1315] The server generates appropriate tags (color, material, category) based on the results of image analysis and stores them in the user's personal closet database. The input is the analyzed item information, and the output is the data of the newly tagged item added to the closet database.

[1316] Step 3:

[1317] The user inputs information such as the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input is the purpose of the outing, date and time, and location, and the output is the data that is sent from the device to the server. The device receives the input information and sends it to the server as structured data. Specific operations include the use of text boxes and drop-down menus.

[1318] Step 4:

[1319] The server retrieves weather forecast data based on the received planned date, time, and location of the outing. The input is the planned date, time, and location of the outing, and the output is the retrieved weather forecast data. Specifically, it sends a request to a weather forecast service (e.g., OpenWeatherMap API) and retrieves the corresponding weather forecast data (e.g., wind speed, probability of precipitation, temperature, etc.).

[1320] Step 5:

[1321] The server uses a generative AI model to generate multiple outfit patterns based on weather forecast data and the user's closet database. The input is weather forecast data and the user's item data, and the output is the generated outfit patterns. Specifically, data such as frequency of use, category, color, material, and tag are input into the generative AI model to generate outfit patterns. The generated patterns are optimized for the user's purposes and conditions.

[1322] Step 6:

[1323] The server sends the generated coordinate pattern to the terminal. The input is the generated coordinate pattern, and the output is the coordinate pattern sent to the user's terminal. The terminal displays the received pattern on the user's screen.

[1324] Step 7:

[1325] The user selects one of several coordination patterns. The input is the selection information for the coordination pattern, and the output is the information for the selected coordination. The device sends the selected information to the server and saves it as feedback data, including data on coordinations that were not selected. Specific actions include selection operations by tapping or clicking.

[1326] Step 8:

[1327] The server analyzes the feedback data and learns the user's preferences and style patterns. The input is the feedback data, and the output is updated user style data. Specifically, the server uses a machine learning algorithm to analyze the feedback data and update the user's style information.

[1328] Step 9:

[1329] When a user wants to organize their closet, they request reuse and recycling suggestions from their terminal. The input is the request content, and the output is the transmission of the request data. The terminal receives the request content and sends it to the server.

[1330] Step 10:

[1331] The server checks the closet database and lists items that have not been used in the past six months. The input is the user's closet data, and the output is the list of unwanted items. The server sends the listed items to the terminal and suggests them to the user.

[1332] Step 11:

[1333] The user selects candidate items for reuse or recycling and sends the selection results to the server. The input is the selected item information, and the output is the selection results sent to the server.

[1334] Step 12:

[1335] The server provides the terminal with information on how to reuse or recycle the selected unwanted items (e.g., contact information for recycle shops or donation destinations). The input is the selection result data, and the output is information on reuse and recycling suggestions. Specifically, the server provides the user with contact information for recycle shops and details of donation destinations.

[1336] (Application example 1)

[1337] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1338] Conventional systems suggest outfits by digitizing the items a user owns and inputting information about outings. However, in physical stores, real-time customer service is required, so a system is needed that efficiently digitizes products in the physical store and allows users to instantly receive outfit suggestions. Furthermore, improving the customer's purchasing experience by providing personalized suggestions that take into account the purpose of the outing and weather forecast is a challenge.

[1339] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1340] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, a reuse / recycle suggestion means, a database construction means for converting products in the physical store into data, and an information presentation means using a display device that can be worn by a user. This makes it possible to quickly convert products in the physical store into data and provide optimal coordinates based on the weather forecast and outing information in real time.

[1341] "Data generation means" refers to the means of registering and managing items from users and physical stores in a database using image analysis, etc.

[1342] The "going out information input means" is a means by which the user inputs information such as the purpose, date and time of going out, and location.

[1343] The "weather forecast acquisition means" is a means for acquiring the weather forecast for the location and date / time when the user plans to go out.

[1344] The "coordination generating means" is a means for generating a coordination pattern based on weather forecast data, outing information, and product data in closets and physical stores.

[1345] The "feedback collection means" is a means for collecting feedback such as evaluations of the coordinated outfits selected by the user.

[1346] The "reuse and recycling suggestion tool" is a method for listing items that have not been used for a certain period of time and providing methods for reusing or recycling them.

[1347] "Means for building a database to digitize products in physical stores" refers to a means for registering and managing product information in physical stores in a database using image analysis, etc.

[1348] The "information presentation means using a display device that can be worn by a user" is a means for presenting information to a user through smart glasses or the like.

[1349] This invention is a system for improving the customer purchasing experience in a physical store, and includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordination generation means, a feedback collection means, a reuse / recycle suggestion means, a database construction means for converting products in the physical store into data, and an information presentation means using a display device that can be worn by the user.

[1350] The server executes a program that integrates these means. The specific processing is as follows:

[1351] 1. Data collection method

[1352] The server uses an image capture device to capture images of products in a physical store and uploads them to a cloud server (e.g., AWS). The uploaded images are analyzed using an image analysis library (e.g., OpenCV) or a machine learning model (e.g., TensorFlow). Information obtained from the analyzed images (e.g., color, material, category) is stored in a database (e.g., MySQL).

[1353] 2. Means of inputting outing information

[1354] The user wears a display device such as smart glasses and inputs the purpose, date, time, and location of their outing through the interface. The input information is then sent from the smart glasses to the server.

[1355] 3. How to get weather forecasts

[1356] Based on the outing information received from the user, the server uses a weather forecast acquisition API (for example, OpenWeatherMap API) to acquire weather forecast data for the location and date / time of the planned outing.

[1357] 4. Coordination Generation Method

[1358] The server generates optimal outfits from a database based on weather forecast data and the user's outing information, and the generated outfit patterns are displayed in real time on the user's smart glasses.

[1359] 5. Feedback Collection Methods

[1360] Users use the smart glasses to evaluate the suggested outfits, and the feedback information is sent to the server. The feedback data is stored in a database and reflected in future suggestions.

[1361] 6. Reuse and recycling proposal methods

[1362] Users can check information about recycled products and campaigns available at physical stores through the smart glasses, and the server will make reuse and recycling suggestions based on the user's preferences.

[1363] Specific examples

[1364] A user uses smart glasses and speaks: "Tomorrow is my date, so please dress casually and stylishly." The server checks the weather forecast, determines that it will be sunny, and suggests outfits such as a white cotton shirt, blue denim pants, casual sneakers, and a simple silver pendant.

[1365] Also, if a recycling campaign is being held in a physical store, the user can say to the smart glasses, "Tell me about the recycling campaign." The smart glasses will then display details of the campaign and eligible products.

[1366] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1367] Step 1:

[1368] The server takes pictures of products in a physical store using an image capture device and uploads the pictures to a cloud server (e.g., AWS). The input is the captured image, and the output is image data stored on the cloud server. Specifically, the captured product images are automatically transferred to the server.

[1369] Step 2:

[1370] The server uses an image analysis library (e.g., OpenCV) and a machine learning model (e.g., TensorFlow) to analyze the image data stored on the cloud server. The input is the stored image data, and the output is the analyzed information (color, material, category, etc.). Specifically, the image analysis algorithm extracts product features from the image and stores this information in a database.

[1371] Step 3:

[1372] The user puts on the smart glasses and inputs the purpose, date, time, and location of their outing by voice. The input is the user's voice information, and the output is the outing information sent to the server. Specifically, the user inputs a voice command such as "Tomorrow is a date, so please dress casually and stylishly" into the smart glasses.

[1373] Step 4:

[1374] The server uses a weather forecast acquisition API (for example, OpenWeatherMap API) to obtain weather forecast data for the planned outing location and date and time. The input is the user's outing information, and the output is the obtained weather forecast data. Specifically, the server sends an API request and analyzes and saves the returned weather forecast data.

[1375] Step 5:

[1376] The server generates optimal outfits from a database based on weather forecast data and outing information. The inputs are weather forecast data, outing information, and item information from the database, and the output is an outfit pattern. Specifically, it uses a generative AI model to suggest outfits based on the items in the database, the weather forecast, and outing information.

[1377] Step 6:

[1378] The server displays the generated coordination patterns on the user's smart glasses in real time. The input is the coordination pattern, and the output is the coordination information presented to the smart glasses. Specifically, the proposed coordination is displayed on the smart glasses display as text and images.

[1379] Step 7:

[1380] The user evaluates the outfits presented to them through the smart glasses and sends feedback information to the server. The input is the user's feedback, and the output is the feedback data sent to the server. Specifically, the user inputs their evaluation, such as "good" or "bad," through the interface of the smart glasses.

[1381] Step 8:

[1382] The server analyzes the feedback data and stores it in a database. The input is feedback data from users, and the output is a database that reflects the feedback. Specifically, the generative AI model is updated based on the feedback information and reflected in the next proposal.

[1383] Step 9:

[1384] This system notifies users of reuse and recycling campaign information at physical stores. The input is the store's campaign information, and the output is the campaign information displayed on the smart glasses. Specifically, the server pushes the campaign information to the smart glasses so that the user can check it.

[1385] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1386] The present invention is a system that digitizes items in a user's closet and suggests outfits based on the purpose of going out, and further combines an emotion engine that recognizes the user's emotions. This system includes a data conversion means, an outing information input means, a weather forecast acquisition means, an outfit creation means, a feedback collection means, a reuse / recycle suggestion means, and an emotion engine.

[1387] 1. Data collection method

[1388] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads the image to their device. The device then sends the image to the server. The server analyzes the image and identifies each item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), which are then stored in a database. Items are tagged and registered in the closet database.

[1389] 2. Means of inputting outing information

[1390] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[1391] 3. How to get weather forecasts

[1392] The server retrieves weather forecast data based on the input date, time, and location of the planned outing. For example, it retrieves the data using a weather forecast API.

[1393] 4. Coordination Generation Method

[1394] The server generates multiple outfit patterns based on the acquired weather forecast data and the user's closet database. The generated outfit patterns are adjusted to suit the user's purpose and conditions for going out. The server then transmits the generated outfit patterns to the terminal.

[1395] 5. Feedback Collection Methods

[1396] The user selects from multiple outfit patterns presented on the device screen. The device then sends the selection results and information on the outfits that were not selected back to the server. The server analyzes the feedback data, learns the user's preferences and style trends, and stores the information in a database.

[1397] 6. Emotion Engine

[1398] The device recognizes the user's emotions using facial expressions and voice data. This emotion data is sent from the device to a server, where it is analyzed. For example, the emotion engine determines whether the user is satisfied or dissatisfied with the outfit suggestions. The emotion data collected by the emotion engine is reflected in the outfit suggestions and learning process.

[1399] 7. Reuse and recycling proposal methods

[1400] When a user wants to organize their closet, they request reuse and recycling suggestions from their device. The request is sent from the device to the server. The server references the closet database and lists items that have not been used for a certain period of time. The listed items are sent to the device and suggested to the user. The user checks the suggested items and selects whether to reuse or recycle them. The selection results are sent from the device to the server, which provides a guide on how to reuse and recycle them.

[1401] Specific examples

[1402] If a user requests a full outfit for a date, the system will suggest outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also presented based on the weather forecast, and the user can choose from them. The emotion engine analyzes the user's facial expressions to determine whether they are satisfied. Based on the results, future suggestions are adjusted. If they are dissatisfied, the system will try a different outfit.

[1403] In the reuse and recycling scenario, the app identifies infrequently used items in the closet, such as "blue denim pants that haven't been worn for six months." When the user selects it, they are given instructions such as contact information for a recycle shop or a donation destination.

[1404] In this way, a system will be realized that allows users to efficiently utilize their closets and, by utilizing emotion recognition to receive more personalized outfit suggestions, will reduce the stress of choosing outfits before going out and contribute to a sustainable society.

[1405] The processing flow will be explained below.

[1406] Step 1:

[1407] A user takes a photo of an item in their closet and uploads the image to their device.

[1408] Step 2:

[1409] The device receives the uploaded image and sends it to the server.

[1410] Step 3:

[1411] The server analyzes the images, identifies the item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), and stores them in a database. Each item is given the appropriate tags.

[1412] Step 4:

[1413] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the device interface. The input information is sent from the device to the server.

[1414] Step 5:

[1415] The server retrieves weather forecast data based on the date, time, and location of the planned outing. For example, it retrieves the data using a weather forecast API.

[1416] Step 6:

[1417] The server generates multiple outfit patterns based on weather forecast data and the user's closet database. The generated outfit patterns are adjusted to suit the user's needs and requirements.

[1418] Step 7:

[1419] The server sends the generated coordinate patterns to the terminal, which displays them on the screen and presents the user with multiple options.

[1420] Step 8:

[1421] The user selects a desired coordinate pattern from the presented patterns, and the terminal sends the selection result to the server.

[1422] Step 9:

[1423] As a means of collecting feedback, the terminal sends information about the coordinated outfits selected and not selected by the user to the server. The server analyzes this information, learns the user's preferences and style trends, and updates the database.

[1424] Step 10:

[1425] The device acts as an emotion engine, recognizing the user's emotions using facial expressions and voice data. This emotion data is sent from the device to a server, where it is analyzed.

[1426] Step 11:

[1427] Based on the emotional data analyzed by the emotion engine, the proposed outfits are evaluated and reflected in future suggestions. For example, if the user is smiling and satisfied, it will be easier to save that style.

[1428] Step 12:

[1429] When a user wants to organize their closet, they request reuse and recycling suggestions from their device, and the request is sent from the device to the server.

[1430] Step 13:

[1431] The server checks the closet database and lists items that have not been used for a certain period of time, for example, items that have not been used in the last six months.

[1432] Step 14:

[1433] The server generates a list of unwanted items and sends it to the terminal, which then presents the list to the user.

[1434] Step 15:

[1435] The user checks the list and selects the items they want to reuse or recycle, and the selection results are sent from the terminal to the server.

[1436] Step 16:

[1437] The server will provide a guide on how to reuse and recycle selected unwanted items (e.g., contact details for recycling shops and donation locations).

[1438] Step 17:

[1439] Users follow the provided guide to reuse and recycle unwanted items.

[1440] Example 2

[1441] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1442] Conventional coordination suggestion systems have had problems in that they are difficult to fully reflect the user's emotions and preferences, and lack the functionality to suggest reuse and recycling to promote sustainable fashion. In order to solve these problems, the present invention aims to provide a system that takes into account the user's emotions and can also suggest reuse and recycling.

[1443] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1444] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, an emotion recognition means, and a reuse / recycle suggestion means. This not only makes it possible to convert the items in the user's closet into data and suggest optimal coordinates based on the purpose of going out while utilizing weather forecast information, but also analyzes the user's emotions to increase satisfaction with the coordinates and make suggestions for reuse / recycle.

[1445] The "data generation means" is a means for registering items in the user's closet in a database through image analysis.

[1446] The "outing information input means" is a means by which a user can input information regarding the purpose, date, time, and location of going out.

[1447] The "weather forecast acquisition means" is a means for acquiring weather forecast information based on the date, time and location of a planned outing.

[1448] The "coordination generating means" is a means for generating an optimal coordination pattern based on the acquired weather forecast data and the user's closet database.

[1449] The "feedback collection means" is a means for allowing a user to select a proposed coordination pattern and collecting information on the selection results and non-selection results.

[1450] The "emotion recognition means" is a means for recognizing emotions by analyzing the user's facial expressions and voice data.

[1451] The "reuse / recycle suggestion tool" is a method for identifying items in the closet that have not been used for a certain period of time and making suggestions for reuse or recycling.

[1452] The present invention is a system that digitizes the items in a user's closet and suggests outfits based on the purpose of going out, and further combines it with an emotion engine that recognizes the user's emotions. This system includes a data generation means, an outing information input means, a weather forecast acquisition means, an outfit creation means, a feedback collection means, a reuse / recycle suggestion means, and an emotion recognition means.

[1453] System Overview

[1454] A user takes a photo of items in their closet (e.g., shirts, pants, skirts, shoes, accessories) and uploads the image to a device, such as a smartphone or tablet. The device then sends the uploaded image to a server. The server analyzes the image and identifies each item's color, material, and category (e.g., tops, bottoms, accessories, shoes, etc.), which are then stored in a database. Image analysis algorithms such as OpenCV and TensorFlow are used for this image analysis.

[1455] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and location through the device interface, and this information is sent from the device to the server.

[1456] The server obtains weather forecast data based on the input date, time, and location of the planned outing. For example, it obtains data using a weather forecast API such as OpenWeatherMap or WeatherAPI. The server generates multiple outfit patterns based on the obtained weather forecast data and the user's closet database. This generation uses a generative AI model (e.g., GPT-3) and takes into account the user's preferences and style. The generated outfit patterns are sent from the server to the device, which then presents them to the user.

[1457] The user selects one of the presented outfit patterns. The device then sends the selection result and information about the outfits that were not selected back to the server. The server analyzes the feedback data, learns the user's preferences and style, and reflects this in its next outfit suggestions.

[1458] The device uses a built-in camera and microphone to collect the user's facial expressions and voice data. This data is analyzed using an emotion recognition algorithm (e.g., EmotionAPI). The emotion data is sent from the device to a server, where it is analyzed. This determines whether the user is satisfied or dissatisfied with the suggested outfit. This emotion data is also reflected in future outfit suggestions.

[1459] When a user wants to organize their closet, they request reuse and recycling suggestions from their device. The request is sent from the device to the server, which analyzes the closet database and lists items that have not been used for a certain period of time. The listed items are sent to the device and suggested to the user. The user reviews the suggested items and chooses whether to reuse or recycle them. The selection results are sent from the device to the server, which provides a guide on how to reuse and recycle them.

[1460] This system not only provides users with efficient and individually optimized outfit suggestions, but also offers highly satisfying suggestions based on emotion recognition.It also promotes the reuse and recycling of closet items, helping to realize a sustainable fashion lifestyle.

[1461] Specific examples

[1462] For example, if a user enters the prompt "I have a date in a week. Please suggest an outfit based on the weather forecast and the items in my closet," the system will suggest outfits such as a white cotton shirt, blue denim pants, black leather boots, and a silver pendant. Different outfits are also presented based on the weather forecast, and the user can select one. The emotion engine analyzes the user's facial expressions to determine whether they are satisfied. Based on the results, future suggestions are adjusted. If they are dissatisfied, the system can try a different outfit.

[1463] In the reuse and recycling scenario, the app identifies infrequently used items in the closet, such as "blue denim pants that haven't been worn for six months." When the user selects it, they are given instructions such as contact information for a recycle shop or a donation destination.

[1464] As a result, users can make effective use of closet items and enjoy more personalized coordination.

[1465] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1466] Step 1:

[1467] A user takes a photo of an item in their closet and uploads the image to their device. This is the input, and the specific data is an image file. The device receives the image and sends it to the server. The device's processing is the transmission of image data.

[1468] Input: Image file

[1469] Output: Send image data to the server

[1470] Step 2:

[1471] The server analyzes the received images and identifies the color, material, and category of each item. This analysis uses image analysis algorithms (e.g., OpenCV, TensorFlow). The analysis results are stored in a database, and each item is assigned a tag. The data obtained from the image analysis process is the item's attribute information.

[1472] Input: Image data

[1473] Output: Item color, material, and category information

[1474] Step 3:

[1475] The user inputs information about the purpose of the outing (e.g., date, work, casual), date and time, and location through the terminal interface. This information is the input data. The terminal sends the input information to the server. The terminal's processing is to send the outing information.

[1476] Input: Purpose of outing, date and time, location information

[1477] Output: Sending out-of-office information to the server

[1478] Step 4:

[1479] The server retrieves weather forecast data based on the input date, time, and location of the planned outing. It sends a request to a weather forecast API (e.g., OpenWeatherMap, WeatherAPI) and receives the forecast data. The retrieved data is the weather forecast. The server stores this data in a database.

[1480] Input: Date and time of outing, location information

[1481] Output: Weather forecast data

[1482] Step 5:

[1483] The server generates optimal outfit patterns based on the acquired weather forecast data and the user's closet database. Here, a generative AI model (e.g., GPT-3) is used, taking into account the user's preferences and style. The generated multiple outfit patterns are the output data.

[1484] Input: Weather forecast data, closet database

[1485] Output: Coordination pattern

[1486] Step 6:

[1487] The server sends the generated coordination patterns to the terminal, which then presents them to the user. The user selects one of the presented coordination patterns, and the selection information is the input data. The terminal then sends the selection result and information on the coordination patterns not selected to the server.

[1488] Input: Coordination pattern, user selection

[1489] Output: Send feedback data to the server

[1490] Step 7:

[1491] The server analyzes the feedback data and learns the user's preferences and style. The analysis results are stored in a database and reflected in the next outfit suggestions. The data obtained through the analysis process is the user's preferences and style trends.

[1492] Input: Feedback data

[1493] Output: User preferences and style trends data

[1494] Step 8:

[1495] The device uses the built-in camera and microphone to collect the user's facial expressions and voice data. An emotion recognition algorithm (e.g., EmotionAPI) is used to identify the user's emotions. This emotion data is the input data. The device sends the emotion data to the server, which then determines the user's satisfaction level based on the analysis results.

[1496] Input: facial expressions, voice data

[1497] Output: Emotion data, satisfaction rating

[1498] Step 9:

[1499] When a user wants to organize their closet, they request reuse and recycling suggestions through the device interface. The content of this request is the input data. The device sends the request to the server, which analyzes the closet database and lists items that have not been used for a certain period of time. The listed items are the output data.

[1500] Input: Reuse / Recycle Request

[1501] Output: List of items suggested for reuse and recycling

[1502] Step 10:

[1503] The terminal presents a list of suggested items to the user, who then selects whether to reuse or recycle. The selection result is the input data. The terminal then sends the selection result to the server, which provides a guide on how to reuse or recycle. The guide information is the output data.

[1504] Input: Reuse / recycle selection results

[1505] Output: Reuse and recycling guide

[1506] (Application example 2)

[1507] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1508] Conventional coordination suggestion systems do not collect enough information to make appropriate suggestions using the items in a user's closet, making it difficult to make personalized suggestions that take into account the user's emotions and external factors such as the weather. Furthermore, there is a lack of systems that effectively suggest ways to organize a user's closet and recycle and reuse items. Therefore, there is a need for improved user experience and environmental considerations.

[1509] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1510] In this invention, the server includes a data conversion means, an outing information input means, a weather forecast acquisition means, a coordinate creation means, a feedback collection means, a reuse / recycle suggestion means, an emotion analysis means, a means for analyzing a user's facial image to identify emotions, a means for creating multiple coordinate patterns based on the weather forecast data and facial image analysis data, and a means for accumulating the feedback emotion data and reflecting it in the next suggestion. This enables personalized coordinate suggestions that take the user's emotions and the weather forecast into consideration, and also enables efficient closet organization and recycling / reuse suggestions.

[1511] The "data generation means" is a means for generating data from items in the user's closet through image analysis.

[1512] The "going out information input means" is a means by which the user inputs information about the purpose, date and time of going out, and the location.

[1513] The "weather forecast acquisition means" is a means for acquiring weather forecast data based on the date, time and location of a planned outing.

[1514] The "coordination generating means" is a means for generating a plurality of coordination patterns based on the acquired weather forecast data and closet database.

[1515] The "feedback collection means" is a means for collecting the user's selection results and feedback on the presented coordination patterns.

[1516] The "reuse / recycle suggestion tool" is a tool that allows users who wish to organize their closets to list items that have not been used for a certain period of time and suggests reuse or recycling.

[1517] The "emotion analysis means" is a means for analyzing a facial image of a user to identify the emotion.

[1518] "Weather forecast data" refers to weather information for a specific date, time and location obtained using a weather forecast API or similar.

[1519] "Facial image analysis data" is emotional data obtained by analyzing the facial image of the user.

[1520] A "closet database" is a database in which items in a user's closet are stored as data.

[1521] The "emotion engine" is an engine that recognizes emotions by analyzing the user's facial expressions and voice.

[1522] A "server" is a computer system that analyzes data, stores data, calls APIs, and generates proposals.

[1523] This invention is implemented by the following means. A system in which a user uses a smartphone or other device to digitize the items in their closet through image analysis, and suggests optimal outfits to the user based on information such as the purpose of the outing and the weather forecast. It is also possible to identify emotions by analyzing the user's facial expressions and voice, and reflect this as feedback in the next suggestions.

[1524] Hardware and software used

[1525] Smartphone: Used by users to take pictures of items, input outing information, and check suggested outfits.

[1526] Server: Analyzes data, obtains weather forecasts, generates coordinates, and accumulates feedback data.

[1527] Keras: Used to run deep learning models for recognizing emotions from facial images.

[1528] OpenCV: Used to perform image analysis of items in the closet.

[1529] Weather forecast API: Obtain weather data based on the date, time, and location you plan to go out.

[1530] Data processing and calculation

[1531] 1. Data collection method: The user takes a photo of the items in their closet with their smartphone and uploads the image to the server. The server analyzes the image, extracts attribute information such as color, material, and category, and stores it in the closet database. This process is performed using OpenCV.

[1532] 2. Means for inputting outing information: The user inputs the purpose, date, time, and location of the outing through the smartphone application. This information is sent to the server.

[1533] 3. Weather forecast acquisition method: The server uses the weather forecast API to acquire the weather forecast for the date, time, and location specified by the user.

[1534] 4. Coordination Generation: The server generates multiple coordination patterns based on weather data and the closet database, taking into account the user's preferences and style trends. The generated coordination patterns are sent to the user's smartphone and displayed to them.

[1535] 5. Feedback collection method: The user selects from the proposed outfits and sends the results back to the server. This allows the server to collect information on outfits that were not selected, and learns the user's style tendencies.

[1536] 6. Emotion analysis: The smartphone captures the user's facial image and analyzes their facial expressions using Keras. The recognized emotion data is sent to the server and used to improve the accuracy of outfit suggestions.

[1537] 7. Reuse / Recycle Suggestion Method: When a user wants to organize their closet, the request is sent to the server. The server refers to the closet database and lists items that have not been used for a certain period of time. This allows the user to make suggestions for reuse or recycling.

[1538] Specific examples

[1539] 1. The user takes a photo of their own shirt or pants with their smartphone and registers these items in a database through the application.

[1540] 2. The user enters an upcoming "casual lunch" into the app, specifying the date, time, and location of the outing.

[1541] 3. The server uses the weather forecast API to get the weather for the specified date, time, and location. If the weather is "sunny," it generates a light outfit.

[1542] 4. The server sends the generated outfit to the user and displays it on their smartphone. The user selects the outfit, and the smartphone captures their face image and analyzes their emotions.

[1543] 5. The emotion engine recognizes "satisfaction" from the user's facial expression and feeds this data back to the server, which will then be reflected in the next suggestion.

[1544] Prompt Sentence Examples

[1545] Closet item image analysis:

[1546] image_path = 'user_shirt.jpg'

[1547] item_data = app.analyze_image(image_path)

[1548] Enter your travel information:

[1549] outing_type = "casual lunch"

[1550] outing_datetime = datetime(2023, 10, 24, 12, 30)

[1551] outing_location = "Tokyo"

[1552] outing_info = app.input_outing_info(outing_type, outing_datetime, outing_location)

[1553] Coordinate generation:

[1554] weather_data = app.get_weather_forecast(outing_location, outing_datetime)

[1555] coordinates = app.generate_coordinate(outing_info, weather_data)

[1556] Collecting emotional feedback:

[1557] user_choice = coordinates[0]

[1558] emotion_feedback = app.provide_feedback(user_choice)

[1559] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1560] Step 1:

[1561] A user takes a photo of an item in their closet with their smartphone and uploads the image to a device. The device then sends the image to the server, which uses OpenCV to analyze the image and extract attribute information such as the item's color, material, and category. This data is then stored in a closet database.

[1562] (Input) Item image

[1563] (Data processing) Image analysis, attribute information extraction

[1564] (Output) Saving attribute information to a database

[1565] Step 2:

[1566] The user enters information about the purpose, date, time, and location of their outing through a smartphone application, and this information is sent from the device to a server.

[1567] (Input) Purpose, date, time, and location information

[1568] (Data calculation) Information storage

[1569] (Output) Send data to the server, save information

[1570] Step 3:

[1571] The server uses the weather forecast API to retrieve weather forecast data for the date, time, and location specified by the user, which involves calling an external weather forecast service and retrieving the required data.

[1572] (Input) Date, time, and location information

[1573] (Data calculation) API call, weather data acquisition

[1574] (Output) Obtaining weather forecast data

[1575] Step 4:

[1576] The server generates multiple outfit patterns suitable for the user's outing based on weather data and a closet database. Past feedback data is also taken into consideration. The generated outfit patterns are sent to the device.

[1577] (Input) Weather data, closet database, usage information

[1578] (Data calculation) Coordination generation, reflecting preference trends

[1579] (Output) Sending coordinate patterns to the device

[1580] Step 5:

[1581] The user selects from multiple outfit patterns displayed on the smartphone screen. The selection result is fed back from the device to the server. The server accumulates this information and learns the user's preferences and style trends.

[1582] (Input) Selection result

[1583] (Data calculation) Feedback collection, trend learning

[1584] (Output) Saving to database

[1585] Step 6:

[1586] The device captures the user's facial image and performs emotion analysis. The emotion engine uses Keras to analyze the facial expressions and sends the recognized emotion data to the server. The server receives this emotion data as feedback and uses it to improve future suggestions.

[1587] (Input) Facial image

[1588] (Data calculation) Emotion analysis, data transmission

[1589] (Output) Acquisition and storage of emotion data

[1590] Step 7:

[1591] When a user wants to organize their closet, they send a request from their device to the server. The server then references the closet database, lists items that have not been used for a certain period of time, and suggests reuse or recycling to the user. The results and instructions are displayed on the device.

[1592] (Input) Arrangement request

[1593] (Data calculation) Data reference, list proposal

[1594] (Output) Reuse and recycling proposals

[1595] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1596] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1597] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1598] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1599] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1600] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1601] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1602] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1603] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1604] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1605] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1606] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1607] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1608] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1609] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1610] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1611] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1612] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1613] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1614] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1615] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1616] The following is further disclosed regarding the above embodiment.

[1617] (Claim 1)

[1618] A data conversion means;

[1619] An outing information input means;

[1620] A means for obtaining weather forecasts;

[1621] Coordinate generation means;

[1622] a feedback collection means;

[1623] Reuse and recycling proposal methods,

[1624] A system including:

[1625] (Claim 2)

[1626] 2. The system according to claim 1, further comprising means for digitizing items in the user's closet through image analysis.

[1627] (Claim 3)

[1628] 2. The system according to claim 1, further comprising means for generating a plurality of coordination patterns based on weather forecast data.

[1629] (Claim 4)

[1630] 2. The system according to claim 1, further comprising means for storing coordinates selected by a user and coordinates not selected by a user in a database.

[1631] (Claim 5)

[1632] 2. The system according to claim 1, further comprising means for identifying infrequently used items and suggesting reuse or recycling of the items.

[1633] "Example 1"

[1634] (Claim 1)

[1635] A way to digitize the items in your closet,

[1636] A means for inputting the purpose, date, time and location of the outing;

[1637] A means for obtaining a weather forecast for when you plan to go out;

[1638] A means for generating a coordinate based on the acquired weather forecast and the digitized items;

[1639] a means for collecting feedback on the generated coordinates;

[1640] A means to propose reuse and recycling for items that have not been used for a certain period of time,

[1641] A system including:

[1642] (Claim 2)

[1643] 2. The system according to claim 1, further comprising means for converting the user's belongings into data through image analysis.

[1644] (Claim 3)

[1645] The system according to claim 1, further comprising means for generating a plurality of coordination patterns based on weather forecast data and a generation AI model.

[1646] "Application Example 1"

[1647] (Claim 1)

[1648] A data conversion means;

[1649] An outing information input means;

[1650] A means for obtaining weather forecasts;

[1651] Coordinate generation means;

[1652] a feedback collection means;

[1653] Reuse and recycling proposal methods,

[1654] A means for building a database to digitize products in physical stores;

[1655] an information presentation means using a display device that can be worn by a user;

[1656] A system including:

[1657] (Claim 2)

[1658] The system according to claim 1, characterized in that it has a means for digitizing items in a user's closet through image analysis and a means for registering and managing products in a physical store in a database.

[1659] (Claim 3)

[1660] 2. The system according to claim 1, further comprising means for generating a plurality of coordination patterns based on weather forecast data and the purpose of going out, and presenting the patterns on a display device.

[1661] "Example 2: Combining Emotion Engines"

[1662] (Claim 1)

[1663] A data conversion means;

[1664] An outing information input means;

[1665] A means for obtaining weather forecasts;

[1666] Coordinate generation means;

[1667] a feedback collection means;

[1668] An emotion recognition means;

[1669] Reuse and recycling proposal methods,

[1670] A system including:

[1671] (Claim 2)

[1672] 2. The system according to claim 1, further comprising means for digitizing items in the user's closet through image analysis.

[1673] (Claim 3)

[1674] 2. The system according to claim 1, further comprising means for generating a plurality of coordination patterns based on weather forecast data.

[1675] "Application example 2 when combining emotion engines"

[1676] (Claim 1)

[1677] A data conversion means;

[1678] An outing information input means;

[1679] A means for obtaining weather forecasts;

[1680] Coordinate generation means;

[1681] a feedback collection means;

[1682] Reuse and recycling proposal methods,

[1683] A sentiment analysis means;

[1684] A means for analyzing a facial image of a user to identify emotions;

[1685] A means for generating a plurality of coordination patterns based on weather forecast data and facial image analysis data;

[1686] A method to accumulate feedback emotional data and reflect it in the next proposal,

[1687] A system including:

[1688] (Claim 2)

[1689] 2. The system according to claim 1, further comprising means for digitizing items in the user's closet through image analysis.

[1690] (Claim 3)

[1691] 2. The system according to claim 1, further comprising means for generating a plurality of coordination patterns based on weather forecast data and facial image analysis data. [Explanation of symbols]

[1692] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A data conversion means; An outing information input means; A means for obtaining weather forecasts; Coordinate generation means; a feedback collection means; Reuse and recycling proposal methods, A system including:

2. 2. The system according to claim 1, further comprising means for converting items in the user's closet into data by image analysis.

3. 2. The system according to claim 1, further comprising means for generating a plurality of coordination patterns based on weather forecast data.

4. 2. The system according to claim 1, further comprising means for storing coordinates selected by a user and coordinates not selected by a user in a database.

5. 2. The system according to claim 1, further comprising means for identifying items that are used less frequently and suggesting reuse or recycling of the items.

Images