System

Abstract

A system is provided.SOLUTION: A system including a means for photographing an image of clothes and uploading the image to a server, a means for analyzing the image by the server and extracting a feature of the clothes, a means for comparing trend data with the feature of the clothes and recommending an optimal combination of clothes, a means for registering information on clothes owned by a user and proposing a combination according to a season or a situation, and a means for displaying an advertisement of a store dealing with similar clothes and enabling online purchase.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the past, it took a lot of time and effort for users to efficiently manage their clothing and find the optimal outfit. In particular, it was difficult to find the optimal combination based on information from recent fashion trends and influencers, and there was a lack of means to quickly create the appropriate outfit. Furthermore, when purchasing new clothes, users also faced challenges in finding items that easily match their existing wardrobe. The objective of the present invention is to solve these problems and provide a system that allows users to easily find the optimal outfit and efficiently purchase new clothes. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems with a system that includes a means for taking images of clothes and uploading them to a server, a means for the server to analyze the images and extract clothing features, a means for comparing the clothing features with trend data and recommending optimal clothing combinations, a means for registering information about the user's clothing and proposing combinations according to the season and situation, and a means for displaying advertisements for stores that carry similar clothing and enabling online purchases. When a user uploads an image of clothing to the server, the color, pattern, and shape of the clothing are analyzed using an image processing library and a machine learning model. The extracted features are compared with a trend database to suggest optimal outfits based on current trends. Furthermore, by pre-registering information about the user's clothing, various combinations according to the season and situation are suggested, and advertisements for stores that carry similar products are displayed, allowing the user to efficiently purchase new clothes online.

[0006] "Means for taking images of clothes" refers to means by which a user can take photos of their own clothes using a photographic device such as a smartphone or camera.

[0007] The "means for uploading to a server" is a means for transmitting the captured image to a remote server via the Internet.

[0008] "Means for extracting clothing features" refers to a means for extracting clothing features such as color, pattern, and shape using image analysis technology and machine learning models.

[0009] "Trend data" refers to data about current trends, such as the latest fashion information and influencer posts.

[0010] "Means for recommending clothing combinations" refers to a means for suggesting optimal clothing combinations to users based on extracted clothing characteristics and trend data.

[0011] "Means for registering information about clothes owned" refers to the means by which users can register the clothes they own in the database.

[0012] "Situation" refers to the circumstances surrounding a particular event or environment, and is a concept that includes seasons, events, weather, etc.

[0013] The "means for suggesting combinations" is a means for suggesting to the user clothing combinations suitable for the season and situation based on registered clothing information.

[0014] "Means for displaying advertisements for stores that sell similar clothing" refers to means for displaying advertisements for stores that sell products similar to the clothing uploaded by the user on the user's device.

[0015] "Means for enabling online purchases" refers to means that allow users to click on a displayed advertisement to access the relevant store's online shop and purchase the product.

[0016] An "image processing library" is a group of programs for preprocessing and analyzing images, and a typical example is OpenCV.

[0017] A "machine learning model" is a mathematical model that learns patterns from large amounts of data and makes predictions and classifications for new data.

[0018] A "database" is a collection of information that systematically stores specific data and can be searched and used as needed. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The system of the present invention allows users to take photos of their clothes, find the best outfits based on the photos, and then purchase similar clothes online. The specific configuration and operation of this system will be described below.

[0041] System configuration

[0042] 1. Device:

[0043] This refers to devices such as smartphones and tablets used by users.

[0044] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[0045] 2. Server:

[0046] This is a remote server that receives and analyzes image data and manages user data.

[0047] An image processing library is used to preprocess and analyze images, and a machine learning model is used to extract clothing features.

[0048] It refers to a trend database and suggests optimal outfits to users.

[0049] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[0050] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[0051] Program processing

[0052] 1. Take and upload an image of your outfit

[0053] The user uses their smartphone to take a photo of newly purchased clothes or clothes they are wearing.

[0054] The device acquires the captured image and sends it to the server using a dedicated app.

[0055] 2. Image Receipt and Analysis

[0056] The server stores the received images and performs preprocessing on them using an image processing library (e.g., OpenCV), such as noise reduction, resizing, and color correction.

[0057] Then, a machine learning model (e.g., TENSORFLOW®) is applied to extract clothing features from the image, including color, pattern, shape, and season.

[0058] 3. Comparison with trend data

[0059] The server compares the extracted clothing characteristics with a trend database, which contains the latest fashion information and influencer posting data.

[0060] Based on the comparison results, the optimal clothing combination is generated and recommended to the user.

[0061] 4. Registering the user in the clothing database

[0062] Users can register information about their clothing using a dedicated app. Information about each piece of clothing can be automatically recognized (image analysis) or manually entered (color, pattern, shape, etc.).

[0063] The server stores the registered information in a database and analyzes the combination of newly registered clothes with existing clothes.

[0064] Based on the user's requests, the system suggests the best outfits for each season and situation.

[0065] 5. Viewing Ads and Purchasing Online

[0066] The server obtains advertising data for stores that sell similar products based on the characteristics of the user's clothes.

[0067] The server sends the acquired advertising data to the user's device and displays it through a dedicated app.

[0068] Users can click on an ad that interests them and access the product page of the online shop to which the ad links, where they can check the product details and make a purchase.

[0069] Specific examples

[0070] For example, consider the case where user A purchases a new red floral dress.

[0071] 1. User A takes a photo of a red floral dress with her smartphone and uploads it to the server using a dedicated app.

[0072] 2. The server receives the image, preprocesses it using an image processing library, and uses a machine learning model to extract the color (red), pattern (floral), and shape (dress) of the dress.

[0073] 3. The server compares the extracted features with a trend database and suggests outfits that combine them with, for example, white sandals and a denim jacket.

[0074] 4. User A has registered other clothes he owns (for example, a white shirt or a black skirt) in the app, and suggestions for combinations with these items are also displayed.

[0075] 5. The server displays an advertisement for a store that sells products similar to the red floral dress, and User A can purchase the product from the online shop.

[0076] In this way, the system of the present invention not only helps users efficiently manage their clothes and find the best outfits, but also allows them to easily purchase related new clothes online.

[0077] The processing flow will be explained below.

[0078] Step 1:

[0079] The user takes a photo of the clothes using the camera on their smartphone or tablet. After taking the photo, they open the dedicated app and tap the "Upload New Clothes" button. This action causes the device to import the image into the app and prepare it for transmission to the server.

[0080] Step 2:

[0081] The device retrieves the captured image and makes an HTTP request to an API endpoint to upload the image file to the server, including the image data and metadata such as the user ID.

[0082] Step 3:

[0083] The server temporarily stores the received image files. Next, it performs preprocessing on the images using an image processing library (e.g., OpenCV). Specifically, it performs image resizing, noise removal, color correction, etc. This generates an image suitable for analysis.

[0084] Step 4:

[0085] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image. This model is pre-trained and can identify clothing features such as color, pattern, shape, and even seasonal characteristics. The analysis results are expressed as, for example, "a dress with a red floral pattern."

[0086] Step 5:

[0087] The server compares the extracted features with a trend database, which stores data on the latest fashion magazines, influencer posts, and popular items from online shops. Using a comparison algorithm, the server generates optimal outfits that match current trends.

[0088] Step 6:

[0089] The server sends the generated outfit suggestions to the device. When the user opens the app, the outfit suggestions are displayed. For example, specific suggestions such as "Pair a red floral dress with white sandals" are made.

[0090] Step 7:

[0091] Users can register information about the clothes they own using a dedicated app. Both automatic recognition (image analysis) and manual input methods are available, making it easy for users to register information about their clothes.

[0092] Step 8:

[0093] The server stores the clothing information sent by the user in a database and analyzes combinations based on existing and newly added clothing information, generating coordination suggestions according to the season and situation.

[0094] Step 9:

[0095] The server retrieves advertising data from stores that sell similar products based on the characteristics of the user's clothing, selects relevant advertisements through advertising networks and partnerships, and prepares to display them to the user.

[0096] Step 10:

[0097] The device displays the advertising data received from the server to the user. When the user clicks on an advertisement that interests them, they are redirected to the product page of the online shop linked to the advertisement. Here, the user can check the product details and purchase it online.

[0098] Through the above processing steps, the user can manage his / her own clothes, receive suggestions for coordinating outfits, and easily purchase related products online.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] Previously, systems that allowed users to manage their clothing and find optimal outfits were either complex or had limited functionality. Furthermore, there were no systems that offered trend-based clothing combination suggestions or a unified online purchasing solution. This made it difficult for users to efficiently manage their clothing, find optimal outfits, and easily purchase related products online.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the server includes a means for a user to take an image and upload it to the server, a means for the server to analyze the image and extract clothing features using an image processing library and a machine learning model, and a means for comparing the extracted clothing features with trend data to generate optimal clothing combinations, thereby enabling users to efficiently manage their own clothing, find optimal combinations, and easily purchase related products online.

[0104] "User" refers to an individual who uses the system to manage their own clothing and receive suggested outfits.

[0105] "Terminal" refers to a portable computing device used by a User, such as a smartphone or tablet.

[0106] "Server" refers to a computer system that remotely receives and analyzes image data and manages user data.

[0107] "Image processing library" refers to software tools and libraries for preprocessing and analyzing images, such as OpenCV.

[0108] A "machine learning model" refers to an algorithm that learns patterns from data and makes predictions or classifications. Specific examples include TensorFlow.

[0109] "Clothing features" refer to attributes of clothing extracted from an image, such as color, pattern, shape, and season.

[0110] "Trend data" refers to a database containing the latest fashion information and influencer posting data.

[0111] A "trend database" refers to a database that accumulates the latest fashion information and trends.

[0112] "Optimal clothing combination" refers to the clothing coordination that is considered optimal for the user, calculated based on the extracted clothing characteristics and the user's preferences and trend data.

[0113] "User's clothing information" refers to data about the clothes owned by the user (color, pattern, shape, etc.).

[0114] "Database" refers to a data storage system for storing and managing information about users' clothing and trend data.

[0115] "Advertising data" refers to advertising information provided by stores and online shops that sell similar products.

[0116] "Online purchase" refers to a user completing a purchase procedure for a product via the Internet.

[0117] MODE FOR CARRYING OUT THE INVENTION

[0118] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the photos, and then help them purchase similar clothes online. The specific configuration and operation of this system are described below.

[0119] System configuration

[0120] 1. Device:

[0121] This refers to devices such as smartphones and tablets used by users.

[0122] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[0123] 2. Server:

[0124] This is a remote server that receives and analyzes image data and manages user data.

[0125] An image processing library (e.g., OpenCV) is used to preprocess and analyze the images, and a machine learning model (e.g., TensorFlow) is used to extract clothing features.

[0126] It refers to a trend database and suggests optimal outfits to users.

[0127] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[0128] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[0129] Overview of program processing

[0130] 1. Take and upload an image of your outfit

[0131] Users take photos of newly purchased or currently wearing clothes with their smartphones, and the device sends the images to a server using a dedicated app.

[0132] 2. Image Receipt and Analysis

[0133] The server stores the received images and performs preprocessing using an image processing library (e.g., OpenCV) to remove noise, resize, and correct color. It then applies a machine learning model (e.g., TensorFlow) to extract clothing features (e.g., color, pattern, shape, season) from the images.

[0134] 3. Comparison with trend data

[0135] The server compares the extracted clothing characteristics with a trend database, which contains the latest fashion information and influencer postings. Based on the comparison results, the server generates optimal clothing combinations and suggests them to the user.

[0136] 4. Registering the user in the clothing database

[0137] Users use a dedicated app to register information about the clothes they own. Information about each piece of clothing can be automatically recognized through image analysis or can be entered manually. The server stores this information in a database and analyzes how newly registered clothes can be combined with existing clothes. The system then provides optimal outfits according to the season and situation.

[0138] 5. Viewing Ads and Purchasing Online

[0139] The server retrieves advertising data for stores that carry similar products based on the characteristics of the user's clothing. The acquired advertising data is sent to the user's device and displayed through a dedicated app. The user can click on an advertisement that interests them, access the product page in the online shop, and purchase the item.

[0140] Specific examples

[0141] For example, a case will be described in which user A purchases a new red floral dress.

[0142] 1. User A takes a photo of a red floral dress with her smartphone and uploads it to the server using a dedicated app.

[0143] 2. The server receives the image, preprocesses it using an image processing library (e.g., OpenCV), and then uses a machine learning model (e.g., TensorFlow) to extract the color (red), pattern (floral), and shape (dress) of the dress.

[0144] 3. The server compares the extracted features with trend data and suggests outfits that combine them with, for example, white sandals and a denim jacket.

[0145] 4. User A has registered other clothes he owns (e.g., white shirts and black skirts) in the app, and suggestions for combinations with these items are also displayed.

[0146] 5. The server displays an advertisement for a store that sells products similar to the red floral dress, and User A can purchase the product from the online shop.

[0147] Prompt Sentence Examples

[0148] "Generate a sentence that explains the system that receives an image of an outfit from the user, analyzes the image, suggests the best outfit, and advertises similar products."

[0149] By configuring it in this way, the system not only helps users efficiently manage their clothes and find the best outfits, but also allows them to easily purchase related new clothes online.

[0150] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0151] Step 1:

[0152] The user takes a photo of the clothes with their smartphone and uploads the image through a dedicated app. The input is the image of the clothes taken by the user, and the output is image data sent to the server. The device uses its camera function to acquire the image and sends it to the server via the Internet.

[0153] Step 2:

[0154] The server saves the image data received from the terminal. The input is the image data sent from the terminal, and the output is the saved image data. Specifically, the server stores the received image data in storage and prepares it for subsequent analysis processing.

[0155] Step 3:

[0156] The server preprocesses the received images using an image processing library (e.g., OpenCV). The input is the stored image data, and the output is the preprocessed image data. Specifically, it performs processes such as noise removal, resizing (e.g., to 256x256 pixels), and color correction.

[0157] Step 4:

[0158] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from preprocessed images. The input is the preprocessed image data, and the output is extracted clothing feature data (e.g., color, pattern, shape, season). Specifically, the image is input into the machine learning model, and clothing features are obtained from the output.

[0159] Step 5:

[0160] The server compares the extracted clothing feature data with a trend database. The input is the clothing feature data and the trend database, and the output is optimal clothing combination suggestion data. The server refers to the trend database and calculates and generates the optimal outfit for the user.

[0161] Step 6:

[0162] The server generates optimal outfit combinations and sends that information to the user's device. The input is the optimal outfit combination suggestion data, and the output is the suggestion information sent to the user's device. Specifically, when the user opens the app, the suggested outfits are displayed.

[0163] Step 7:

[0164] The user registers information about their clothing using a dedicated app. The input is the clothing information entered or photographed by the user, and the output is the registration information sent to the server. The device acquires this information and sends it to the server.

[0165] Step 8:

[0166] The server stores the user's clothing information in a database. The input is the registered clothing information, and the output is the stored database. The server compares the user's new clothing data with existing data and analyzes the combinations.

[0167] Step 9:

[0168] The server retrieves advertising data for stores that sell similar products based on the characteristics of the user's clothing. The input is the characteristic data of the user's clothing, and the output is the retrieved advertising data. The server then refers to the advertising service to collect advertising information for related products.

[0169] Step 10:

[0170] The server sends the acquired advertising data to the user's device. The input is the collected advertising data, and the output is advertising information sent to the user's device. Users can view these advertisements through a dedicated app and purchase products they are interested in online.

[0171] (Application example 1)

[0172] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0173] Conventional clothing coordination suggestion systems have difficulty easily linking information about the clothes a user owns with the latest trends, and have also faced challenges in efficiently finding and purchasing similar products. Furthermore, they lacked personalized suggestions tailored to the user's preferences and requirements, requiring users to expend a great deal of time and effort. Our goal is to provide a system that solves these issues, allowing users to more easily and instantly find appropriate coordinations and purchase similar products.

[0174] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0175] In this invention, the server includes means for taking images of clothes and uploading them to a remote server, means for the remote server to analyze the images and extract clothing characteristics, means for comparing the clothing characteristics with trend data and presenting optimal clothing combinations, means for registering information about clothes owned by the user and proposing combinations according to seasons and events, means for displaying advertisements for stores that carry similar clothes and enabling online purchases, means for proposing new clothes using a generative AI model based on optimal coordination, and means for proposing clothes based on specified requirements by inputting a prompt text. This allows users to not only easily analyze and manage the characteristics of their own clothes, but also enables efficient coordination suggestions linked to the latest trend information and allows them to quickly find and purchase similar products.

[0176] "Means for taking images of clothes and uploading them to a remote server" refers to a function that allows a user to take an image of clothes using a device such as a smartphone or tablet, and then send the image data to a remote server via the Internet.

[0177] "Means for the remote server to analyze the image and extract characteristics of the clothing" refers to a function in which the remote server analyzes the image data received and extracts characteristics such as the color, pattern, shape, and seasonality of the clothing.

[0178] The "means for comparing trend data with clothing characteristics and presenting optimal clothing combinations" is a function that compares extracted clothing characteristics with the latest trend data and presents optimal outfits to the user.

[0179] "A means for users to register information about the clothes they own and suggest combinations appropriate for the season or event" is a function that allows users to register information about the clothes they own in the system and then suggests outfits appropriate for the season or event based on that information.

[0180] "Means for displaying advertisements for stores that sell similar clothing and enabling online purchases" is a function that displays advertisements for stores that sell similar products based on the characteristics of the clothing registered by the user, allowing the user to purchase directly from those stores.

[0181] "Means for proposing new clothing using a generative AI model based on optimal coordination" is a function that uses a generative AI model to suggest new clothing based on extracted clothing characteristics and optimal coordination information.

[0182] "A means for suggesting clothing based on specified requirements by inputting a prompt sentence" is a function that allows the user to input specific requirements or preferences as a prompt sentence and suggests appropriate clothing based on that information.

[0183] The system of the present invention allows users to take photos of their own clothes using a smartphone or tablet, find optimal outfits based on the photos, and then help them purchase similar clothes online. The specific configuration and operation of this system are described in detail below.

[0184] System configuration

[0185] 1. Device (smartphone, tablet, etc.):

[0186] It has a camera function for taking images and a function for connecting to the Internet and communicating with a remote server.

[0187] Using a dedicated application, users can take photos of their clothes and upload them to a remote server.

[0188] 2. Remote Server:

[0189] Receives and analyzes image data, and manages user data.

[0190] An image processing library (e.g., OpenCV) is used to preprocess and analyze the images, and a machine learning model (e.g., TensorFlow) is used to extract clothing features.

[0191] The system refers to a trend database and compares the extracted clothing characteristics to suggest the optimal outfit.

[0192] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[0193] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[0194] Based on the optimal coordination, new clothing suggestions are made using a generative AI model.

[0195] By entering a prompt, clothing suggestions will be made based on the specified requirements.

[0196] How it works

[0197] 1. Take and upload your outfit image:

[0198] The user takes a photo of the new or currently worn clothes using a smartphone, and then uploads the image to a remote server using a dedicated application.

[0199] 2. Image Receipt and Analysis:

[0200] The remote server receives the uploaded images and performs preprocessing (noise removal, resizing, color correction, etc.) using an image processing library (OpenCV), then extracts clothing features (color, pattern, shape, season, etc.) using a machine learning model (TensorFlow).

[0201] 3. Comparison with trend data:

[0202] The extracted clothing characteristics are compared with a trend database, the latest fashion information, and influencer posts. Based on the comparison results, optimal clothing combinations are generated and recommended to the user.

[0203] 4. Registering the user in the clothing database:

[0204] Users register information about their clothing using a dedicated app. The registered information is stored in a database on a remote server. The server then analyzes how the newly registered clothing can be combined with existing clothing to suggest optimal outfits for each season and situation.

[0205] 5. Viewing Ads and Online Purchases:

[0206] The remote server retrieves advertising data for stores that sell similar products based on the extracted clothing characteristics. The acquired advertising data is sent to the user's device and displayed through a dedicated app. Users can click on an advertisement that interests them and purchase the product from the online shop.

[0207] 6. New clothing suggestions using generative AI models:

[0208] Based on the extracted clothing features and optimal coordination information, a generative AI model is used to suggest new clothing to the user. For example, if a user takes a photo of a red floral dress, the system will suggest new clothing designs based on those features.

[0209] 7. Specifying requirements by prompt sentence:

[0210] Users can specify their specific requirements and preferences by entering prompts in a dedicated application, and the server will then suggest clothing that matches the requirements.

[0211] Specific examples

[0212] Consider the case where User A purchases a new red floral dress, takes a photo of it with her smartphone, and uploads it to the server using a dedicated app. The server analyzes the image and extracts the color (red), pattern (floral), and shape (dress). It then compares the extracted features with a trend database and suggests combinations, such as white sandals and a denim jacket. Furthermore, if User A enters the prompt "casual outdoor style," the server uses the generative AI model to suggest new clothing based on those requirements.

[0213] Prompt Sentence Examples

[0214] text

[0215] Analyze an image of a red floral dress and suggest the best outfit for it. Also, show similar items from an online shop.

[0216] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0217] Step 1:

[0218] The device takes pictures of the clothes the user is wearing or newly purchased clothes. When the user clicks the capture button in the dedicated app, the device's camera function is activated and the image is captured. The input is the image data captured by the camera, and the output is an image file stored on the device.

[0219] Step 2:

[0220] The device uploads the captured images to a remote server via a dedicated app. When the user clicks the upload button, the image data is sent to the remote server via the Internet. The input is the image file in the device, and the output is the image data sent to the remote server.

[0221] Step 3:

[0222] The remote server stores the received image data and performs preprocessing on the image using an image processing library (e.g., OpenCV). Specifically, it performs processes such as noise reduction, resizing, and color correction. The input is the image data uploaded to the server, and the output is the image data after preprocessing.

[0223] Step 4:

[0224] The remote server inputs the preprocessed image data into a machine learning model (e.g., TensorFlow) to extract clothing features. Features include color, pattern, shape, season, etc. The input is the preprocessed image data, and the output is the extracted clothing feature data.

[0225] Step 5:

[0226] The remote server compares the extracted clothing features with a trend database, referring to the latest fashion information and influencer posts to generate optimal clothing combinations. The input is clothing feature data and trend data, and the output is optimal clothing combination data.

[0227] Step 6:

[0228] The remote server retrieves information about clothing registered by the user using a dedicated app from a database and analyzes how the newly registered clothing can be combined with existing clothing. As a specific example, it suggests optimal outfits according to the season or event. The input is the user's clothing data, and the output is combination data according to the season or event.

[0229] Step 7:

[0230] The remote server retrieves advertising data for stores that sell similar products based on the extracted clothing features. The advertising data is provided by online shops and displayed to users. The input is clothing feature data and store advertising data, and the output is advertising data displayed on the user's device.

[0231] Step 8:

[0232] The remote server uses a generative AI model based on the optimal outfit to suggest new outfits. The generative AI model uses deep learning, for example. The input is optimal outfit data, and the output is new outfit suggestion data.

[0233] Step 9:

[0234] The user inputs a prompt on a dedicated application and sends specific requirements and preferences to the server, which then suggests clothes that fit the specified requirements based on the prompt. The input is the prompt from the user, and the output is the suggested clothing data.

[0235] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0236] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the images, and assist users in purchasing similar clothes online. By combining it with an emotion engine, the system suggests the best clothing combinations and advertisements according to the user's emotions. The specific configuration and operation of this system will be described below.

[0237] System configuration

[0238] 1. Device:

[0239] A device such as a smartphone or tablet used by a user.

[0240] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[0241] It has the ability to capture the user's facial expressions and voice.

[0242] 2. Server:

[0243] This is a remote server that receives and analyzes image data and manages user data.

[0244] An image processing library is used to preprocess and analyze images, and a machine learning model is used to extract clothing features.

[0245] It refers to a trend database and suggests optimal outfits to users.

[0246] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[0247] The advertising data of stores that sell similar products is acquired and displayed to the user.

[0248] It uses an emotion engine to analyze user emotions and tailor suggestions and advertisements.

[0249] Program processing

[0250] When a user takes a photo of an outfit using the camera on their smartphone or tablet, opens the app, and taps the "Upload New Outfit" button, the image is sent to the server. The device retrieves the image and executes an HTTP request.

[0251] The server temporarily stores the received image files and performs preprocessing using an image processing library (e.g., OpenCV). An appropriate analysis state is generated by resizing, noise removal, color correction, etc. Next, a machine learning model (e.g., TensorFlow) is used to extract features such as the color, pattern, shape, and season of the clothing. This results in an analysis result such as "a dress with a red floral pattern."

[0252] The server compares the extracted features with a trend database, which includes data from magazines and social media, to generate optimal outfits based on the latest fashion information and influencer posts.

[0253] The server then sends the generated outfit suggestions to the user's device, where the user can receive the suggestions via the app. For example, a specific suggestion such as "pair a red floral dress with white sandals" is made.

[0254] Users can use a dedicated app to register information about the clothes they own. Registration methods include automatic recognition (image analysis) and manual input (color, pattern, shape, etc.), making it easy to add clothing information to the database. The server saves the registered clothing information in the database and analyzes how newly added clothes can be combined with existing clothes to make optimal suggestions.

[0255] The server collects advertising data from stores that sell similar products based on the user's clothing characteristics. It then selects relevant ads through advertising networks and partnerships and displays them to the user. When the user clicks on an ad that interests them, they are redirected to the product page in the online shop, where they can view product details and make a purchase.

[0256] Introducing the Emotion Engine

[0257] The system incorporates an emotion engine, and the following specific processes are added:

[0258] The emotion engine analyzes the user's facial expressions and voice to identify their current emotions. For example, it uses a camera to collect the user's facial expression data and uses voice recognition technology to analyze the user's speech.

[0259] The server uses the analysis results obtained from the emotion engine to suggest optimal outfits and advertisements according to the user's emotions. For example, if the user is happy, it will suggest bright-colored clothes, and if they are depressed, it will suggest comfortable, relaxing clothes. It is also possible to dynamically change relevant product advertisements according to the user's emotions.

[0260] Specific examples

[0261] For example, consider the case where User B purchases a new blue denim dress, takes a photo of it, and uploads it to the server.

[0262] 1. User B takes a photo of a denim dress with their smartphone and uploads it to the server using a dedicated app.

[0263] 2. The server receives the image, preprocesses it using an image processing library, and then uses a machine learning model to extract the color (blue), pattern (denim), and shape (dress) of the dress.

[0264] 3. The server compares the extracted features with a trend database and suggests outfits, such as "combining white sneakers with a red bag."

[0265] 4. If user B is found to be happy through the emotion engine, suggest more casual and brightly colored items.

[0266] 5. User B has already registered a white shirt and a black skirt in the app, and suggestions for combinations with them are also displayed.

[0267] 6. The server displays an advertisement for a store that sells a product similar to the blue denim dress, and User B purchases the product from an online shop.

[0268] In this way, the system of the present invention can dynamically respond to the user's emotions and can also provide personalized coordination and advertising suggestions.

[0269] The processing flow will be explained below.

[0270] Step 1:

[0271] The user takes a photo of the clothes using the camera on their smartphone or tablet. After taking the photo, they open the dedicated app and tap the "Upload New Clothes" button. This action causes the device to import the image into the app and prepare it for transmission to the server.

[0272] Step 2:

[0273] The device retrieves the captured image and makes an HTTP request to an API endpoint to upload the image file to the server, including the image data and metadata such as the user ID.

[0274] Step 3:

[0275] The server temporarily stores the received image files. Next, it performs preprocessing on the images using an image processing library (e.g., OpenCV). Specifically, it performs image resizing, noise removal, color correction, etc. This generates an image suitable for analysis.

[0276] Step 4:

[0277] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image. This model is pre-trained and can identify clothing features such as color, pattern, shape, and season. The analysis results are expressed as, for example, "a red floral dress."

[0278] Step 5:

[0279] The server compares the extracted features with a trend database, which stores data on the latest fashion magazines, influencer posts, and popular items from online shops. Using a comparison algorithm, the server generates optimal outfits that match current trends.

[0280] Step 6:

[0281] The server sends the generated outfit suggestions to the device. When the user opens the app, the outfit suggestions are displayed. For example, specific suggestions such as "Pair a red floral dress with white sandals" are made.

[0282] Step 7:

[0283] Users can register information about the clothes they own using a dedicated app. Both automatic recognition (image analysis) and manual input (color, pattern, shape, etc.) are available, making it easy for users to register information about their clothes.

[0284] Step 8:

[0285] The server stores the clothing information sent by the user in a database and analyzes combinations based on existing clothing information and newly added information, thereby generating coordination suggestions according to the season and situation.

[0286] Step 9:

[0287] The device captures the user's facial expressions and voice and sends them to an emotion engine, which uses a model trained on a dataset for emotion analysis to analyze the user's emotions in real time.

[0288] Step 10:

[0289] The emotion engine analyzes the user's emotions and sends the results to the server. For example, it responds when the user is judged to be happy or depressed.

[0290] Step 11:

[0291] The server uses the emotional data sent from the emotion engine to adjust the recommended outfits and advertisements. For example, if the user is happy, it will suggest brightly colored clothes and casual items, and if they are depressed, it will suggest relaxing clothes. It also selects the most appropriate advertisement based on the user's emotions and sends it to the user's device.

[0292] Step 12:

[0293] The device displays coordination suggestions and advertisements to the user based on the emotions received from the server. Because the suggestions displayed are suited to the user's emotions, the user can receive more personalized services.

[0294] Through the above processing steps, the user can receive suggestions for optimal clothing management and coordination based on their emotions, and can also easily purchase related products online.

[0295] Example 2

[0296] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0297] Conventional coordination support systems were unable to provide suggestions based on the user's emotions and mood regarding clothing, and were unable to sufficiently increase user satisfaction. Furthermore, when purchasing similar products online, advertisements displayed did not dynamically change based on the user's emotions and preferences, preventing effective marketing. Furthermore, it was difficult to provide suggestions for combining outfits with the user's existing clothes to suit the season or specific situations.

[0298] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for taking an image of clothes and uploading it to the server, a means for the server to analyze the image and extract characteristics of the clothes, a means for comparing the characteristics of the clothes with trend data and recommending optimal clothing combinations, a means for registering information on clothes owned by the user and proposing combinations according to the season and situation, a means for displaying advertisements for stores that carry similar clothes and enabling online purchases, and a means for analyzing the user's emotions using an emotion analysis engine and adjusting optimal clothing combinations and advertisements based on the emotions. This makes it possible to suggest coordinations and display advertisements that match the user's emotions and mood.

[0299] "Clothing image" refers to photographic data of clothing that the user will wear.

[0300] "Means for uploading" refers to the ability for a user to send images taken by the user to a remote server via the Internet.

[0301] "Means for analyzing images and extracting clothing features" refers to the process of identifying and extracting attributes such as clothing color, pattern, and shape from the received image data.

[0302] "Trend data" refers to data that reflects the latest fashion information and market trends.

[0303] "Means of recommending optimal clothing combinations" refers to a function that suggests outfits that suit the user based on trend data and clothing characteristics.

[0304] "Means for registering information about clothes owned by the user" refers to a function that allows a user to input or read data about clothes they own into the system.

[0305] "Means of suggesting combinations according to the season and situation" refers to a function that suggests clothing combinations that suit the season, specific events, or occasions.

[0306] "Means of displaying advertisements for stores that sell similar clothing and enabling online purchases" refers to a function that displays advertisements for stores that sell products similar to the clothing photographed by the user and allows the user to purchase those products online.

[0307] An "emotion analysis engine" refers to a system that analyzes a user's facial expressions and voice to identify their current emotional state.

[0308] "Means for adjusting optimal clothing combinations and advertisements based on emotions" refers to a function that dynamically changes coordination suggestions and advertisement content to match the identified user's emotions.

[0309] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the photos, and help them purchase similar clothes online. Furthermore, by combining it with an emotion engine, the system suggests the best clothing combinations and advertisements based on the user's emotions.

[0310] System configuration

[0311] 1. Device:

[0312] A device such as a smartphone or tablet used by a user.

[0313] It has a camera function for taking pictures and the ability to connect to the Internet and communicate with a server.

[0314] It has the ability to capture the user's facial expressions and voice.

[0315] 2. Server:

[0316] This is a remote server that receives and analyzes image data and manages user data.

[0317] Specifically, the image is preprocessed using an image processing library (e.g., OpenCV), and clothing features are extracted using a machine learning model (e.g., TensorFlow).

[0318] The system references a trend database and generates the optimal coordination.

[0319] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[0320] Advertising data is collected and displayed to the user.

[0321] It uses an emotion engine to analyze user emotions and tailor suggestions and advertisements.

[0322] Example of operation

[0323] 1. The user takes a photo of their clothes using the device's camera.

[0324] Example: A user takes a picture of a new blue denim dress.

[0325] 2. Upload the images taken by the device to the server using a dedicated app.

[0326] 3. The server receives the images and performs preprocessing using an image processing library (e.g., OpenCV).

[0327] Pre-processing includes resizing, noise reduction, color correction, etc.

[0328] 4. The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image.

[0329] Example: Extract features such as color (blue), pattern (denim), and shape (dress) from a blue denim dress.

[0330] 5. The server compares the extracted features with a trend database and suggests optimal outfits.

[0331] Example: A specific coordination suggestion such as "pair it with white sneakers and a red bag."

[0332] 6. The server sends the generated coordination proposal to the user's device.

[0333] 7. The device displays the suggestions to the user through a dedicated app.

[0334] For example, you might see suggestions like "Pair a blue denim dress with white sneakers."

[0335] 8. The user uses a dedicated app to register information about the clothes they own.

[0336] Example: Register information such as a white shirt or black skirt you own.

[0337] 9. The server analyzes the combination of new clothes with existing clothes based on the registered clothing information and makes optimal suggestions.

[0338] For example: a blue denim dress paired with a white shirt.

[0339] 10. The server collects advertising data from stores that sell similar products and displays it to the user.

[0340] 11. The device displays advertisements, and when the user clicks on an advertisement that interests them, they are redirected to the product page of the online shop.

[0341] Example: You see products similar to a blue denim dress and can buy them online.

[0342] Prompt Sentence Examples

[0343] "Suggest outfits that go well with the blue denim dress the user recently purchased. Also, if the user is happy with the outfit, suggest items in brighter colors."

[0344] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0345] Step 1:

[0346] The user takes a photo of the clothes using the device's camera.

[0347] Input: The user activates the camera and takes a picture of the clothes.

[0348] Output: The captured image data is saved on the device.

[0349] What happens: A user takes a picture of a new blue denim dress with their smartphone camera.

[0350] Step 2:

[0351] The device uploads the images taken to the server using a dedicated app.

[0352] Input: Image data stored on the device.

[0353] Output: An HTTP POST request containing the image data is sent to the server.

[0354] Specific operation: The user taps the "Upload new clothes" button, and the device sends image data to the server.

[0355] Step 3:

[0356] The server temporarily stores the received image files and performs preprocessing using an image processing library (e.g., OpenCV).

[0357] Input: Image data received by the server.

[0358] Output: Preprocessed image data.

[0359] Data processing or data calculation: image resizing, noise reduction, color correction, etc.

[0360] What happens: The server resizes the image, removes noise, and performs color correction.

[0361] Step 4:

[0362] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image.

[0363] Input: Preprocessed image data.

[0364] Output: Extracted clothing feature data (e.g. color, pattern, shape).

[0365] Data processing or data calculation: Identifying color, pattern, and shape from image data and generating feature data.

[0366] Specific operation: The server extracts the color (blue), pattern (denim), and shape (dress) of the blue denim dress.

[0367] Step 5:

[0368] The server compares the extracted features with a trend database and generates the optimal outfit.

[0369] Input: Extracted clothing feature data.

[0370] Output: Optimal outfit suggestion data.

[0371] Data processing or data calculation: Refer to the feature data and trend database to determine the optimal coordination.

[0372] Specific behavior: The server generates suggestions such as "pair with white sneakers and a red bag."

[0373] Step 6:

[0374] The server transmits the generated coordination proposal to the user's terminal.

[0375] Input: Coordination suggestion data.

[0376] Output: The proposal data sent to the user's device.

[0377] Specific operation: The server sends the generated coordination proposal to the user's device.

[0378] Step 7:

[0379] The coordinated suggestions received by the terminal are displayed to the user via a dedicated app.

[0380] Input: The coordinate suggestion data passed in.

[0381] Output: The suggestions displayed in the app.

[0382] Specific behavior: The device displays a suggestion on the app screen: "Pair a blue denim dress with white sneakers."

[0383] Step 8:

[0384] Users register information about the clothes they own using a dedicated app.

[0385] Input: Clothing information entered or uploaded by the user.

[0386] Output: Clothing information from the registered database.

[0387] Specific operation: The user enters information about a white shirt and a black skirt they own into the app and registers them.

[0388] Step 9:

[0389] Based on the registered clothing information, the server analyzes the combination of new clothing with existing clothing and makes optimal suggestions.

[0390] Input: Registered clothing information.

[0391] Output: Combination suggestion data of newly added clothes and existing clothes.

[0392] Data processing or data calculation: Combining and analyzing existing clothing information with new clothing information.

[0393] What it does: The server suggests a combination of a blue denim dress and a white shirt.

[0394] Step 10:

[0395] The server collects advertising data from stores that sell similar products and transmits it to the user's terminal.

[0396] Input: Registered clothing feature data.

[0397] Output: Collected advertising data.

[0398] What it does: The server collects relevant ads through ad networks.

[0399] Step 11:

[0400] The advertisements received by the device are displayed to the user via a dedicated app.

[0401] Input: Ad data sent by the server.

[0402] Output: The ad displayed within the app.

[0403] What happens: The device displays an ad for a product similar to the blue denim dress on the app screen.

[0404] Step 12:

[0405] The emotion engine analyzes the user's facial expressions and voice to identify their current emotion.

[0406] Input: Facial expression and audio data captured by the camera.

[0407] Output: Parsed emotion data.

[0408] Data processing or data calculation: Performing facial expression recognition and voice analysis to identify emotions.

[0409] Specific operation: Collects the user's facial expression data using a camera and analyzes the user's speech using voice recognition technology.

[0410] Step 13:

[0411] Based on the analysis results obtained from the emotion engine, the server adjusts the optimal coordination and advertisements according to the user's emotions.

[0412] Input: Parsed emotion data.

[0413] Output: Reconciled coordinate and advertising data.

[0414] Data processing or data calculation: Dynamically changing outfits or advertising content based on emotional data.

[0415] What it does: If the user is happy, suggest brighter colored items and show more relevant ads.

[0416] (Application example 2)

[0417] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0418] Until now, when users purchased clothes in a physical store, there was no way to instantly get suggestions on how to coordinate them with the clothes they already own or the best combinations that match their current mood. There is also a demand for personalized suggestions based on the user's emotions to provide a more comfortable shopping experience. This will increase product purchases in physical stores and improve customer satisfaction.

[0419] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for taking an image of clothes and uploading it to the server, means for the server to analyze the image and extract characteristics of the clothes, means for comparing the characteristics of the clothes with trend data and recommending optimal clothing combinations, means for registering information about clothes owned by the user and proposing combinations according to the season and situation, means for displaying advertisements for stores that carry similar clothes and enabling online purchases, and means for analyzing the user's facial expressions and voice to identify emotions and dynamically adjusting coordinations and advertisements based on the emotions. This allows users to take photos of clothes they like in a physical store, receive optimal coordination suggestions in real time, and enjoy a personalized shopping experience according to their emotions.

[0420] "Clothing images" are photos of clothing taken by the user using the camera function.

[0421] "Server" means a remotely located computer system that receives and processes data sent by users and returns analytical results.

[0422] "Clothing characteristics" are visual and physical attributes of clothing such as color, pattern, shape, and season.

[0423] "Trend Data" is a database containing information about current trends and popular styles.

[0424] The "optimal clothing combination" is the most appropriate fashion coordination for the user, generated based on the analyzed clothing characteristics and trend data.

[0425] "User information" refers to personal information such as data on the clothes owned by the user, their preferences, and purchase history.

[0426] "Season" refers to a specific season such as spring, summer, autumn, or winter, and is a standard for selecting clothing appropriate for each season.

[0427] "Situation" refers to a specific event, purpose, weather, or other environmental factors, and is the condition for choosing appropriate clothing.

[0428] "Store advertising" refers to promotional information for similar or related products sold in brick-and-mortar stores or online shops.

[0429] "Online shopping" is a shopping method that allows you to order and pay for products over the Internet.

[0430] "Facial expression" is an element that indicates the expression or emotion of the user's face.

[0431] "Voice" refers to the words and tone of voice spoken by the user, and is information that is part of emotion analysis.

[0432] "Emotion" refers to a user's psychological state, such as happiness, sadness, excitement, or depression.

[0433] "Identifying emotions" means identifying the user's psychological state at that time from their facial expressions and voice.

[0434] "Dynamic adjustment" means automatically changing and optimizing coordination and advertising content in real time based on the analysis results.

[0435] An embodiment of the present invention will now be described. The present invention is a system that allows a user to take a photo of clothes they like in a physical store and suggests optimal outfits based on the image and emotional information.

[0436] Hardware and Software Configuration

[0437] Device: A smartphone used by a user, equipped with a camera and internet connection, and capable of capturing the user's facial expressions and voice.

[0438] server:

[0439] Receives and analyzes image data and audio data, and manages user data.

[0440] Preprocess the images using an image processing library (OpenCV).

[0441] A machine learning model (TensorFlow) is used to extract clothing features.

[0442] An emotion analysis engine (Microsoft® Azure® Emotion API) is used to identify emotions from the user's facial expressions and voice through emotion analysis.

[0443] The system compares the outfit with a trend database to generate the optimal outfit.

[0444] Advertising data for similar products is collected and displayed to the user.

[0445] Database: MongoDB is used to store user clothing information and trend data.

[0446] DETAILED DESCRIPTION OF THE EMBODIMENTS

[0447] Program processing:

[0448] 1. The user takes a photo of the clothes using their smartphone camera, opens the app, and taps the "Upload new clothes" button. The image is sent to the server.

[0449] 2. The server temporarily stores the received image files and performs preprocessing using an image processing library (OpenCV), such as resizing, noise reduction, and color correction.

[0450] 3. Next, the server uses a machine learning model (TensorFlow) to extract features such as the color, pattern, shape, and season of the clothing, resulting in an analysis result such as "a red floral dress."

[0451] 4. The server compares the extracted features with a trend database and generates the optimal outfit based on the latest fashion information.

[0452] 5. The server sends the generated outfit suggestions to the user's device, and the user receives the suggestions via the app. The user's facial expressions and voice are also captured, and their emotions are identified by an emotion analysis engine.

[0453] 6. The server then uses the results of the emotion analysis to suggest outfits and advertisements that match the user's emotions. For example, if the user is happy, it will suggest bright-colored clothes, and if they are depressed, it will suggest relaxing clothes.

[0454] 7. It collects advertising data from stores that sell similar products and displays ads that users may be interested in. When clicked, users are redirected to the product page of the online shop.

[0455] Specific examples

[0456] For example, user A can take a photo of a red floral dress they like in a physical store and upload it to the server using a dedicated app. The server performs image and emotional analysis, and if it determines that user A is pleased, it will suggest more casual and brightly colored items (such as white sandals or a yellow bag). It will also display related products available in the store, allowing the user to try them on and purchase them immediately.

[0457] Example prompt for a generative AI model:

[0458] 1. Prompt for clothing feature extraction:

[0459] "Please extract the color, pattern, shape, and season of the clothes in this image."

[0460] 2. Prompts for sentiment analysis:

[0461] "Analyze the user's facial features from this image and identify their current emotion."

[0462] This allows users to enhance their in-store shopping experience and receive more personalized offers.

[0463] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0464] Step 1:

[0465] A user takes a photo of an outfit using their smartphone camera, opens the app, and taps the "Upload New Outfit" button. The input is the photo of the outfit taken by the user, and the output is the photo file being sent to the server. At this time, the app executes an HTTP request to send the image to the server. Specifically, the operation involves taking a photo using the smartphone's camera function and uploading the image file from within the app.

[0466] Step 2:

[0467] The server temporarily stores the image files it receives and performs preprocessing using an image processing library (OpenCV). The input is the image file of the clothes sent to the server, and the output is the preprocessed image data. This preprocessing includes resizing, noise removal, color correction, etc. Specifically, the server receives the images and performs resizing and noise removal using OpenCV.

[0468] Step 3:

[0469] The server uses a machine learning model (TensorFlow) to extract features such as the color, pattern, shape, and season of the clothing. The input is preprocessed image data, and the output is extracted clothing feature data. Specifically, analysis is performed using the TensorFlow model to extract specific features such as "a dress with a red floral pattern."

[0470] Step 4:

[0471] The server compares the extracted features with a trend database and generates optimal outfits based on the latest fashion information. The input is the extracted clothing feature data, and the output is optimal outfit suggestions. Specifically, a matching algorithm with the trend database is used to perform processing to suggest appropriate combinations.

[0472] Step 5:

[0473] The server sends the generated outfit suggestions to the user's device, and the user receives the suggestions via the app. The input is the optimal outfit suggestion, and the output is the display of outfit information on the user's device. Specifically, this process sends a notification to the app and displays the suggestion content.

[0474] Step 6:

[0475] The system captures the user's facial expressions and voice and identifies their emotions using an emotion analysis engine (Microsoft Azure Emotion API). The input is the user's facial and voice data, and the output is analyzed emotional information. Specifically, the system captures facial expressions and voice using the smartphone's camera and microphone, and identifies emotions through the Azure Emotion API.

[0476] Step 7:

[0477] The server then uses the results of the emotion analysis to suggest outfits and advertisements that match the user's emotions. The input is the analyzed emotion information and clothing feature data, and the output is emotion-based outfit and advertisement suggestions. Specifically, if the user is happy, it suggests brightly colored items, and if they are depressed, it suggests relaxing clothes.

[0478] Step 8:

[0479] The server collects advertising data for similar products and displays it to the user. The input is advertising data related to clothing feature data, and the output is the display of advertisements on the user's device. Specifically, the process involves collecting related products through an advertising network and displaying advertisements that the user is likely to be interested in on the app.

[0480] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0481] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0482] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0483] [Second embodiment]

[0484] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0485] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0486] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0487] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0488] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0489] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0490] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0491] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0492] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0493] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0494] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0495] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0496] The system of the present invention allows users to take photos of their clothes, find the best outfits based on the photos, and then purchase similar clothes online. The specific configuration and operation of this system will be described below.

[0497] System configuration

[0498] 1. Device:

[0499] This refers to devices such as smartphones and tablets used by users.

[0500] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[0501] 2. Server:

[0502] This is a remote server that receives and analyzes image data and manages user data.

[0503] An image processing library is used to preprocess and analyze images, and a machine learning model is used to extract clothing features.

[0504] It refers to a trend database and suggests optimal outfits to users.

[0505] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[0506] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[0507] Program processing

[0508] 1. Take and upload an image of your outfit

[0509] The user uses their smartphone to take a photo of newly purchased clothes or clothes they are wearing.

[0510] The device acquires the captured image and sends it to the server using a dedicated app.

[0511] 2. Image Receipt and Analysis

[0512] The server stores the received images and performs preprocessing on them using an image processing library (e.g., OpenCV), such as noise reduction, resizing, and color correction.

[0513] Then, a machine learning model (e.g., TensorFlow) is applied to extract clothing features from the image, including color, pattern, shape, and season.

[0514] 3. Comparison with trend data

[0515] The server compares the extracted clothing characteristics with a trend database, which contains the latest fashion information and influencer posting data.

[0516] Based on the comparison results, the optimal clothing combination is generated and recommended to the user.

[0517] 4. Registering the user in the clothing database

[0518] Users can register information about their clothing using a dedicated app. Information about each piece of clothing can be automatically recognized (image analysis) or manually entered (color, pattern, shape, etc.).

[0519] The server stores the registered information in a database and analyzes the combination of newly registered clothes with existing clothes.

[0520] Based on the user's requests, the system suggests the best outfits for each season and situation.

[0521] 5. Viewing Ads and Purchasing Online

[0522] The server obtains advertising data for stores that sell similar products based on the characteristics of the user's clothes.

[0523] The server sends the acquired advertising data to the user's device and displays it through a dedicated app.

[0524] Users can click on an ad that interests them and access the product page of the online shop to which the ad links, where they can check the product details and make a purchase.

[0525] Specific examples

[0526] For example, consider the case where user A purchases a new red floral dress.

[0527] 1. User A takes a photo of a red floral dress with her smartphone and uploads it to the server using a dedicated app.

[0528] 2. The server receives the image, preprocesses it using an image processing library, and uses a machine learning model to extract the color (red), pattern (floral), and shape (dress) of the dress.

[0529] 3. The server compares the extracted features with a trend database and suggests outfits that combine them with, for example, white sandals and a denim jacket.

[0530] 4. User A has registered other clothes he owns (for example, a white shirt or a black skirt) in the app, and suggestions for combinations with these items are also displayed.

[0531] 5. The server displays an advertisement for a store that sells products similar to the red floral dress, and User A can purchase the product from the online shop.

[0532] In this way, the system of the present invention not only helps users efficiently manage their clothes and find the best outfits, but also allows them to easily purchase related new clothes online.

[0533] The processing flow will be explained below.

[0534] Step 1:

[0535] The user takes a photo of the clothes using the camera on their smartphone or tablet. After taking the photo, they open the dedicated app and tap the "Upload New Clothes" button. This action causes the device to import the image into the app and prepare it for transmission to the server.

[0536] Step 2:

[0537] The device retrieves the captured image and makes an HTTP request to an API endpoint to upload the image file to the server, including the image data and metadata such as the user ID.

[0538] Step 3:

[0539] The server temporarily stores the received image files. Next, it performs preprocessing on the images using an image processing library (e.g., OpenCV). Specifically, it performs image resizing, noise removal, color correction, etc. This generates an image suitable for analysis.

[0540] Step 4:

[0541] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image. This model is pre-trained and can identify clothing features such as color, pattern, shape, and even seasonal characteristics. The analysis results are expressed as, for example, "a dress with a red floral pattern."

[0542] Step 5:

[0543] The server compares the extracted features with a trend database, which stores data on the latest fashion magazines, influencer posts, and popular items from online shops. Using a comparison algorithm, the server generates optimal outfits that match current trends.

[0544] Step 6:

[0545] The server sends the generated outfit suggestions to the device. When the user opens the app, the outfit suggestions are displayed. For example, specific suggestions such as "Pair a red floral dress with white sandals" are made.

[0546] Step 7:

[0547] Users can register information about the clothes they own using a dedicated app. Both automatic recognition (image analysis) and manual input methods are available, making it easy for users to register information about their clothes.

[0548] Step 8:

[0549] The server stores the clothing information sent by the user in a database and analyzes combinations based on existing and newly added clothing information, generating coordination suggestions according to the season and situation.

[0550] Step 9:

[0551] The server retrieves advertising data from stores that sell similar products based on the characteristics of the user's clothing, selects relevant advertisements through advertising networks and partnerships, and prepares to display them to the user.

[0552] Step 10:

[0553] The device displays the advertising data received from the server to the user. When the user clicks on an advertisement that interests them, they are redirected to the product page of the online shop linked to the advertisement. Here, the user can check the product details and purchase it online.

[0554] Through the above processing steps, the user can manage his / her own clothes, receive suggestions for coordinating outfits, and easily purchase related products online.

[0555] Example 1

[0556] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0557] Previously, systems that allowed users to manage their clothing and find optimal outfits were either complex or had limited functionality. Furthermore, there were no systems that offered trend-based clothing combination suggestions or a unified online purchasing solution. This made it difficult for users to efficiently manage their clothing, find optimal outfits, and easily purchase related products online.

[0558] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0559] In this invention, the server includes a means for a user to take an image and upload it to the server, a means for the server to analyze the image and extract clothing features using an image processing library and a machine learning model, and a means for comparing the extracted clothing features with trend data to generate optimal clothing combinations, thereby enabling users to efficiently manage their own clothing, find optimal combinations, and easily purchase related products online.

[0560] "User" refers to an individual who uses the system to manage their own clothing and receive suggested outfits.

[0561] "Terminal" refers to a portable computing device used by a User, such as a smartphone or tablet.

[0562] "Server" refers to a computer system that remotely receives and analyzes image data and manages user data.

[0563] "Image processing library" refers to software tools and libraries for preprocessing and analyzing images, such as OpenCV.

[0564] A "machine learning model" refers to an algorithm that learns patterns from data and makes predictions or classifications. Specific examples include TensorFlow.

[0565] "Clothing features" refer to attributes of clothing extracted from an image, such as color, pattern, shape, and season.

[0566] "Trend data" refers to a database containing the latest fashion information and influencer posting data.

[0567] A "trend database" refers to a database that accumulates the latest fashion information and trends.

[0568] "Optimal clothing combination" refers to the clothing coordination that is considered optimal for the user, calculated based on the extracted clothing characteristics and the user's preferences and trend data.

[0569] "User's clothing information" refers to data about the clothes owned by the user (color, pattern, shape, etc.).

[0570] "Database" refers to a data storage system for storing and managing information about users' clothing and trend data.

[0571] "Advertising data" refers to advertising information provided by stores and online shops that sell similar products.

[0572] "Online purchase" refers to a user completing a purchase procedure for a product via the Internet.

[0573] MODE FOR CARRYING OUT THE INVENTION

[0574] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the photos, and then help them purchase similar clothes online. The specific configuration and operation of this system are described below.

[0575] System configuration

[0576] 1. Device:

[0577] This refers to devices such as smartphones and tablets used by users.

[0578] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[0579] 2. Server:

[0580] This is a remote server that receives and analyzes image data and manages user data.

[0581] An image processing library (e.g., OpenCV) is used to preprocess and analyze the images, and a machine learning model (e.g., TensorFlow) is used to extract clothing features.

[0582] It refers to a trend database and suggests optimal outfits to users.

[0583] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[0584] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[0585] Overview of program processing

[0586] 1. Take and upload an image of your outfit

[0587] Users take photos of newly purchased or currently wearing clothes with their smartphones, and the device sends the images to a server using a dedicated app.

[0588] 2. Image Receipt and Analysis

[0589] The server stores the received images and performs preprocessing using an image processing library (e.g., OpenCV) to remove noise, resize, and correct color. It then applies a machine learning model (e.g., TensorFlow) to extract clothing features (e.g., color, pattern, shape, season) from the images.

[0590] 3. Comparison with trend data

[0591] The server compares the extracted clothing characteristics with a trend database, which contains the latest fashion information and influencer postings. Based on the comparison results, the server generates optimal clothing combinations and suggests them to the user.

[0592] 4. Registering the user in the clothing database

[0593] Users use a dedicated app to register information about the clothes they own. Information about each piece of clothing can be automatically recognized through image analysis or can be entered manually. The server stores this information in a database and analyzes how newly registered clothes can be combined with existing clothes. The system then provides optimal outfits according to the season and situation.

[0594] 5. Viewing Ads and Purchasing Online

[0595] The server retrieves advertising data for stores that carry similar products based on the characteristics of the user's clothing. The acquired advertising data is sent to the user's device and displayed through a dedicated app. The user can click on an advertisement that interests them, access the product page in the online shop, and purchase the item.

[0596] Specific examples

[0597] For example, a case will be described in which user A purchases a new red floral dress.

[0598] 1. User A takes a photo of a red floral dress with her smartphone and uploads it to the server using a dedicated app.

[0599] 2. The server receives the image, preprocesses it using an image processing library (e.g., OpenCV), and then uses a machine learning model (e.g., TensorFlow) to extract the color (red), pattern (floral), and shape (dress) of the dress.

[0600] 3. The server compares the extracted features with trend data and suggests outfits that combine them with, for example, white sandals and a denim jacket.

[0601] 4. User A has registered other clothes he owns (e.g., white shirts and black skirts) in the app, and suggestions for combinations with these items are also displayed.

[0602] 5. The server displays an advertisement for a store that sells products similar to the red floral dress, and User A can purchase the product from the online shop.

[0603] Prompt Sentence Examples

[0604] "Generate a sentence that explains the system that receives an image of an outfit from the user, analyzes the image, suggests the best outfit, and advertises similar products."

[0605] By configuring it in this way, the system not only helps users efficiently manage their clothes and find the best outfits, but also allows them to easily purchase related new clothes online.

[0606] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0607] Step 1:

[0608] The user takes a photo of the clothes with their smartphone and uploads the image through a dedicated app. The input is the image of the clothes taken by the user, and the output is image data sent to the server. The device uses its camera function to acquire the image and sends it to the server via the Internet.

[0609] Step 2:

[0610] The server saves the image data received from the terminal. The input is the image data sent from the terminal, and the output is the saved image data. Specifically, the server stores the received image data in storage and prepares it for subsequent analysis processing.

[0611] Step 3:

[0612] The server preprocesses the received images using an image processing library (e.g., OpenCV). The input is the stored image data, and the output is the preprocessed image data. Specifically, it performs processes such as noise removal, resizing (e.g., to 256x256 pixels), and color correction.

[0613] Step 4:

[0614] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from preprocessed images. The input is the preprocessed image data, and the output is extracted clothing feature data (e.g., color, pattern, shape, season). Specifically, the image is input into the machine learning model, and clothing features are obtained from the output.

[0615] Step 5:

[0616] The server compares the extracted clothing feature data with a trend database. The input is the clothing feature data and the trend database, and the output is optimal clothing combination suggestion data. The server refers to the trend database and calculates and generates the optimal outfit for the user.

[0617] Step 6:

[0618] The server generates optimal outfit combinations and sends that information to the user's device. The input is the optimal outfit combination suggestion data, and the output is the suggestion information sent to the user's device. Specifically, when the user opens the app, the suggested outfits are displayed.

[0619] Step 7:

[0620] The user registers information about their clothing using a dedicated app. The input is the clothing information entered or photographed by the user, and the output is the registration information sent to the server. The device acquires this information and sends it to the server.

[0621] Step 8:

[0622] The server stores the user's clothing information in a database. The input is the registered clothing information, and the output is the stored database. The server compares the user's new clothing data with existing data and analyzes the combinations.

[0623] Step 9:

[0624] The server retrieves advertising data for stores that sell similar products based on the characteristics of the user's clothing. The input is the characteristic data of the user's clothing, and the output is the retrieved advertising data. The server then refers to the advertising service to collect advertising information for related products.

[0625] Step 10:

[0626] The server sends the acquired advertising data to the user's device. The input is the collected advertising data, and the output is advertising information sent to the user's device. Users can view these advertisements through a dedicated app and purchase products they are interested in online.

[0627] (Application example 1)

[0628] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0629] Conventional clothing coordination suggestion systems have difficulty easily linking information about the clothes a user owns with the latest trends, and have also faced challenges in efficiently finding and purchasing similar products. Furthermore, they lacked personalized suggestions tailored to the user's preferences and requirements, requiring users to expend a great deal of time and effort. Our goal is to provide a system that solves these issues, allowing users to more easily and instantly find appropriate coordinations and purchase similar products.

[0630] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0631] In this invention, the server includes means for taking images of clothes and uploading them to a remote server, means for the remote server to analyze the images and extract clothing characteristics, means for comparing the clothing characteristics with trend data and presenting optimal clothing combinations, means for registering information about clothes owned by the user and proposing combinations according to seasons and events, means for displaying advertisements for stores that carry similar clothes and enabling online purchases, means for proposing new clothes using a generative AI model based on optimal coordination, and means for proposing clothes based on specified requirements by inputting a prompt text. This allows users to not only easily analyze and manage the characteristics of their own clothes, but also enables efficient coordination suggestions linked to the latest trend information and allows them to quickly find and purchase similar products.

[0632] "Means for taking images of clothes and uploading them to a remote server" refers to a function that allows a user to take an image of clothes using a device such as a smartphone or tablet, and then send the image data to a remote server via the Internet.

[0633] "Means for the remote server to analyze the image and extract characteristics of the clothing" refers to a function in which the remote server analyzes the image data received and extracts characteristics such as the color, pattern, shape, and seasonality of the clothing.

[0634] The "means for comparing trend data with clothing characteristics and presenting optimal clothing combinations" is a function that compares extracted clothing characteristics with the latest trend data and presents optimal outfits to the user.

[0635] "A means for users to register information about the clothes they own and suggest combinations appropriate for the season or event" is a function that allows users to register information about the clothes they own in the system and then suggests outfits appropriate for the season or event based on that information.

[0636] "Means for displaying advertisements for stores that sell similar clothing and enabling online purchases" is a function that displays advertisements for stores that sell similar products based on the characteristics of the clothing registered by the user, allowing the user to purchase directly from those stores.

[0637] "Means for proposing new clothing using a generative AI model based on optimal coordination" is a function that uses a generative AI model to suggest new clothing based on extracted clothing characteristics and optimal coordination information.

[0638] "A means for suggesting clothing based on specified requirements by inputting a prompt sentence" is a function that allows the user to input specific requirements or preferences as a prompt sentence and suggests appropriate clothing based on that information.

[0639] The system of the present invention allows users to take photos of their own clothes using a smartphone or tablet, find optimal outfits based on the photos, and then help them purchase similar clothes online. The specific configuration and operation of this system are described in detail below.

[0640] System configuration

[0641] 1. Device (smartphone, tablet, etc.):

[0642] It has a camera function for taking images and a function for connecting to the Internet and communicating with a remote server.

[0643] Using a dedicated application, users can take photos of their clothes and upload them to a remote server.

[0644] 2. Remote Server:

[0645] Receives and analyzes image data, and manages user data.

[0646] An image processing library (e.g., OpenCV) is used to preprocess and analyze the images, and a machine learning model (e.g., TensorFlow) is used to extract clothing features.

[0647] The system refers to a trend database and compares the extracted clothing characteristics to suggest the optimal outfit.

[0648] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[0649] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[0650] Based on the optimal coordination, new clothing suggestions are made using a generative AI model.

[0651] By entering a prompt, clothing suggestions will be made based on the specified requirements.

[0652] How it works

[0653] 1. Take and upload your outfit image:

[0654] The user takes a photo of the new or currently worn clothes using a smartphone, and then uploads the image to a remote server using a dedicated application.

[0655] 2. Image Receipt and Analysis:

[0656] The remote server receives the uploaded images and performs preprocessing (noise removal, resizing, color correction, etc.) using an image processing library (OpenCV), then extracts clothing features (color, pattern, shape, season, etc.) using a machine learning model (TensorFlow).

[0657] 3. Comparison with trend data:

[0658] The extracted clothing characteristics are compared with a trend database, the latest fashion information, and influencer posts. Based on the comparison results, optimal clothing combinations are generated and recommended to the user.

[0659] 4. Registering the user in the clothing database:

[0660] Users register information about their clothing using a dedicated app. The registered information is stored in a database on a remote server. The server then analyzes how the newly registered clothing can be combined with existing clothing to suggest optimal outfits for each season and situation.

[0661] 5. Viewing Ads and Online Purchases:

[0662] The remote server retrieves advertising data for stores that sell similar products based on the extracted clothing characteristics. The acquired advertising data is sent to the user's device and displayed through a dedicated app. Users can click on an advertisement that interests them and purchase the product from the online shop.

[0663] 6. New clothing suggestions using generative AI models:

[0664] Based on the extracted clothing features and optimal coordination information, a generative AI model is used to suggest new clothing to the user. For example, if a user takes a photo of a red floral dress, the system will suggest new clothing designs based on those features.

[0665] 7. Specifying requirements by prompt sentence:

[0666] Users can specify their specific requirements and preferences by entering prompts in a dedicated application, and the server will then suggest clothing that matches the requirements.

[0667] Specific examples

[0668] Consider the case where User A purchases a new red floral dress, takes a photo of it with her smartphone, and uploads it to the server using a dedicated app. The server analyzes the image and extracts the color (red), pattern (floral), and shape (dress). It then compares the extracted features with a trend database and suggests combinations, such as white sandals and a denim jacket. Furthermore, if User A enters the prompt "casual outdoor style," the server uses the generative AI model to suggest new clothing based on those requirements.

[0669] Prompt Sentence Examples

[0670] text

[0671] Analyze an image of a red floral dress and suggest the best outfit for it. Also, show similar items from an online shop.

[0672] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0673] Step 1:

[0674] The device takes pictures of the clothes the user is wearing or newly purchased clothes. When the user clicks the capture button in the dedicated app, the device's camera function is activated and the image is captured. The input is the image data captured by the camera, and the output is an image file stored on the device.

[0675] Step 2:

[0676] The device uploads the captured images to a remote server via a dedicated app. When the user clicks the upload button, the image data is sent to the remote server via the Internet. The input is the image file in the device, and the output is the image data sent to the remote server.

[0677] Step 3:

[0678] The remote server stores the received image data and performs preprocessing on the image using an image processing library (e.g., OpenCV). Specifically, it performs processes such as noise reduction, resizing, and color correction. The input is the image data uploaded to the server, and the output is the image data after preprocessing.

[0679] Step 4:

[0680] The remote server inputs the preprocessed image data into a machine learning model (e.g., TensorFlow) to extract clothing features. Features include color, pattern, shape, season, etc. The input is the preprocessed image data, and the output is the extracted clothing feature data.

[0681] Step 5:

[0682] The remote server compares the extracted clothing features with a trend database, referring to the latest fashion information and influencer posts to generate optimal clothing combinations. The input is clothing feature data and trend data, and the output is optimal clothing combination data.

[0683] Step 6:

[0684] The remote server retrieves information about clothing registered by the user using a dedicated app from a database and analyzes how the newly registered clothing can be combined with existing clothing. As a specific example, it suggests optimal outfits according to the season or event. The input is the user's clothing data, and the output is combination data according to the season or event.

[0685] Step 7:

[0686] The remote server retrieves advertising data for stores that sell similar products based on the extracted clothing features. The advertising data is provided by online shops and displayed to users. The input is clothing feature data and store advertising data, and the output is advertising data displayed on the user's device.

[0687] Step 8:

[0688] The remote server uses a generative AI model based on the optimal outfit to suggest new outfits. The generative AI model uses deep learning, for example. The input is optimal outfit data, and the output is new outfit suggestion data.

[0689] Step 9:

[0690] The user inputs a prompt on a dedicated application and sends specific requirements and preferences to the server, which then suggests clothes that fit the specified requirements based on the prompt. The input is the prompt from the user, and the output is the suggested clothing data.

[0691] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0692] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the images, and assist users in purchasing similar clothes online. By combining it with an emotion engine, the system suggests the best clothing combinations and advertisements according to the user's emotions. The specific configuration and operation of this system will be described below.

[0693] System configuration

[0694] 1. Device:

[0695] A device such as a smartphone or tablet used by a user.

[0696] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[0697] It has the ability to capture the user's facial expressions and voice.

[0698] 2. Server:

[0699] This is a remote server that receives and analyzes image data and manages user data.

[0700] An image processing library is used to preprocess and analyze images, and a machine learning model is used to extract clothing features.

[0701] It refers to a trend database and suggests optimal outfits to users.

[0702] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[0703] The advertising data of stores that sell similar products is acquired and displayed to the user.

[0704] It uses an emotion engine to analyze user emotions and tailor suggestions and advertisements.

[0705] Program processing

[0706] When a user takes a photo of an outfit using the camera on their smartphone or tablet, opens the app, and taps the "Upload New Outfit" button, the image is sent to the server. The device retrieves the image and executes an HTTP request.

[0707] The server temporarily stores the received image files and performs preprocessing using an image processing library (e.g., OpenCV). An appropriate analysis state is generated by resizing, noise removal, color correction, etc. Next, a machine learning model (e.g., TensorFlow) is used to extract features such as the color, pattern, shape, and season of the clothing. This results in an analysis result such as "a dress with a red floral pattern."

[0708] The server compares the extracted features with a trend database, which includes data from magazines and social media, to generate optimal outfits based on the latest fashion information and influencer posts.

[0709] The server then sends the generated outfit suggestions to the user's device, where the user can receive the suggestions via the app. For example, a specific suggestion such as "pair a red floral dress with white sandals" is made.

[0710] Users can use a dedicated app to register information about the clothes they own. Registration methods include automatic recognition (image analysis) and manual input (color, pattern, shape, etc.), making it easy to add clothing information to the database. The server saves the registered clothing information in the database and analyzes how newly added clothes can be combined with existing clothes to make optimal suggestions.

[0711] The server collects advertising data from stores that sell similar products based on the user's clothing characteristics. It then selects relevant ads through advertising networks and partnerships and displays them to the user. When the user clicks on an ad that interests them, they are redirected to the product page in the online shop, where they can view product details and make a purchase.

[0712] Introducing the Emotion Engine

[0713] The system incorporates an emotion engine, and the following specific processes are added:

[0714] The emotion engine analyzes the user's facial expressions and voice to identify their current emotions. For example, it uses a camera to collect the user's facial expression data and uses voice recognition technology to analyze the user's speech.

[0715] The server uses the analysis results obtained from the emotion engine to suggest optimal outfits and advertisements according to the user's emotions. For example, if the user is happy, it will suggest bright-colored clothes, and if they are depressed, it will suggest comfortable, relaxing clothes. It is also possible to dynamically change relevant product advertisements according to the user's emotions.

[0716] Specific examples

[0717] For example, consider the case where User B purchases a new blue denim dress, takes a photo of it, and uploads it to the server.

[0718] 1. User B takes a photo of a denim dress with their smartphone and uploads it to the server using a dedicated app.

[0719] 2. The server receives the image, preprocesses it using an image processing library, and then uses a machine learning model to extract the color (blue), pattern (denim), and shape (dress) of the dress.

[0720] 3. The server compares the extracted features with a trend database and suggests outfits, such as "combining white sneakers with a red bag."

[0721] 4. If user B is found to be happy through the emotion engine, suggest more casual and brightly colored items.

[0722] 5. User B has already registered a white shirt and a black skirt in the app, and suggestions for combinations with them are also displayed.

[0723] 6. The server displays an advertisement for a store that sells a product similar to the blue denim dress, and User B purchases the product from an online shop.

[0724] In this way, the system of the present invention can dynamically respond to the user's emotions and can also provide personalized coordination and advertising suggestions.

[0725] The processing flow will be explained below.

[0726] Step 1:

[0727] The user takes a photo of the clothes using the camera on their smartphone or tablet. After taking the photo, they open the dedicated app and tap the "Upload New Clothes" button. This action causes the device to import the image into the app and prepare it for transmission to the server.

[0728] Step 2:

[0729] The device retrieves the captured image and makes an HTTP request to an API endpoint to upload the image file to the server, including the image data and metadata such as the user ID.

[0730] Step 3:

[0731] The server temporarily stores the received image files. Next, it performs preprocessing on the images using an image processing library (e.g., OpenCV). Specifically, it performs image resizing, noise removal, color correction, etc. This generates an image suitable for analysis.

[0732] Step 4:

[0733] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image. This model is pre-trained and can identify clothing features such as color, pattern, shape, and season. The analysis results are expressed as, for example, "a red floral dress."

[0734] Step 5:

[0735] The server compares the extracted features with a trend database, which stores data on the latest fashion magazines, influencer posts, and popular items from online shops. Using a comparison algorithm, the server generates optimal outfits that match current trends.

[0736] Step 6:

[0737] The server sends the generated outfit suggestions to the device. When the user opens the app, the outfit suggestions are displayed. For example, specific suggestions such as "Pair a red floral dress with white sandals" are made.

[0738] Step 7:

[0739] Users can register information about the clothes they own using a dedicated app. Both automatic recognition (image analysis) and manual input (color, pattern, shape, etc.) are available, making it easy for users to register information about their clothes.

[0740] Step 8:

[0741] The server stores the clothing information sent by the user in a database and analyzes combinations based on existing clothing information and newly added information, thereby generating coordination suggestions according to the season and situation.

[0742] Step 9:

[0743] The device captures the user's facial expressions and voice and sends them to an emotion engine, which uses a model trained on a dataset for emotion analysis to analyze the user's emotions in real time.

[0744] Step 10:

[0745] The emotion engine analyzes the user's emotions and sends the results to the server. For example, it responds when the user is judged to be happy or depressed.

[0746] Step 11:

[0747] The server uses the emotional data sent from the emotion engine to adjust the recommended outfits and advertisements. For example, if the user is happy, it will suggest brightly colored clothes and casual items, and if they are depressed, it will suggest relaxing clothes. It also selects the most appropriate advertisement based on the user's emotions and sends it to the user's device.

[0748] Step 12:

[0749] The device displays coordination suggestions and advertisements to the user based on the emotions received from the server. Because the suggestions displayed are suited to the user's emotions, the user can receive more personalized services.

[0750] Through the above processing steps, the user can receive suggestions for optimal clothing management and coordination based on their emotions, and can also easily purchase related products online.

[0751] Example 2

[0752] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0753] Conventional coordination support systems were unable to provide suggestions based on the user's emotions and mood regarding clothing, and were unable to sufficiently increase user satisfaction. Furthermore, when purchasing similar products online, advertisements displayed did not dynamically change based on the user's emotions and preferences, preventing effective marketing. Furthermore, it was difficult to provide suggestions for combining outfits with the user's existing clothes to suit the season or specific situations.

[0754] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for taking an image of clothes and uploading it to the server, a means for the server to analyze the image and extract characteristics of the clothes, a means for comparing the characteristics of the clothes with trend data and recommending optimal clothing combinations, a means for registering information on clothes owned by the user and proposing combinations according to the season and situation, a means for displaying advertisements for stores that carry similar clothes and enabling online purchases, and a means for analyzing the user's emotions using an emotion analysis engine and adjusting optimal clothing combinations and advertisements based on the emotions. This makes it possible to suggest coordinations and display advertisements that match the user's emotions and mood.

[0755] "Clothing image" refers to photographic data of clothing that the user will wear.

[0756] "Means for uploading" refers to the ability for a user to send images taken by the user to a remote server via the Internet.

[0757] "Means for analyzing images and extracting clothing features" refers to the process of identifying and extracting attributes such as clothing color, pattern, and shape from the received image data.

[0758] "Trend data" refers to data that reflects the latest fashion information and market trends.

[0759] "Means of recommending optimal clothing combinations" refers to a function that suggests outfits that suit the user based on trend data and clothing characteristics.

[0760] "Means for registering information about clothes owned by the user" refers to a function that allows a user to input or read data about clothes they own into the system.

[0761] "Means of suggesting combinations according to the season and situation" refers to a function that suggests clothing combinations that suit the season, specific events, or occasions.

[0762] "Means of displaying advertisements for stores that sell similar clothing and enabling online purchases" refers to a function that displays advertisements for stores that sell products similar to the clothing photographed by the user and allows the user to purchase those products online.

[0763] An "emotion analysis engine" refers to a system that analyzes a user's facial expressions and voice to identify their current emotional state.

[0764] "Means for adjusting optimal clothing combinations and advertisements based on emotions" refers to a function that dynamically changes coordination suggestions and advertisement content to match the identified user's emotions.

[0765] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the photos, and help them purchase similar clothes online. Furthermore, by combining it with an emotion engine, the system suggests the best clothing combinations and advertisements based on the user's emotions.

[0766] System configuration

[0767] 1. Device:

[0768] A device such as a smartphone or tablet used by a user.

[0769] It has a camera function for taking pictures and the ability to connect to the Internet and communicate with a server.

[0770] It has the ability to capture the user's facial expressions and voice.

[0771] 2. Server:

[0772] This is a remote server that receives and analyzes image data and manages user data.

[0773] Specifically, the image is preprocessed using an image processing library (e.g., OpenCV), and clothing features are extracted using a machine learning model (e.g., TensorFlow).

[0774] The system references a trend database and generates the optimal coordination.

[0775] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[0776] Advertising data is collected and displayed to the user.

[0777] It uses an emotion engine to analyze user emotions and tailor suggestions and advertisements.

[0778] Example of operation

[0779] 1. The user takes a photo of their clothes using the device's camera.

[0780] Example: A user takes a picture of a new blue denim dress.

[0781] 2. Upload the images taken by the device to the server using a dedicated app.

[0782] 3. The server receives the images and performs preprocessing using an image processing library (e.g., OpenCV).

[0783] Pre-processing includes resizing, noise reduction, color correction, etc.

[0784] 4. The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image.

[0785] Example: Extract features such as color (blue), pattern (denim), and shape (dress) from a blue denim dress.

[0786] 5. The server compares the extracted features with a trend database and suggests optimal outfits.

[0787] Example: A specific coordination suggestion such as "pair it with white sneakers and a red bag."

[0788] 6. The server sends the generated coordination proposal to the user's device.

[0789] 7. The device displays the suggestions to the user through a dedicated app.

[0790] For example, you might see suggestions like "Pair a blue denim dress with white sneakers."

[0791] 8. The user uses a dedicated app to register information about the clothes they own.

[0792] Example: Register information such as a white shirt or black skirt you own.

[0793] 9. The server analyzes the combination of new clothes with existing clothes based on the registered clothing information and makes optimal suggestions.

[0794] For example: a blue denim dress paired with a white shirt.

[0795] 10. The server collects advertising data from stores that sell similar products and displays it to the user.

[0796] 11. The device displays advertisements, and when the user clicks on an advertisement that interests them, they are redirected to the product page of the online shop.

[0797] Example: You see products similar to a blue denim dress and can buy them online.

[0798] Prompt Sentence Examples

[0799] "Suggest outfits that go well with the blue denim dress the user recently purchased. Also, if the user is happy with the outfit, suggest items in brighter colors."

[0800] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0801] Step 1:

[0802] The user takes a photo of the clothes using the device's camera.

[0803] Input: The user activates the camera and takes a picture of the clothes.

[0804] Output: The captured image data is saved on the device.

[0805] What happens: A user takes a picture of a new blue denim dress with their smartphone camera.

[0806] Step 2:

[0807] The device uploads the images taken to the server using a dedicated app.

[0808] Input: Image data stored on the device.

[0809] Output: An HTTP POST request containing the image data is sent to the server.

[0810] Specific operation: The user taps the "Upload new clothes" button, and the device sends image data to the server.

[0811] Step 3:

[0812] The server temporarily stores the received image files and performs preprocessing using an image processing library (e.g., OpenCV).

[0813] Input: Image data received by the server.

[0814] Output: Preprocessed image data.

[0815] Data processing or data calculation: image resizing, noise reduction, color correction, etc.

[0816] What happens: The server resizes the image, removes noise, and performs color correction.

[0817] Step 4:

[0818] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image.

[0819] Input: Preprocessed image data.

[0820] Output: Extracted clothing feature data (e.g. color, pattern, shape).

[0821] Data processing or data calculation: Identifying color, pattern, and shape from image data and generating feature data.

[0822] Specific operation: The server extracts the color (blue), pattern (denim), and shape (dress) of the blue denim dress.

[0823] Step 5:

[0824] The server compares the extracted features with a trend database and generates the optimal outfit.

[0825] Input: Extracted clothing feature data.

[0826] Output: Optimal outfit suggestion data.

[0827] Data processing or data calculation: Refer to the feature data and trend database to determine the optimal coordination.

[0828] Specific behavior: The server generates suggestions such as "pair with white sneakers and a red bag."

[0829] Step 6:

[0830] The server transmits the generated coordination proposal to the user's terminal.

[0831] Input: Coordination suggestion data.

[0832] Output: The proposal data sent to the user's device.

[0833] Specific operation: The server sends the generated coordination proposal to the user's device.

[0834] Step 7:

[0835] The coordinated suggestions received by the terminal are displayed to the user via a dedicated app.

[0836] Input: The coordinate suggestion data passed in.

[0837] Output: The suggestions displayed in the app.

[0838] Specific behavior: The device displays a suggestion on the app screen: "Pair a blue denim dress with white sneakers."

[0839] Step 8:

[0840] Users register information about the clothes they own using a dedicated app.

[0841] Input: Clothing information entered or uploaded by the user.

[0842] Output: Clothing information from the registered database.

[0843] Specific operation: The user enters information about a white shirt and a black skirt they own into the app and registers them.

[0844] Step 9:

[0845] Based on the registered clothing information, the server analyzes the combination of new clothing with existing clothing and makes optimal suggestions.

[0846] Input: Registered clothing information.

[0847] Output: Combination suggestion data of newly added clothes and existing clothes.

[0848] Data processing or data calculation: Combining and analyzing existing clothing information with new clothing information.

[0849] What it does: The server suggests a combination of a blue denim dress and a white shirt.

[0850] Step 10:

[0851] The server collects advertising data from stores that sell similar products and transmits it to the user's terminal.

[0852] Input: Registered clothing feature data.

[0853] Output: Collected advertising data.

[0854] What it does: The server collects relevant ads through ad networks.

[0855] Step 11:

[0856] The advertisements received by the device are displayed to the user via a dedicated app.

[0857] Input: Ad data sent by the server.

[0858] Output: The ad displayed within the app.

[0859] What happens: The device displays an ad for a product similar to the blue denim dress on the app screen.

[0860] Step 12:

[0861] The emotion engine analyzes the user's facial expressions and voice to identify their current emotion.

[0862] Input: Facial expression and audio data captured by the camera.

[0863] Output: Parsed emotion data.

[0864] Data processing or data calculation: Performing facial expression recognition and voice analysis to identify emotions.

[0865] Specific operation: Collects the user's facial expression data using a camera and analyzes the user's speech using voice recognition technology.

[0866] Step 13:

[0867] Based on the analysis results obtained from the emotion engine, the server adjusts the optimal coordination and advertisements according to the user's emotions.

[0868] Input: Parsed emotion data.

[0869] Output: Reconciled coordinate and advertising data.

[0870] Data processing or data calculation: Dynamically changing outfits or advertising content based on emotional data.

[0871] What it does: If the user is happy, suggest brighter colored items and show more relevant ads.

[0872] (Application example 2)

[0873] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0874] Until now, when users purchased clothes in a physical store, there was no way to instantly get suggestions on how to coordinate them with the clothes they already own or the best combinations that match their current mood. There is also a demand for personalized suggestions based on the user's emotions to provide a more comfortable shopping experience. This will increase product purchases in physical stores and improve customer satisfaction.

[0875] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for taking an image of clothes and uploading it to the server, means for the server to analyze the image and extract characteristics of the clothes, means for comparing the characteristics of the clothes with trend data and recommending optimal clothing combinations, means for registering information about clothes owned by the user and proposing combinations according to the season and situation, means for displaying advertisements for stores that carry similar clothes and enabling online purchases, and means for analyzing the user's facial expressions and voice to identify emotions and dynamically adjusting coordinations and advertisements based on the emotions. This allows users to take photos of clothes they like in a physical store, receive optimal coordination suggestions in real time, and enjoy a personalized shopping experience according to their emotions.

[0876] "Clothing images" are photos of clothing taken by the user using the camera function.

[0877] "Server" means a remotely located computer system that receives and processes data sent by users and returns analytical results.

[0878] "Clothing characteristics" are visual and physical attributes of clothing such as color, pattern, shape, and season.

[0879] "Trend Data" is a database containing information about current trends and popular styles.

[0880] The "optimal clothing combination" is the most appropriate fashion coordination for the user, generated based on the analyzed clothing characteristics and trend data.

[0881] "User information" refers to personal information such as data on the clothes owned by the user, their preferences, and purchase history.

[0882] "Season" refers to a specific season such as spring, summer, autumn, or winter, and is a standard for selecting clothing appropriate for each season.

[0883] "Situation" refers to a specific event, purpose, weather, or other environmental factors, and is the condition for choosing appropriate clothing.

[0884] "Store advertising" refers to promotional information for similar or related products sold in brick-and-mortar stores or online shops.

[0885] "Online shopping" is a shopping method that allows you to order and pay for products over the Internet.

[0886] "Facial expression" is an element that indicates the expression or emotion of the user's face.

[0887] "Voice" refers to the words and tone of voice spoken by the user, and is information that is part of emotion analysis.

[0888] "Emotion" refers to a user's psychological state, such as happiness, sadness, excitement, or depression.

[0889] "Identifying emotions" means identifying the user's psychological state at that time from their facial expressions and voice.

[0890] "Dynamic adjustment" means automatically changing and optimizing coordination and advertising content in real time based on the analysis results.

[0891] An embodiment of the present invention will now be described. The present invention is a system that allows a user to take a photo of clothes they like in a physical store and suggests optimal outfits based on the image and emotional information.

[0892] Hardware and Software Configuration

[0893] Device: A smartphone used by a user, equipped with a camera and internet connection, and capable of capturing the user's facial expressions and voice.

[0894] server:

[0895] Receives and analyzes image data and audio data, and manages user data.

[0896] Preprocess the images using an image processing library (OpenCV).

[0897] A machine learning model (TensorFlow) is used to extract clothing features.

[0898] We use an emotion analysis engine (Microsoft Azure Emotion API) to identify emotions from the user's facial expressions and voice through emotion analysis.

[0899] The system compares the outfit with a trend database to generate the optimal outfit.

[0900] Advertising data for similar products is collected and displayed to the user.

[0901] Database: MongoDB is used to store user clothing information and trend data.

[0902] DETAILED DESCRIPTION OF THE EMBODIMENTS

[0903] Program processing:

[0904] 1. The user takes a photo of the clothes using their smartphone camera, opens the app, and taps the "Upload new clothes" button. The image is sent to the server.

[0905] 2. The server temporarily stores the received image files and performs preprocessing using an image processing library (OpenCV), such as resizing, noise reduction, and color correction.

[0906] 3. Next, the server uses a machine learning model (TensorFlow) to extract features such as the color, pattern, shape, and season of the clothing, resulting in an analysis result such as "a red floral dress."

[0907] 4. The server compares the extracted features with a trend database and generates the optimal outfit based on the latest fashion information.

[0908] 5. The server sends the generated outfit suggestions to the user's device, and the user receives the suggestions via the app. The user's facial expressions and voice are also captured, and their emotions are identified by an emotion analysis engine.

[0909] 6. The server then uses the results of the emotion analysis to suggest outfits and advertisements that match the user's emotions. For example, if the user is happy, it will suggest bright-colored clothes, and if they are depressed, it will suggest relaxing clothes.

[0910] 7. It collects advertising data from stores that sell similar products and displays ads that users may be interested in. When clicked, users are redirected to the product page of the online shop.

[0911] Specific examples

[0912] For example, user A can take a photo of a red floral dress they like in a physical store and upload it to the server using a dedicated app. The server performs image and emotional analysis, and if it determines that user A is pleased, it will suggest more casual and brightly colored items (such as white sandals or a yellow bag). It will also display related products available in the store, allowing the user to try them on and purchase them immediately.

[0913] Example prompt for a generative AI model:

[0914] 1. Prompt for clothing feature extraction:

[0915] "Please extract the color, pattern, shape, and season of the clothes in this image."

[0916] 2. Prompts for sentiment analysis:

[0917] "Analyze the user's facial features from this image and identify their current emotion."

[0918] This allows users to enhance their in-store shopping experience and receive more personalized offers.

[0919] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0920] Step 1:

[0921] A user takes a photo of an outfit using their smartphone camera, opens the app, and taps the "Upload New Outfit" button. The input is the photo of the outfit taken by the user, and the output is the photo file being sent to the server. At this time, the app executes an HTTP request to send the image to the server. Specifically, the operation involves taking a photo using the smartphone's camera function and uploading the image file from within the app.

[0922] Step 2:

[0923] The server temporarily stores the image files it receives and performs preprocessing using an image processing library (OpenCV). The input is the image file of the clothes sent to the server, and the output is the preprocessed image data. This preprocessing includes resizing, noise removal, color correction, etc. Specifically, the server receives the images and performs resizing and noise removal using OpenCV.

[0924] Step 3:

[0925] The server uses a machine learning model (TensorFlow) to extract features such as the color, pattern, shape, and season of the clothing. The input is preprocessed image data, and the output is extracted clothing feature data. Specifically, analysis is performed using the TensorFlow model to extract specific features such as "a dress with a red floral pattern."

[0926] Step 4:

[0927] The server compares the extracted features with a trend database and generates optimal outfits based on the latest fashion information. The input is the extracted clothing feature data, and the output is optimal outfit suggestions. Specifically, a matching algorithm with the trend database is used to perform processing to suggest appropriate combinations.

[0928] Step 5:

[0929] The server sends the generated outfit suggestions to the user's device, and the user receives the suggestions via the app. The input is the optimal outfit suggestion, and the output is the display of outfit information on the user's device. Specifically, this process sends a notification to the app and displays the suggestion content.

[0930] Step 6:

[0931] The system captures the user's facial expressions and voice and identifies their emotions using an emotion analysis engine (Microsoft Azure Emotion API). The input is the user's facial and voice data, and the output is analyzed emotional information. Specifically, the system captures facial expressions and voice using the smartphone's camera and microphone, and identifies emotions through the Azure Emotion API.

[0932] Step 7:

[0933] The server then uses the results of the emotion analysis to suggest outfits and advertisements that match the user's emotions. The input is the analyzed emotion information and clothing feature data, and the output is emotion-based outfit and advertisement suggestions. Specifically, if the user is happy, it suggests brightly colored items, and if they are depressed, it suggests relaxing clothes.

[0934] Step 8:

[0935] The server collects advertising data for similar products and displays it to the user. The input is advertising data related to clothing feature data, and the output is the display of advertisements on the user's device. Specifically, the process involves collecting related products through an advertising network and displaying advertisements that the user is likely to be interested in on the app.

[0936] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0937] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0938] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0939] [Third embodiment]

[0940] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0941] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0942] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0943] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0944] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0945] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0946] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0947] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0948] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0949] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0950] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0951] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0952] The system of the present invention allows users to take photos of their clothes, find the best outfits based on the photos, and then purchase similar clothes online. The specific configuration and operation of this system will be described below.

[0953] System configuration

[0954] 1. Device:

[0955] This refers to devices such as smartphones and tablets used by users.

[0956] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[0957] 2. Server:

[0958] This is a remote server that receives and analyzes image data and manages user data.

[0959] An image processing library is used to preprocess and analyze images, and a machine learning model is used to extract clothing features.

[0960] It refers to a trend database and suggests optimal outfits to users.

[0961] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[0962] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[0963] Program processing

[0964] 1. Take and upload an image of your outfit

[0965] The user uses their smartphone to take a photo of newly purchased clothes or clothes they are wearing.

[0966] The device acquires the captured image and sends it to the server using a dedicated app.

[0967] 2. Image Receipt and Analysis

[0968] The server stores the received images and performs preprocessing on them using an image processing library (e.g., OpenCV), such as noise reduction, resizing, and color correction.

[0969] Then, a machine learning model (e.g., TensorFlow) is applied to extract clothing features from the image, including color, pattern, shape, and season.

[0970] 3. Comparison with trend data

[0971] The server compares the extracted clothing characteristics with a trend database, which contains the latest fashion information and influencer posting data.

[0972] Based on the comparison results, the optimal clothing combination is generated and recommended to the user.

[0973] 4. Registering the user in the clothing database

[0974] Users can register information about their clothing using a dedicated app. Information about each piece of clothing can be automatically recognized (image analysis) or manually entered (color, pattern, shape, etc.).

[0975] The server stores the registered information in a database and analyzes the combination of newly registered clothes with existing clothes.

[0976] Based on the user's requests, the system suggests the best outfits for each season and situation.

[0977] 5. Viewing Ads and Purchasing Online

[0978] The server obtains advertising data for stores that sell similar products based on the characteristics of the user's clothes.

[0979] The server sends the acquired advertising data to the user's device and displays it through a dedicated app.

[0980] Users can click on an ad that interests them and access the product page of the online shop to which the ad links, where they can check the product details and make a purchase.

[0981] Specific examples

[0982] For example, consider the case where user A purchases a new red floral dress.

[0983] 1. User A takes a photo of a red floral dress with her smartphone and uploads it to the server using a dedicated app.

[0984] 2. The server receives the image, preprocesses it using an image processing library, and uses a machine learning model to extract the color (red), pattern (floral), and shape (dress) of the dress.

[0985] 3. The server compares the extracted features with a trend database and suggests outfits that combine them with, for example, white sandals and a denim jacket.

[0986] 4. User A has registered other clothes he owns (for example, a white shirt or a black skirt) in the app, and suggestions for combinations with these items are also displayed.

[0987] 5. The server displays an advertisement for a store that sells products similar to the red floral dress, and User A can purchase the product from the online shop.

[0988] In this way, the system of the present invention not only helps users efficiently manage their clothes and find the best outfits, but also allows them to easily purchase related new clothes online.

[0989] The processing flow will be explained below.

[0990] Step 1:

[0991] The user takes a photo of the clothes using the camera on their smartphone or tablet. After taking the photo, they open the dedicated app and tap the "Upload New Clothes" button. This action causes the device to import the image into the app and prepare it for transmission to the server.

[0992] Step 2:

[0993] The device retrieves the captured image and makes an HTTP request to an API endpoint to upload the image file to the server, including the image data and metadata such as the user ID.

[0994] Step 3:

[0995] The server temporarily stores the received image files. Next, it performs preprocessing on the images using an image processing library (e.g., OpenCV). Specifically, it performs image resizing, noise removal, color correction, etc. This generates an image suitable for analysis.

[0996] Step 4:

[0997] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image. This model is pre-trained and can identify clothing features such as color, pattern, shape, and even seasonal characteristics. The analysis results are expressed as, for example, "a dress with a red floral pattern."

[0998] Step 5:

[0999] The server compares the extracted features with a trend database, which stores data on the latest fashion magazines, influencer posts, and popular items from online shops. Using a comparison algorithm, the server generates optimal outfits that match current trends.

[1000] Step 6:

[1001] The server sends the generated outfit suggestions to the device. When the user opens the app, the outfit suggestions are displayed. For example, specific suggestions such as "Pair a red floral dress with white sandals" are made.

[1002] Step 7:

[1003] Users can register information about the clothes they own using a dedicated app. Both automatic recognition (image analysis) and manual input methods are available, making it easy for users to register information about their clothes.

[1004] Step 8:

[1005] The server stores the clothing information sent by the user in a database and analyzes combinations based on existing and newly added clothing information, generating coordination suggestions according to the season and situation.

[1006] Step 9:

[1007] The server retrieves advertising data from stores that sell similar products based on the characteristics of the user's clothing, selects relevant advertisements through advertising networks and partnerships, and prepares to display them to the user.

[1008] Step 10:

[1009] The device displays the advertising data received from the server to the user. When the user clicks on an advertisement that interests them, they are redirected to the product page of the online shop linked to the advertisement. Here, the user can check the product details and purchase it online.

[1010] Through the above processing steps, the user can manage his / her own clothes, receive suggestions for coordinating outfits, and easily purchase related products online.

[1011] Example 1

[1012] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1013] Previously, systems that allowed users to manage their clothing and find optimal outfits were either complex or had limited functionality. Furthermore, there were no systems that offered trend-based clothing combination suggestions or a unified online purchasing solution. This made it difficult for users to efficiently manage their clothing, find optimal outfits, and easily purchase related products online.

[1014] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1015] In this invention, the server includes a means for a user to take an image and upload it to the server, a means for the server to analyze the image and extract clothing features using an image processing library and a machine learning model, and a means for comparing the extracted clothing features with trend data to generate optimal clothing combinations, thereby enabling users to efficiently manage their own clothing, find optimal combinations, and easily purchase related products online.

[1016] "User" refers to an individual who uses the system to manage their own clothing and receive suggested outfits.

[1017] "Terminal" refers to a portable computing device used by a User, such as a smartphone or tablet.

[1018] "Server" refers to a computer system that remotely receives and analyzes image data and manages user data.

[1019] "Image processing library" refers to software tools and libraries for preprocessing and analyzing images, such as OpenCV.

[1020] A "machine learning model" refers to an algorithm that learns patterns from data and makes predictions or classifications. Specific examples include TensorFlow.

[1021] "Clothing features" refer to attributes of clothing extracted from an image, such as color, pattern, shape, and season.

[1022] "Trend data" refers to a database containing the latest fashion information and influencer posting data.

[1023] A "trend database" refers to a database that accumulates the latest fashion information and trends.

[1024] "Optimal clothing combination" refers to the clothing coordination that is considered optimal for the user, calculated based on the extracted clothing characteristics and the user's preferences and trend data.

[1025] "User's clothing information" refers to data about the clothes owned by the user (color, pattern, shape, etc.).

[1026] "Database" refers to a data storage system for storing and managing information about users' clothing and trend data.

[1027] "Advertising data" refers to advertising information provided by stores and online shops that sell similar products.

[1028] "Online purchase" refers to a user completing a purchase procedure for a product via the Internet.

[1029] MODE FOR CARRYING OUT THE INVENTION

[1030] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the photos, and then help them purchase similar clothes online. The specific configuration and operation of this system are described below.

[1031] System configuration

[1032] 1. Device:

[1033] This refers to devices such as smartphones and tablets used by users.

[1034] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[1035] 2. Server:

[1036] This is a remote server that receives and analyzes image data and manages user data.

[1037] An image processing library (e.g., OpenCV) is used to preprocess and analyze the images, and a machine learning model (e.g., TensorFlow) is used to extract clothing features.

[1038] It refers to a trend database and suggests optimal outfits to users.

[1039] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[1040] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[1041] Overview of program processing

[1042] 1. Take and upload an image of your outfit

[1043] Users take photos of newly purchased or currently wearing clothes with their smartphones, and the device sends the images to a server using a dedicated app.

[1044] 2. Image Receipt and Analysis

[1045] The server stores the received images and performs preprocessing using an image processing library (e.g., OpenCV) to remove noise, resize, and correct color. It then applies a machine learning model (e.g., TensorFlow) to extract clothing features (e.g., color, pattern, shape, season) from the images.

[1046] 3. Comparison with trend data

[1047] The server compares the extracted clothing characteristics with a trend database, which contains the latest fashion information and influencer postings. Based on the comparison results, the server generates optimal clothing combinations and suggests them to the user.

[1048] 4. Registering the user in the clothing database

[1049] Users use a dedicated app to register information about the clothes they own. Information about each piece of clothing can be automatically recognized through image analysis or can be entered manually. The server stores this information in a database and analyzes how newly registered clothes can be combined with existing clothes. The system then provides optimal outfits according to the season and situation.

[1050] 5. Viewing Ads and Purchasing Online

[1051] The server retrieves advertising data for stores that carry similar products based on the characteristics of the user's clothing. The acquired advertising data is sent to the user's device and displayed through a dedicated app. The user can click on an advertisement that interests them, access the product page in the online shop, and purchase the item.

[1052] Specific examples

[1053] For example, a case will be described in which user A purchases a new red floral dress.

[1054] 1. User A takes a photo of a red floral dress with her smartphone and uploads it to the server using a dedicated app.

[1055] 2. The server receives the image, preprocesses it using an image processing library (e.g., OpenCV), and then uses a machine learning model (e.g., TensorFlow) to extract the color (red), pattern (floral), and shape (dress) of the dress.

[1056] 3. The server compares the extracted features with trend data and suggests outfits that combine them with, for example, white sandals and a denim jacket.

[1057] 4. User A has registered other clothes he owns (e.g., white shirts and black skirts) in the app, and suggestions for combinations with these items are also displayed.

[1058] 5. The server displays an advertisement for a store that sells products similar to the red floral dress, and User A can purchase the product from the online shop.

[1059] Prompt Sentence Examples

[1060] "Generate a sentence that explains the system that receives an image of an outfit from the user, analyzes the image, suggests the best outfit, and advertises similar products."

[1061] By configuring it in this way, the system not only helps users efficiently manage their clothes and find the best outfits, but also allows them to easily purchase related new clothes online.

[1062] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1063] Step 1:

[1064] The user takes a photo of the clothes with their smartphone and uploads the image through a dedicated app. The input is the image of the clothes taken by the user, and the output is image data sent to the server. The device uses its camera function to acquire the image and sends it to the server via the Internet.

[1065] Step 2:

[1066] The server saves the image data received from the terminal. The input is the image data sent from the terminal, and the output is the saved image data. Specifically, the server stores the received image data in storage and prepares it for subsequent analysis processing.

[1067] Step 3:

[1068] The server preprocesses the received images using an image processing library (e.g., OpenCV). The input is the stored image data, and the output is the preprocessed image data. Specifically, it performs processes such as noise removal, resizing (e.g., to 256x256 pixels), and color correction.

[1069] Step 4:

[1070] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from preprocessed images. The input is the preprocessed image data, and the output is extracted clothing feature data (e.g., color, pattern, shape, season). Specifically, the image is input into the machine learning model, and clothing features are obtained from the output.

[1071] Step 5:

[1072] The server compares the extracted clothing feature data with a trend database. The input is the clothing feature data and the trend database, and the output is optimal clothing combination suggestion data. The server refers to the trend database and calculates and generates the optimal outfit for the user.

[1073] Step 6:

[1074] The server generates optimal outfit combinations and sends that information to the user's device. The input is the optimal outfit combination suggestion data, and the output is the suggestion information sent to the user's device. Specifically, when the user opens the app, the suggested outfits are displayed.

[1075] Step 7:

[1076] The user registers information about their clothing using a dedicated app. The input is the clothing information entered or photographed by the user, and the output is the registration information sent to the server. The device acquires this information and sends it to the server.

[1077] Step 8:

[1078] The server stores the user's clothing information in a database. The input is the registered clothing information, and the output is the stored database. The server compares the user's new clothing data with existing data and analyzes the combinations.

[1079] Step 9:

[1080] The server retrieves advertising data for stores that sell similar products based on the characteristics of the user's clothing. The input is the characteristic data of the user's clothing, and the output is the retrieved advertising data. The server then refers to the advertising service to collect advertising information for related products.

[1081] Step 10:

[1082] The server sends the acquired advertising data to the user's device. The input is the collected advertising data, and the output is advertising information sent to the user's device. Users can view these advertisements through a dedicated app and purchase products they are interested in online.

[1083] (Application example 1)

[1084] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1085] Conventional clothing coordination suggestion systems have difficulty easily linking information about the clothes a user owns with the latest trends, and have also faced challenges in efficiently finding and purchasing similar products. Furthermore, they lacked personalized suggestions tailored to the user's preferences and requirements, requiring users to expend a great deal of time and effort. Our goal is to provide a system that solves these issues, allowing users to more easily and instantly find appropriate coordinations and purchase similar products.

[1086] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1087] In this invention, the server includes means for taking images of clothes and uploading them to a remote server, means for the remote server to analyze the images and extract clothing characteristics, means for comparing the clothing characteristics with trend data and presenting optimal clothing combinations, means for registering information about clothes owned by the user and proposing combinations according to seasons and events, means for displaying advertisements for stores that carry similar clothes and enabling online purchases, means for proposing new clothes using a generative AI model based on optimal coordination, and means for proposing clothes based on specified requirements by inputting a prompt text. This allows users to not only easily analyze and manage the characteristics of their own clothes, but also enables efficient coordination suggestions linked to the latest trend information and allows them to quickly find and purchase similar products.

[1088] "Means for taking images of clothes and uploading them to a remote server" refers to a function that allows a user to take an image of clothes using a device such as a smartphone or tablet, and then send the image data to a remote server via the Internet.

[1089] "Means for the remote server to analyze the image and extract characteristics of the clothing" refers to a function in which the remote server analyzes the image data received and extracts characteristics such as the color, pattern, shape, and seasonality of the clothing.

[1090] The "means for comparing trend data with clothing characteristics and presenting optimal clothing combinations" is a function that compares extracted clothing characteristics with the latest trend data and presents optimal outfits to the user.

[1091] "A means for users to register information about the clothes they own and suggest combinations appropriate for the season or event" is a function that allows users to register information about the clothes they own in the system and then suggests outfits appropriate for the season or event based on that information.

[1092] "Means for displaying advertisements for stores that sell similar clothing and enabling online purchases" is a function that displays advertisements for stores that sell similar products based on the characteristics of the clothing registered by the user, allowing the user to purchase directly from those stores.

[1093] "Means for proposing new clothing using a generative AI model based on optimal coordination" is a function that uses a generative AI model to suggest new clothing based on extracted clothing characteristics and optimal coordination information.

[1094] "A means for suggesting clothing based on specified requirements by inputting a prompt sentence" is a function that allows the user to input specific requirements or preferences as a prompt sentence and suggests appropriate clothing based on that information.

[1095] The system of the present invention allows users to take photos of their own clothes using a smartphone or tablet, find optimal outfits based on the photos, and then help them purchase similar clothes online. The specific configuration and operation of this system are described in detail below.

[1096] System configuration

[1097] 1. Device (smartphone, tablet, etc.):

[1098] It has a camera function for taking images and a function for connecting to the Internet and communicating with a remote server.

[1099] Using a dedicated application, users can take photos of their clothes and upload them to a remote server.

[1100] 2. Remote Server:

[1101] Receives and analyzes image data, and manages user data.

[1102] An image processing library (e.g., OpenCV) is used to preprocess and analyze the images, and a machine learning model (e.g., TensorFlow) is used to extract clothing features.

[1103] The system refers to a trend database and compares the extracted clothing characteristics to suggest the optimal outfit.

[1104] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[1105] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[1106] Based on the optimal coordination, new clothing suggestions are made using a generative AI model.

[1107] By entering a prompt, clothing suggestions will be made based on the specified requirements.

[1108] How it works

[1109] 1. Take and upload your outfit image:

[1110] The user takes a photo of the new or currently worn clothes using a smartphone, and then uploads the image to a remote server using a dedicated application.

[1111] 2. Image Receipt and Analysis:

[1112] The remote server receives the uploaded images and performs preprocessing (noise removal, resizing, color correction, etc.) using an image processing library (OpenCV), then extracts clothing features (color, pattern, shape, season, etc.) using a machine learning model (TensorFlow).

[1113] 3. Comparison with trend data:

[1114] The extracted clothing characteristics are compared with a trend database, the latest fashion information, and influencer posts. Based on the comparison results, optimal clothing combinations are generated and recommended to the user.

[1115] 4. Registering the user in the clothing database:

[1116] Users register information about their clothing using a dedicated app. The registered information is stored in a database on a remote server. The server then analyzes how the newly registered clothing can be combined with existing clothing to suggest optimal outfits for each season and situation.

[1117] 5. Viewing Ads and Online Purchases:

[1118] The remote server retrieves advertising data for stores that sell similar products based on the extracted clothing characteristics. The acquired advertising data is sent to the user's device and displayed through a dedicated app. Users can click on an advertisement that interests them and purchase the product from the online shop.

[1119] 6. New clothing suggestions using generative AI models:

[1120] Based on the extracted clothing features and optimal coordination information, a generative AI model is used to suggest new clothing to the user. For example, if a user takes a photo of a red floral dress, the system will suggest new clothing designs based on those features.

[1121] 7. Specifying requirements by prompt sentence:

[1122] Users can specify their specific requirements and preferences by entering prompts in a dedicated application, and the server will then suggest clothing that matches the requirements.

[1123] Specific examples

[1124] Consider the case where User A purchases a new red floral dress, takes a photo of it with her smartphone, and uploads it to the server using a dedicated app. The server analyzes the image and extracts the color (red), pattern (floral), and shape (dress). It then compares the extracted features with a trend database and suggests combinations, such as white sandals and a denim jacket. Furthermore, if User A enters the prompt "casual outdoor style," the server uses the generative AI model to suggest new clothing based on those requirements.

[1125] Prompt Sentence Examples

[1126] text

[1127] Analyze an image of a red floral dress and suggest the best outfit for it. Also, show similar items from an online shop.

[1128] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1129] Step 1:

[1130] The device takes pictures of the clothes the user is wearing or newly purchased clothes. When the user clicks the capture button in the dedicated app, the device's camera function is activated and the image is captured. The input is the image data captured by the camera, and the output is an image file stored on the device.

[1131] Step 2:

[1132] The device uploads the captured images to a remote server via a dedicated app. When the user clicks the upload button, the image data is sent to the remote server via the Internet. The input is the image file in the device, and the output is the image data sent to the remote server.

[1133] Step 3:

[1134] The remote server stores the received image data and performs preprocessing on the image using an image processing library (e.g., OpenCV). Specifically, it performs processes such as noise reduction, resizing, and color correction. The input is the image data uploaded to the server, and the output is the image data after preprocessing.

[1135] Step 4:

[1136] The remote server inputs the preprocessed image data into a machine learning model (e.g., TensorFlow) to extract clothing features. Features include color, pattern, shape, season, etc. The input is the preprocessed image data, and the output is the extracted clothing feature data.

[1137] Step 5:

[1138] The remote server compares the extracted clothing features with a trend database, referring to the latest fashion information and influencer posts to generate optimal clothing combinations. The input is clothing feature data and trend data, and the output is optimal clothing combination data.

[1139] Step 6:

[1140] The remote server retrieves information about clothing registered by the user using a dedicated app from a database and analyzes how the newly registered clothing can be combined with existing clothing. As a specific example, it suggests optimal outfits according to the season or event. The input is the user's clothing data, and the output is combination data according to the season or event.

[1141] Step 7:

[1142] The remote server retrieves advertising data for stores that sell similar products based on the extracted clothing features. The advertising data is provided by online shops and displayed to users. The input is clothing feature data and store advertising data, and the output is advertising data displayed on the user's device.

[1143] Step 8:

[1144] The remote server uses a generative AI model based on the optimal outfit to suggest new outfits. The generative AI model uses deep learning, for example. The input is optimal outfit data, and the output is new outfit suggestion data.

[1145] Step 9:

[1146] The user inputs a prompt on a dedicated application and sends specific requirements and preferences to the server, which then suggests clothes that fit the specified requirements based on the prompt. The input is the prompt from the user, and the output is the suggested clothing data.

[1147] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1148] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the images, and assist users in purchasing similar clothes online. By combining it with an emotion engine, the system suggests the best clothing combinations and advertisements according to the user's emotions. The specific configuration and operation of this system will be described below.

[1149] System configuration

[1150] 1. Device:

[1151] A device such as a smartphone or tablet used by a user.

[1152] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[1153] It has the ability to capture the user's facial expressions and voice.

[1154] 2. Server:

[1155] This is a remote server that receives and analyzes image data and manages user data.

[1156] An image processing library is used to preprocess and analyze images, and a machine learning model is used to extract clothing features.

[1157] It refers to a trend database and suggests optimal outfits to users.

[1158] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[1159] The advertising data of stores that sell similar products is acquired and displayed to the user.

[1160] It uses an emotion engine to analyze user emotions and tailor suggestions and advertisements.

[1161] Program processing

[1162] When a user takes a photo of an outfit using the camera on their smartphone or tablet, opens the app, and taps the "Upload New Outfit" button, the image is sent to the server. The device retrieves the image and executes an HTTP request.

[1163] The server temporarily stores the received image files and performs preprocessing using an image processing library (e.g., OpenCV). An appropriate analysis state is generated by resizing, noise removal, color correction, etc. Next, a machine learning model (e.g., TensorFlow) is used to extract features such as the color, pattern, shape, and season of the clothing. This results in an analysis result such as "a dress with a red floral pattern."

[1164] The server compares the extracted features with a trend database, which includes data from magazines and social media, to generate optimal outfits based on the latest fashion information and influencer posts.

[1165] The server then sends the generated outfit suggestions to the user's device, where the user can receive the suggestions via the app. For example, a specific suggestion such as "pair a red floral dress with white sandals" is made.

[1166] Users can use a dedicated app to register information about the clothes they own. Registration methods include automatic recognition (image analysis) and manual input (color, pattern, shape, etc.), making it easy to add clothing information to the database. The server saves the registered clothing information in the database and analyzes how newly added clothes can be combined with existing clothes to make optimal suggestions.

[1167] The server collects advertising data from stores that sell similar products based on the user's clothing characteristics. It then selects relevant ads through advertising networks and partnerships and displays them to the user. When the user clicks on an ad that interests them, they are redirected to the product page in the online shop, where they can view product details and make a purchase.

[1168] Introducing the Emotion Engine

[1169] The system incorporates an emotion engine, and the following specific processes are added:

[1170] The emotion engine analyzes the user's facial expressions and voice to identify their current emotions. For example, it uses a camera to collect the user's facial expression data and uses voice recognition technology to analyze the user's speech.

[1171] The server uses the analysis results obtained from the emotion engine to suggest optimal outfits and advertisements according to the user's emotions. For example, if the user is happy, it will suggest bright-colored clothes, and if they are depressed, it will suggest comfortable, relaxing clothes. It is also possible to dynamically change relevant product advertisements according to the user's emotions.

[1172] Specific examples

[1173] For example, consider the case where User B purchases a new blue denim dress, takes a photo of it, and uploads it to the server.

[1174] 1. User B takes a photo of a denim dress with their smartphone and uploads it to the server using a dedicated app.

[1175] 2. The server receives the image, preprocesses it using an image processing library, and then uses a machine learning model to extract the color (blue), pattern (denim), and shape (dress) of the dress.

[1176] 3. The server compares the extracted features with a trend database and suggests outfits, such as "combining white sneakers with a red bag."

[1177] 4. If user B is found to be happy through the emotion engine, suggest more casual and brightly colored items.

[1178] 5. User B has already registered a white shirt and a black skirt in the app, and suggestions for combinations with them are also displayed.

[1179] 6. The server displays an advertisement for a store that sells a product similar to the blue denim dress, and User B purchases the product from an online shop.

[1180] In this way, the system of the present invention can dynamically respond to the user's emotions and can also provide personalized coordination and advertising suggestions.

[1181] The processing flow will be explained below.

[1182] Step 1:

[1183] The user takes a photo of the clothes using the camera on their smartphone or tablet. After taking the photo, they open the dedicated app and tap the "Upload New Clothes" button. This action causes the device to import the image into the app and prepare it for transmission to the server.

[1184] Step 2:

[1185] The device retrieves the captured image and makes an HTTP request to an API endpoint to upload the image file to the server, including the image data and metadata such as the user ID.

[1186] Step 3:

[1187] The server temporarily stores the received image files. Next, it performs preprocessing on the images using an image processing library (e.g., OpenCV). Specifically, it performs image resizing, noise removal, color correction, etc. This generates an image suitable for analysis.

[1188] Step 4:

[1189] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image. This model is pre-trained and can identify clothing features such as color, pattern, shape, and season. The analysis results are expressed as, for example, "a red floral dress."

[1190] Step 5:

[1191] The server compares the extracted features with a trend database, which stores data on the latest fashion magazines, influencer posts, and popular items from online shops. Using a comparison algorithm, the server generates optimal outfits that match current trends.

[1192] Step 6:

[1193] The server sends the generated outfit suggestions to the device. When the user opens the app, the outfit suggestions are displayed. For example, specific suggestions such as "Pair a red floral dress with white sandals" are made.

[1194] Step 7:

[1195] Users can register information about the clothes they own using a dedicated app. Both automatic recognition (image analysis) and manual input (color, pattern, shape, etc.) are available, making it easy for users to register information about their clothes.

[1196] Step 8:

[1197] The server stores the clothing information sent by the user in a database and analyzes combinations based on existing clothing information and newly added information, thereby generating coordination suggestions according to the season and situation.

[1198] Step 9:

[1199] The device captures the user's facial expressions and voice and sends them to an emotion engine, which uses a model trained on a dataset for emotion analysis to analyze the user's emotions in real time.

[1200] Step 10:

[1201] The emotion engine analyzes the user's emotions and sends the results to the server. For example, it responds when the user is judged to be happy or depressed.

[1202] Step 11:

[1203] The server uses the emotional data sent from the emotion engine to adjust the recommended outfits and advertisements. For example, if the user is happy, it will suggest brightly colored clothes and casual items, and if they are depressed, it will suggest relaxing clothes. It also selects the most appropriate advertisement based on the user's emotions and sends it to the user's device.

[1204] Step 12:

[1205] The device displays coordination suggestions and advertisements to the user based on the emotions received from the server. Because the suggestions displayed are suited to the user's emotions, the user can receive more personalized services.

[1206] Through the above processing steps, the user can receive suggestions for optimal clothing management and coordination based on their emotions, and can also easily purchase related products online.

[1207] Example 2

[1208] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1209] Conventional coordination support systems were unable to provide suggestions based on the user's emotions and mood regarding clothing, and were unable to sufficiently increase user satisfaction. Furthermore, when purchasing similar products online, advertisements displayed did not dynamically change based on the user's emotions and preferences, preventing effective marketing. Furthermore, it was difficult to provide suggestions for combining outfits with the user's existing clothes to suit the season or specific situations.

[1210] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for taking an image of clothes and uploading it to the server, a means for the server to analyze the image and extract characteristics of the clothes, a means for comparing the characteristics of the clothes with trend data and recommending optimal clothing combinations, a means for registering information on clothes owned by the user and proposing combinations according to the season and situation, a means for displaying advertisements for stores that carry similar clothes and enabling online purchases, and a means for analyzing the user's emotions using an emotion analysis engine and adjusting optimal clothing combinations and advertisements based on the emotions. This makes it possible to suggest coordinations and display advertisements that match the user's emotions and mood.

[1211] "Clothing image" refers to photographic data of clothing that the user will wear.

[1212] "Means for uploading" refers to the ability for a user to send images taken by the user to a remote server via the Internet.

[1213] "Means for analyzing images and extracting clothing features" refers to the process of identifying and extracting attributes such as clothing color, pattern, and shape from the received image data.

[1214] "Trend data" refers to data that reflects the latest fashion information and market trends.

[1215] "Means of recommending optimal clothing combinations" refers to a function that suggests outfits that suit the user based on trend data and clothing characteristics.

[1216] "Means for registering information about clothes owned by the user" refers to a function that allows a user to input or read data about clothes they own into the system.

[1217] "Means of suggesting combinations according to the season and situation" refers to a function that suggests clothing combinations that suit the season, specific events, or occasions.

[1218] "Means of displaying advertisements for stores that sell similar clothing and enabling online purchases" refers to a function that displays advertisements for stores that sell products similar to the clothing photographed by the user and allows the user to purchase those products online.

[1219] An "emotion analysis engine" refers to a system that analyzes a user's facial expressions and voice to identify their current emotional state.

[1220] "Means for adjusting optimal clothing combinations and advertisements based on emotions" refers to a function that dynamically changes coordination suggestions and advertisement content to match the identified user's emotions.

[1221] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the photos, and help them purchase similar clothes online. Furthermore, by combining it with an emotion engine, the system suggests the best clothing combinations and advertisements based on the user's emotions.

[1222] System configuration

[1223] 1. Device:

[1224] A device such as a smartphone or tablet used by a user.

[1225] It has a camera function for taking pictures and the ability to connect to the Internet and communicate with a server.

[1226] It has the ability to capture the user's facial expressions and voice.

[1227] 2. Server:

[1228] This is a remote server that receives and analyzes image data and manages user data.

[1229] Specifically, the image is preprocessed using an image processing library (e.g., OpenCV), and clothing features are extracted using a machine learning model (e.g., TensorFlow).

[1230] The system references a trend database and generates the optimal coordination.

[1231] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[1232] Advertising data is collected and displayed to the user.

[1233] It uses an emotion engine to analyze user emotions and tailor suggestions and advertisements.

[1234] Example of operation

[1235] 1. The user takes a photo of their clothes using the device's camera.

[1236] Example: A user takes a picture of a new blue denim dress.

[1237] 2. Upload the images taken by the device to the server using a dedicated app.

[1238] 3. The server receives the images and performs preprocessing using an image processing library (e.g., OpenCV).

[1239] Pre-processing includes resizing, noise reduction, color correction, etc.

[1240] 4. The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image.

[1241] Example: Extract features such as color (blue), pattern (denim), and shape (dress) from a blue denim dress.

[1242] 5. The server compares the extracted features with a trend database and suggests optimal outfits.

[1243] Example: A specific coordination suggestion such as "pair it with white sneakers and a red bag."

[1244] 6. The server sends the generated coordination proposal to the user's device.

[1245] 7. The device displays the suggestions to the user through a dedicated app.

[1246] For example, you might see suggestions like "Pair a blue denim dress with white sneakers."

[1247] 8. The user uses a dedicated app to register information about the clothes they own.

[1248] Example: Register information such as a white shirt or black skirt you own.

[1249] 9. The server analyzes the combination of new clothes with existing clothes based on the registered clothing information and makes optimal suggestions.

[1250] For example: a blue denim dress paired with a white shirt.

[1251] 10. The server collects advertising data from stores that sell similar products and displays it to the user.

[1252] 11. The device displays advertisements, and when the user clicks on an advertisement that interests them, they are redirected to the product page of the online shop.

[1253] Example: You see products similar to a blue denim dress and can buy them online.

[1254] Prompt Sentence Examples

[1255] "Suggest outfits that go well with the blue denim dress the user recently purchased. Also, if the user is happy with the outfit, suggest items in brighter colors."

[1256] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1257] Step 1:

[1258] The user takes a photo of the clothes using the device's camera.

[1259] Input: The user activates the camera and takes a picture of the clothes.

[1260] Output: The captured image data is saved on the device.

[1261] What happens: A user takes a picture of a new blue denim dress with their smartphone camera.

[1262] Step 2:

[1263] The device uploads the images taken to the server using a dedicated app.

[1264] Input: Image data stored on the device.

[1265] Output: An HTTP POST request containing the image data is sent to the server.

[1266] Specific operation: The user taps the "Upload new clothes" button, and the device sends image data to the server.

[1267] Step 3:

[1268] The server temporarily stores the received image files and performs preprocessing using an image processing library (e.g., OpenCV).

[1269] Input: Image data received by the server.

[1270] Output: Preprocessed image data.

[1271] Data processing or data calculation: image resizing, noise reduction, color correction, etc.

[1272] What happens: The server resizes the image, removes noise, and performs color correction.

[1273] Step 4:

[1274] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image.

[1275] Input: Preprocessed image data.

[1276] Output: Extracted clothing feature data (e.g. color, pattern, shape).

[1277] Data processing or data calculation: Identifying color, pattern, and shape from image data and generating feature data.

[1278] Specific operation: The server extracts the color (blue), pattern (denim), and shape (dress) of the blue denim dress.

[1279] Step 5:

[1280] The server compares the extracted features with a trend database and generates the optimal outfit.

[1281] Input: Extracted clothing feature data.

[1282] Output: Optimal outfit suggestion data.

[1283] Data processing or data calculation: Refer to the feature data and trend database to determine the optimal coordination.

[1284] Specific behavior: The server generates suggestions such as "pair with white sneakers and a red bag."

[1285] Step 6:

[1286] The server transmits the generated coordination proposal to the user's terminal.

[1287] Input: Coordination suggestion data.

[1288] Output: The proposal data sent to the user's device.

[1289] Specific operation: The server sends the generated coordination proposal to the user's device.

[1290] Step 7:

[1291] The coordinated suggestions received by the terminal are displayed to the user via a dedicated app.

[1292] Input: The coordinate suggestion data passed in.

[1293] Output: The suggestions displayed in the app.

[1294] Specific behavior: The device displays a suggestion on the app screen: "Pair a blue denim dress with white sneakers."

[1295] Step 8:

[1296] Users register information about the clothes they own using a dedicated app.

[1297] Input: Clothing information entered or uploaded by the user.

[1298] Output: Clothing information from the registered database.

[1299] Specific operation: The user enters information about a white shirt and a black skirt they own into the app and registers them.

[1300] Step 9:

[1301] Based on the registered clothing information, the server analyzes the combination of new clothing with existing clothing and makes optimal suggestions.

[1302] Input: Registered clothing information.

[1303] Output: Combination suggestion data of newly added clothes and existing clothes.

[1304] Data processing or data calculation: Combining and analyzing existing clothing information with new clothing information.

[1305] What it does: The server suggests a combination of a blue denim dress and a white shirt.

[1306] Step 10:

[1307] The server collects advertising data from stores that sell similar products and transmits it to the user's terminal.

[1308] Input: Registered clothing feature data.

[1309] Output: Collected advertising data.

[1310] What it does: The server collects relevant ads through ad networks.

[1311] Step 11:

[1312] The advertisements received by the device are displayed to the user via a dedicated app.

[1313] Input: Ad data sent by the server.

[1314] Output: The ad displayed within the app.

[1315] What happens: The device displays an ad for a product similar to the blue denim dress on the app screen.

[1316] Step 12:

[1317] The emotion engine analyzes the user's facial expressions and voice to identify their current emotion.

[1318] Input: Facial expression and audio data captured by the camera.

[1319] Output: Parsed emotion data.

[1320] Data processing or data calculation: Performing facial expression recognition and voice analysis to identify emotions.

[1321] Specific operation: Collects the user's facial expression data using a camera and analyzes the user's speech using voice recognition technology.

[1322] Step 13:

[1323] Based on the analysis results obtained from the emotion engine, the server adjusts the optimal coordination and advertisements according to the user's emotions.

[1324] Input: Parsed emotion data.

[1325] Output: Reconciled coordinate and advertising data.

[1326] Data processing or data calculation: Dynamically changing outfits or advertising content based on emotional data.

[1327] What it does: If the user is happy, suggest brighter colored items and show more relevant ads.

[1328] (Application example 2)

[1329] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1330] Until now, when users purchased clothes in a physical store, there was no way to instantly get suggestions on how to coordinate them with the clothes they already own or the best combinations that match their current mood. There is also a demand for personalized suggestions based on the user's emotions to provide a more comfortable shopping experience. This will increase product purchases in physical stores and improve customer satisfaction.

[1331] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for taking an image of clothes and uploading it to the server, means for the server to analyze the image and extract characteristics of the clothes, means for comparing the characteristics of the clothes with trend data and recommending optimal clothing combinations, means for registering information about clothes owned by the user and proposing combinations according to the season and situation, means for displaying advertisements for stores that carry similar clothes and enabling online purchases, and means for analyzing the user's facial expressions and voice to identify emotions and dynamically adjusting coordinations and advertisements based on the emotions. This allows users to take photos of clothes they like in a physical store, receive optimal coordination suggestions in real time, and enjoy a personalized shopping experience according to their emotions.

[1332] "Clothing images" are photos of clothing taken by the user using the camera function.

[1333] "Server" means a remotely located computer system that receives and processes data sent by users and returns analytical results.

[1334] "Clothing characteristics" are visual and physical attributes of clothing such as color, pattern, shape, and season.

[1335] "Trend Data" is a database containing information about current trends and popular styles.

[1336] The "optimal clothing combination" is the most appropriate fashion coordination for the user, generated based on the analyzed clothing characteristics and trend data.

[1337] "User information" refers to personal information such as data on the clothes owned by the user, their preferences, and purchase history.

[1338] "Season" refers to a specific season such as spring, summer, autumn, or winter, and is a standard for selecting clothing appropriate for each season.

[1339] "Situation" refers to a specific event, purpose, weather, or other environmental factors, and is the condition for choosing appropriate clothing.

[1340] "Store advertising" refers to promotional information for similar or related products sold in brick-and-mortar stores or online shops.

[1341] "Online shopping" is a shopping method that allows you to order and pay for products over the Internet.

[1342] "Facial expression" is an element that indicates the expression or emotion of the user's face.

[1343] "Voice" refers to the words and tone of voice spoken by the user, and is information that is part of emotion analysis.

[1344] "Emotion" refers to a user's psychological state, such as happiness, sadness, excitement, or depression.

[1345] "Identifying emotions" means identifying the user's psychological state at that time from their facial expressions and voice.

[1346] "Dynamic adjustment" means automatically changing and optimizing coordination and advertising content in real time based on the analysis results.

[1347] An embodiment of the present invention will now be described. The present invention is a system that allows a user to take a photo of clothes they like in a physical store and suggests optimal outfits based on the image and emotional information.

[1348] Hardware and Software Configuration

[1349] Device: A smartphone used by a user, equipped with a camera and internet connection, and capable of capturing the user's facial expressions and voice.

[1350] server:

[1351] Receives and analyzes image data and audio data, and manages user data.

[1352] Preprocess the images using an image processing library (OpenCV).

[1353] A machine learning model (TensorFlow) is used to extract clothing features.

[1354] We use an emotion analysis engine (Microsoft Azure Emotion API) to identify emotions from the user's facial expressions and voice through emotion analysis.

[1355] The system compares the outfit with a trend database to generate the optimal outfit.

[1356] Advertising data for similar products is collected and displayed to the user.

[1357] Database: MongoDB is used to store user clothing information and trend data.

[1358] DETAILED DESCRIPTION OF THE EMBODIMENTS

[1359] Program processing:

[1360] 1. The user takes a photo of the clothes using their smartphone camera, opens the app, and taps the "Upload new clothes" button. The image is sent to the server.

[1361] 2. The server temporarily stores the received image files and performs preprocessing using an image processing library (OpenCV), such as resizing, noise reduction, and color correction.

[1362] 3. Next, the server uses a machine learning model (TensorFlow) to extract features such as the color, pattern, shape, and season of the clothing, resulting in an analysis result such as "a red floral dress."

[1363] 4. The server compares the extracted features with a trend database and generates the optimal outfit based on the latest fashion information.

[1364] 5. The server sends the generated outfit suggestions to the user's device, and the user receives the suggestions via the app. The user's facial expressions and voice are also captured, and their emotions are identified by an emotion analysis engine.

[1365] 6. The server then uses the results of the emotion analysis to suggest outfits and advertisements that match the user's emotions. For example, if the user is happy, it will suggest bright-colored clothes, and if they are depressed, it will suggest relaxing clothes.

[1366] 7. It collects advertising data from stores that sell similar products and displays ads that users may be interested in. When clicked, users are redirected to the product page of the online shop.

[1367] Specific examples

[1368] For example, user A can take a photo of a red floral dress they like in a physical store and upload it to the server using a dedicated app. The server performs image and emotional analysis, and if it determines that user A is pleased, it will suggest more casual and brightly colored items (such as white sandals or a yellow bag). It will also display related products available in the store, allowing the user to try them on and purchase them immediately.

[1369] Example prompt for a generative AI model:

[1370] 1. Prompt for clothing feature extraction:

[1371] "Please extract the color, pattern, shape, and season of the clothes in this image."

[1372] 2. Prompts for sentiment analysis:

[1373] "Analyze the user's facial features from this image and identify their current emotion."

[1374] This allows users to enhance their in-store shopping experience and receive more personalized offers.

[1375] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1376] Step 1:

[1377] A user takes a photo of an outfit using their smartphone camera, opens the app, and taps the "Upload New Outfit" button. The input is the photo of the outfit taken by the user, and the output is the photo file being sent to the server. At this time, the app executes an HTTP request to send the image to the server. Specifically, the operation involves taking a photo using the smartphone's camera function and uploading the image file from within the app.

[1378] Step 2:

[1379] The server temporarily stores the image files it receives and performs preprocessing using an image processing library (OpenCV). The input is the image file of the clothes sent to the server, and the output is the preprocessed image data. This preprocessing includes resizing, noise removal, color correction, etc. Specifically, the server receives the images and performs resizing and noise removal using OpenCV.

[1380] Step 3:

[1381] The server uses a machine learning model (TensorFlow) to extract features such as the color, pattern, shape, and season of the clothing. The input is preprocessed image data, and the output is extracted clothing feature data. Specifically, analysis is performed using the TensorFlow model to extract specific features such as "a dress with a red floral pattern."

[1382] Step 4:

[1383] The server compares the extracted features with a trend database and generates optimal outfits based on the latest fashion information. The input is the extracted clothing feature data, and the output is optimal outfit suggestions. Specifically, a matching algorithm with the trend database is used to perform processing to suggest appropriate combinations.

[1384] Step 5:

[1385] The server sends the generated outfit suggestions to the user's device, and the user receives the suggestions via the app. The input is the optimal outfit suggestion, and the output is the display of outfit information on the user's device. Specifically, this process sends a notification to the app and displays the suggestion content.

[1386] Step 6:

[1387] The system captures the user's facial expressions and voice and identifies their emotions using an emotion analysis engine (Microsoft Azure Emotion API). The input is the user's facial and voice data, and the output is analyzed emotional information. Specifically, the system captures facial expressions and voice using the smartphone's camera and microphone, and identifies emotions through the Azure Emotion API.

[1388] Step 7:

[1389] The server then uses the results of the emotion analysis to suggest outfits and advertisements that match the user's emotions. The input is the analyzed emotion information and clothing feature data, and the output is emotion-based outfit and advertisement suggestions. Specifically, if the user is happy, it suggests brightly colored items, and if they are depressed, it suggests relaxing clothes.

[1390] Step 8:

[1391] The server collects advertising data for similar products and displays it to the user. The input is advertising data related to clothing feature data, and the output is the display of advertisements on the user's device. Specifically, the process involves collecting related products through an advertising network and displaying advertisements that the user is likely to be interested in on the app.

[1392] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1393] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1394] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1395] [Fourth embodiment]

[1396] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1397] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1398] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1399] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1400] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1401] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1402] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1403] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1404] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1405] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1406] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1407] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1408] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1409] The system of the present invention allows users to take photos of their clothes, find the best outfits based on the photos, and then purchase similar clothes online. The specific configuration and operation of this system will be described below.

[1410] System configuration

[1411] 1. Device:

[1412] This refers to devices such as smartphones and tablets used by users.

[1413] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[1414] 2. Server:

[1415] This is a remote server that receives and analyzes image data and manages user data.

[1416] An image processing library is used to preprocess and analyze images, and a machine learning model is used to extract clothing features.

[1417] It refers to a trend database and suggests optimal outfits to users.

[1418] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[1419] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[1420] Program processing

[1421] 1. Take and upload an image of your outfit

[1422] The user uses their smartphone to take a photo of newly purchased clothes or clothes they are wearing.

[1423] The device acquires the captured image and sends it to the server using a dedicated app.

[1424] 2. Image Receipt and Analysis

[1425] The server stores the received images and performs preprocessing on them using an image processing library (e.g., OpenCV), such as noise reduction, resizing, and color correction.

[1426] Then, a machine learning model (e.g., TensorFlow) is applied to extract clothing features from the image, including color, pattern, shape, and season.

[1427] 3. Comparison with trend data

[1428] The server compares the extracted clothing characteristics with a trend database, which contains the latest fashion information and influencer posting data.

[1429] Based on the comparison results, the optimal clothing combination is generated and recommended to the user.

[1430] 4. Registering the user in the clothing database

[1431] Users can register information about their clothing using a dedicated app. Information about each piece of clothing can be automatically recognized (image analysis) or manually entered (color, pattern, shape, etc.).

[1432] The server stores the registered information in a database and analyzes the combination of newly registered clothes with existing clothes.

[1433] Based on the user's requests, the system suggests the best outfits for each season and situation.

[1434] 5. Viewing Ads and Purchasing Online

[1435] The server obtains advertising data for stores that sell similar products based on the characteristics of the user's clothes.

[1436] The server sends the acquired advertising data to the user's device and displays it through a dedicated app.

[1437] Users can click on an ad that interests them and access the product page of the online shop to which the ad links, where they can check the product details and make a purchase.

[1438] Specific examples

[1439] For example, consider the case where user A purchases a new red floral dress.

[1440] 1. User A takes a photo of a red floral dress with her smartphone and uploads it to the server using a dedicated app.

[1441] 2. The server receives the image, preprocesses it using an image processing library, and uses a machine learning model to extract the color (red), pattern (floral), and shape (dress) of the dress.

[1442] 3. The server compares the extracted features with a trend database and suggests outfits that combine them with, for example, white sandals and a denim jacket.

[1443] 4. User A has registered other clothes he owns (for example, a white shirt or a black skirt) in the app, and suggestions for combinations with these items are also displayed.

[1444] 5. The server displays an advertisement for a store that sells products similar to the red floral dress, and User A can purchase the product from the online shop.

[1445] In this way, the system of the present invention not only helps users efficiently manage their clothes and find the best outfits, but also allows them to easily purchase related new clothes online.

[1446] The processing flow will be explained below.

[1447] Step 1:

[1448] The user takes a photo of the clothes using the camera on their smartphone or tablet. After taking the photo, they open the dedicated app and tap the "Upload New Clothes" button. This action causes the device to import the image into the app and prepare it for transmission to the server.

[1449] Step 2:

[1450] The device retrieves the captured image and makes an HTTP request to an API endpoint to upload the image file to the server, including the image data and metadata such as the user ID.

[1451] Step 3:

[1452] The server temporarily stores the received image files. Next, it performs preprocessing on the images using an image processing library (e.g., OpenCV). Specifically, it performs image resizing, noise removal, color correction, etc. This generates an image suitable for analysis.

[1453] Step 4:

[1454] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image. This model is pre-trained and can identify clothing features such as color, pattern, shape, and even seasonal characteristics. The analysis results are expressed as, for example, "a dress with a red floral pattern."

[1455] Step 5:

[1456] The server compares the extracted features with a trend database, which stores data on the latest fashion magazines, influencer posts, and popular items from online shops. Using a comparison algorithm, the server generates optimal outfits that match current trends.

[1457] Step 6:

[1458] The server sends the generated outfit suggestions to the device. When the user opens the app, the outfit suggestions are displayed. For example, specific suggestions such as "Pair a red floral dress with white sandals" are made.

[1459] Step 7:

[1460] Users can register information about the clothes they own using a dedicated app. Both automatic recognition (image analysis) and manual input methods are available, making it easy for users to register information about their clothes.

[1461] Step 8:

[1462] The server stores the clothing information sent by the user in a database and analyzes combinations based on existing and newly added clothing information, generating coordination suggestions according to the season and situation.

[1463] Step 9:

[1464] The server retrieves advertising data from stores that sell similar products based on the characteristics of the user's clothing, selects relevant advertisements through advertising networks and partnerships, and prepares to display them to the user.

[1465] Step 10:

[1466] The device displays the advertising data received from the server to the user. When the user clicks on an advertisement that interests them, they are redirected to the product page of the online shop linked to the advertisement. Here, the user can check the product details and purchase it online.

[1467] Through the above processing steps, the user can manage his / her own clothes, receive suggestions for coordinating outfits, and easily purchase related products online.

[1468] Example 1

[1469] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1470] Previously, systems that allowed users to manage their clothing and find optimal outfits were either complex or had limited functionality. Furthermore, there were no systems that offered trend-based clothing combination suggestions or a unified online purchasing solution. This made it difficult for users to efficiently manage their clothing, find optimal outfits, and easily purchase related products online.

[1471] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1472] In this invention, the server includes a means for a user to take an image and upload it to the server, a means for the server to analyze the image and extract clothing features using an image processing library and a machine learning model, and a means for comparing the extracted clothing features with trend data to generate optimal clothing combinations, thereby enabling users to efficiently manage their own clothing, find optimal combinations, and easily purchase related products online.

[1473] "User" refers to an individual who uses the system to manage their own clothing and receive suggested outfits.

[1474] "Terminal" refers to a portable computing device used by a User, such as a smartphone or tablet.

[1475] "Server" refers to a computer system that remotely receives and analyzes image data and manages user data.

[1476] "Image processing library" refers to software tools and libraries for preprocessing and analyzing images, such as OpenCV.

[1477] A "machine learning model" refers to an algorithm that learns patterns from data and makes predictions or classifications. Specific examples include TensorFlow.

[1478] "Clothing features" refer to attributes of clothing extracted from an image, such as color, pattern, shape, and season.

[1479] "Trend data" refers to a database containing the latest fashion information and influencer posting data.

[1480] A "trend database" refers to a database that accumulates the latest fashion information and trends.

[1481] "Optimal clothing combination" refers to the clothing coordination that is considered optimal for the user, calculated based on the extracted clothing characteristics and the user's preferences and trend data.

[1482] "User's clothing information" refers to data about the clothes owned by the user (color, pattern, shape, etc.).

[1483] "Database" refers to a data storage system for storing and managing information about users' clothing and trend data.

[1484] "Advertising data" refers to advertising information provided by stores and online shops that sell similar products.

[1485] "Online purchase" refers to a user completing a purchase procedure for a product via the Internet.

[1486] MODE FOR CARRYING OUT THE INVENTION

[1487] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the photos, and then help them purchase similar clothes online. The specific configuration and operation of this system are described below.

[1488] System configuration

[1489] 1. Device:

[1490] This refers to devices such as smartphones and tablets used by users.

[1491] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[1492] 2. Server:

[1493] This is a remote server that receives and analyzes image data and manages user data.

[1494] An image processing library (e.g., OpenCV) is used to preprocess and analyze the images, and a machine learning model (e.g., TensorFlow) is used to extract clothing features.

[1495] It refers to a trend database and suggests optimal outfits to users.

[1496] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[1497] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[1498] Overview of program processing

[1499] 1. Take and upload an image of your outfit

[1500] Users take photos of newly purchased or currently wearing clothes with their smartphones, and the device sends the images to a server using a dedicated app.

[1501] 2. Image Receipt and Analysis

[1502] The server stores the received images and performs preprocessing using an image processing library (e.g., OpenCV) to remove noise, resize, and correct color. It then applies a machine learning model (e.g., TensorFlow) to extract clothing features (e.g., color, pattern, shape, season) from the images.

[1503] 3. Comparison with trend data

[1504] The server compares the extracted clothing characteristics with a trend database, which contains the latest fashion information and influencer postings. Based on the comparison results, the server generates optimal clothing combinations and suggests them to the user.

[1505] 4. Registering the user in the clothing database

[1506] Users use a dedicated app to register information about the clothes they own. Information about each piece of clothing can be automatically recognized through image analysis or can be entered manually. The server stores this information in a database and analyzes how newly registered clothes can be combined with existing clothes. The system then provides optimal outfits according to the season and situation.

[1507] 5. Viewing Ads and Purchasing Online

[1508] The server retrieves advertising data for stores that carry similar products based on the characteristics of the user's clothing. The acquired advertising data is sent to the user's device and displayed through a dedicated app. The user can click on an advertisement that interests them, access the product page in the online shop, and purchase the item.

[1509] Specific examples

[1510] For example, a case will be described in which user A purchases a new red floral dress.

[1511] 1. User A takes a photo of a red floral dress with her smartphone and uploads it to the server using a dedicated app.

[1512] 2. The server receives the image, preprocesses it using an image processing library (e.g., OpenCV), and then uses a machine learning model (e.g., TensorFlow) to extract the color (red), pattern (floral), and shape (dress) of the dress.

[1513] 3. The server compares the extracted features with trend data and suggests outfits that combine them with, for example, white sandals and a denim jacket.

[1514] 4. User A has registered other clothes he owns (e.g., white shirts and black skirts) in the app, and suggestions for combinations with these items are also displayed.

[1515] 5. The server displays an advertisement for a store that sells products similar to the red floral dress, and User A can purchase the product from the online shop.

[1516] Prompt Sentence Examples

[1517] "Generate a sentence that explains the system that receives an image of an outfit from the user, analyzes the image, suggests the best outfit, and advertises similar products."

[1518] By configuring it in this way, the system not only helps users efficiently manage their clothes and find the best outfits, but also allows them to easily purchase related new clothes online.

[1519] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1520] Step 1:

[1521] The user takes a photo of the clothes with their smartphone and uploads the image through a dedicated app. The input is the image of the clothes taken by the user, and the output is image data sent to the server. The device uses its camera function to acquire the image and sends it to the server via the Internet.

[1522] Step 2:

[1523] The server saves the image data received from the terminal. The input is the image data sent from the terminal, and the output is the saved image data. Specifically, the server stores the received image data in storage and prepares it for subsequent analysis processing.

[1524] Step 3:

[1525] The server preprocesses the received images using an image processing library (e.g., OpenCV). The input is the stored image data, and the output is the preprocessed image data. Specifically, it performs processes such as noise removal, resizing (e.g., to 256x256 pixels), and color correction.

[1526] Step 4:

[1527] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from preprocessed images. The input is the preprocessed image data, and the output is extracted clothing feature data (e.g., color, pattern, shape, season). Specifically, the image is input into the machine learning model, and clothing features are obtained from the output.

[1528] Step 5:

[1529] The server compares the extracted clothing feature data with a trend database. The input is the clothing feature data and the trend database, and the output is optimal clothing combination suggestion data. The server refers to the trend database and calculates and generates the optimal outfit for the user.

[1530] Step 6:

[1531] The server generates optimal outfit combinations and sends that information to the user's device. The input is the optimal outfit combination suggestion data, and the output is the suggestion information sent to the user's device. Specifically, when the user opens the app, the suggested outfits are displayed.

[1532] Step 7:

[1533] The user registers information about their clothing using a dedicated app. The input is the clothing information entered or photographed by the user, and the output is the registration information sent to the server. The device acquires this information and sends it to the server.

[1534] Step 8:

[1535] The server stores the user's clothing information in a database. The input is the registered clothing information, and the output is the stored database. The server compares the user's new clothing data with existing data and analyzes the combinations.

[1536] Step 9:

[1537] The server retrieves advertising data for stores that sell similar products based on the characteristics of the user's clothing. The input is the characteristic data of the user's clothing, and the output is the retrieved advertising data. The server then refers to the advertising service to collect advertising information for related products.

[1538] Step 10:

[1539] The server sends the acquired advertising data to the user's device. The input is the collected advertising data, and the output is advertising information sent to the user's device. Users can view these advertisements through a dedicated app and purchase products they are interested in online.

[1540] (Application example 1)

[1541] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1542] Conventional clothing coordination suggestion systems have difficulty easily linking information about the clothes a user owns with the latest trends, and have also faced challenges in efficiently finding and purchasing similar products. Furthermore, they lacked personalized suggestions tailored to the user's preferences and requirements, requiring users to expend a great deal of time and effort. Our goal is to provide a system that solves these issues, allowing users to more easily and instantly find appropriate coordinations and purchase similar products.

[1543] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1544] In this invention, the server includes means for taking images of clothes and uploading them to a remote server, means for the remote server to analyze the images and extract clothing characteristics, means for comparing the clothing characteristics with trend data and presenting optimal clothing combinations, means for registering information about clothes owned by the user and proposing combinations according to seasons and events, means for displaying advertisements for stores that carry similar clothes and enabling online purchases, means for proposing new clothes using a generative AI model based on optimal coordination, and means for proposing clothes based on specified requirements by inputting a prompt text. This allows users to not only easily analyze and manage the characteristics of their own clothes, but also enables efficient coordination suggestions linked to the latest trend information and allows them to quickly find and purchase similar products.

[1545] "Means for taking images of clothes and uploading them to a remote server" refers to a function that allows a user to take an image of clothes using a device such as a smartphone or tablet, and then send the image data to a remote server via the Internet.

[1546] "Means for the remote server to analyze the image and extract characteristics of the clothing" refers to a function in which the remote server analyzes the image data received and extracts characteristics such as the color, pattern, shape, and seasonality of the clothing.

[1547] The "means for comparing trend data with clothing characteristics and presenting optimal clothing combinations" is a function that compares extracted clothing characteristics with the latest trend data and presents optimal outfits to the user.

[1548] "A means for users to register information about the clothes they own and suggest combinations appropriate for the season or event" is a function that allows users to register information about the clothes they own in the system and then suggests outfits appropriate for the season or event based on that information.

[1549] "Means for displaying advertisements for stores that sell similar clothing and enabling online purchases" is a function that displays advertisements for stores that sell similar products based on the characteristics of the clothing registered by the user, allowing the user to purchase directly from those stores.

[1550] "Means for proposing new clothing using a generative AI model based on optimal coordination" is a function that uses a generative AI model to suggest new clothing based on extracted clothing characteristics and optimal coordination information.

[1551] "A means for suggesting clothing based on specified requirements by inputting a prompt sentence" is a function that allows the user to input specific requirements or preferences as a prompt sentence and suggests appropriate clothing based on that information.

[1552] The system of the present invention allows users to take photos of their own clothes using a smartphone or tablet, find optimal outfits based on the photos, and then help them purchase similar clothes online. The specific configuration and operation of this system are described in detail below.

[1553] System configuration

[1554] 1. Device (smartphone, tablet, etc.):

[1555] It has a camera function for taking images and a function for connecting to the Internet and communicating with a remote server.

[1556] Using a dedicated application, users can take photos of their clothes and upload them to a remote server.

[1557] 2. Remote Server:

[1558] Receives and analyzes image data, and manages user data.

[1559] An image processing library (e.g., OpenCV) is used to preprocess and analyze the images, and a machine learning model (e.g., TensorFlow) is used to extract clothing features.

[1560] The system refers to a trend database and compares the extracted clothing characteristics to suggest the optimal outfit.

[1561] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[1562] The advertisement data of stores that sell similar products is acquired and displayed to the user.

[1563] Based on the optimal coordination, new clothing suggestions are made using a generative AI model.

[1564] By entering a prompt, clothing suggestions will be made based on the specified requirements.

[1565] How it works

[1566] 1. Take and upload your outfit image:

[1567] The user takes a photo of the new or currently worn clothes using a smartphone, and then uploads the image to a remote server using a dedicated application.

[1568] 2. Image Receipt and Analysis:

[1569] The remote server receives the uploaded images and performs preprocessing (noise removal, resizing, color correction, etc.) using an image processing library (OpenCV), then extracts clothing features (color, pattern, shape, season, etc.) using a machine learning model (TensorFlow).

[1570] 3. Comparison with trend data:

[1571] The extracted clothing characteristics are compared with a trend database, the latest fashion information, and influencer posts. Based on the comparison results, optimal clothing combinations are generated and recommended to the user.

[1572] 4. Registering the user in the clothing database:

[1573] Users register information about their clothing using a dedicated app. The registered information is stored in a database on a remote server. The server then analyzes how the newly registered clothing can be combined with existing clothing to suggest optimal outfits for each season and situation.

[1574] 5. Viewing Ads and Online Purchases:

[1575] The remote server retrieves advertising data for stores that sell similar products based on the extracted clothing characteristics. The acquired advertising data is sent to the user's device and displayed through a dedicated app. Users can click on an advertisement that interests them and purchase the product from the online shop.

[1576] 6. New clothing suggestions using generative AI models:

[1577] Based on the extracted clothing features and optimal coordination information, a generative AI model is used to suggest new clothing to the user. For example, if a user takes a photo of a red floral dress, the system will suggest new clothing designs based on those features.

[1578] 7. Specifying requirements by prompt sentence:

[1579] Users can specify their specific requirements and preferences by entering prompts in a dedicated application, and the server will then suggest clothing that matches the requirements.

[1580] Specific examples

[1581] Consider the case where User A purchases a new red floral dress, takes a photo of it with her smartphone, and uploads it to the server using a dedicated app. The server analyzes the image and extracts the color (red), pattern (floral), and shape (dress). It then compares the extracted features with a trend database and suggests combinations, such as white sandals and a denim jacket. Furthermore, if User A enters the prompt "casual outdoor style," the server uses the generative AI model to suggest new clothing based on those requirements.

[1582] Prompt Sentence Examples

[1583] text

[1584] Analyze an image of a red floral dress and suggest the best outfit for it. Also, show similar items from an online shop.

[1585] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1586] Step 1:

[1587] The device takes pictures of the clothes the user is wearing or newly purchased clothes. When the user clicks the capture button in the dedicated app, the device's camera function is activated and the image is captured. The input is the image data captured by the camera, and the output is an image file stored on the device.

[1588] Step 2:

[1589] The device uploads the captured images to a remote server via a dedicated app. When the user clicks the upload button, the image data is sent to the remote server via the Internet. The input is the image file in the device, and the output is the image data sent to the remote server.

[1590] Step 3:

[1591] The remote server stores the received image data and performs preprocessing on the image using an image processing library (e.g., OpenCV). Specifically, it performs processes such as noise reduction, resizing, and color correction. The input is the image data uploaded to the server, and the output is the image data after preprocessing.

[1592] Step 4:

[1593] The remote server inputs the preprocessed image data into a machine learning model (e.g., TensorFlow) to extract clothing features. Features include color, pattern, shape, season, etc. The input is the preprocessed image data, and the output is the extracted clothing feature data.

[1594] Step 5:

[1595] The remote server compares the extracted clothing features with a trend database, referring to the latest fashion information and influencer posts to generate optimal clothing combinations. The input is clothing feature data and trend data, and the output is optimal clothing combination data.

[1596] Step 6:

[1597] The remote server retrieves information about clothing registered by the user using a dedicated app from a database and analyzes how the newly registered clothing can be combined with existing clothing. As a specific example, it suggests optimal outfits according to the season or event. The input is the user's clothing data, and the output is combination data according to the season or event.

[1598] Step 7:

[1599] The remote server retrieves advertising data for stores that sell similar products based on the extracted clothing features. The advertising data is provided by online shops and displayed to users. The input is clothing feature data and store advertising data, and the output is advertising data displayed on the user's device.

[1600] Step 8:

[1601] The remote server uses a generative AI model based on the optimal outfit to suggest new outfits. The generative AI model uses deep learning, for example. The input is optimal outfit data, and the output is new outfit suggestion data.

[1602] Step 9:

[1603] The user inputs a prompt on a dedicated application and sends specific requirements and preferences to the server, which then suggests clothes that fit the specified requirements based on the prompt. The input is the prompt from the user, and the output is the suggested clothing data.

[1604] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1605] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the images, and assist users in purchasing similar clothes online. By combining it with an emotion engine, the system suggests the best clothing combinations and advertisements according to the user's emotions. The specific configuration and operation of this system will be described below.

[1606] System configuration

[1607] 1. Device:

[1608] A device such as a smartphone or tablet used by a user.

[1609] It has a camera function for taking pictures and a function for connecting to the Internet and communicating with a server.

[1610] It has the ability to capture the user's facial expressions and voice.

[1611] 2. Server:

[1612] This is a remote server that receives and analyzes image data and manages user data.

[1613] An image processing library is used to preprocess and analyze images, and a machine learning model is used to extract clothing features.

[1614] It refers to a trend database and suggests optimal outfits to users.

[1615] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[1616] The advertising data of stores that sell similar products is acquired and displayed to the user.

[1617] It uses an emotion engine to analyze user emotions and tailor suggestions and advertisements.

[1618] Program processing

[1619] When a user takes a photo of an outfit using the camera on their smartphone or tablet, opens the app, and taps the "Upload New Outfit" button, the image is sent to the server. The device retrieves the image and executes an HTTP request.

[1620] The server temporarily stores the received image files and performs preprocessing using an image processing library (e.g., OpenCV). An appropriate analysis state is generated by resizing, noise removal, color correction, etc. Next, a machine learning model (e.g., TensorFlow) is used to extract features such as the color, pattern, shape, and season of the clothing. This results in an analysis result such as "a dress with a red floral pattern."

[1621] The server compares the extracted features with a trend database, which includes data from magazines and social media, to generate optimal outfits based on the latest fashion information and influencer posts.

[1622] The server then sends the generated outfit suggestions to the user's device, where the user can receive the suggestions via the app. For example, a specific suggestion such as "pair a red floral dress with white sandals" is made.

[1623] Users can use a dedicated app to register information about the clothes they own. Registration methods include automatic recognition (image analysis) and manual input (color, pattern, shape, etc.), making it easy to add clothing information to the database. The server saves the registered clothing information in the database and analyzes how newly added clothes can be combined with existing clothes to make optimal suggestions.

[1624] The server collects advertising data from stores that sell similar products based on the user's clothing characteristics. It then selects relevant ads through advertising networks and partnerships and displays them to the user. When the user clicks on an ad that interests them, they are redirected to the product page in the online shop, where they can view product details and make a purchase.

[1625] Introducing the Emotion Engine

[1626] The system incorporates an emotion engine, and the following specific processes are added:

[1627] The emotion engine analyzes the user's facial expressions and voice to identify their current emotions. For example, it uses a camera to collect the user's facial expression data and uses voice recognition technology to analyze the user's speech.

[1628] The server uses the analysis results obtained from the emotion engine to suggest optimal outfits and advertisements according to the user's emotions. For example, if the user is happy, it will suggest bright-colored clothes, and if they are depressed, it will suggest comfortable, relaxing clothes. It is also possible to dynamically change relevant product advertisements according to the user's emotions.

[1629] Specific examples

[1630] For example, consider the case where User B purchases a new blue denim dress, takes a photo of it, and uploads it to the server.

[1631] 1. User B takes a photo of a denim dress with their smartphone and uploads it to the server using a dedicated app.

[1632] 2. The server receives the image, preprocesses it using an image processing library, and then uses a machine learning model to extract the color (blue), pattern (denim), and shape (dress) of the dress.

[1633] 3. The server compares the extracted features with a trend database and suggests outfits, such as "combining white sneakers with a red bag."

[1634] 4. If user B is found to be happy through the emotion engine, suggest more casual and brightly colored items.

[1635] 5. User B has already registered a white shirt and a black skirt in the app, and suggestions for combinations with them are also displayed.

[1636] 6. The server displays an advertisement for a store that sells a product similar to the blue denim dress, and User B purchases the product from an online shop.

[1637] In this way, the system of the present invention can dynamically respond to the user's emotions and can also provide personalized coordination and advertising suggestions.

[1638] The processing flow will be explained below.

[1639] Step 1:

[1640] The user takes a photo of the clothes using the camera on their smartphone or tablet. After taking the photo, they open the dedicated app and tap the "Upload New Clothes" button. This action causes the device to import the image into the app and prepare it for transmission to the server.

[1641] Step 2:

[1642] The device retrieves the captured image and makes an HTTP request to an API endpoint to upload the image file to the server, including the image data and metadata such as the user ID.

[1643] Step 3:

[1644] The server temporarily stores the received image files. Next, it performs preprocessing on the images using an image processing library (e.g., OpenCV). Specifically, it performs image resizing, noise removal, color correction, etc. This generates an image suitable for analysis.

[1645] Step 4:

[1646] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image. This model is pre-trained and can identify clothing features such as color, pattern, shape, and season. The analysis results are expressed as, for example, "a red floral dress."

[1647] Step 5:

[1648] The server compares the extracted features with a trend database, which stores data on the latest fashion magazines, influencer posts, and popular items from online shops. Using a comparison algorithm, the server generates optimal outfits that match current trends.

[1649] Step 6:

[1650] The server sends the generated outfit suggestions to the device. When the user opens the app, the outfit suggestions are displayed. For example, specific suggestions such as "Pair a red floral dress with white sandals" are made.

[1651] Step 7:

[1652] Users can register information about the clothes they own using a dedicated app. Both automatic recognition (image analysis) and manual input (color, pattern, shape, etc.) are available, making it easy for users to register information about their clothes.

[1653] Step 8:

[1654] The server stores the clothing information sent by the user in a database and analyzes combinations based on existing clothing information and newly added information, thereby generating coordination suggestions according to the season and situation.

[1655] Step 9:

[1656] The device captures the user's facial expressions and voice and sends them to an emotion engine, which uses a model trained on a dataset for emotion analysis to analyze the user's emotions in real time.

[1657] Step 10:

[1658] The emotion engine analyzes the user's emotions and sends the results to the server. For example, it responds when the user is judged to be happy or depressed.

[1659] Step 11:

[1660] The server uses the emotional data sent from the emotion engine to adjust the recommended outfits and advertisements. For example, if the user is happy, it will suggest brightly colored clothes and casual items, and if they are depressed, it will suggest relaxing clothes. It also selects the most appropriate advertisement based on the user's emotions and sends it to the user's device.

[1661] Step 12:

[1662] The device displays coordination suggestions and advertisements to the user based on the emotions received from the server. Because the suggestions displayed are suited to the user's emotions, the user can receive more personalized services.

[1663] Through the above processing steps, the user can receive suggestions for optimal clothing management and coordination based on their emotions, and can also easily purchase related products online.

[1664] Example 2

[1665] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1666] Conventional coordination support systems were unable to provide suggestions based on the user's emotions and mood regarding clothing, and were unable to sufficiently increase user satisfaction. Furthermore, when purchasing similar products online, advertisements displayed did not dynamically change based on the user's emotions and preferences, preventing effective marketing. Furthermore, it was difficult to provide suggestions for combining outfits with the user's existing clothes to suit the season or specific situations.

[1667] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for taking an image of clothes and uploading it to the server, a means for the server to analyze the image and extract characteristics of the clothes, a means for comparing the characteristics of the clothes with trend data and recommending optimal clothing combinations, a means for registering information on clothes owned by the user and proposing combinations according to the season and situation, a means for displaying advertisements for stores that carry similar clothes and enabling online purchases, and a means for analyzing the user's emotions using an emotion analysis engine and adjusting optimal clothing combinations and advertisements based on the emotions. This makes it possible to suggest coordinations and display advertisements that match the user's emotions and mood.

[1668] "Clothing image" refers to photographic data of clothing that the user will wear.

[1669] "Means for uploading" refers to the ability for a user to send images taken by the user to a remote server via the Internet.

[1670] "Means for analyzing images and extracting clothing features" refers to the process of identifying and extracting attributes such as clothing color, pattern, and shape from the received image data.

[1671] "Trend data" refers to data that reflects the latest fashion information and market trends.

[1672] "Means of recommending optimal clothing combinations" refers to a function that suggests outfits that suit the user based on trend data and clothing characteristics.

[1673] "Means for registering information about clothes owned by the user" refers to a function that allows a user to input or read data about clothes they own into the system.

[1674] "Means of suggesting combinations according to the season and situation" refers to a function that suggests clothing combinations that suit the season, specific events, or occasions.

[1675] "Means of displaying advertisements for stores that sell similar clothing and enabling online purchases" refers to a function that displays advertisements for stores that sell products similar to the clothing photographed by the user and allows the user to purchase those products online.

[1676] An "emotion analysis engine" refers to a system that analyzes a user's facial expressions and voice to identify their current emotional state.

[1677] "Means for adjusting optimal clothing combinations and advertisements based on emotions" refers to a function that dynamically changes coordination suggestions and advertisement content to match the identified user's emotions.

[1678] The system of the present invention allows users to take photos of their own clothes, find the best outfits based on the photos, and help them purchase similar clothes online. Furthermore, by combining it with an emotion engine, the system suggests the best clothing combinations and advertisements based on the user's emotions.

[1679] System configuration

[1680] 1. Device:

[1681] A device such as a smartphone or tablet used by a user.

[1682] It has a camera function for taking pictures and the ability to connect to the Internet and communicate with a server.

[1683] It has the ability to capture the user's facial expressions and voice.

[1684] 2. Server:

[1685] This is a remote server that receives and analyzes image data and manages user data.

[1686] Specifically, the image is preprocessed using an image processing library (e.g., OpenCV), and clothing features are extracted using a machine learning model (e.g., TensorFlow).

[1687] The system references a trend database and generates the optimal coordination.

[1688] It manages the user's clothing information registered in a database and suggests combinations based on that information.

[1689] Advertising data is collected and displayed to the user.

[1690] It uses an emotion engine to analyze user emotions and tailor suggestions and advertisements.

[1691] Example of operation

[1692] 1. The user takes a photo of their clothes using the device's camera.

[1693] Example: A user takes a picture of a new blue denim dress.

[1694] 2. Upload the images taken by the device to the server using a dedicated app.

[1695] 3. The server receives the images and performs preprocessing using an image processing library (e.g., OpenCV).

[1696] Pre-processing includes resizing, noise reduction, color correction, etc.

[1697] 4. The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image.

[1698] Example: Extract features such as color (blue), pattern (denim), and shape (dress) from a blue denim dress.

[1699] 5. The server compares the extracted features with a trend database and suggests optimal outfits.

[1700] Example: A specific coordination suggestion such as "pair it with white sneakers and a red bag."

[1701] 6. The server sends the generated coordination proposal to the user's device.

[1702] 7. The device displays the suggestions to the user through a dedicated app.

[1703] For example, you might see suggestions like "Pair a blue denim dress with white sneakers."

[1704] 8. The user uses a dedicated app to register information about the clothes they own.

[1705] Example: Register information such as a white shirt or black skirt you own.

[1706] 9. The server analyzes the combination of new clothes with existing clothes based on the registered clothing information and makes optimal suggestions.

[1707] For example: a blue denim dress paired with a white shirt.

[1708] 10. The server collects advertising data from stores that sell similar products and displays it to the user.

[1709] 11. The device displays advertisements, and when the user clicks on an advertisement that interests them, they are redirected to the product page of the online shop.

[1710] Example: You see products similar to a blue denim dress and can buy them online.

[1711] Prompt Sentence Examples

[1712] "Suggest outfits that go well with the blue denim dress the user recently purchased. Also, if the user is happy with the outfit, suggest items in brighter colors."

[1713] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1714] Step 1:

[1715] The user takes a photo of the clothes using the device's camera.

[1716] Input: The user activates the camera and takes a picture of the clothes.

[1717] Output: The captured image data is saved on the device.

[1718] What happens: A user takes a picture of a new blue denim dress with their smartphone camera.

[1719] Step 2:

[1720] The device uploads the images taken to the server using a dedicated app.

[1721] Input: Image data stored on the device.

[1722] Output: An HTTP POST request containing the image data is sent to the server.

[1723] Specific operation: The user taps the "Upload new clothes" button, and the device sends image data to the server.

[1724] Step 3:

[1725] The server temporarily stores the received image files and performs preprocessing using an image processing library (e.g., OpenCV).

[1726] Input: Image data received by the server.

[1727] Output: Preprocessed image data.

[1728] Data processing or data calculation: image resizing, noise reduction, color correction, etc.

[1729] What happens: The server resizes the image, removes noise, and performs color correction.

[1730] Step 4:

[1731] The server uses a machine learning model (e.g., TensorFlow) to extract clothing features from the image.

[1732] Input: Preprocessed image data.

[1733] Output: Extracted clothing feature data (e.g. color, pattern, shape).

[1734] Data processing or data calculation: Identifying color, pattern, and shape from image data and generating feature data.

[1735] Specific operation: The server extracts the color (blue), pattern (denim), and shape (dress) of the blue denim dress.

[1736] Step 5:

[1737] The server compares the extracted features with a trend database and generates the optimal outfit.

[1738] Input: Extracted clothing feature data.

[1739] Output: Optimal outfit suggestion data.

[1740] Data processing or data calculation: Refer to the feature data and trend database to determine the optimal coordination.

[1741] Specific behavior: The server generates suggestions such as "pair with white sneakers and a red bag."

[1742] Step 6:

[1743] The server transmits the generated coordination proposal to the user's terminal.

[1744] Input: Coordination suggestion data.

[1745] Output: The proposal data sent to the user's device.

[1746] Specific operation: The server sends the generated coordination proposal to the user's device.

[1747] Step 7:

[1748] The coordinated suggestions received by the terminal are displayed to the user via a dedicated app.

[1749] Input: The coordinate suggestion data passed in.

[1750] Output: The suggestions displayed in the app.

[1751] Specific behavior: The device displays a suggestion on the app screen: "Pair a blue denim dress with white sneakers."

[1752] Step 8:

[1753] Users register information about the clothes they own using a dedicated app.

[1754] Input: Clothing information entered or uploaded by the user.

[1755] Output: Clothing information from the registered database.

[1756] Specific operation: The user enters information about a white shirt and a black skirt they own into the app and registers them.

[1757] Step 9:

[1758] Based on the registered clothing information, the server analyzes the combination of new clothing with existing clothing and makes optimal suggestions.

[1759] Input: Registered clothing information.

[1760] Output: Combination suggestion data of newly added clothes and existing clothes.

[1761] Data processing or data calculation: Combining and analyzing existing clothing information with new clothing information.

[1762] What it does: The server suggests a combination of a blue denim dress and a white shirt.

[1763] Step 10:

[1764] The server collects advertising data from stores that sell similar products and transmits it to the user's terminal.

[1765] Input: Registered clothing feature data.

[1766] Output: Collected advertising data.

[1767] What it does: The server collects relevant ads through ad networks.

[1768] Step 11:

[1769] The advertisements received by the device are displayed to the user via a dedicated app.

[1770] Input: Ad data sent by the server.

[1771] Output: The ad displayed within the app.

[1772] What happens: The device displays an ad for a product similar to the blue denim dress on the app screen.

[1773] Step 12:

[1774] The emotion engine analyzes the user's facial expressions and voice to identify their current emotion.

[1775] Input: Facial expression and audio data captured by the camera.

[1776] Output: Parsed emotion data.

[1777] Data processing or data calculation: Performing facial expression recognition and voice analysis to identify emotions.

[1778] Specific operation: Collects the user's facial expression data using a camera and analyzes the user's speech using voice recognition technology.

[1779] Step 13:

[1780] Based on the analysis results obtained from the emotion engine, the server adjusts the optimal coordination and advertisements according to the user's emotions.

[1781] Input: Parsed emotion data.

[1782] Output: Reconciled coordinate and advertising data.

[1783] Data processing or data calculation: Dynamically changing outfits or advertising content based on emotional data.

[1784] What it does: If the user is happy, suggest brighter colored items and show more relevant ads.

[1785] (Application example 2)

[1786] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1787] Until now, when users purchased clothes in a physical store, there was no way to instantly get suggestions on how to coordinate them with the clothes they already own or the best combinations that match their current mood. There is also a demand for personalized suggestions based on the user's emotions to provide a more comfortable shopping experience. This will increase product purchases in physical stores and improve customer satisfaction.

[1788] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for taking an image of clothes and uploading it to the server, means for the server to analyze the image and extract characteristics of the clothes, means for comparing the characteristics of the clothes with trend data and recommending optimal clothing combinations, means for registering information about clothes owned by the user and proposing combinations according to the season and situation, means for displaying advertisements for stores that carry similar clothes and enabling online purchases, and means for analyzing the user's facial expressions and voice to identify emotions and dynamically adjusting coordinations and advertisements based on the emotions. This allows users to take photos of clothes they like in a physical store, receive optimal coordination suggestions in real time, and enjoy a personalized shopping experience according to their emotions.

[1789] "Clothing images" are photos of clothing taken by the user using the camera function.

[1790] "Server" means a remotely located computer system that receives and processes data sent by users and returns analytical results.

[1791] "Clothing characteristics" are visual and physical attributes of clothing such as color, pattern, shape, and season.

[1792] "Trend Data" is a database containing information about current trends and popular styles.

[1793] The "optimal clothing combination" is the most appropriate fashion coordination for the user, generated based on the analyzed clothing characteristics and trend data.

[1794] "User information" refers to personal information such as data on the clothes owned by the user, their preferences, and purchase history.

[1795] "Season" refers to a specific season such as spring, summer, autumn, or winter, and is a standard for selecting clothing appropriate for each season.

[1796] "Situation" refers to a specific event, purpose, weather, or other environmental factors, and is the condition for choosing appropriate clothing.

[1797] "Store advertising" refers to promotional information for similar or related products sold in brick-and-mortar stores or online shops.

[1798] "Online shopping" is a shopping method that allows you to order and pay for products over the Internet.

[1799] "Facial expression" is an element that indicates the expression or emotion of the user's face.

[1800] "Voice" refers to the words and tone of voice spoken by the user, and is information that is part of emotion analysis.

[1801] "Emotion" refers to a user's psychological state, such as happiness, sadness, excitement, or depression.

[1802] "Identifying emotions" means identifying the user's psychological state at that time from their facial expressions and voice.

[1803] "Dynamic adjustment" means automatically changing and optimizing coordination and advertising content in real time based on the analysis results.

[1804] An embodiment of the present invention will now be described. The present invention is a system that allows a user to take a photo of clothes they like in a physical store and suggests optimal outfits based on the image and emotional information.

[1805] Hardware and Software Configuration

[1806] Device: A smartphone used by a user, equipped with a camera and internet connection, and capable of capturing the user's facial expressions and voice.

[1807] server:

[1808] Receives and analyzes image data and audio data, and manages user data.

[1809] Preprocess the images using an image processing library (OpenCV).

[1810] A machine learning model (TensorFlow) is used to extract clothing features.

[1811] We use an emotion analysis engine (Microsoft Azure Emotion API) to identify emotions from the user's facial expressions and voice through emotion analysis.

[1812] The system compares the outfit with a trend database to generate the optimal outfit.

[1813] Advertising data for similar products is collected and displayed to the user.

[1814] Database: MongoDB is used to store user clothing information and trend data.

[1815] DETAILED DESCRIPTION OF THE EMBODIMENTS

[1816] Program processing:

[1817] 1. The user takes a photo of the clothes using their smartphone camera, opens the app, and taps the "Upload new clothes" button. The image is sent to the server.

[1818] 2. The server temporarily stores the received image files and performs preprocessing using an image processing library (OpenCV), such as resizing, noise reduction, and color correction.

[1819] 3. Next, the server uses a machine learning model (TensorFlow) to extract features such as the color, pattern, shape, and season of the clothing, resulting in an analysis result such as "a red floral dress."

[1820] 4. The server compares the extracted features with a trend database and generates the optimal outfit based on the latest fashion information.

[1821] 5. The server sends the generated outfit suggestions to the user's device, and the user receives the suggestions via the app. The user's facial expressions and voice are also captured, and their emotions are identified by an emotion analysis engine.

[1822] 6. The server then uses the results of the emotion analysis to suggest outfits and advertisements that match the user's emotions. For example, if the user is happy, it will suggest bright-colored clothes, and if they are depressed, it will suggest relaxing clothes.

[1823] 7. It collects advertising data from stores that sell similar products and displays ads that users may be interested in. When clicked, users are redirected to the product page of the online shop.

[1824] Specific examples

[1825] For example, user A can take a photo of a red floral dress they like in a physical store and upload it to the server using a dedicated app. The server performs image and emotional analysis, and if it determines that user A is pleased, it will suggest more casual and brightly colored items (such as white sandals or a yellow bag). It will also display related products available in the store, allowing the user to try them on and purchase them immediately.

[1826] Example prompt for a generative AI model:

[1827] 1. Prompt for clothing feature extraction:

[1828] "Please extract the color, pattern, shape, and season of the clothes in this image."

[1829] 2. Prompts for sentiment analysis:

[1830] "Analyze the user's facial features from this image and identify their current emotion."

[1831] This allows users to enhance their in-store shopping experience and receive more personalized offers.

[1832] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1833] Step 1:

[1834] A user takes a photo of an outfit using their smartphone camera, opens the app, and taps the "Upload New Outfit" button. The input is the photo of the outfit taken by the user, and the output is the photo file being sent to the server. At this time, the app executes an HTTP request to send the image to the server. Specifically, the operation involves taking a photo using the smartphone's camera function and uploading the image file from within the app.

[1835] Step 2:

[1836] The server temporarily stores the image files it receives and performs preprocessing using an image processing library (OpenCV). The input is the image file of the clothes sent to the server, and the output is the preprocessed image data. This preprocessing includes resizing, noise removal, color correction, etc. Specifically, the server receives the images and performs resizing and noise removal using OpenCV.

[1837] Step 3:

[1838] The server uses a machine learning model (TensorFlow) to extract features such as the color, pattern, shape, and season of the clothing. The input is preprocessed image data, and the output is extracted clothing feature data. Specifically, analysis is performed using the TensorFlow model to extract specific features such as "a dress with a red floral pattern."

[1839] Step 4:

[1840] The server compares the extracted features with a trend database and generates optimal outfits based on the latest fashion information. The input is the extracted clothing feature data, and the output is optimal outfit suggestions. Specifically, a matching algorithm with the trend database is used to perform processing to suggest appropriate combinations.

[1841] Step 5:

[1842] The server sends the generated outfit suggestions to the user's device, and the user receives the suggestions via the app. The input is the optimal outfit suggestion, and the output is the display of outfit information on the user's device. Specifically, this process sends a notification to the app and displays the suggestion content.

[1843] Step 6:

[1844] The system captures the user's facial expressions and voice and identifies their emotions using an emotion analysis engine (Microsoft Azure Emotion API). The input is the user's facial and voice data, and the output is analyzed emotional information. Specifically, the system captures facial expressions and voice using the smartphone's camera and microphone, and identifies emotions through the Azure Emotion API.

[1845] Step 7:

[1846] The server then uses the results of the emotion analysis to suggest outfits and advertisements that match the user's emotions. The input is the analyzed emotion information and clothing feature data, and the output is emotion-based outfit and advertisement suggestions. Specifically, if the user is happy, it suggests brightly colored items, and if they are depressed, it suggests relaxing clothes.

[1847] Step 8:

[1848] The server collects advertising data for similar products and displays it to the user. The input is advertising data related to clothing feature data, and the output is the display of advertisements on the user's device. Specifically, the process involves collecting related products through an advertising network and displaying advertisements that the user is likely to be interested in on the app.

[1849] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1850] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1851] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1852] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1853] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1854] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1855] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1856] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1857] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1858] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1859] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1860] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1861] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory s...

Claims

1. A means to take pictures of the clothes and upload them to a server; A server analyzes the image and extracts the characteristics of the clothing. A method to compare trend data with clothing characteristics and recommend optimal clothing combinations, A way for users to register information about their clothing and have the app suggest combinations that suit the season and situation. A means to display advertisements for stores that carry similar clothing and enable online purchases; A system including:

2. 10. The system of claim 1, wherein the system uses an image processing library and a machine learning model to analyze images of clothing and extract color, pattern, and shape.

3. 2. The system of claim 1, further comprising trend data acquisition means for comparing the extracted clothing features with a trend database.

Images