system

Abstract

We provide the system. [Solution] A system comprising: means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple brands; means for the server to learn the acquired product data; means for the server to acquire the user's behavior history; means for the server to analyze preferences and trends based on the user's behavior history; means for the server to generate outfits based on the user's conditions; means for the server to create images of the generated outfits; means for the terminal to present the generated outfits to the user; means for the terminal to receive feedback from the user; and means for the server to generate outfits again based on the feedback.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional coordination systems are often limited to providing coordination proposals within a single brand and have difficulty flexibly meeting diverse customer needs and preferences. Also, data analysis for eliciting potential customer desires has been insufficient, resulting in a possible decline in customer satisfaction. Therefore, there is a need to develop a system that integrates products of multiple brands and utilizes user behavior history and feedback to propose more accurate coordination.

Means for Solving the Problems

[0005] This invention provides a system that uses user-entered profile information, preferences, and budget to enable a server to acquire product data from multiple brands, learn from this data, and then make product suggestions tailored to the user's needs. Furthermore, it acquires the user's behavioral history through the LINE interface and analyzes the user's preferences and tendencies based on this data. The server generates an optimal outfit based on the user's conditions and sends the image to the user's terminal. The user can review the outfit through the terminal and send feedback. The server can then regenerate the outfit based on the feedback, enabling it to make more accurate suggestions. This improves customer satisfaction and revitalizes the customer base of both brands and department stores.

[0006] "Profile information" refers to basic user information, including name, age, gender, and other information used to identify an individual.

[0007] "Preferences" refer to information about a user's personal tastes, such as preferred styles, colors, and brands.

[0008] "Budget" refers to the maximum amount of money a user is willing to spend on a purchase.

[0009] A "server" refers to a computer system that acts as a central processing unit, performing tasks such as acquiring, learning, storing, analyzing, and generating various types of data.

[0010] "Product data" refers to information about a product, including data such as product name, image, price, category, and features.

[0011] "Learning" refers to the process of using machine learning algorithms to extract product characteristics from acquired product data and then performing data analysis based on that.

[0012] The "LINE interface" refers to a communication method for acquiring user behavior history via the external system LINE and facilitating integration through a data cleanroom.

[0013] "Behavioral history" refers to recorded data about a user's actions, including their past search history and purchase history.

[0014] "User conditions" refer to the purchasing requirements entered by the user, such as budget, preferences, and style.

[0015] "Coordination" refers to a fashion or style suggestion created by combining multiple items.

[0016] "Image generation" refers to the process of creating images that visually represent the coordinated outfits based on the selected products.

[0017] "Device" refers to a device actually used by the user, and includes personal computers, smartphones, tablets, etc.

[0018] "Feedback" refers to opinions and requests regarding suggestions provided by users.

[0019] "Regenerating the outfit" refers to the process by which the server, having received feedback, creates a new outfit that reflects the user's new opinions and requests. [Brief explanation of the drawing]

[0020] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5]It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0021] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0022] First, the language used in the following description will be explained.

[0023] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0024] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0025] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0026] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0028] [First Embodiment]

[0029] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0030] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0033] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0036] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0040] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0041] This invention is a system that generates and proposes the optimal outfit to a user based on their profile information, preferences, and budget. Specifically, the server acquires product data from multiple brands, learns from it using a machine learning algorithm, and analyzes it based on the user's behavior history and feedback data to generate the optimal outfit for the user's conditions. The following describes the processing of this system's program in natural language.

[0042] Program processing flow

[0043] 1. User Registration

[0044] Device: Users create an account using a dedicated application or website. Here, users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[0045] User: Follow the appropriate prompts to enter your information and complete the registration process.

[0046] Terminal: Sends the entered information to the server.

[0047] 2. Acquisition and learning of product data

[0048] Server: Uses each brand's API to retrieve product data (images, price, category, features, etc.).

[0049] Server: Stores the retrieved product data in the database.

[0050] Server: Uses machine learning algorithms to learn from acquired product data and extracts the characteristics of each product.

[0051] 3. Integration with the LINE interface

[0052] Server: Acquires user activity history (search history, purchase history, etc.) through the LINE interface (data cleanroom).

[0053] Server: Aggregates acquired behavioral history data and analyzes user preferences and trends.

[0054] 4. Coordination generation

[0055] User: Enter your budget and desired style using the dedicated app or website.

[0056] Terminal: Sends the entered budget and style information to the server.

[0057] Server: Selects the most suitable product from the database based on the user's conditions (budget, style, preferences).

[0058] Server: Combines selected products to generate the optimal outfit.

[0059] Server: Creates an image of the generated outfit and sends it to the terminal.

[0060] 5. Suggestions and feedback for users

[0061] Terminal: Displays the generated outfit image to the user.

[0062] User: Check the outfit and send feedback if there's anything you don't like.

[0063] Terminal: Sends user feedback to the server.

[0064] Server: Based on feedback, it regenerates the coordination and makes new suggestions that meet the user's requests.

[0065] Specific example

[0066] Let's say a user who has just created an account wants casual outfits. The user creates an account using a dedicated app and enters profile information, preferences, and budget. For example, if the user's budget is under 50,000 yen and their preferences are denim jackets and sneakers, this information is sent to the server.

[0067] The server retrieves the latest product data from each brand and stores it in a database. Next, it uses machine learning algorithms to learn from the product data and understand the characteristics of each product. Using the LINE interface, it retrieves products that the user has previously searched for and purchased within LINE, and analyzes the user's preferences in more detail.

[0068] Based on the user's budget and preferences, the server selects appropriate items from its database and combines them to generate the optimal outfit. The generated outfit image is sent to the terminal and displayed to the user. The user reviews the outfit and sends feedback if they are dissatisfied with it. The server receives the feedback, generates a new outfit, and provides the user with an optimized new suggestion.

[0069] Thus, the present invention is a system that integrates data from multiple brands and utilizes user behavior history and feedback to provide highly accurate, personalized outfits. This system allows users to easily find the optimal fashion that suits their preferences and budget, leading to increased satisfaction.

[0070] The following describes the processing flow.

[0071] Step 1:

[0072] User Registration

[0073] Device: The user launches the app or website and accesses the account registration screen.

[0074] User: Enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[0075] Terminal: Sends the entered information to the server.

[0076] Step 2:

[0077] Product data acquisition

[0078] Server: Calls APIs from multiple brands to retrieve the latest product data (images, prices, categories, features, etc.).

[0079] Server: Stores the retrieved product data in the database.

[0080] Step 3:

[0081] Learning about product specifications

[0082] Server: Starts a machine learning algorithm and learns from product data stored in the database.

[0083] Server: Extracts the characteristics of each product and updates the model to accommodate user preferences.

[0084] Step 4:

[0085] Integration with LINE interface

[0086] Server: Obtains user activity history (search history, purchase history, etc.) via the LINE interface.

[0087] Server: Integrates acquired behavioral history into a database and analyzes user preferences and trends.

[0088] Step 5:

[0089] Coordination generation

[0090] User: Enter your budget and desired style using the dedicated app or website.

[0091] Terminal: Sends information about the budget and style entered by the user to the server.

[0092] Server: Selects the most suitable product based on the user's conditions (budget, style, preferences).

[0093] Server: Combines selected products to generate outfit combinations.

[0094] Server: Creates an image of the generated outfit and sends it to the terminal.

[0095] Step 6:

[0096] Suggestions for users

[0097] Terminal: Displays the generated outfit image to the user.

[0098] User: Review the outfit and provide feedback as needed.

[0099] Step 7:

[0100] Feedback processing

[0101] Terminal: Sends user feedback to the server.

[0102] Server: Analyzes feedback and generates new coordinates that reflect new user requests and improvements.

[0103] Server: Sends the newly generated outfit image to the terminal.

[0104] Step 8:

[0105] Final confirmation and proposal

[0106] Terminal: Presents a new outfit to the user and allows for final confirmation.

[0107] User: Review the new outfit and proceed to purchase if satisfied.

[0108] In this way, a system is realized in which the server, terminal, and user work together at each step to provide highly personalized coordination.

[0109] (Example 1)

[0110] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0111] Traditional fashion coordination systems often struggled to provide suggestions that matched users' preferences and budgets. Furthermore, there was a lack of systems that effectively utilized product data from multiple brands and generated personalized outfits based on user behavior history and feedback. There was also a need for a system that could efficiently provide new outfit suggestions by reflecting user feedback in real time.

[0112] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0113] In this invention, the server includes means for acquiring product data from multiple online shops, means for analyzing the product data, and means for acquiring the user's behavior history. This makes it possible to select the most suitable products for the user and generate personalized outfits based on the user's profile information, preferences, and budget. Furthermore, by analyzing the user's behavior history and understanding the user's preferences and tendencies, more accurate suggestions become possible. In addition, by presenting the generated outfit images to the user and receiving feedback from the user, it is possible to generate new outfits in real time that meet the user's requests. In this way, users can easily find fashion outfits that match their preferences and budget, and their satisfaction level increases.

[0114] A "user" refers to an individual or group that uses this system to generate and receive fashion coordination suggestions.

[0115] A "server" refers to a computer system that acquires and analyzes product data, collects and analyzes behavioral history, and generates product coordination.

[0116] "Device" refers to a device used by users to input profile information, preferences, and budget, as well as a device for receiving styling suggestions and feedback.

[0117] An "online shop" refers to a store or platform that sells products via the internet.

[0118] "Product data" refers to information such as product images, prices, categories, and features obtained from online shops.

[0119] "Behavioral history" refers to records of actions a user has taken in the past, such as search history and purchase history.

[0120] "Profile information" refers to personal information such as the user's name, age, and gender.

[0121] "Preferences" refer to the user's tastes in fashion styles, colors, brands, and other preferences.

[0122] "Budget" refers to the amount of money a user plans to spend on fashion coordination.

[0123] "Feedback" refers to the opinions and requests that users provide regarding the suggested outfits.

[0124] A "messaging platform interface" refers to a means of data communication used to obtain a user's behavioral history.

[0125] "Coordination" refers to a suggestion created by combining multiple fashion items.

[0126] This invention is a system that generates and proposes optimal fashion coordinates to users based on their profile information, preferences, and budget. Specific embodiments of this system are described in detail below.

[0127] Hardware and software configuration

[0128] 1. Server

[0129] Hardware: Server machines with high-performance processors, sufficient memory and storage (e.g., Dell PowerEdge series or HPE ProLiant series)

[0130] software:

[0131] API access: REST API provided by each online shop

[0132] Database: NoSQL database (e.g., MongoDB)

[0133] Machine learning algorithms: TENSORFLOW® or PyTorch

[0134] Big data analysis tool: Apache Spark®

[0135] Image processing: Adobe Photoshop API

[0136] 2. Terminal

[0137] Hardware: Devices used by users to input information (e.g., smartphones, tablets, desktop computers)

[0138] software:

[0139] User Interface: Dedicated application or web browser (e.g., iOS / Android® app, Google Chrome®)

[0140] Operating procedure and specific examples

[0141] User Registration

[0142] Users enter profile information such as their name, age, gender, fashion preferences (e.g., casual, formal), favorite brands, and budget using a dedicated application or website. For example, if a user enters "Name: Ichiro Tanaka," "Age: 30," "Gender: Male," "Preference: Casual," "Brands: Any," and "Budget: 50,000 yen," this information is sent from the device to the server.

[0143] Product data acquisition and analysis

[0144] The server retrieves product data through the online shop's API. The retrieved data is stored in a NoSQL database and analyzed using machine learning algorithms. For example, the server retrieves data for "denim jacket" and "price: 10,000 yen" from "online shop A" and extracts its features using a machine learning model.

[0145] Acquisition and analysis of behavioral history

[0146] The server uses a messaging platform interface to retrieve user activity history (e.g., purchase history, search history). This retrieved activity history is analyzed using big data analytics tools to understand user preferences and trends. For example, if a user has frequently purchased "denim jackets" and "sneakers" in the past, this information is aggregated on the server.

[0147] Coordination generation

[0148] The user enters "casual style" and a "budget of 50,000 yen" into a dedicated app or website. The device sends this information to the server. The server queries the database and selects items that fit the "casual style" and "budget of 50,000 yen or less." The selected items are coordinated to the user's preferences, such as a "denim jacket" and "sneakers." An image of the generated coordinated outfit is created and sent to the device.

[0149] Obtaining feedback and making revisions

[0150] The generated outfit is displayed on the device, and the user reviews it. If the user provides feedback such as "I'd like the sneakers to be a different brand," this information is sent to the server. Based on the feedback, the server re-selects items and generates outfits, creating new suggestions. For example, it might select sneakers from a different brand and generate a new outfit idea.

[0151] Examples of prompt statements

[0152] "Based on the user's profile information and budget, which indicate they prefer a casual style, please provide the best possible suggestions."

[0153] "Please generate new outfits, taking into account the user's past behavior history and feedback."

[0154] Through the above procedure, the system of the present invention can effectively integrate data from multiple online shops and generate and provide optimal fashion coordinates tailored to the user's preferences and budget.

[0155] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0156] Step 1: User Registration

[0157] Input: The user enters their name, age, gender, fashion preferences, favorite brands, and budget into a dedicated application or website.

[0158] Operation: The terminal collects information entered by the user and sends it to the server in JSON format. The user interface provided by the terminal has an input form, and the user enters information using buttons or text fields for each item.

[0159] Output: User profile information, preferences, and budget are sent to the server. The server receives this information and stores it in the user database.

[0160] Step 2: Acquisition and analysis of product data

[0161] Input: API endpoint information for each online shop

[0162] Operation: The server uses a REST API to retrieve product data (images, price, category, features, etc.) from each online shop. The retrieved data is stored in a NoSQL database (e.g., MongoDB). The server then uses TensorFlow or PyTorch to train a machine learning model and analyze the product data.

[0163] Output: Product data is stored on the server, and the analyzed results are reflected in the machine learning model.

[0164] Step 3: Acquisition and analysis of behavioral history

[0165] Input: User activity history (search history, purchase history, etc.)

[0166] Operation: The server retrieves user behavior history data through a messaging platform interface. The retrieved data is aggregated and analyzed using big data analytics tools (e.g., Apache Spark) to analyze user preferences and trends. For example, data on products the user has purchased or searched for in the past six months is analyzed.

[0167] Output: The server stores the user's behavioral history, and analysis results of preferences and trends based on that history are obtained.

[0168] Step 4: Creating the outfit

[0169] Input: User criteria (budget, style, preferences), product data, behavioral history data

[0170] Operation: The user enters criteria such as "casual style" and "budget of 50,000 yen" in a dedicated app or website. The device sends this information to the server. The server queries the database and selects products that match the user's criteria. It combines the selected products to generate the optimal outfit and creates an outfit image using the Adobe Photoshop API, etc.

[0171] Output: The generated outfit and its image are created on the server and sent to the terminal.

[0172] Step 5: Obtain feedback and make revisions

[0173] Input: User feedback

[0174] Operation: The device displays the generated outfit image to the user. When the user enters feedback such as "I want the sneakers to be a different brand," the device sends that feedback to the server. The server receives the feedback, selects new items, and generates the outfit again.

[0175] Output: The newly generated outfit image is created on the server, sent back to the terminal, and presented to the user.

[0176] Through the processing steps described above, this system can efficiently generate and provide fashion coordinates that match the user's preferences and budget.

[0177] (Application Example 1)

[0178] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0179] Currently, many users spend time and effort finding the perfect outfit that suits their preferences and budget when purchasing fashion items online. Furthermore, the inability to try on items online leads to concerns about their appearance and fit, resulting in low post-purchase satisfaction. Therefore, there is a need for a system that suggests optimal outfits based on user preferences and budget, and also provides a virtual try-on experience.

[0180] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0181] In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple brands; and means for the server to learn the acquired product data. This makes it possible not only to generate an optimal outfit based on the user's preferences and budget, but also to confirm the outfit and receive feedback through virtual try-on.

[0182] A "user" is someone who uses the system to input their profile information, preferences, and budget, and then receives suggestions for the most suitable outfit.

[0183] "Profile information" refers to basic personal information entered by the user, including, for example, name, age, and gender.

[0184] "Preferences" refer to a user's personal tastes in fashion, such as their favorite styles, colors, and brands.

[0185] "Budget" refers to the maximum amount of money a user can spend on coordinating an outfit.

[0186] A "server" is a central computer system that handles tasks such as acquiring product data, learning, analyzing user behavior history, and generating coordinated product combinations.

[0187] A "brand" is a group of products offered by a specific company or designer.

[0188] "Product data" refers to information about a product, including images, price, category, and features.

[0189] "Learning" refers to the process where the server uses machine learning algorithms to extract features from product data and generate outfits that match the user's preferences.

[0190] "Behavioral history" refers to data that includes the user's search history and purchase history when using the system.

[0191] "Analysis" is the process of understanding user preferences and trends based on behavioral history data acquired by the server.

[0192] A "coordinate" is a combination of multiple fashion items selected based on the user's preferences and budget.

[0193] An "image" is a visual representation of an outfit and is presented to the user.

[0194] A "terminal" is a device that allows users to access the system, receive coordination suggestions, and send feedback.

[0195] "Feedback" refers to the evaluations and comments that users make regarding the suggested outfits.

[0196] "Virtual try-on" is a feature that allows users to try out outfits in a virtual space, and check the appearance and fit in real time.

[0197] A "communication interface" is a means of connection that allows a server to obtain a user's behavior history from an external system.

[0198] This invention relates to a system that allows users to input their profile information, preferences, and budget using a smartphone or smart glasses, and virtually try on the optimal outfit. This system provides the optimal outfit based on the user's preferences and budget, and further allows them to try on the actual look and fit through virtual try-on.

[0199] The server performs the following processes: First, it receives the user's profile information, preferences, and budget, and uses this information to retrieve product data from each brand's API. Next, it uses a machine learning algorithm to learn from the retrieved product data and extract the characteristics of each product. This learning process uses the Python library scikit-learn. Furthermore, it obtains the user's behavioral history (search history, purchase history, etc.) through the communication interface and uses this to analyze the user's preferences and tendencies.

[0200] In the outfit generation process, the system selects the most suitable items from a database based on the user's entered budget and style, and then combines them to generate an outfit. The server creates an image of the generated outfit and sends it to the user's device. The device then displays the generated outfit to the user and provides a virtual try-on function. The ability to check the appearance and fit of the outfit in real time operates on smart glasses or smartphones.

[0201] If a user is dissatisfied with the outfit after a virtual try-on, they can submit feedback. The device sends the feedback to the server, which then regenerates the outfit based on the feedback and provides an optimized new suggestion.

[0202] As a concrete example, consider a case where a user desires a casual style and has a budget of 50,000 yen or less. In this case, the user opens the application and enters their profile information, preferences, and budget. The server then retrieves appropriate product data from the database and learns from it using a machine learning algorithm. Based on the user's past behavior history, an optimal outfit is generated. The generated outfit is provided to the user using a virtual try-on function, allowing the user to check the outfit as if they were actually trying it on.

[0203] Example of a prompt:

[0204] "Based on the user's profile information, please suggest the best casual outfit within a budget of 50,000 yen. Please also consider their past search and purchase history."

[0205] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0206] Step 1: User Registration

[0207] Users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget using a dedicated application or website. The entered information is sent from the device to the server. This registers the user's basic information and purchase conditions on the server.

[0208] Input: Profile information, preferences, budget

[0209] Output: User ID, registration completion confirmation

[0210] Step 2: Acquire and learn product data

[0211] The server uses each brand's API to retrieve product data (images, price, category, features, etc.) and stores it in a database. Next, a machine learning algorithm (e.g., scikit-learn's RandomForestClassifier) ​​is used to train the server on the product data and extract the features of each product.

[0212] Input: Product data from brand API

[0213] Output: Trained model, feature data

[0214] Step 3: Acquisition and analysis of behavioral history

[0215] The server obtains user behavior history (search history, purchase history, etc.) through a communication interface. Based on this data, user preferences and trends are analyzed using machine learning algorithms.

[0216] Input: Behavioral history data

[0217] Output: Analysis results of preferences and tendencies

[0218] Step 4: Creating the outfit

[0219] When a user enters their budget and desired style through a dedicated app or website, that information is sent to the server. Based on the user's conditions (budget, style, and preferences), the server uses a trained model to select the most suitable items from its database and generate an outfit.

[0220] Input: Budget, Style (user input)

[0221] Output: Optimal coordination suggestion

[0222] Step 5: Virtual fitting provided

[0223] The server creates an image of the outfit and sends it to the device. The device then provides the user with a virtual try-on experience by displaying the outfit in real time via smart glasses or a smartphone.

[0224] Input: Generated coordinate data

[0225] Output: Virtual try-on images and feedback screen

[0226] Step 6: Collecting User Feedback

[0227] Users review suggested outfits through virtual try-on and submit feedback if they are dissatisfied. The device then forwards this feedback to the server.

[0228] Input: User Feedback

[0229] Output: Receiving and transferring feedback data

[0230] Step 7: Generating a re-coordinate based on feedback

[0231] The server regenerates the coordination based on the user's feedback data. This generates new suggestions that meet the user's requests and sends them to the terminal.

[0232] Input: Feedback data

[0233] Output: Newly optimized outfit suggestions

[0234] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0235] This invention is a system that generates and proposes optimal outfits based on user-entered profile information, preferences, and budget, and further analyzes the user's emotional data using an emotion engine to provide even more personalized suggestions. Specifically, the server acquires product data from multiple brands, learns from it using a machine learning algorithm, and analyzes it based on the user's behavior history and feedback data. Furthermore, by analyzing the user's emotional data using an emotion engine, the system generates outfits that are optimal for the user's conditions and emotional state. The processing of this system's program is described below in natural language.

[0236] Program processing flow

[0237] 1. User Registration

[0238] Device: Users create an account using a dedicated application or website. Here, users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[0239] User: Follow the appropriate prompts to enter your information and complete the registration process.

[0240] Terminal: Sends the entered information to the server.

[0241] 2. Acquisition of product data

[0242] Server: Calls APIs from multiple brands to retrieve the latest product data (images, prices, categories, features, etc.).

[0243] Server: Stores the retrieved product data in the database.

[0244] 3. Learning from product data

[0245] Server: Starts a machine learning algorithm and learns from product data stored in the database.

[0246] Server: Extracts the characteristics of each product and updates the model to accommodate user preferences.

[0247] 4. Integration with the LINE interface

[0248] Server: Obtains user activity history (search history, purchase history, etc.) via the LINE interface.

[0249] Server: Integrates acquired behavioral history into a database and analyzes user preferences and trends.

[0250] 5. Coordination generation

[0251] User: Enter your budget and desired style using the dedicated app or website.

[0252] Terminal: Sends information about the budget and style entered by the user to the server.

[0253] Server: Selects the most suitable product based on the user's conditions (budget, style, preferences).

[0254] Server: Combines selected products to generate outfit combinations.

[0255] Server: Creates an image of the generated outfit and sends it to the terminal.

[0256] 6. Acquisition of emotional data

[0257] Terminal: When suggesting outfits, the emotion engine acquires the user's emotional data (e.g., facial recognition and voice analysis).

[0258] Terminal: Sends acquired emotion data to the server.

[0259] 7. Suggestions for Users

[0260] Terminal: Displays the generated outfit image to the user.

[0261] User: Review the outfit and provide feedback as needed.

[0262] Terminal: Sends emotional data acquired when suggesting outfits to the server.

[0263] 8. Processing Feedback

[0264] Terminal: Sends user feedback to the server.

[0265] Server: Analyzes feedback and sentiment data to generate new outfits that reflect the user's new requests and areas for improvement.

[0266] Server: Sends the newly generated outfit image to the terminal.

[0267] 9. Final confirmation and proposal

[0268] Terminal: Presents a new outfit to the user and allows for final confirmation.

[0269] User: Review the new outfit and proceed to purchase if satisfied.

[0270] Specific example

[0271] For example, let's say a user who has just created an account wants a casual outfit. The user creates an account using the app and enters profile information, preferences, and budget. In this case, the budget is under 50,000 yen, and the favorite items are a denim jacket and sneakers. This information is sent to the server.

[0272] The server retrieves the latest product data from each brand and stores it in a database. Then, it uses machine learning algorithms to learn from the product data and understand the characteristics of each product. Furthermore, it uses the LINE interface to retrieve products that users have previously searched for and purchased within LINE, and analyzes user preferences.

[0273] Based on the user's budget and preferences, the server selects appropriate items from a database and combines them to generate the optimal outfit. The generated outfit image is sent to the device and displayed to the user. At this time, the emotion engine acquires emotion data from the user's facial expressions and voice and sends it to the server. If the user is not satisfied with the outfit, they can send feedback from their device.

[0274] The server analyzes emotional data and feedback, generates a new outfit reflecting the user's new requests and suggestions for improvement, and sends the image to the device. The device then presents the new outfit to the user, and if the user is satisfied, it proceeds to the purchase process.

[0275] Thus, the present invention is a system that provides highly personalized coordination based on the user's emotional state by combining an emotion engine, thereby further improving user satisfaction.

[0276] The following describes the processing flow.

[0277] Step 1:

[0278] User Registration

[0279] Terminal: The user launches the app or website and accesses the account registration screen.

[0280] User: Enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[0281] Terminal: Send the entered information to the server.

[0282] Step 2:

[0283] Obtaining product data

[0284] Server: Invoke the APIs of multiple brands to obtain product data (images, prices, categories, features, etc.).

[0285] Server: Store the obtained product data in the database.

[0286] Step 3:

[0287] Learning product data

[0288] Server: Activate the machine learning algorithm and learn the product data stored in the database.

[0289] Server: Extract the features of each product and update the model.

[0290] Step 4:

[0291] Integration with the LINE interface

[0292] Server: Obtain the user's behavior history (search history, purchase history, etc.) via the LINE interface.

[0293] Server: Integrate the obtained behavior history into the database and analyze the user's preferences and trends.

[0294] Step 5:

[0295] Obtaining emotional data

[0296] Terminal: When displaying the proposed coordinates, the emotion engine obtains the user's emotional data (such as facial expression recognition and voice analysis).

[0297] Terminal: Transmits the obtained emotional data to the server.

[0298] Step 6:

[0299] Generating coordinates

[0300] User: Inputs the budget and desired style on a dedicated app or website.

[0301] Terminal: Transmits the information about the budget and style input by the user to the server.

[0302] Server: Selects the optimal products based on the user's conditions (budget, style, preferences) and emotional data.

[0303] Server: Combines the selected products to generate coordinates.

[0304] Server: Creates an image of the generated coordinates and transmits it to the terminal.

[0305] Step 7:

[0306] Proposal to the user

[0307] Terminal: Presents the image of the generated coordinates to the user.

[0308] User: Checks the coordinates and inputs feedback if necessary.

[0309] Terminal: The emotion engine obtains the user's emotional data regarding the proposal and transmits it to the server.

[0310] Step 8:

[0311] Feedback processing

[0312] Terminal: Sends user feedback to the server.

[0313] Server: Analyzes feedback and sentiment data to generate new outfits that reflect the user's new requests and areas for improvement.

[0314] Server: Sends the newly generated outfit image to the terminal.

[0315] Step 9:

[0316] Final confirmation and proposal

[0317] Terminal: Presents a new outfit to the user and allows for final confirmation.

[0318] User: Review the new outfit and proceed to purchase if satisfied.

[0319] This allows us to comprehensively utilize user profile information, preferences, budget, behavioral history, and even emotional data to provide users with the most suitable outfits. As a result, highly personalized suggestions become possible, improving user satisfaction.

[0320] (Example 2)

[0321] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0322] Conventional outfit suggestion systems generate outfits based on the user's basic information and behavioral history, but they do not provide personalized suggestions that take into account the user's emotional state. As a result, user satisfaction does not improve sufficiently, and user engagement in the process leading to a purchase decision sometimes decreases.

[0323] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple data sources; means for the server to learn the acquired product data; means for the server to acquire the user's behavior history; means for the server to analyze preferences and tendencies based on the user's behavior history; means for the server to generate outfits based on the user's conditions; means for the server to create images of the generated outfits; means for the terminal to present the generated outfits to the user; means for the terminal to receive feedback from the user; means for the terminal to acquire the user's emotional data; means for the server to analyze the user's emotional data using an emotional engine; and means for the server to generate outfits again based on the feedback and emotional data. This makes it possible to provide highly personalized outfits based on emotional states.

[0324] A "user" refers to an individual who uses this system to receive styling suggestions.

[0325] "Profile information" refers to basic personal information that users enter, such as their name, age, and gender.

[0326] "Preferences" refer to the user's input regarding style, color, brand, and other tastes.

[0327] "Budget" refers to the range of money users are willing to spend on an outfit.

[0328] A "server" refers to a central computer that controls the entire system and processes, stores, and analyzes data.

[0329] A "data source" refers to a source of information from which product data is obtained from multiple brands and other information providers.

[0330] "Product data" refers to information about each product, including images, price, category, and features.

[0331] "Learning" refers to the process of analyzing product data acquired using machine learning algorithms, extracting features, and updating the model.

[0332] "Behavioral history" refers to data such as search history and purchase history that a user has performed within the system.

[0333] "Preferences and tendencies" refer to the user's interests and preferences, which are analyzed based on the user's behavioral history.

[0334] "Conditions" refer to constraints and requests such as budget, style, and preferences specified by the user.

[0335] "Coordination" refers to a set of fashion items created by combining multiple products selected based on the user's criteria.

[0336] "Emotional data" refers to data about the user's emotional state obtained through methods such as facial recognition and voice analysis.

[0337] An "emotion engine" refers to a system or algorithm used to analyze user emotional data.

[0338] "Feedback" refers to the user's opinion and evaluation of the suggested outfit.

[0339] "Terminal" refers to a device that a user interacts with, including applications and websites.

[0340] This invention is a system that generates and proposes optimal outfits to users based on their profile information, preferences, and budget. Furthermore, it can analyze the user's emotional data using an emotion engine to provide personalized suggestions. The specific configuration and operation procedure of this system are described below.

[0341] System Configuration

[0342] This system consists of the following main components:

[0343] 1. User terminal:

[0344] A dedicated application or website is used. Users enter their profile information, preferences, and budget here.

[0345] It is equipped with cameras and microphones to acquire emotional data.

[0346] 2. Server:

[0347] A server for acquiring and learning from product data. It collects product data from multiple data sources.

[0348] The system stores and analyzes user behavior history and feedback data.

[0349] The emotion engine is activated to analyze the user's emotional data.

[0350] Data processing flow

[0351] 1. User registration:

[0352] Device: Users enter their profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget using a dedicated application or website. This information is then sent to the server.

[0353] Server: Stores the transmitted information in the database.

[0354] 2. Acquisition of product data:

[0355] Server: Retrieves the latest product data (images, prices, categories, features, etc.) from multiple brands via API. This data is stored in a database.

[0356] 3. Learning from product data:

[0357] Server: Applies machine learning algorithms and trains the model on product data. The Python library Scikit-learn can be used here. Extracts product features and updates the model.

[0358] 4. Acquisition and integration of user behavior history:

[0359] Server: Obtains user activity history (search history, purchase history, etc.). This data is obtained through messaging interfaces such as the LINE interface.

[0360] Server: Integrates and analyzes behavioral history data into a database. This allows for the understanding of user preferences and trends.

[0361] 5. Coordination generation:

[0362] User: Enter your desired style and budget using the dedicated app or website.

[0363] Terminal: Sends this information to the server.

[0364] Server: Based on the input conditions, the server selects the most suitable products and generates a coordinated outfit. Then, it uses an image processing library (e.g., OpenCV) to create an image of the generated outfit. The image is sent to the terminal and presented to the user.

[0365] 6. Acquisition and analysis of emotional data:

[0366] Device: Uses a camera and microphone to acquire user emotion data (facial expressions, voice, etc.) when suggesting outfit combinations.

[0367] Terminal: Sends emotional data to the server.

[0368] Server: Analyzes sentiment data using an emotion engine (e.g., Amazon Rekognition or Google Cloud Speech-to-Text).

[0369] 7. Processing feedback:

[0370] Terminal: Receives user feedback and sends it to the server. If the user is not satisfied with the coordination, they can enter that information.

[0371] Server: Analyzes feedback and emotion data to generate a new outfit. The generated new outfit is then sent back to the terminal.

[0372] 8. Final confirmation and purchase:

[0373] User: Review the new outfit and proceed to purchase if satisfied. This cycle is repeated until the user is satisfied.

[0374] Specific example

[0375] For example, let's say a user wants a casual outfit. The user enters information such as their name, age, gender, style preferences, and budget on the app. The user then enters a prompt message such as, "I'd like a casual outfit. My budget is under 50,000 yen, and I like denim jackets and sneakers."

[0376] The server retrieves the latest product data from multiple data sources and stores it in a database. It uses machine learning algorithms to learn from the data and understand the characteristics of each product. When a user enters their desired style and budget in the app, that information is sent to the server. Based on these conditions, the server selects the most suitable products and generates outfit ideas.

[0377] The generated outfit is sent as an image to the device and presented to the user. During this process, the device uses its camera and microphone to capture the user's emotional data and send it to the server. If the user is not satisfied with the suggested outfit, they provide feedback. The server analyzes the feedback and emotional data to generate a new outfit. This process is repeated until the user is satisfied.

[0378] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0379] Step 1: User Registration

[0380] Terminal: Users enter profile information such as name, age, gender, style preferences, and budget using a dedicated application or website. The entered information is organized according to a database structure and sent to the server.

[0381] Server: Stores the submitted user profile information in the database and creates records to manage information for each user. The input is the user's profile information, and the output is the user information stored in the database.

[0382] Step 2: Obtain product data

[0383] Server: The server calls APIs from multiple data sources to retrieve the latest product data. The retrieved product data includes images, prices, categories, features, etc. The input is product data retrieved from the APIs, and the output is product data stored in the database in a standardized format.

[0384] Step 3: Learn product data

[0385] Server: The server launches a machine learning algorithm and learns from product data stored in the database. Specifically, it uses libraries such as Python's Scikit-learn to extract product features. The input is product data, and the output is updated information for the model that has grasped the features.

[0386] Step 4: Acquisition and integration of behavioral history

[0387] Server: The server uses a messaging interface (e.g., LINE interface) to retrieve user activity history. This activity history includes search history and purchase history. The input is activity history data from the interface, and the output is activity history information integrated into a database.

[0388] Step 5: Creating the outfit

[0389] User: Enter your desired style and budget using the dedicated app or website.

[0390] Terminal: Sends this information to the server.

[0391] Server: The server selects the most suitable products based on the input conditions. After selection, it creates an image of the coordinated outfit using an image processing library such as OpenCV. The input is the user's condition information, and the output is the generated image of the coordinated outfit.

[0392] Step 6: Acquisition and analysis of emotional data

[0393] Device: When suggesting outfit combinations, the camera and microphone are used to acquire user emotion data (facial expressions, voice, etc.).

[0394] Terminal: Sends acquired emotion data to the server. Input is emotion data acquired from the camera and microphone, and output is emotion data sent to the server.

[0395] Server: Analyzes sentiment data using an emotion engine. For example, Amazon Rekognition or Google Cloud Speech-to-Text are used. The analysis results are stored as analysis data.

[0396] Step 7: Proposal to the user

[0397] Terminal: Displays the generated outfit image to the user.

[0398] User: Review the outfit and enter feedback. Input is the outfit image and user feedback, and output is the user's feedback information.

[0399] Terminal: Sends feedback information to the server. Input is user feedback information, and output is the transmission of feedback to the server.

[0400] Step 8: Processing Feedback

[0401] Server: The server analyzes feedback and sentiment data and generates new outfits that reflect the user's new requests and areas for improvement. The input is user feedback information and sentiment data, and the output is new outfit information.

[0402] Server: Sends the newly generated outfit image to the terminal. The input is the new outfit information, and the output is the outfit image sent to the terminal.

[0403] Step 9: Final Review and Proposal

[0404] Terminal: Presents a new outfit to the user and allows for final confirmation.

[0405] User: Review the new outfit and proceed to purchase if satisfied. The input is the image of the new outfit, and the output is the user's final confirmation information.

[0406] Terminal: Sends the user's final confirmation information to the server to proceed with the purchase process. The input is the final confirmation information, and the output is the progress of the purchase process.

[0407] (Application Example 2)

[0408] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0409] Traditional styling suggestion systems, while based on user profile information, preferences, and budget, had the challenge of not being able to provide personalized suggestions that took into account the user's instantaneous emotions and reactions. As a result, it was difficult to provide suggestions that truly satisfied users and to fully stimulate their purchasing intent. Furthermore, it was difficult to efficiently learn from product data of multiple brands and to conduct detailed analysis of user behavior history.

[0410] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0411] In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple brands; means for the server to learn the acquired product data; means for the server to acquire the user's behavior history; means for the server to analyze preferences and tendencies based on the user's behavior history; means for the server to generate outfits based on the user's conditions; means for the server to create images of the generated outfits; means for the terminal to present the generated outfits to the user; means for the terminal to receive feedback from the user; means for the server to generate outfits again based on the feedback; means for using an emotion engine to acquire the user's emotional data when suggesting outfits; and means for the server to analyze the acquired emotional data and make suggestions that are optimal for the user's emotional state. This makes it possible to provide more personalized outfit suggestions that take into account the user's instantaneous emotional state.

[0412] "A means for users to input at least profile information, preferences, and budget" refers to a system consisting of an interface for users to input their basic information, clothing preferences, and the amount of money they are willing to spend.

[0413] "A means for a server to acquire product data from multiple brands" refers to a system in which a server has the function of collecting the latest product information from multiple brands via APIs, etc., and storing it in a database.

[0414] "Means for a server to learn from acquired product data" refers to a system that has the function of analyzing product data collected by the server using machine learning algorithms, and understanding and learning its characteristics.

[0415] "Means by which a server obtains a user's behavioral history" refers to a system that includes interfaces and functions for a server to collect a user's past behavioral history, such as search history and purchase history.

[0416] "Means by which a server analyzes user preferences and trends based on user behavior history" refers to a system that includes algorithms and analytical functions for analyzing the behavior history collected by the server and deriving user preferences and purchasing trends.

[0417] "A means by which a server generates outfits based on user conditions" refers to a system equipped with an algorithm that generates the optimal outfit based on the conditions (profile information, preferences, budget) entered by the user.

[0418] "Means for creating images of server-generated outfits" refers to a system that has the function of converting server-generated outfits into images so that they can be presented to the user in a visual format.

[0419] "Means for the terminal to present the generated coordinate to the user" refers to a system that includes an interface for the user's terminal to display the coordinate image sent from the server.

[0420] "A means for a device to receive user feedback" refers to a system that includes an interface for users to input their opinions and impressions of the proposed outfits.

[0421] "Means for the server to regenerate the coordination based on feedback" refers to a system equipped with an algorithm that allows the server to analyze user feedback and regenerate the optimal coordination based on the results.

[0422] "Methods for using an emotion engine to acquire user emotion data when suggesting outfits" refers to a system that includes an engine that recognizes the user's emotions towards the suggested outfits from their facial expressions and voice, and collects that data.

[0423] "A means of analyzing emotional data acquired by a server to provide optimal suggestions for the user's emotional state" refers to a system that includes an algorithm and analysis function for analyzing emotional data collected by a server and providing optimal coordination suggestions for the user's emotional state.

[0424] This invention is a system that suggests the optimal outfit based on the user's profile information, preferences, budget, behavioral history, and emotional data. The system has the following configuration:

[0425] First, users create an account using a dedicated application or website. They enter profile information such as their name, age, gender, style preferences, and budget. This sends the user's basic data to the server.

[0426] The server retrieves product data (images, prices, categories, features, etc.) from multiple brands via APIs. This product data is stored in a database. The server then uses machine learning algorithms to learn from this data and extract the features of each product. Specifically, the Python programming language and the Keras library are used in this process.

[0427] Next, the server retrieves the user's activity history via the API interface. This activity history includes past search and purchase history. This information is also stored in the database and used to analyze the user's preferences and trends.

[0428] To generate an outfit, the user enters their desired style and budget via a dedicated app or website. The server receives this information and selects the most suitable items based on the user's criteria. The selected items are combined into an outfit, and an image of the outfit is generated. The generated outfit image is sent to the device and presented to the user.

[0429] When an outfit is presented to the user, the emotion engine acquires the user's emotion data. This emotion data is analyzed from the user's facial expressions and voice. This includes analyzing facial expressions from camera frames using the OpenCV library. The acquired emotion data is sent to the server for analysis.

[0430] The server analyzes user feedback and sentiment data to generate new outfits that reflect the user's new requests and reactions. This allows for personalized suggestions to be made to the user. This process is repeated until an outfit that the user is ultimately satisfied with is generated.

[0431] To give a concrete example, a user creates an account and enters profile information, preferences, and budget. For example, if the budget is under 50,000 yen and the user's favorite items are denim jackets and sneakers, the server retrieves the latest product data from each brand and stores it in a database. A machine learning algorithm is used to learn from the product data and understand its characteristics. The server also analyzes the user's preferences using their past search and purchase history. Then, it generates the optimal outfit and presents the image to the user. At this time, the emotion engine captures the user's reaction and sends it to the server. If the user is not satisfied with the outfit, they send feedback from their device. The server analyzes the emotion data and feedback, generates a new outfit, and presents it.

[0432] An example of an input prompt for a generative AI model is as follows:

[0433] "User profile information: name, age, gender, style preferences, budget. Product data: brand, price, category, features. Sentiment data: emotional state, confidence level."

[0434] This makes it possible to offer more personalized outfit suggestions that take into account the user's emotional state at that moment.

[0435] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0436] Step 1:

[0437] User Registration

[0438] Users create an account using a dedicated application or website. They enter profile information such as their name, age, gender, style preferences, and budget.

[0439] (Input): User profile information

[0440] (Processing): Send the entered information to the server.

[0441] (Output): User information stored on the server

[0442] Step 2:

[0443] Product data acquisition

[0444] The server retrieves product data from multiple brands via APIs. This product data includes images, prices, categories, and features.

[0445] (Input): API endpoint, brand information

[0446] (Processing): Retrieve data via API call and store it in the database.

[0447] (Output): Product data stored in the database

[0448] Step 3:

[0449] Learning about product specifications

[0450] The server learns from acquired product data using machine learning algorithms. This allows it to understand and classify the characteristics of each product.

[0451] (Input): Product data

[0452] (Processing): Analyze and learn data using machine learning algorithms (using Python and Keras).

[0453] (Output): Product Feature Model

[0454] Step 4:

[0455] Acquisition of user behavior history

[0456] The server retrieves the user's behavioral history (past search history and purchase history) via an API interface. This information is also stored in the database.

[0457] (Input): User ID, API endpoint

[0458] (Processing): Obtain behavioral history via API and store it in the database.

[0459] (Output): Behavioral history data stored in the database

[0460] Step 5:

[0461] Analysis of preferences and tendencies

[0462] The server analyzes user preferences and tendencies based on the acquired behavioral history. This reveals the user's basic preferences and purchasing trends.

[0463] (Input): Behavioral history data

[0464] (Processing): Analyze user preferences and trends using data analysis algorithms.

[0465] (Output): User-preferred model

[0466] Step 6:

[0467] Coordination generation

[0468] The server selects the most suitable products based on the user's conditions (profile information, preferences, budget) and generates outfit combinations using a generative AI model.

[0469] (Input): User information, product feature model, user preference model

[0470] (Processing): The AI ​​model generates the outfits and produces an image.

[0471] (Output): Image of the generated coordinate

[0472] Step 7:

[0473] Coordination suggestions

[0474] The terminal presents the generated outfit to the user. During this process, the emotion engine acquires the user's emotion data. This emotion data is sent to the server.

[0475] (Input): Coordinated image, camera frame

[0476] (Processing): Display the coordinated image, acquire emotion data using the emotion engine, and send it to the server (using OpenCV).

[0477] (Output): User sentiment data

[0478] Step 8:

[0479] Obtaining feedback

[0480] The device receives feedback from the user. The user enters their opinions and impressions of the suggested outfit. The feedback data is also sent to the server.

[0481] (Input): User Feedback

[0482] (Processing): Send feedback to the server

[0483] (Output): Feedback data stored on the server

[0484] Step 9:

[0485] Generating a new outfit

[0486] The server analyzes feedback and sentiment data and generates new outfits that reflect the user's new requests and reactions.

[0487] (Input): Sentiment data, feedback data

[0488] (Processing): Analyze this data using a data analysis algorithm and generate a new coordinate.

[0489] (Output): Image of the new outfit

[0490] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0491] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0492] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0493] [Second Embodiment]

[0494] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0495] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0496] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0497] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0498] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0499] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0500] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0501] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0502] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0503] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0504] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0505] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0506] This invention is a system that generates and proposes the optimal outfit to a user based on their profile information, preferences, and budget. Specifically, the server acquires product data from multiple brands, learns from it using a machine learning algorithm, and analyzes it based on the user's behavior history and feedback data to generate the optimal outfit for the user's conditions. The following describes the processing of this system's program in natural language.

[0507] Program processing flow

[0508] 1. User Registration

[0509] Device: Users create an account using a dedicated application or website. Here, users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[0510] User: Follow the appropriate prompts to enter your information and complete the registration process.

[0511] Terminal: Sends the entered information to the server.

[0512] 2. Acquisition and learning of product data

[0513] Server: Uses each brand's API to retrieve product data (images, price, category, features, etc.).

[0514] Server: Stores the retrieved product data in the database.

[0515] Server: Uses machine learning algorithms to learn from acquired product data and extracts the characteristics of each product.

[0516] 3. Integration with the LINE interface

[0517] Server: Acquires user activity history (search history, purchase history, etc.) through the LINE interface (data cleanroom).

[0518] Server: Aggregates acquired behavioral history data and analyzes user preferences and trends.

[0519] 4. Coordination generation

[0520] User: Enter your budget and desired style using the dedicated app or website.

[0521] Terminal: Sends the entered budget and style information to the server.

[0522] Server: Selects the most suitable product from the database based on the user's conditions (budget, style, preferences).

[0523] Server: Combines selected products to generate the optimal outfit.

[0524] Server: Creates an image of the generated outfit and sends it to the terminal.

[0525] 5. Suggestions and feedback for users

[0526] Terminal: Displays the generated outfit image to the user.

[0527] User: Check the outfit and send feedback if there's anything you don't like.

[0528] Terminal: Sends user feedback to the server.

[0529] Server: Based on feedback, it regenerates the coordination and makes new suggestions that meet the user's requests.

[0530] Specific example

[0531] Let's say a user who has just created an account wants casual outfits. The user creates an account using a dedicated app and enters profile information, preferences, and budget. For example, if the user's budget is under 50,000 yen and their preferences are denim jackets and sneakers, this information is sent to the server.

[0532] The server retrieves the latest product data from each brand and stores it in a database. Next, it uses machine learning algorithms to learn from the product data and understand the characteristics of each product. Using the LINE interface, it retrieves products that the user has previously searched for and purchased within LINE, and analyzes the user's preferences in more detail.

[0533] Based on the user's budget and preferences, the server selects appropriate items from its database and combines them to generate the optimal outfit. The generated outfit image is sent to the terminal and displayed to the user. The user reviews the outfit and sends feedback if they are dissatisfied with it. The server receives the feedback, generates a new outfit, and provides the user with an optimized new suggestion.

[0534] Thus, the present invention is a system that integrates data from multiple brands and utilizes user behavior history and feedback to provide highly accurate, personalized outfits. This system allows users to easily find the optimal fashion that suits their preferences and budget, leading to increased satisfaction.

[0535] The following describes the processing flow.

[0536] Step 1:

[0537] User Registration

[0538] Device: The user launches the app or website and accesses the account registration screen.

[0539] User: Enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[0540] Terminal: Sends the entered information to the server.

[0541] Step 2:

[0542] Product data acquisition

[0543] Server: Calls APIs from multiple brands to retrieve the latest product data (images, prices, categories, features, etc.).

[0544] Server: Stores the retrieved product data in the database.

[0545] Step 3:

[0546] Learning about product specifications

[0547] Server: Starts a machine learning algorithm and learns from product data stored in the database.

[0548] Server: Extracts the characteristics of each product and updates the model to accommodate user preferences.

[0549] Step 4:

[0550] Integration with LINE interface

[0551] Server: Obtains user activity history (search history, purchase history, etc.) via the LINE interface.

[0552] Server: Integrates acquired behavioral history into a database and analyzes user preferences and trends.

[0553] Step 5:

[0554] Coordination generation

[0555] User: Enter your budget and desired style using the dedicated app or website.

[0556] Terminal: Sends information about the budget and style entered by the user to the server.

[0557] Server: Selects the most suitable product based on the user's conditions (budget, style, preferences).

[0558] Server: Combines selected products to generate outfit combinations.

[0559] Server: Creates an image of the generated outfit and sends it to the terminal.

[0560] Step 6:

[0561] Suggestions for users

[0562] Terminal: Displays the generated outfit image to the user.

[0563] User: Review the outfit and provide feedback as needed.

[0564] Step 7:

[0565] Feedback processing

[0566] Terminal: Sends user feedback to the server.

[0567] Server: Analyzes feedback and generates new coordinates that reflect new user requests and improvements.

[0568] Server: Sends the newly generated outfit image to the terminal.

[0569] Step 8:

[0570] Final confirmation and proposal

[0571] Terminal: Presents a new outfit to the user and allows for final confirmation.

[0572] User: Review the new outfit and proceed to purchase if satisfied.

[0573] In this way, a system is realized in which the server, terminal, and user work together at each step to provide highly personalized coordination.

[0574] (Example 1)

[0575] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0576] Traditional fashion coordination systems often struggled to provide suggestions that matched users' preferences and budgets. Furthermore, there was a lack of systems that effectively utilized product data from multiple brands and generated personalized outfits based on user behavior history and feedback. There was also a need for a system that could efficiently provide new outfit suggestions by reflecting user feedback in real time.

[0577] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0578] In this invention, the server includes means for acquiring product data from multiple online shops, means for analyzing the product data, and means for acquiring the user's behavior history. This makes it possible to select the most suitable products for the user and generate personalized outfits based on the user's profile information, preferences, and budget. Furthermore, by analyzing the user's behavior history and understanding the user's preferences and tendencies, more accurate suggestions become possible. In addition, by presenting the generated outfit images to the user and receiving feedback from the user, it is possible to generate new outfits in real time that meet the user's requests. In this way, users can easily find fashion outfits that match their preferences and budget, and their satisfaction level increases.

[0579] A "user" refers to an individual or group that uses this system to generate and receive fashion coordination suggestions.

[0580] A "server" refers to a computer system that acquires and analyzes product data, collects and analyzes behavioral history, and generates product coordination.

[0581] "Device" refers to a device used by users to input profile information, preferences, and budget, as well as a device for receiving styling suggestions and feedback.

[0582] An "online shop" refers to a store or platform that sells products via the internet.

[0583] "Product data" refers to information such as product images, prices, categories, and features obtained from online shops.

[0584] "Behavioral history" refers to records of actions a user has taken in the past, such as search history and purchase history.

[0585] "Profile information" refers to personal information such as the user's name, age, and gender.

[0586] "Preferences" refer to the user's tastes in fashion styles, colors, brands, and other preferences.

[0587] "Budget" refers to the amount of money a user plans to spend on fashion coordination.

[0588] "Feedback" refers to the opinions and requests that users provide regarding the suggested outfits.

[0589] A "messaging platform interface" refers to a means of data communication used to obtain a user's behavioral history.

[0590] "Coordination" refers to a suggestion created by combining multiple fashion items.

[0591] This invention is a system that generates and proposes optimal fashion coordinates to users based on their profile information, preferences, and budget. Specific embodiments of this system are described in detail below.

[0592] Hardware and software configuration

[0593] 1. Server

[0594] Hardware: Server machines with high-performance processors, sufficient memory and storage (e.g., Dell PowerEdge series or HPE ProLiant series)

[0595] software:

[0596] API access: REST API provided by each online shop

[0597] Database: NoSQL database (e.g., MongoDB)

[0598] Machine learning algorithms: TensorFlow or PyTorch

[0599] Big data analysis tool: Apache Spark

[0600] Image processing: Adobe Photoshop API

[0601] 2. Terminal

[0602] Hardware: Devices used by users to input information (e.g., smartphones, tablets, desktop computers)

[0603] software:

[0604] User Interface: Dedicated application or web browser (e.g., iOS / Android app, Google Chrome)

[0605] Operating procedure and specific examples

[0606] User Registration

[0607] Users enter profile information such as their name, age, gender, fashion preferences (e.g., casual, formal), favorite brands, and budget using a dedicated application or website. For example, if a user enters "Name: Ichiro Tanaka," "Age: 30," "Gender: Male," "Preference: Casual," "Brands: Any," and "Budget: 50,000 yen," this information is sent from the device to the server.

[0608] Product data acquisition and analysis

[0609] The server retrieves product data through the online shop's API. The retrieved data is stored in a NoSQL database and analyzed using machine learning algorithms. For example, the server retrieves data for "denim jacket" and "price: 10,000 yen" from "online shop A" and extracts its features using a machine learning model.

[0610] Acquisition and analysis of behavioral history

[0611] The server uses a messaging platform interface to retrieve user activity history (e.g., purchase history, search history). This retrieved activity history is analyzed using big data analytics tools to understand user preferences and trends. For example, if a user has frequently purchased "denim jackets" and "sneakers" in the past, this information is aggregated on the server.

[0612] Coordination generation

[0613] The user enters "casual style" and a "budget of 50,000 yen" into a dedicated app or website. The device sends this information to the server. The server queries the database and selects items that fit the "casual style" and "budget of 50,000 yen or less." The selected items are coordinated to the user's preferences, such as a "denim jacket" and "sneakers." An image of the generated coordinated outfit is created and sent to the device.

[0614] Obtaining feedback and making revisions

[0615] The generated outfit is displayed on the device, and the user reviews it. If the user provides feedback such as "I'd like the sneakers to be a different brand," this information is sent to the server. Based on the feedback, the server re-selects items and generates outfits, creating new suggestions. For example, it might select sneakers from a different brand and generate a new outfit idea.

[0616] Examples of prompt statements

[0617] "Based on the user's profile information and budget, which indicate they prefer a casual style, please provide the best possible suggestions."

[0618] "Please generate new outfits, taking into account the user's past behavior history and feedback."

[0619] Through the above procedure, the system of the present invention can effectively integrate data from multiple online shops and generate and provide optimal fashion coordinates tailored to the user's preferences and budget.

[0620] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0621] Step 1: User Registration

[0622] Input: The user enters their name, age, gender, fashion preferences, favorite brands, and budget into a dedicated application or website.

[0623] Operation: The terminal collects information entered by the user and sends it to the server in JSON format. The user interface provided by the terminal has an input form, and the user enters information using buttons or text fields for each item.

[0624] Output: User profile information, preferences, and budget are sent to the server. The server receives this information and stores it in the user database.

[0625] Step 2: Acquisition and analysis of product data

[0626] Input: API endpoint information for each online shop

[0627] Operation: The server uses a REST API to retrieve product data (images, price, category, features, etc.) from each online shop. The retrieved data is stored in a NoSQL database (e.g., MongoDB). The server then uses TensorFlow or PyTorch to train a machine learning model and analyze the product data.

[0628] Output: Product data is stored on the server, and the analyzed results are reflected in the machine learning model.

[0629] Step 3: Acquisition and analysis of behavioral history

[0630] Input: User activity history (search history, purchase history, etc.)

[0631] Operation: The server retrieves user behavior history data through a messaging platform interface. The retrieved data is aggregated and analyzed using big data analytics tools (e.g., Apache Spark) to analyze user preferences and trends. For example, data on products the user has purchased or searched for in the past six months is analyzed.

[0632] Output: The server stores the user's behavioral history, and analysis results of preferences and trends based on that history are obtained.

[0633] Step 4: Creating the outfit

[0634] Input: User criteria (budget, style, preferences), product data, behavioral history data

[0635] Operation: The user enters criteria such as "casual style" and "budget of 50,000 yen" in a dedicated app or website. The device sends this information to the server. The server queries the database and selects products that match the user's criteria. It combines the selected products to generate the optimal outfit and creates an outfit image using the Adobe Photoshop API, etc.

[0636] Output: The generated outfit and its image are created on the server and sent to the terminal.

[0637] Step 5: Obtain feedback and make revisions

[0638] Input: User feedback

[0639] Operation: The device displays the generated outfit image to the user. When the user enters feedback such as "I want the sneakers to be a different brand," the device sends that feedback to the server. The server receives the feedback, selects new items, and generates the outfit again.

[0640] Output: The newly generated outfit image is created on the server, sent back to the terminal, and presented to the user.

[0641] Through the processing steps described above, this system can efficiently generate and provide fashion coordinates that match the user's preferences and budget.

[0642] (Application Example 1)

[0643] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0644] Currently, many users spend time and effort finding the perfect outfit that suits their preferences and budget when purchasing fashion items online. Furthermore, the inability to try on items online leads to concerns about their appearance and fit, resulting in low post-purchase satisfaction. Therefore, there is a need for a system that suggests optimal outfits based on user preferences and budget, and also provides a virtual try-on experience.

[0645] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0646] In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple brands; and means for the server to learn the acquired product data. This makes it possible not only to generate an optimal outfit based on the user's preferences and budget, but also to confirm the outfit and receive feedback through virtual try-on.

[0647] A "user" is someone who uses the system to input their profile information, preferences, and budget, and then receives suggestions for the most suitable outfit.

[0648] "Profile information" refers to basic personal information entered by the user, including, for example, name, age, and gender.

[0649] "Preferences" refer to a user's personal tastes in fashion, such as their favorite styles, colors, and brands.

[0650] "Budget" refers to the maximum amount of money a user can spend on coordinating an outfit.

[0651] A "server" is a central computer system that handles tasks such as acquiring product data, learning, analyzing user behavior history, and generating coordinated product combinations.

[0652] A "brand" is a group of products offered by a specific company or designer.

[0653] "Product data" refers to information about a product, including images, price, category, and features.

[0654] "Learning" refers to the process where the server uses machine learning algorithms to extract features from product data and generate outfits that match the user's preferences.

[0655] "Behavioral history" refers to data that includes the user's search history and purchase history when using the system.

[0656] "Analysis" is the process of understanding user preferences and trends based on behavioral history data acquired by the server.

[0657] A "coordinate" is a combination of multiple fashion items selected based on the user's preferences and budget.

[0658] An "image" is a visual representation of an outfit and is presented to the user.

[0659] A "terminal" is a device that allows users to access the system, receive coordination suggestions, and send feedback.

[0660] "Feedback" refers to the evaluations and comments that users make regarding the suggested outfits.

[0661] "Virtual try-on" is a feature that allows users to try out outfits in a virtual space, and check the appearance and fit in real time.

[0662] A "communication interface" is a means of connection that allows a server to obtain a user's behavior history from an external system.

[0663] This invention relates to a system that allows users to input their profile information, preferences, and budget using a smartphone or smart glasses, and virtually try on the optimal outfit. This system provides the optimal outfit based on the user's preferences and budget, and further allows them to try on the actual look and fit through virtual try-on.

[0664] The server performs the following processes: First, it receives the user's profile information, preferences, and budget, and uses this information to retrieve product data from each brand's API. Next, it uses a machine learning algorithm to learn from the retrieved product data and extract the characteristics of each product. This learning process uses the Python library scikit-learn. Furthermore, it obtains the user's behavioral history (search history, purchase history, etc.) through the communication interface and uses this to analyze the user's preferences and tendencies.

[0665] In the outfit generation process, the system selects the most suitable items from a database based on the user's entered budget and style, and then combines them to generate an outfit. The server creates an image of the generated outfit and sends it to the user's device. The device then displays the generated outfit to the user and provides a virtual try-on function. The ability to check the appearance and fit of the outfit in real time operates on smart glasses or smartphones.

[0666] If a user is dissatisfied with the outfit after a virtual try-on, they can submit feedback. The device sends the feedback to the server, which then regenerates the outfit based on the feedback and provides an optimized new suggestion.

[0667] As a concrete example, consider a case where a user desires a casual style and has a budget of 50,000 yen or less. In this case, the user opens the application and enters their profile information, preferences, and budget. The server then retrieves appropriate product data from the database and learns from it using a machine learning algorithm. Based on the user's past behavior history, an optimal outfit is generated. The generated outfit is provided to the user using a virtual try-on function, allowing the user to check the outfit as if they were actually trying it on.

[0668] Example of a prompt:

[0669] "Based on the user's profile information, please suggest the best casual outfit within a budget of 50,000 yen. Please also consider their past search and purchase history."

[0670] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0671] Step 1: User Registration

[0672] Users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget using a dedicated application or website. The entered information is sent from the device to the server. This registers the user's basic information and purchase conditions on the server.

[0673] Input: Profile information, preferences, budget

[0674] Output: User ID, registration completion confirmation

[0675] Step 2: Acquire and learn product data

[0676] The server uses each brand's API to retrieve product data (images, price, category, features, etc.) and stores it in a database. Next, a machine learning algorithm (e.g., scikit-learn's RandomForestClassifier) ​​is used to train the server on the product data and extract the features of each product.

[0677] Input: Product data from brand API

[0678] Output: Trained model, feature data

[0679] Step 3: Acquisition and analysis of behavioral history

[0680] The server obtains user behavior history (search history, purchase history, etc.) through a communication interface. Based on this data, user preferences and trends are analyzed using machine learning algorithms.

[0681] Input: Behavioral history data

[0682] Output: Analysis results of preferences and tendencies

[0683] Step 4: Creating the outfit

[0684] When a user enters their budget and desired style through a dedicated app or website, that information is sent to the server. Based on the user's conditions (budget, style, and preferences), the server uses a trained model to select the most suitable items from its database and generate an outfit.

[0685] Input: Budget, Style (user input)

[0686] Output: Optimal coordination suggestion

[0687] Step 5: Virtual fitting provided

[0688] The server creates an image of the outfit and sends it to the device. The device then provides the user with a virtual try-on experience by displaying the outfit in real time via smart glasses or a smartphone.

[0689] Input: Generated coordinate data

[0690] Output: Virtual try-on images and feedback screen

[0691] Step 6: Collecting User Feedback

[0692] Users review suggested outfits through virtual try-on and submit feedback if they are dissatisfied. The device then forwards this feedback to the server.

[0693] Input: User Feedback

[0694] Output: Receiving and transferring feedback data

[0695] Step 7: Generating a re-coordinate based on feedback

[0696] The server regenerates the coordination based on the user's feedback data. This generates new suggestions that meet the user's requests and sends them to the terminal.

[0697] Input: Feedback data

[0698] Output: Newly optimized outfit suggestions

[0699] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0700] This invention is a system that generates and proposes optimal outfits based on user-entered profile information, preferences, and budget, and further analyzes the user's emotional data using an emotion engine to provide even more personalized suggestions. Specifically, the server acquires product data from multiple brands, learns from it using a machine learning algorithm, and analyzes it based on the user's behavior history and feedback data. Furthermore, by analyzing the user's emotional data using an emotion engine, the system generates outfits that are optimal for the user's conditions and emotional state. The processing of this system's program is described below in natural language.

[0701] Program processing flow

[0702] 1. User Registration

[0703] Device: Users create an account using a dedicated application or website. Here, users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[0704] User: Follow the appropriate prompts to enter your information and complete the registration process.

[0705] Terminal: Sends the entered information to the server.

[0706] 2. Acquisition of product data

[0707] Server: Calls APIs from multiple brands to retrieve the latest product data (images, prices, categories, features, etc.).

[0708] Server: Stores the retrieved product data in the database.

[0709] 3. Learning from product data

[0710] Server: Starts a machine learning algorithm and learns from product data stored in the database.

[0711] Server: Extracts the characteristics of each product and updates the model to accommodate user preferences.

[0712] 4. Integration with the LINE interface

[0713] Server: Obtains user activity history (search history, purchase history, etc.) via the LINE interface.

[0714] Server: Integrates acquired behavioral history into a database and analyzes user preferences and trends.

[0715] 5. Coordination generation

[0716] User: Enter your budget and desired style using the dedicated app or website.

[0717] Terminal: Sends information about the budget and style entered by the user to the server.

[0718] Server: Selects the most suitable product based on the user's conditions (budget, style, preferences).

[0719] Server: Combines selected products to generate outfit combinations.

[0720] Server: Creates an image of the generated outfit and sends it to the terminal.

[0721] 6. Acquisition of emotional data

[0722] Terminal: When suggesting outfits, the emotion engine acquires the user's emotional data (e.g., facial recognition and voice analysis).

[0723] Terminal: Sends acquired emotion data to the server.

[0724] 7. Suggestions for Users

[0725] Terminal: Displays the generated outfit image to the user.

[0726] User: Review the outfit and provide feedback as needed.

[0727] Terminal: Sends emotional data acquired when suggesting outfits to the server.

[0728] 8. Processing Feedback

[0729] Terminal: Sends user feedback to the server.

[0730] Server: Analyzes feedback and sentiment data to generate new outfits that reflect the user's new requests and areas for improvement.

[0731] Server: Sends the newly generated outfit image to the terminal.

[0732] 9. Final confirmation and proposal

[0733] Terminal: Presents a new outfit to the user and allows for final confirmation.

[0734] User: Review the new outfit and proceed to purchase if satisfied.

[0735] Specific example

[0736] For example, let's say a user who has just created an account wants a casual outfit. The user creates an account using the app and enters profile information, preferences, and budget. In this case, the budget is under 50,000 yen, and the favorite items are a denim jacket and sneakers. This information is sent to the server.

[0737] The server retrieves the latest product data from each brand and stores it in a database. Then, it uses machine learning algorithms to learn from the product data and understand the characteristics of each product. Furthermore, it uses the LINE interface to retrieve products that users have previously searched for and purchased within LINE, and analyzes user preferences.

[0738] Based on the user's budget and preferences, the server selects appropriate items from a database and combines them to generate the optimal outfit. The generated outfit image is sent to the device and displayed to the user. At this time, the emotion engine acquires emotion data from the user's facial expressions and voice and sends it to the server. If the user is not satisfied with the outfit, they can send feedback from their device.

[0739] The server analyzes emotional data and feedback, generates a new outfit reflecting the user's new requests and suggestions for improvement, and sends the image to the device. The device then presents the new outfit to the user, and if the user is satisfied, it proceeds to the purchase process.

[0740] Thus, the present invention is a system that provides highly personalized coordination based on the user's emotional state by combining an emotion engine, thereby further improving user satisfaction.

[0741] The following describes the processing flow.

[0742] Step 1:

[0743] User Registration

[0744] Device: The user launches the app or website and accesses the account registration screen.

[0745] User: Enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[0746] Terminal: Sends the entered information to the server.

[0747] Step 2:

[0748] Product data acquisition

[0749] Server: Calls APIs from multiple brands to retrieve product data (images, prices, categories, features, etc.).

[0750] Server: Stores the retrieved product data in the database.

[0751] Step 3:

[0752] Learning about product specifications

[0753] Server: Starts a machine learning algorithm and learns from product data stored in the database.

[0754] Server: Extracts the characteristics of each product and updates the model.

[0755] Step 4:

[0756] Integration with LINE interface

[0757] Server: Obtains user activity history (search history, purchase history, etc.) via the LINE interface.

[0758] Server: Integrates acquired behavioral history into a database and analyzes user preferences and trends.

[0759] Step 5:

[0760] Acquisition of emotional data

[0761] Terminal: When displaying suggested outfits, the emotion engine acquires the user's emotion data (such as facial recognition and voice analysis).

[0762] Terminal: Sends acquired emotion data to the server.

[0763] Step 6:

[0764] Coordination generation

[0765] User: Enter your budget and desired style using the dedicated app or website.

[0766] Terminal: Sends information about the budget and style entered by the user to the server.

[0767] Server: Selects the optimal product based on user criteria (budget, style, preferences) and sentiment data.

[0768] Server: Combines selected products to generate outfit combinations.

[0769] Server: Creates an image of the generated outfit and sends it to the terminal.

[0770] Step 7:

[0771] Suggestions for users

[0772] Terminal: Displays the generated outfit image to the user.

[0773] User: Review the outfit and provide feedback as needed.

[0774] Terminal: The emotion engine retrieves the user's emotion data regarding the suggestion and sends it to the server.

[0775] Step 8:

[0776] Feedback processing

[0777] Terminal: Sends user feedback to the server.

[0778] Server: Analyzes feedback and sentiment data to generate new outfits that reflect the user's new requests and areas for improvement.

[0779] Server: Sends the newly generated outfit image to the terminal.

[0780] Step 9:

[0781] Final confirmation and proposal

[0782] Terminal: Presents a new outfit to the user and allows for final confirmation.

[0783] User: Review the new outfit and proceed to purchase if satisfied.

[0784] This allows us to comprehensively utilize user profile information, preferences, budget, behavioral history, and even emotional data to provide users with the most suitable outfits. As a result, highly personalized suggestions become possible, improving user satisfaction.

[0785] (Example 2)

[0786] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0787] Conventional outfit suggestion systems generate outfits based on the user's basic information and behavioral history, but they do not provide personalized suggestions that take into account the user's emotional state. As a result, user satisfaction does not improve sufficiently, and user engagement in the process leading to a purchase decision sometimes decreases.

[0788] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple data sources; means for the server to learn the acquired product data; means for the server to acquire the user's behavior history; means for the server to analyze preferences and tendencies based on the user's behavior history; means for the server to generate outfits based on the user's conditions; means for the server to create images of the generated outfits; means for the terminal to present the generated outfits to the user; means for the terminal to receive feedback from the user; means for the terminal to acquire the user's emotional data; means for the server to analyze the user's emotional data using an emotional engine; and means for the server to generate outfits again based on the feedback and emotional data. This makes it possible to provide highly personalized outfits based on emotional states.

[0789] A "user" refers to an individual who uses this system to receive styling suggestions.

[0790] "Profile information" refers to basic personal information that users enter, such as their name, age, and gender.

[0791] "Preferences" refer to the user's input regarding style, color, brand, and other tastes.

[0792] "Budget" refers to the range of money users are willing to spend on an outfit.

[0793] A "server" refers to a central computer that controls the entire system and processes, stores, and analyzes data.

[0794] A "data source" refers to a source of information from which product data is obtained from multiple brands and other information providers.

[0795] "Product data" refers to information about each product, including images, price, category, and features.

[0796] "Learning" refers to the process of analyzing product data acquired using machine learning algorithms, extracting features, and updating the model.

[0797] "Behavioral history" refers to data such as search history and purchase history that a user has performed within the system.

[0798] "Preferences and tendencies" refer to the user's interests and preferences, which are analyzed based on the user's behavioral history.

[0799] "Conditions" refer to constraints and requests such as budget, style, and preferences specified by the user.

[0800] "Coordination" refers to a set of fashion items created by combining multiple products selected based on the user's criteria.

[0801] "Emotional data" refers to data about the user's emotional state obtained through methods such as facial recognition and voice analysis.

[0802] An "emotion engine" refers to a system or algorithm used to analyze user emotional data.

[0803] "Feedback" refers to the user's opinion and evaluation of the suggested outfit.

[0804] "Terminal" refers to a device that a user interacts with, including applications and websites.

[0805] This invention is a system that generates and proposes optimal outfits to users based on their profile information, preferences, and budget. Furthermore, it can analyze the user's emotional data using an emotion engine to provide personalized suggestions. The specific configuration and operation procedure of this system are described below.

[0806] System Configuration

[0807] This system consists of the following main components:

[0808] 1. User terminal:

[0809] A dedicated application or website is used. Users enter their profile information, preferences, and budget here.

[0810] It is equipped with cameras and microphones to acquire emotional data.

[0811] 2. Server:

[0812] A server for acquiring and learning from product data. It collects product data from multiple data sources.

[0813] The system stores and analyzes user behavior history and feedback data.

[0814] The emotion engine is activated to analyze the user's emotional data.

[0815] Data processing flow

[0816] 1. User registration:

[0817] Device: Users enter their profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget using a dedicated application or website. This information is then sent to the server.

[0818] Server: Stores the transmitted information in the database.

[0819] 2. Acquisition of product data:

[0820] Server: Retrieves the latest product data (images, prices, categories, features, etc.) from multiple brands via API. This data is stored in a database.

[0821] 3. Learning from product data:

[0822] Server: Applies machine learning algorithms and trains the model on product data. The Python library Scikit-learn can be used here. Extracts product features and updates the model.

[0823] 4. Acquisition and integration of user behavior history:

[0824] Server: Obtains user activity history (search history, purchase history, etc.). This data is obtained through messaging interfaces such as the LINE interface.

[0825] Server: Integrates and analyzes behavioral history data into a database. This allows for the understanding of user preferences and trends.

[0826] 5. Coordination generation:

[0827] User: Enter your desired style and budget using the dedicated app or website.

[0828] Terminal: Sends this information to the server.

[0829] Server: Based on the input conditions, the server selects the most suitable products and generates a coordinated outfit. Then, it uses an image processing library (e.g., OpenCV) to create an image of the generated outfit. The image is sent to the terminal and presented to the user.

[0830] 6. Acquisition and analysis of emotional data:

[0831] Device: Uses a camera and microphone to acquire user emotion data (facial expressions, voice, etc.) when suggesting outfit combinations.

[0832] Terminal: Sends emotional data to the server.

[0833] Server: Analyzes sentiment data using an emotion engine (e.g., Amazon Rekognition or Google Cloud Speech-to-Text).

[0834] 7. Processing feedback:

[0835] Terminal: Receives user feedback and sends it to the server. If the user is not satisfied with the coordination, they can enter that information.

[0836] Server: Analyzes feedback and emotion data to generate a new outfit. The generated new outfit is then sent back to the terminal.

[0837] 8. Final confirmation and purchase:

[0838] User: Review the new outfit and proceed to purchase if satisfied. This cycle is repeated until the user is satisfied.

[0839] Specific example

[0840] For example, let's say a user wants a casual outfit. The user enters information such as their name, age, gender, style preferences, and budget on the app. The user then enters a prompt message such as, "I'd like a casual outfit. My budget is under 50,000 yen, and I like denim jackets and sneakers."

[0841] The server retrieves the latest product data from multiple data sources and stores it in a database. It uses machine learning algorithms to learn from the data and understand the characteristics of each product. When a user enters their desired style and budget in the app, that information is sent to the server. Based on these conditions, the server selects the most suitable products and generates outfit ideas.

[0842] The generated outfit is sent as an image to the device and presented to the user. During this process, the device uses its camera and microphone to capture the user's emotional data and send it to the server. If the user is not satisfied with the suggested outfit, they provide feedback. The server analyzes the feedback and emotional data to generate a new outfit. This process is repeated until the user is satisfied.

[0843] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0844] Step 1: User Registration

[0845] Terminal: Users enter profile information such as name, age, gender, style preferences, and budget using a dedicated application or website. The entered information is organized according to a database structure and sent to the server.

[0846] Server: Stores the submitted user profile information in the database and creates records to manage information for each user. The input is the user's profile information, and the output is the user information stored in the database.

[0847] Step 2: Obtain product data

[0848] Server: The server calls APIs from multiple data sources to retrieve the latest product data. The retrieved product data includes images, prices, categories, features, etc. The input is product data retrieved from the APIs, and the output is product data stored in the database in a standardized format.

[0849] Step 3: Learn product data

[0850] Server: The server launches a machine learning algorithm and learns from product data stored in the database. Specifically, it uses libraries such as Python's Scikit-learn to extract product features. The input is product data, and the output is updated information for the model that has grasped the features.

[0851] Step 4: Acquisition and integration of behavioral history

[0852] Server: The server uses a messaging interface (e.g., LINE interface) to retrieve user activity history. This activity history includes search history and purchase history. The input is activity history data from the interface, and the output is activity history information integrated into a database.

[0853] Step 5: Creating the outfit

[0854] User: Enter your desired style and budget using the dedicated app or website.

[0855] Terminal: Sends this information to the server.

[0856] Server: The server selects the most suitable products based on the input conditions. After selection, it creates an image of the coordinated outfit using an image processing library such as OpenCV. The input is the user's condition information, and the output is the generated image of the coordinated outfit.

[0857] Step 6: Acquisition and analysis of emotional data

[0858] Device: When suggesting outfit combinations, the camera and microphone are used to acquire user emotion data (facial expressions, voice, etc.).

[0859] Terminal: Sends acquired emotion data to the server. Input is emotion data acquired from the camera and microphone, and output is emotion data sent to the server.

[0860] Server: Analyzes sentiment data using an emotion engine. For example, Amazon Rekognition or Google Cloud Speech-to-Text are used. The analysis results are stored as analysis data.

[0861] Step 7: Proposal to the user

[0862] Terminal: Displays the generated outfit image to the user.

[0863] User: Review the outfit and enter feedback. Input is the outfit image and user feedback, and output is the user's feedback information.

[0864] Terminal: Sends feedback information to the server. Input is user feedback information, and output is the transmission of feedback to the server.

[0865] Step 8: Processing Feedback

[0866] Server: The server analyzes feedback and sentiment data and generates new outfits that reflect the user's new requests and areas for improvement. The input is user feedback information and sentiment data, and the output is new outfit information.

[0867] Server: Sends the newly generated outfit image to the terminal. The input is the new outfit information, and the output is the outfit image sent to the terminal.

[0868] Step 9: Final Review and Proposal

[0869] Terminal: Presents a new outfit to the user and allows for final confirmation.

[0870] User: Review the new outfit and proceed to purchase if satisfied. The input is the image of the new outfit, and the output is the user's final confirmation information.

[0871] Terminal: Sends the user's final confirmation information to the server to proceed with the purchase process. The input is the final confirmation information, and the output is the progress of the purchase process.

[0872] (Application Example 2)

[0873] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0874] Traditional styling suggestion systems, while based on user profile information, preferences, and budget, had the challenge of not being able to provide personalized suggestions that took into account the user's instantaneous emotions and reactions. As a result, it was difficult to provide suggestions that truly satisfied users and to fully stimulate their purchasing intent. Furthermore, it was difficult to efficiently learn from product data of multiple brands and to conduct detailed analysis of user behavior history.

[0875] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0876] In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple brands; means for the server to learn the acquired product data; means for the server to acquire the user's behavior history; means for the server to analyze preferences and tendencies based on the user's behavior history; means for the server to generate outfits based on the user's conditions; means for the server to create images of the generated outfits; means for the terminal to present the generated outfits to the user; means for the terminal to receive feedback from the user; means for the server to generate outfits again based on the feedback; means for using an emotion engine to acquire the user's emotional data when suggesting outfits; and means for the server to analyze the acquired emotional data and make suggestions that are optimal for the user's emotional state. This makes it possible to provide more personalized outfit suggestions that take into account the user's instantaneous emotional state.

[0877] "A means for users to input at least profile information, preferences, and budget" refers to a system consisting of an interface for users to input their basic information, clothing preferences, and the amount of money they are willing to spend.

[0878] "A means for a server to acquire product data from multiple brands" refers to a system in which a server has the function of collecting the latest product information from multiple brands via APIs, etc., and storing it in a database.

[0879] "Means for a server to learn from acquired product data" refers to a system that has the function of analyzing product data collected by the server using machine learning algorithms, and understanding and learning its characteristics.

[0880] "Means by which a server obtains a user's behavioral history" refers to a system that includes interfaces and functions for a server to collect a user's past behavioral history, such as search history and purchase history.

[0881] "Means by which a server analyzes user preferences and trends based on user behavior history" refers to a system that includes algorithms and analytical functions for analyzing the behavior history collected by the server and deriving user preferences and purchasing trends.

[0882] "A means by which a server generates outfits based on user conditions" refers to a system equipped with an algorithm that generates the optimal outfit based on the conditions (profile information, preferences, budget) entered by the user.

[0883] "Means for creating images of server-generated outfits" refers to a system that has the function of converting server-generated outfits into images so that they can be presented to the user in a visual format.

[0884] "Means for the terminal to present the generated coordinate to the user" refers to a system that includes an interface for the user's terminal to display the coordinate image sent from the server.

[0885] "A means for a device to receive user feedback" refers to a system that includes an interface for users to input their opinions and impressions of the proposed outfits.

[0886] "Means for the server to regenerate the coordination based on feedback" refers to a system equipped with an algorithm that allows the server to analyze user feedback and regenerate the optimal coordination based on the results.

[0887] "Methods for using an emotion engine to acquire user emotion data when suggesting outfits" refers to a system that includes an engine that recognizes the user's emotions towards the suggested outfits from their facial expressions and voice, and collects that data.

[0888] "A means of analyzing emotional data acquired by a server to provide optimal suggestions for the user's emotional state" refers to a system that includes an algorithm and analysis function for analyzing emotional data collected by a server and providing optimal coordination suggestions for the user's emotional state.

[0889] This invention is a system that suggests the optimal outfit based on the user's profile information, preferences, budget, behavioral history, and emotional data. The system has the following configuration:

[0890] First, users create an account using a dedicated application or website. They enter profile information such as their name, age, gender, style preferences, and budget. This sends the user's basic data to the server.

[0891] The server retrieves product data (images, prices, categories, features, etc.) from multiple brands via APIs. This product data is stored in a database. The server then uses machine learning algorithms to learn from this data and extract the features of each product. Specifically, the Python programming language and the Keras library are used in this process.

[0892] Next, the server retrieves the user's activity history via the API interface. This activity history includes past search and purchase history. This information is also stored in the database and used to analyze the user's preferences and trends.

[0893] To generate an outfit, the user enters their desired style and budget via a dedicated app or website. The server receives this information and selects the most suitable items based on the user's criteria. The selected items are combined into an outfit, and an image of the outfit is generated. The generated outfit image is sent to the device and presented to the user.

[0894] When an outfit is presented to the user, the emotion engine acquires the user's emotion data. This emotion data is analyzed from the user's facial expressions and voice. This includes analyzing facial expressions from camera frames using the OpenCV library. The acquired emotion data is sent to the server for analysis.

[0895] The server analyzes user feedback and sentiment data to generate new outfits that reflect the user's new requests and reactions. This allows for personalized suggestions to be made to the user. This process is repeated until an outfit that the user is ultimately satisfied with is generated.

[0896] To give a concrete example, a user creates an account and enters profile information, preferences, and budget. For example, if the budget is under 50,000 yen and the user's favorite items are denim jackets and sneakers, the server retrieves the latest product data from each brand and stores it in a database. A machine learning algorithm is used to learn from the product data and understand its characteristics. The server also analyzes the user's preferences using their past search and purchase history. Then, it generates the optimal outfit and presents the image to the user. At this time, the emotion engine captures the user's reaction and sends it to the server. If the user is not satisfied with the outfit, they send feedback from their device. The server analyzes the emotion data and feedback, generates a new outfit, and presents it.

[0897] An example of an input prompt for a generative AI model is as follows:

[0898] "User profile information: name, age, gender, style preferences, budget. Product data: brand, price, category, features. Sentiment data: emotional state, confidence level."

[0899] This makes it possible to offer more personalized outfit suggestions that take into account the user's emotional state at that moment.

[0900] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0901] Step 1:

[0902] User Registration

[0903] Users create an account using a dedicated application or website. They enter profile information such as their name, age, gender, style preferences, and budget.

[0904] (Input): User profile information

[0905] (Processing): Send the entered information to the server.

[0906] (Output): User information stored on the server

[0907] Step 2:

[0908] Product data acquisition

[0909] The server retrieves product data from multiple brands via APIs. This product data includes images, prices, categories, and features.

[0910] (Input): API endpoint, brand information

[0911] (Processing): Retrieve data via API call and store it in the database.

[0912] (Output): Product data stored in the database

[0913] Step 3:

[0914] Learning about product specifications

[0915] The server learns from acquired product data using machine learning algorithms. This allows it to understand and classify the characteristics of each product.

[0916] (Input): Product data

[0917] (Processing): Analyze and learn data using machine learning algorithms (using Python and Keras).

[0918] (Output): Product Feature Model

[0919] Step 4:

[0920] Acquisition of user behavior history

[0921] The server retrieves the user's behavioral history (past search history and purchase history) via an API interface. This information is also stored in the database.

[0922] (Input): User ID, API endpoint

[0923] (Processing): Obtain behavioral history via API and store it in the database.

[0924] (Output): Behavioral history data stored in the database

[0925] Step 5:

[0926] Analysis of preferences and tendencies

[0927] The server analyzes user preferences and tendencies based on the acquired behavioral history. This reveals the user's basic preferences and purchasing trends.

[0928] (Input): Behavioral history data

[0929] (Processing): Analyze user preferences and trends using data analysis algorithms.

[0930] (Output): User-preferred model

[0931] Step 6:

[0932] Coordination generation

[0933] The server selects the most suitable products based on the user's conditions (profile information, preferences, budget) and generates outfit combinations using a generative AI model.

[0934] (Input): User information, product feature model, user preference model

[0935] (Processing): The AI ​​model generates the outfits and produces an image.

[0936] (Output): Image of the generated coordinate

[0937] Step 7:

[0938] Coordination suggestions

[0939] The terminal presents the generated outfit to the user. During this process, the emotion engine acquires the user's emotion data. This emotion data is sent to the server.

[0940] (Input): Coordinated image, camera frame

[0941] (Processing): Display the coordinated image, acquire emotion data using the emotion engine, and send it to the server (using OpenCV).

[0942] (Output): User sentiment data

[0943] Step 8:

[0944] Obtaining feedback

[0945] The device receives feedback from the user. The user enters their opinions and impressions of the suggested outfit. The feedback data is also sent to the server.

[0946] (Input): User Feedback

[0947] (Processing): Send feedback to the server

[0948] (Output): Feedback data stored on the server

[0949] Step 9:

[0950] Generating a new outfit

[0951] The server analyzes feedback and sentiment data and generates new outfits that reflect the user's new requests and reactions.

[0952] (Input): Sentiment data, feedback data

[0953] (Processing): Analyze this data using a data analysis algorithm and generate a new coordinate.

[0954] (Output): Image of the new outfit

[0955] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0956] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0957] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0958] [Third Embodiment]

[0959] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0960] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0961] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0962] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0963] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0964] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0965] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0966] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0967] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0968] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0969] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0970] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0971] This invention is a system that generates and proposes the optimal outfit to a user based on their profile information, preferences, and budget. Specifically, the server acquires product data from multiple brands, learns from it using a machine learning algorithm, and analyzes it based on the user's behavior history and feedback data to generate the optimal outfit for the user's conditions. The following describes the processing of this system's program in natural language.

[0972] Program processing flow

[0973] 1. User Registration

[0974] Device: Users create an account using a dedicated application or website. Here, users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[0975] User: Follow the appropriate prompts to enter your information and complete the registration process.

[0976] Terminal: Sends the entered information to the server.

[0977] 2. Acquisition and learning of product data

[0978] Server: Uses each brand's API to retrieve product data (images, price, category, features, etc.).

[0979] Server: Stores the retrieved product data in the database.

[0980] Server: Uses machine learning algorithms to learn from acquired product data and extracts the characteristics of each product.

[0981] 3. Integration with the LINE interface

[0982] Server: Acquires user activity history (search history, purchase history, etc.) through the LINE interface (data cleanroom).

[0983] Server: Aggregates acquired behavioral history data and analyzes user preferences and trends.

[0984] 4. Coordination generation

[0985] User: Enter your budget and desired style using the dedicated app or website.

[0986] Terminal: Sends the entered budget and style information to the server.

[0987] Server: Selects the most suitable product from the database based on the user's conditions (budget, style, preferences).

[0988] Server: Combines selected products to generate the optimal outfit.

[0989] Server: Creates an image of the generated outfit and sends it to the terminal.

[0990] 5. Suggestions and feedback for users

[0991] Terminal: Displays the generated outfit image to the user.

[0992] User: Check the outfit and send feedback if there's anything you don't like.

[0993] Terminal: Sends user feedback to the server.

[0994] Server: Based on feedback, it regenerates the coordination and makes new suggestions that meet the user's requests.

[0995] Specific example

[0996] Let's say a user who has just created an account wants casual outfits. The user creates an account using a dedicated app and enters profile information, preferences, and budget. For example, if the user's budget is under 50,000 yen and their preferences are denim jackets and sneakers, this information is sent to the server.

[0997] The server retrieves the latest product data from each brand and stores it in a database. Next, it uses machine learning algorithms to learn from the product data and understand the characteristics of each product. Using the LINE interface, it retrieves products that the user has previously searched for and purchased within LINE, and analyzes the user's preferences in more detail.

[0998] Based on the user's budget and preferences, the server selects appropriate items from its database and combines them to generate the optimal outfit. The generated outfit image is sent to the terminal and displayed to the user. The user reviews the outfit and sends feedback if they are dissatisfied with it. The server receives the feedback, generates a new outfit, and provides the user with an optimized new suggestion.

[0999] Thus, the present invention is a system that integrates data from multiple brands and utilizes user behavior history and feedback to provide highly accurate, personalized outfits. This system allows users to easily find the optimal fashion that suits their preferences and budget, leading to increased satisfaction.

[1000] The following describes the processing flow.

[1001] Step 1:

[1002] User Registration

[1003] Device: The user launches the app or website and accesses the account registration screen.

[1004] User: Enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[1005] Terminal: Sends the entered information to the server.

[1006] Step 2:

[1007] Product data acquisition

[1008] Server: Calls APIs from multiple brands to retrieve the latest product data (images, prices, categories, features, etc.).

[1009] Server: Stores the retrieved product data in the database.

[1010] Step 3:

[1011] Learning about product specifications

[1012] Server: Starts a machine learning algorithm and learns from product data stored in the database.

[1013] Server: Extracts the characteristics of each product and updates the model to accommodate user preferences.

[1014] Step 4:

[1015] Integration with LINE interface

[1016] Server: Obtains user activity history (search history, purchase history, etc.) via the LINE interface.

[1017] Server: Integrates acquired behavioral history into a database and analyzes user preferences and trends.

[1018] Step 5:

[1019] Coordination generation

[1020] User: Enter your budget and desired style using the dedicated app or website.

[1021] Terminal: Sends information about the budget and style entered by the user to the server.

[1022] Server: Selects the most suitable product based on the user's conditions (budget, style, preferences).

[1023] Server: Combines selected products to generate outfit combinations.

[1024] Server: Creates an image of the generated outfit and sends it to the terminal.

[1025] Step 6:

[1026] Suggestions for users

[1027] Terminal: Displays the generated outfit image to the user.

[1028] User: Review the outfit and provide feedback as needed.

[1029] Step 7:

[1030] Feedback processing

[1031] Terminal: Sends user feedback to the server.

[1032] Server: Analyzes feedback and generates new coordinates that reflect new user requests and improvements.

[1033] Server: Sends the newly generated outfit image to the terminal.

[1034] Step 8:

[1035] Final confirmation and proposal

[1036] Terminal: Presents a new outfit to the user and allows for final confirmation.

[1037] User: Review the new outfit and proceed to purchase if satisfied.

[1038] In this way, a system is realized in which the server, terminal, and user work together at each step to provide highly personalized coordination.

[1039] (Example 1)

[1040] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1041] Traditional fashion coordination systems often struggled to provide suggestions that matched users' preferences and budgets. Furthermore, there was a lack of systems that effectively utilized product data from multiple brands and generated personalized outfits based on user behavior history and feedback. There was also a need for a system that could efficiently provide new outfit suggestions by reflecting user feedback in real time.

[1042] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1043] In this invention, the server includes means for acquiring product data from multiple online shops, means for analyzing the product data, and means for acquiring the user's behavior history. This makes it possible to select the most suitable products for the user and generate personalized outfits based on the user's profile information, preferences, and budget. Furthermore, by analyzing the user's behavior history and understanding the user's preferences and tendencies, more accurate suggestions become possible. In addition, by presenting the generated outfit images to the user and receiving feedback from the user, it is possible to generate new outfits in real time that meet the user's requests. In this way, users can easily find fashion outfits that match their preferences and budget, and their satisfaction level increases.

[1044] A "user" refers to an individual or group that uses this system to generate and receive fashion coordination suggestions.

[1045] A "server" refers to a computer system that acquires and analyzes product data, collects and analyzes behavioral history, and generates product coordination.

[1046] "Device" refers to a device used by users to input profile information, preferences, and budget, as well as a device for receiving styling suggestions and feedback.

[1047] An "online shop" refers to a store or platform that sells products via the internet.

[1048] "Product data" refers to information such as product images, prices, categories, and features obtained from online shops.

[1049] "Behavioral history" refers to records of actions a user has taken in the past, such as search history and purchase history.

[1050] "Profile information" refers to personal information such as the user's name, age, and gender.

[1051] "Preferences" refer to the user's tastes in fashion styles, colors, brands, and other preferences.

[1052] "Budget" refers to the amount of money a user plans to spend on fashion coordination.

[1053] "Feedback" refers to the opinions and requests that users provide regarding the suggested outfits.

[1054] A "messaging platform interface" refers to a means of data communication used to obtain a user's behavioral history.

[1055] "Coordination" refers to a suggestion created by combining multiple fashion items.

[1056] This invention is a system that generates and proposes optimal fashion coordinates to users based on their profile information, preferences, and budget. Specific embodiments of this system are described in detail below.

[1057] Hardware and software configuration

[1058] 1. Server

[1059] Hardware: Server machines with high-performance processors, sufficient memory and storage (e.g., Dell PowerEdge series or HPE ProLiant series)

[1060] software:

[1061] API access: REST API provided by each online shop

[1062] Database: NoSQL database (e.g., MongoDB)

[1063] Machine learning algorithms: TensorFlow or PyTorch

[1064] Big data analysis tool: Apache Spark

[1065] Image processing: Adobe Photoshop API

[1066] 2. Terminal

[1067] Hardware: Devices used by users to input information (e.g., smartphones, tablets, desktop computers)

[1068] software:

[1069] User Interface: Dedicated application or web browser (e.g., iOS / Android app, Google Chrome)

[1070] Operating procedure and specific examples

[1071] User Registration

[1072] Users enter profile information such as their name, age, gender, fashion preferences (e.g., casual, formal), favorite brands, and budget using a dedicated application or website. For example, if a user enters "Name: Ichiro Tanaka," "Age: 30," "Gender: Male," "Preference: Casual," "Brands: Any," and "Budget: 50,000 yen," this information is sent from the device to the server.

[1073] Product data acquisition and analysis

[1074] The server retrieves product data through the online shop's API. The retrieved data is stored in a NoSQL database and analyzed using machine learning algorithms. For example, the server retrieves data for "denim jacket" and "price: 10,000 yen" from "online shop A" and extracts its features using a machine learning model.

[1075] Acquisition and analysis of behavioral history

[1076] The server uses a messaging platform interface to retrieve user activity history (e.g., purchase history, search history). This retrieved activity history is analyzed using big data analytics tools to understand user preferences and trends. For example, if a user has frequently purchased "denim jackets" and "sneakers" in the past, this information is aggregated on the server.

[1077] Coordination generation

[1078] The user enters "casual style" and a "budget of 50,000 yen" into a dedicated app or website. The device sends this information to the server. The server queries the database and selects items that fit the "casual style" and "budget of 50,000 yen or less." The selected items are coordinated to the user's preferences, such as a "denim jacket" and "sneakers." An image of the generated coordinated outfit is created and sent to the device.

[1079] Obtaining feedback and making revisions

[1080] The generated outfit is displayed on the device, and the user reviews it. If the user provides feedback such as "I'd like the sneakers to be a different brand," this information is sent to the server. Based on the feedback, the server re-selects items and generates outfits, creating new suggestions. For example, it might select sneakers from a different brand and generate a new outfit idea.

[1081] Examples of prompt statements

[1082] "Based on the user's profile information and budget, which indicate they prefer a casual style, please provide the best possible suggestions."

[1083] "Please generate new outfits, taking into account the user's past behavior history and feedback."

[1084] Through the above procedure, the system of the present invention can effectively integrate data from multiple online shops and generate and provide optimal fashion coordinates tailored to the user's preferences and budget.

[1085] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1086] Step 1: User Registration

[1087] Input: The user enters their name, age, gender, fashion preferences, favorite brands, and budget into a dedicated application or website.

[1088] Operation: The terminal collects information entered by the user and sends it to the server in JSON format. The user interface provided by the terminal has an input form, and the user enters information using buttons or text fields for each item.

[1089] Output: User profile information, preferences, and budget are sent to the server. The server receives this information and stores it in the user database.

[1090] Step 2: Acquisition and analysis of product data

[1091] Input: API endpoint information for each online shop

[1092] Operation: The server uses a REST API to retrieve product data (images, price, category, features, etc.) from each online shop. The retrieved data is stored in a NoSQL database (e.g., MongoDB). The server then uses TensorFlow or PyTorch to train a machine learning model and analyze the product data.

[1093] Output: Product data is stored on the server, and the analyzed results are reflected in the machine learning model.

[1094] Step 3: Acquisition and analysis of behavioral history

[1095] Input: User activity history (search history, purchase history, etc.)

[1096] Operation: The server retrieves user behavior history data through a messaging platform interface. The retrieved data is aggregated and analyzed using big data analytics tools (e.g., Apache Spark) to analyze user preferences and trends. For example, data on products the user has purchased or searched for in the past six months is analyzed.

[1097] Output: The server stores the user's behavioral history, and analysis results of preferences and trends based on that history are obtained.

[1098] Step 4: Creating the outfit

[1099] Input: User criteria (budget, style, preferences), product data, behavioral history data

[1100] Operation: The user enters criteria such as "casual style" and "budget of 50,000 yen" in a dedicated app or website. The device sends this information to the server. The server queries the database and selects products that match the user's criteria. It combines the selected products to generate the optimal outfit and creates an outfit image using the Adobe Photoshop API, etc.

[1101] Output: The generated outfit and its image are created on the server and sent to the terminal.

[1102] Step 5: Obtain feedback and make revisions

[1103] Input: User feedback

[1104] Operation: The device displays the generated outfit image to the user. When the user enters feedback such as "I want the sneakers to be a different brand," the device sends that feedback to the server. The server receives the feedback, selects new items, and generates the outfit again.

[1105] Output: The newly generated outfit image is created on the server, sent back to the terminal, and presented to the user.

[1106] Through the processing steps described above, this system can efficiently generate and provide fashion coordinates that match the user's preferences and budget.

[1107] (Application Example 1)

[1108] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1109] Currently, many users spend time and effort finding the perfect outfit that suits their preferences and budget when purchasing fashion items online. Furthermore, the inability to try on items online leads to concerns about their appearance and fit, resulting in low post-purchase satisfaction. Therefore, there is a need for a system that suggests optimal outfits based on user preferences and budget, and also provides a virtual try-on experience.

[1110] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1111] In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple brands; and means for the server to learn the acquired product data. This makes it possible not only to generate an optimal outfit based on the user's preferences and budget, but also to confirm the outfit and receive feedback through virtual try-on.

[1112] A "user" is someone who uses the system to input their profile information, preferences, and budget, and then receives suggestions for the most suitable outfit.

[1113] "Profile information" refers to basic personal information entered by the user, including, for example, name, age, and gender.

[1114] "Preferences" refer to a user's personal tastes in fashion, such as their favorite styles, colors, and brands.

[1115] "Budget" refers to the maximum amount of money a user can spend on coordinating an outfit.

[1116] A "server" is a central computer system that handles tasks such as acquiring product data, learning, analyzing user behavior history, and generating coordinated product combinations.

[1117] A "brand" is a group of products offered by a specific company or designer.

[1118] "Product data" refers to information about a product, including images, price, category, and features.

[1119] "Learning" refers to the process where the server uses machine learning algorithms to extract features from product data and generate outfits that match the user's preferences.

[1120] "Behavioral history" refers to data that includes the user's search history and purchase history when using the system.

[1121] "Analysis" is the process of understanding user preferences and trends based on behavioral history data acquired by the server.

[1122] A "coordinate" is a combination of multiple fashion items selected based on the user's preferences and budget.

[1123] An "image" is a visual representation of an outfit and is presented to the user.

[1124] A "terminal" is a device that allows users to access the system, receive coordination suggestions, and send feedback.

[1125] "Feedback" refers to the evaluations and comments that users make regarding the suggested outfits.

[1126] "Virtual try-on" is a feature that allows users to try out outfits in a virtual space, and check the appearance and fit in real time.

[1127] A "communication interface" is a means of connection that allows a server to obtain a user's behavior history from an external system.

[1128] This invention relates to a system that allows users to input their profile information, preferences, and budget using a smartphone or smart glasses, and virtually try on the optimal outfit. This system provides the optimal outfit based on the user's preferences and budget, and further allows them to try on the actual look and fit through virtual try-on.

[1129] The server performs the following processes: First, it receives the user's profile information, preferences, and budget, and uses this information to retrieve product data from each brand's API. Next, it uses a machine learning algorithm to learn from the retrieved product data and extract the characteristics of each product. This learning process uses the Python library scikit-learn. Furthermore, it obtains the user's behavioral history (search history, purchase history, etc.) through the communication interface and uses this to analyze the user's preferences and tendencies.

[1130] In the outfit generation process, the system selects the most suitable items from a database based on the user's entered budget and style, and then combines them to generate an outfit. The server creates an image of the generated outfit and sends it to the user's device. The device then displays the generated outfit to the user and provides a virtual try-on function. The ability to check the appearance and fit of the outfit in real time operates on smart glasses or smartphones.

[1131] If a user is dissatisfied with the outfit after a virtual try-on, they can submit feedback. The device sends the feedback to the server, which then regenerates the outfit based on the feedback and provides an optimized new suggestion.

[1132] As a concrete example, consider a case where a user desires a casual style and has a budget of 50,000 yen or less. In this case, the user opens the application and enters their profile information, preferences, and budget. The server then retrieves appropriate product data from the database and learns from it using a machine learning algorithm. Based on the user's past behavior history, an optimal outfit is generated. The generated outfit is provided to the user using a virtual try-on function, allowing the user to check the outfit as if they were actually trying it on.

[1133] Example of a prompt:

[1134] "Based on the user's profile information, please suggest the best casual outfit within a budget of 50,000 yen. Please also consider their past search and purchase history."

[1135] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1136] Step 1: User Registration

[1137] Users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget using a dedicated application or website. The entered information is sent from the device to the server. This registers the user's basic information and purchase conditions on the server.

[1138] Input: Profile information, preferences, budget

[1139] Output: User ID, registration completion confirmation

[1140] Step 2: Acquire and learn product data

[1141] The server uses each brand's API to retrieve product data (images, price, category, features, etc.) and stores it in a database. Next, a machine learning algorithm (e.g., scikit-learn's RandomForestClassifier) ​​is used to train the server on the product data and extract the features of each product.

[1142] Input: Product data from brand API

[1143] Output: Trained model, feature data

[1144] Step 3: Acquisition and analysis of behavioral history

[1145] The server obtains user behavior history (search history, purchase history, etc.) through a communication interface. Based on this data, user preferences and trends are analyzed using machine learning algorithms.

[1146] Input: Behavioral history data

[1147] Output: Analysis results of preferences and tendencies

[1148] Step 4: Creating the outfit

[1149] When a user enters their budget and desired style through a dedicated app or website, that information is sent to the server. Based on the user's conditions (budget, style, and preferences), the server uses a trained model to select the most suitable items from its database and generate an outfit.

[1150] Input: Budget, Style (user input)

[1151] Output: Optimal coordination suggestion

[1152] Step 5: Virtual fitting provided

[1153] The server creates an image of the outfit and sends it to the device. The device then provides the user with a virtual try-on experience by displaying the outfit in real time via smart glasses or a smartphone.

[1154] Input: Generated coordinate data

[1155] Output: Virtual try-on images and feedback screen

[1156] Step 6: Collecting User Feedback

[1157] Users review suggested outfits through virtual try-on and submit feedback if they are dissatisfied. The device then forwards this feedback to the server.

[1158] Input: User Feedback

[1159] Output: Receiving and transferring feedback data

[1160] Step 7: Generating a re-coordinate based on feedback

[1161] The server regenerates the coordination based on the user's feedback data. This generates new suggestions that meet the user's requests and sends them to the terminal.

[1162] Input: Feedback data

[1163] Output: Newly optimized outfit suggestions

[1164] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1165] This invention is a system that generates and proposes optimal outfits based on user-entered profile information, preferences, and budget, and further analyzes the user's emotional data using an emotion engine to provide even more personalized suggestions. Specifically, the server acquires product data from multiple brands, learns from it using a machine learning algorithm, and analyzes it based on the user's behavior history and feedback data. Furthermore, by analyzing the user's emotional data using an emotion engine, the system generates outfits that are optimal for the user's conditions and emotional state. The processing of this system's program is described below in natural language.

[1166] Program processing flow

[1167] 1. User Registration

[1168] Device: Users create an account using a dedicated application or website. Here, users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[1169] User: Follow the appropriate prompts to enter your information and complete the registration process.

[1170] Terminal: Sends the entered information to the server.

[1171] 2. Acquisition of product data

[1172] Server: Calls APIs from multiple brands to retrieve the latest product data (images, prices, categories, features, etc.).

[1173] Server: Stores the retrieved product data in the database.

[1174] 3. Learning from product data

[1175] Server: Starts a machine learning algorithm and learns from product data stored in the database.

[1176] Server: Extracts the characteristics of each product and updates the model to accommodate user preferences.

[1177] 4. Integration with the LINE interface

[1178] Server: Obtains user activity history (search history, purchase history, etc.) via the LINE interface.

[1179] Server: Integrates acquired behavioral history into a database and analyzes user preferences and trends.

[1180] 5. Coordination generation

[1181] User: Enter your budget and desired style using the dedicated app or website.

[1182] Terminal: Sends information about the budget and style entered by the user to the server.

[1183] Server: Selects the most suitable product based on the user's conditions (budget, style, preferences).

[1184] Server: Combines selected products to generate outfit combinations.

[1185] Server: Creates an image of the generated outfit and sends it to the terminal.

[1186] 6. Acquisition of emotional data

[1187] Terminal: When suggesting outfits, the emotion engine acquires the user's emotional data (e.g., facial recognition and voice analysis).

[1188] Terminal: Sends acquired emotion data to the server.

[1189] 7. Suggestions for Users

[1190] Terminal: Displays the generated outfit image to the user.

[1191] User: Review the outfit and provide feedback as needed.

[1192] Terminal: Sends emotional data acquired when suggesting outfits to the server.

[1193] 8. Processing Feedback

[1194] Terminal: Sends user feedback to the server.

[1195] Server: Analyzes feedback and sentiment data to generate new outfits that reflect the user's new requests and areas for improvement.

[1196] Server: Sends the newly generated outfit image to the terminal.

[1197] 9. Final confirmation and proposal

[1198] Terminal: Presents a new outfit to the user and allows for final confirmation.

[1199] User: Review the new outfit and proceed to purchase if satisfied.

[1200] Specific example

[1201] For example, let's say a user who has just created an account wants a casual outfit. The user creates an account using the app and enters profile information, preferences, and budget. In this case, the budget is under 50,000 yen, and the favorite items are a denim jacket and sneakers. This information is sent to the server.

[1202] The server retrieves the latest product data from each brand and stores it in a database. Then, it uses machine learning algorithms to learn from the product data and understand the characteristics of each product. Furthermore, it uses the LINE interface to retrieve products that users have previously searched for and purchased within LINE, and analyzes user preferences.

[1203] Based on the user's budget and preferences, the server selects appropriate items from a database and combines them to generate the optimal outfit. The generated outfit image is sent to the device and displayed to the user. At this time, the emotion engine acquires emotion data from the user's facial expressions and voice and sends it to the server. If the user is not satisfied with the outfit, they can send feedback from their device.

[1204] The server analyzes emotional data and feedback, generates a new outfit reflecting the user's new requests and suggestions for improvement, and sends the image to the device. The device then presents the new outfit to the user, and if the user is satisfied, it proceeds to the purchase process.

[1205] Thus, the present invention is a system that provides highly personalized coordination based on the user's emotional state by combining an emotion engine, thereby further improving user satisfaction.

[1206] The following describes the processing flow.

[1207] Step 1:

[1208] User Registration

[1209] Device: The user launches the app or website and accesses the account registration screen.

[1210] User: Enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[1211] Terminal: Sends the entered information to the server.

[1212] Step 2:

[1213] Product data acquisition

[1214] Server: Calls APIs from multiple brands to retrieve product data (images, prices, categories, features, etc.).

[1215] Server: Stores the retrieved product data in the database.

[1216] Step 3:

[1217] Learning about product specifications

[1218] Server: Starts a machine learning algorithm and learns from product data stored in the database.

[1219] Server: Extracts the characteristics of each product and updates the model.

[1220] Step 4:

[1221] Integration with LINE interface

[1222] Server: Obtains user activity history (search history, purchase history, etc.) via the LINE interface.

[1223] Server: Integrates acquired behavioral history into a database and analyzes user preferences and trends.

[1224] Step 5:

[1225] Acquisition of emotional data

[1226] Terminal: When displaying suggested outfits, the emotion engine acquires the user's emotion data (such as facial recognition and voice analysis).

[1227] Terminal: Sends acquired emotion data to the server.

[1228] Step 6:

[1229] Coordination generation

[1230] User: Enter your budget and desired style using the dedicated app or website.

[1231] Terminal: Sends information about the budget and style entered by the user to the server.

[1232] Server: Selects the optimal product based on user criteria (budget, style, preferences) and sentiment data.

[1233] Server: Combines selected products to generate outfit combinations.

[1234] Server: Creates an image of the generated outfit and sends it to the terminal.

[1235] Step 7:

[1236] Suggestions for users

[1237] Terminal: Displays the generated outfit image to the user.

[1238] User: Review the outfit and provide feedback as needed.

[1239] Terminal: The emotion engine retrieves the user's emotion data regarding the suggestion and sends it to the server.

[1240] Step 8:

[1241] Feedback processing

[1242] Terminal: Sends user feedback to the server.

[1243] Server: Analyzes feedback and sentiment data to generate new outfits that reflect the user's new requests and areas for improvement.

[1244] Server: Sends the newly generated outfit image to the terminal.

[1245] Step 9:

[1246] Final confirmation and proposal

[1247] Terminal: Presents a new outfit to the user and allows for final confirmation.

[1248] User: Review the new outfit and proceed to purchase if satisfied.

[1249] This allows us to comprehensively utilize user profile information, preferences, budget, behavioral history, and even emotional data to provide users with the most suitable outfits. As a result, highly personalized suggestions become possible, improving user satisfaction.

[1250] (Example 2)

[1251] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1252] Conventional outfit suggestion systems generate outfits based on the user's basic information and behavioral history, but they do not provide personalized suggestions that take into account the user's emotional state. As a result, user satisfaction does not improve sufficiently, and user engagement in the process leading to a purchase decision sometimes decreases.

[1253] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple data sources; means for the server to learn the acquired product data; means for the server to acquire the user's behavior history; means for the server to analyze preferences and tendencies based on the user's behavior history; means for the server to generate outfits based on the user's conditions; means for the server to create images of the generated outfits; means for the terminal to present the generated outfits to the user; means for the terminal to receive feedback from the user; means for the terminal to acquire the user's emotional data; means for the server to analyze the user's emotional data using an emotional engine; and means for the server to generate outfits again based on the feedback and emotional data. This makes it possible to provide highly personalized outfits based on emotional states.

[1254] A "user" refers to an individual who uses this system to receive styling suggestions.

[1255] "Profile information" refers to basic personal information that users enter, such as their name, age, and gender.

[1256] "Preferences" refer to the user's input regarding style, color, brand, and other tastes.

[1257] "Budget" refers to the range of money users are willing to spend on an outfit.

[1258] A "server" refers to a central computer that controls the entire system and processes, stores, and analyzes data.

[1259] A "data source" refers to a source of information from which product data is obtained from multiple brands and other information providers.

[1260] "Product data" refers to information about each product, including images, price, category, and features.

[1261] "Learning" refers to the process of analyzing product data acquired using machine learning algorithms, extracting features, and updating the model.

[1262] "Behavioral history" refers to data such as search history and purchase history that a user has performed within the system.

[1263] "Preferences and tendencies" refer to the user's interests and preferences, which are analyzed based on the user's behavioral history.

[1264] "Conditions" refer to constraints and requests such as budget, style, and preferences specified by the user.

[1265] "Coordination" refers to a set of fashion items created by combining multiple products selected based on the user's criteria.

[1266] "Emotional data" refers to data about the user's emotional state obtained through methods such as facial recognition and voice analysis.

[1267] An "emotion engine" refers to a system or algorithm used to analyze user emotional data.

[1268] "Feedback" refers to the user's opinion and evaluation of the suggested outfit.

[1269] "Terminal" refers to a device that a user interacts with, including applications and websites.

[1270] This invention is a system that generates and proposes optimal outfits to users based on their profile information, preferences, and budget. Furthermore, it can analyze the user's emotional data using an emotion engine to provide personalized suggestions. The specific configuration and operation procedure of this system are described below.

[1271] System Configuration

[1272] This system consists of the following main components:

[1273] 1. User terminal:

[1274] A dedicated application or website is used. Users enter their profile information, preferences, and budget here.

[1275] It is equipped with cameras and microphones to acquire emotional data.

[1276] 2. Server:

[1277] A server for acquiring and learning from product data. It collects product data from multiple data sources.

[1278] The system stores and analyzes user behavior history and feedback data.

[1279] The emotion engine is activated to analyze the user's emotional data.

[1280] Data processing flow

[1281] 1. User registration:

[1282] Device: Users enter their profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget using a dedicated application or website. This information is then sent to the server.

[1283] Server: Stores the transmitted information in the database.

[1284] 2. Acquisition of product data:

[1285] Server: Retrieves the latest product data (images, prices, categories, features, etc.) from multiple brands via API. This data is stored in a database.

[1286] 3. Learning from product data:

[1287] Server: Applies machine learning algorithms and trains the model on product data. The Python library Scikit-learn can be used here. Extracts product features and updates the model.

[1288] 4. Acquisition and integration of user behavior history:

[1289] Server: Obtains user activity history (search history, purchase history, etc.). This data is obtained through messaging interfaces such as the LINE interface.

[1290] Server: Integrates and analyzes behavioral history data into a database. This allows for the understanding of user preferences and trends.

[1291] 5. Coordination generation:

[1292] User: Enter your desired style and budget using the dedicated app or website.

[1293] Terminal: Sends this information to the server.

[1294] Server: Based on the input conditions, the server selects the most suitable products and generates a coordinated outfit. Then, it uses an image processing library (e.g., OpenCV) to create an image of the generated outfit. The image is sent to the terminal and presented to the user.

[1295] 6. Acquisition and analysis of emotional data:

[1296] Device: Uses a camera and microphone to acquire user emotion data (facial expressions, voice, etc.) when suggesting outfit combinations.

[1297] Terminal: Sends emotional data to the server.

[1298] Server: Analyzes sentiment data using an emotion engine (e.g., Amazon Rekognition or Google Cloud Speech-to-Text).

[1299] 7. Processing feedback:

[1300] Terminal: Receives user feedback and sends it to the server. If the user is not satisfied with the coordination, they can enter that information.

[1301] Server: Analyzes feedback and emotion data to generate a new outfit. The generated new outfit is then sent back to the terminal.

[1302] 8. Final confirmation and purchase:

[1303] User: Review the new outfit and proceed to purchase if satisfied. This cycle is repeated until the user is satisfied.

[1304] Specific example

[1305] For example, let's say a user wants a casual outfit. The user enters information such as their name, age, gender, style preferences, and budget on the app. The user then enters a prompt message such as, "I'd like a casual outfit. My budget is under 50,000 yen, and I like denim jackets and sneakers."

[1306] The server retrieves the latest product data from multiple data sources and stores it in a database. It uses machine learning algorithms to learn from the data and understand the characteristics of each product. When a user enters their desired style and budget in the app, that information is sent to the server. Based on these conditions, the server selects the most suitable products and generates outfit ideas.

[1307] The generated outfit is sent as an image to the device and presented to the user. During this process, the device uses its camera and microphone to capture the user's emotional data and send it to the server. If the user is not satisfied with the suggested outfit, they provide feedback. The server analyzes the feedback and emotional data to generate a new outfit. This process is repeated until the user is satisfied.

[1308] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1309] Step 1: User Registration

[1310] Terminal: Users enter profile information such as name, age, gender, style preferences, and budget using a dedicated application or website. The entered information is organized according to a database structure and sent to the server.

[1311] Server: Stores the submitted user profile information in the database and creates records to manage information for each user. The input is the user's profile information, and the output is the user information stored in the database.

[1312] Step 2: Obtain product data

[1313] Server: The server calls APIs from multiple data sources to retrieve the latest product data. The retrieved product data includes images, prices, categories, features, etc. The input is product data retrieved from the APIs, and the output is product data stored in the database in a standardized format.

[1314] Step 3: Learn product data

[1315] Server: The server launches a machine learning algorithm and learns from product data stored in the database. Specifically, it uses libraries such as Python's Scikit-learn to extract product features. The input is product data, and the output is updated information for the model that has grasped the features.

[1316] Step 4: Acquisition and integration of behavioral history

[1317] Server: The server uses a messaging interface (e.g., LINE interface) to retrieve user activity history. This activity history includes search history and purchase history. The input is activity history data from the interface, and the output is activity history information integrated into a database.

[1318] Step 5: Creating the outfit

[1319] User: Enter your desired style and budget using the dedicated app or website.

[1320] Terminal: Sends this information to the server.

[1321] Server: The server selects the most suitable products based on the input conditions. After selection, it creates an image of the coordinated outfit using an image processing library such as OpenCV. The input is the user's condition information, and the output is the generated image of the coordinated outfit.

[1322] Step 6: Acquisition and analysis of emotional data

[1323] Device: When suggesting outfit combinations, the camera and microphone are used to acquire user emotion data (facial expressions, voice, etc.).

[1324] Terminal: Sends acquired emotion data to the server. Input is emotion data acquired from the camera and microphone, and output is emotion data sent to the server.

[1325] Server: Analyzes sentiment data using an emotion engine. For example, Amazon Rekognition or Google Cloud Speech-to-Text are used. The analysis results are stored as analysis data.

[1326] Step 7: Proposal to the user

[1327] Terminal: Displays the generated outfit image to the user.

[1328] User: Review the outfit and enter feedback. Input is the outfit image and user feedback, and output is the user's feedback information.

[1329] Terminal: Sends feedback information to the server. Input is user feedback information, and output is the transmission of feedback to the server.

[1330] Step 8: Processing Feedback

[1331] Server: The server analyzes feedback and sentiment data and generates new outfits that reflect the user's new requests and areas for improvement. The input is user feedback information and sentiment data, and the output is new outfit information.

[1332] Server: Sends the newly generated outfit image to the terminal. The input is the new outfit information, and the output is the outfit image sent to the terminal.

[1333] Step 9: Final Review and Proposal

[1334] Terminal: Presents a new outfit to the user and allows for final confirmation.

[1335] User: Review the new outfit and proceed to purchase if satisfied. The input is the image of the new outfit, and the output is the user's final confirmation information.

[1336] Terminal: Sends the user's final confirmation information to the server to proceed with the purchase process. The input is the final confirmation information, and the output is the progress of the purchase process.

[1337] (Application Example 2)

[1338] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1339] Traditional styling suggestion systems, while based on user profile information, preferences, and budget, had the challenge of not being able to provide personalized suggestions that took into account the user's instantaneous emotions and reactions. As a result, it was difficult to provide suggestions that truly satisfied users and to fully stimulate their purchasing intent. Furthermore, it was difficult to efficiently learn from product data of multiple brands and to conduct detailed analysis of user behavior history.

[1340] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1341] In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple brands; means for the server to learn the acquired product data; means for the server to acquire the user's behavior history; means for the server to analyze preferences and tendencies based on the user's behavior history; means for the server to generate outfits based on the user's conditions; means for the server to create images of the generated outfits; means for the terminal to present the generated outfits to the user; means for the terminal to receive feedback from the user; means for the server to generate outfits again based on the feedback; means for using an emotion engine to acquire the user's emotional data when suggesting outfits; and means for the server to analyze the acquired emotional data and make suggestions that are optimal for the user's emotional state. This makes it possible to provide more personalized outfit suggestions that take into account the user's instantaneous emotional state.

[1342] "A means for users to input at least profile information, preferences, and budget" refers to a system consisting of an interface for users to input their basic information, clothing preferences, and the amount of money they are willing to spend.

[1343] "A means for a server to acquire product data from multiple brands" refers to a system in which a server has the function of collecting the latest product information from multiple brands via APIs, etc., and storing it in a database.

[1344] "Means for a server to learn from acquired product data" refers to a system that has the function of analyzing product data collected by the server using machine learning algorithms, and understanding and learning its characteristics.

[1345] "Means by which a server obtains a user's behavioral history" refers to a system that includes interfaces and functions for a server to collect a user's past behavioral history, such as search history and purchase history.

[1346] "Means by which a server analyzes user preferences and trends based on user behavior history" refers to a system that includes algorithms and analytical functions for analyzing the behavior history collected by the server and deriving user preferences and purchasing trends.

[1347] "A means by which a server generates outfits based on user conditions" refers to a system equipped with an algorithm that generates the optimal outfit based on the conditions (profile information, preferences, budget) entered by the user.

[1348] "Means for creating images of server-generated outfits" refers to a system that has the function of converting server-generated outfits into images so that they can be presented to the user in a visual format.

[1349] "Means for the terminal to present the generated coordinate to the user" refers to a system that includes an interface for the user's terminal to display the coordinate image sent from the server.

[1350] "A means for a device to receive user feedback" refers to a system that includes an interface for users to input their opinions and impressions of the proposed outfits.

[1351] "Means for the server to regenerate the coordination based on feedback" refers to a system equipped with an algorithm that allows the server to analyze user feedback and regenerate the optimal coordination based on the results.

[1352] "Methods for using an emotion engine to acquire user emotion data when suggesting outfits" refers to a system that includes an engine that recognizes the user's emotions towards the suggested outfits from their facial expressions and voice, and collects that data.

[1353] "A means of analyzing emotional data acquired by a server to provide optimal suggestions for the user's emotional state" refers to a system that includes an algorithm and analysis function for analyzing emotional data collected by a server and providing optimal coordination suggestions for the user's emotional state.

[1354] This invention is a system that suggests the optimal outfit based on the user's profile information, preferences, budget, behavioral history, and emotional data. The system has the following configuration:

[1355] First, users create an account using a dedicated application or website. They enter profile information such as their name, age, gender, style preferences, and budget. This sends the user's basic data to the server.

[1356] The server retrieves product data (images, prices, categories, features, etc.) from multiple brands via APIs. This product data is stored in a database. The server then uses machine learning algorithms to learn from this data and extract the features of each product. Specifically, the Python programming language and the Keras library are used in this process.

[1357] Next, the server retrieves the user's activity history via the API interface. This activity history includes past search and purchase history. This information is also stored in the database and used to analyze the user's preferences and trends.

[1358] To generate an outfit, the user enters their desired style and budget via a dedicated app or website. The server receives this information and selects the most suitable items based on the user's criteria. The selected items are combined into an outfit, and an image of the outfit is generated. The generated outfit image is sent to the device and presented to the user.

[1359] When an outfit is presented to the user, the emotion engine acquires the user's emotion data. This emotion data is analyzed from the user's facial expressions and voice. This includes analyzing facial expressions from camera frames using the OpenCV library. The acquired emotion data is sent to the server for analysis.

[1360] The server analyzes user feedback and sentiment data to generate new outfits that reflect the user's new requests and reactions. This allows for personalized suggestions to be made to the user. This process is repeated until an outfit that the user is ultimately satisfied with is generated.

[1361] To give a concrete example, a user creates an account and enters profile information, preferences, and budget. For example, if the budget is under 50,000 yen and the user's favorite items are denim jackets and sneakers, the server retrieves the latest product data from each brand and stores it in a database. A machine learning algorithm is used to learn from the product data and understand its characteristics. The server also analyzes the user's preferences using their past search and purchase history. Then, it generates the optimal outfit and presents the image to the user. At this time, the emotion engine captures the user's reaction and sends it to the server. If the user is not satisfied with the outfit, they send feedback from their device. The server analyzes the emotion data and feedback, generates a new outfit, and presents it.

[1362] An example of an input prompt for a generative AI model is as follows:

[1363] "User profile information: name, age, gender, style preferences, budget. Product data: brand, price, category, features. Sentiment data: emotional state, confidence level."

[1364] This makes it possible to offer more personalized outfit suggestions that take into account the user's emotional state at that moment.

[1365] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1366] Step 1:

[1367] User Registration

[1368] Users create an account using a dedicated application or website. They enter profile information such as their name, age, gender, style preferences, and budget.

[1369] (Input): User profile information

[1370] (Processing): Send the entered information to the server.

[1371] (Output): User information stored on the server

[1372] Step 2:

[1373] Product data acquisition

[1374] The server retrieves product data from multiple brands via APIs. This product data includes images, prices, categories, and features.

[1375] (Input): API endpoint, brand information

[1376] (Processing): Retrieve data via API call and store it in the database.

[1377] (Output): Product data stored in the database

[1378] Step 3:

[1379] Learning about product specifications

[1380] The server learns from acquired product data using machine learning algorithms. This allows it to understand and classify the characteristics of each product.

[1381] (Input): Product data

[1382] (Processing): Analyze and learn data using machine learning algorithms (using Python and Keras).

[1383] (Output): Product Feature Model

[1384] Step 4:

[1385] Acquisition of user behavior history

[1386] The server retrieves the user's behavioral history (past search history and purchase history) via an API interface. This information is also stored in the database.

[1387] (Input): User ID, API endpoint

[1388] (Processing): Obtain behavioral history via API and store it in the database.

[1389] (Output): Behavioral history data stored in the database

[1390] Step 5:

[1391] Analysis of preferences and tendencies

[1392] The server analyzes user preferences and tendencies based on the acquired behavioral history. This reveals the user's basic preferences and purchasing trends.

[1393] (Input): Behavioral history data

[1394] (Processing): Analyze user preferences and trends using data analysis algorithms.

[1395] (Output): User-preferred model

[1396] Step 6:

[1397] Coordination generation

[1398] The server selects the most suitable products based on the user's conditions (profile information, preferences, budget) and generates outfit combinations using a generative AI model.

[1399] (Input): User information, product feature model, user preference model

[1400] (Processing): The AI ​​model generates the outfits and produces an image.

[1401] (Output): Image of the generated coordinate

[1402] Step 7:

[1403] Coordination suggestions

[1404] The terminal presents the generated outfit to the user. During this process, the emotion engine acquires the user's emotion data. This emotion data is sent to the server.

[1405] (Input): Coordinated image, camera frame

[1406] (Processing): Display the coordinated image, acquire emotion data using the emotion engine, and send it to the server (using OpenCV).

[1407] (Output): User sentiment data

[1408] Step 8:

[1409] Obtaining feedback

[1410] The device receives feedback from the user. The user enters their opinions and impressions of the suggested outfit. The feedback data is also sent to the server.

[1411] (Input): User Feedback

[1412] (Processing): Send feedback to the server

[1413] (Output): Feedback data stored on the server

[1414] Step 9:

[1415] Generating a new outfit

[1416] The server analyzes feedback and sentiment data and generates new outfits that reflect the user's new requests and reactions.

[1417] (Input): Sentiment data, feedback data

[1418] (Processing): Analyze this data using a data analysis algorithm and generate a new coordinate.

[1419] (Output): Image of the new outfit

[1420] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1421] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1422] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1423] [Fourth Embodiment]

[1424] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1425] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1426] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1427] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1428] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1429] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1430] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1431] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1432] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1433] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1434] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1435] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1436] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1437] This invention is a system that generates and proposes the optimal outfit to a user based on their profile information, preferences, and budget. Specifically, the server acquires product data from multiple brands, learns from it using a machine learning algorithm, and analyzes it based on the user's behavior history and feedback data to generate the optimal outfit for the user's conditions. The following describes the processing of this system's program in natural language.

[1438] Program processing flow

[1439] 1. User Registration

[1440] Device: Users create an account using a dedicated application or website. Here, users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[1441] User: Follow the appropriate prompts to enter your information and complete the registration process.

[1442] Terminal: Sends the entered information to the server.

[1443] 2. Acquisition and learning of product data

[1444] Server: Uses each brand's API to retrieve product data (images, price, category, features, etc.).

[1445] Server: Stores the retrieved product data in the database.

[1446] Server: Uses machine learning algorithms to learn from acquired product data and extracts the characteristics of each product.

[1447] 3. Integration with the LINE interface

[1448] Server: Acquires user activity history (search history, purchase history, etc.) through the LINE interface (data cleanroom).

[1449] Server: Aggregates acquired behavioral history data and analyzes user preferences and trends.

[1450] 4. Coordination generation

[1451] User: Enter your budget and desired style using the dedicated app or website.

[1452] Terminal: Sends the entered budget and style information to the server.

[1453] Server: Selects the most suitable product from the database based on the user's conditions (budget, style, preferences).

[1454] Server: Combines selected products to generate the optimal outfit.

[1455] Server: Creates an image of the generated outfit and sends it to the terminal.

[1456] 5. Suggestions and feedback for users

[1457] Terminal: Displays the generated outfit image to the user.

[1458] User: Check the outfit and send feedback if there's anything you don't like.

[1459] Terminal: Sends user feedback to the server.

[1460] Server: Based on feedback, it regenerates the coordination and makes new suggestions that meet the user's requests.

[1461] Specific example

[1462] Let's say a user who has just created an account wants casual outfits. The user creates an account using a dedicated app and enters profile information, preferences, and budget. For example, if the user's budget is under 50,000 yen and their preferences are denim jackets and sneakers, this information is sent to the server.

[1463] The server retrieves the latest product data from each brand and stores it in a database. Next, it uses machine learning algorithms to learn from the product data and understand the characteristics of each product. Using the LINE interface, it retrieves products that the user has previously searched for and purchased within LINE, and analyzes the user's preferences in more detail.

[1464] Based on the user's budget and preferences, the server selects appropriate items from its database and combines them to generate the optimal outfit. The generated outfit image is sent to the terminal and displayed to the user. The user reviews the outfit and sends feedback if they are dissatisfied with it. The server receives the feedback, generates a new outfit, and provides the user with an optimized new suggestion.

[1465] Thus, the present invention is a system that integrates data from multiple brands and utilizes user behavior history and feedback to provide highly accurate, personalized outfits. This system allows users to easily find the optimal fashion that suits their preferences and budget, leading to increased satisfaction.

[1466] The following describes the processing flow.

[1467] Step 1:

[1468] User Registration

[1469] Device: The user launches the app or website and accesses the account registration screen.

[1470] User: Enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[1471] Terminal: Sends the entered information to the server.

[1472] Step 2:

[1473] Product data acquisition

[1474] Server: Calls APIs from multiple brands to retrieve the latest product data (images, prices, categories, features, etc.).

[1475] Server: Stores the retrieved product data in the database.

[1476] Step 3:

[1477] Learning about product specifications

[1478] Server: Starts a machine learning algorithm and learns from product data stored in the database.

[1479] Server: Extracts the characteristics of each product and updates the model to accommodate user preferences.

[1480] Step 4:

[1481] Integration with LINE interface

[1482] Server: Obtains user activity history (search history, purchase history, etc.) via the LINE interface.

[1483] Server: Integrates acquired behavioral history into a database and analyzes user preferences and trends.

[1484] Step 5:

[1485] Coordination generation

[1486] User: Enter your budget and desired style using the dedicated app or website.

[1487] Terminal: Sends information about the budget and style entered by the user to the server.

[1488] Server: Selects the most suitable product based on the user's conditions (budget, style, preferences).

[1489] Server: Combines selected products to generate outfit combinations.

[1490] Server: Creates an image of the generated outfit and sends it to the terminal.

[1491] Step 6:

[1492] Suggestions for users

[1493] Terminal: Displays the generated outfit image to the user.

[1494] User: Review the outfit and provide feedback as needed.

[1495] Step 7:

[1496] Feedback processing

[1497] Terminal: Sends user feedback to the server.

[1498] Server: Analyzes feedback and generates new coordinates that reflect new user requests and improvements.

[1499] Server: Sends the newly generated outfit image to the terminal.

[1500] Step 8:

[1501] Final confirmation and proposal

[1502] Terminal: Presents a new outfit to the user and allows for final confirmation.

[1503] User: Review the new outfit and proceed to purchase if satisfied.

[1504] In this way, a system is realized in which the server, terminal, and user work together at each step to provide highly personalized coordination.

[1505] (Example 1)

[1506] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1507] Traditional fashion coordination systems often struggled to provide suggestions that matched users' preferences and budgets. Furthermore, there was a lack of systems that effectively utilized product data from multiple brands and generated personalized outfits based on user behavior history and feedback. There was also a need for a system that could efficiently provide new outfit suggestions by reflecting user feedback in real time.

[1508] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1509] In this invention, the server includes means for acquiring product data from multiple online shops, means for analyzing the product data, and means for acquiring the user's behavior history. This makes it possible to select the most suitable products for the user and generate personalized outfits based on the user's profile information, preferences, and budget. Furthermore, by analyzing the user's behavior history and understanding the user's preferences and tendencies, more accurate suggestions become possible. In addition, by presenting the generated outfit images to the user and receiving feedback from the user, it is possible to generate new outfits in real time that meet the user's requests. In this way, users can easily find fashion outfits that match their preferences and budget, and their satisfaction level increases.

[1510] A "user" refers to an individual or group that uses this system to generate and receive fashion coordination suggestions.

[1511] A "server" refers to a computer system that acquires and analyzes product data, collects and analyzes behavioral history, and generates product coordination.

[1512] "Device" refers to a device used by users to input profile information, preferences, and budget, as well as a device for receiving styling suggestions and feedback.

[1513] An "online shop" refers to a store or platform that sells products via the internet.

[1514] "Product data" refers to information such as product images, prices, categories, and features obtained from online shops.

[1515] "Behavioral history" refers to records of actions a user has taken in the past, such as search history and purchase history.

[1516] "Profile information" refers to personal information such as the user's name, age, and gender.

[1517] "Preferences" refer to the user's tastes in fashion styles, colors, brands, and other preferences.

[1518] "Budget" refers to the amount of money a user plans to spend on fashion coordination.

[1519] "Feedback" refers to the opinions and requests that users provide regarding the suggested outfits.

[1520] A "messaging platform interface" refers to a means of data communication used to obtain a user's behavioral history.

[1521] "Coordination" refers to a suggestion created by combining multiple fashion items.

[1522] This invention is a system that generates and proposes optimal fashion coordinates to users based on their profile information, preferences, and budget. Specific embodiments of this system are described in detail below.

[1523] Hardware and software configuration

[1524] 1. Server

[1525] Hardware: Server machines with high-performance processors, sufficient memory and storage (e.g., Dell PowerEdge series or HPE ProLiant series)

[1526] software:

[1527] API access: REST API provided by each online shop

[1528] Database: NoSQL database (e.g., MongoDB)

[1529] Machine learning algorithms: TensorFlow or PyTorch

[1530] Big data analysis tool: Apache Spark

[1531] Image processing: Adobe Photoshop API

[1532] 2. Terminal

[1533] Hardware: Devices used by users to input information (e.g., smartphones, tablets, desktop computers)

[1534] software:

[1535] User Interface: Dedicated application or web browser (e.g., iOS / Android app, Google Chrome)

[1536] Operating procedure and specific examples

[1537] User Registration

[1538] Users enter profile information such as their name, age, gender, fashion preferences (e.g., casual, formal), favorite brands, and budget using a dedicated application or website. For example, if a user enters "Name: Ichiro Tanaka," "Age: 30," "Gender: Male," "Preference: Casual," "Brands: Any," and "Budget: 50,000 yen," this information is sent from the device to the server.

[1539] Product data acquisition and analysis

[1540] The server retrieves product data through the online shop's API. The retrieved data is stored in a NoSQL database and analyzed using machine learning algorithms. For example, the server retrieves data for "denim jacket" and "price: 10,000 yen" from "online shop A" and extracts its features using a machine learning model.

[1541] Acquisition and analysis of behavioral history

[1542] The server uses a messaging platform interface to retrieve user activity history (e.g., purchase history, search history). This retrieved activity history is analyzed using big data analytics tools to understand user preferences and trends. For example, if a user has frequently purchased "denim jackets" and "sneakers" in the past, this information is aggregated on the server.

[1543] Coordination generation

[1544] The user enters "casual style" and a "budget of 50,000 yen" into a dedicated app or website. The device sends this information to the server. The server queries the database and selects items that fit the "casual style" and "budget of 50,000 yen or less." The selected items are coordinated to the user's preferences, such as a "denim jacket" and "sneakers." An image of the generated coordinated outfit is created and sent to the device.

[1545] Obtaining feedback and making revisions

[1546] The generated outfit is displayed on the device, and the user reviews it. If the user provides feedback such as "I'd like the sneakers to be a different brand," this information is sent to the server. Based on the feedback, the server re-selects items and generates outfits, creating new suggestions. For example, it might select sneakers from a different brand and generate a new outfit idea.

[1547] Examples of prompt statements

[1548] "Based on the user's profile information and budget, which indicate they prefer a casual style, please provide the best possible suggestions."

[1549] "Please generate new outfits, taking into account the user's past behavior history and feedback."

[1550] Through the above procedure, the system of the present invention can effectively integrate data from multiple online shops and generate and provide optimal fashion coordinates tailored to the user's preferences and budget.

[1551] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1552] Step 1: User Registration

[1553] Input: The user enters their name, age, gender, fashion preferences, favorite brands, and budget into a dedicated application or website.

[1554] Operation: The terminal collects information entered by the user and sends it to the server in JSON format. The user interface provided by the terminal has an input form, and the user enters information using buttons or text fields for each item.

[1555] Output: User profile information, preferences, and budget are sent to the server. The server receives this information and stores it in the user database.

[1556] Step 2: Acquisition and analysis of product data

[1557] Input: API endpoint information for each online shop

[1558] Operation: The server uses a REST API to retrieve product data (images, price, category, features, etc.) from each online shop. The retrieved data is stored in a NoSQL database (e.g., MongoDB). The server then uses TensorFlow or PyTorch to train a machine learning model and analyze the product data.

[1559] Output: Product data is stored on the server, and the analyzed results are reflected in the machine learning model.

[1560] Step 3: Acquisition and analysis of behavioral history

[1561] Input: User activity history (search history, purchase history, etc.)

[1562] Operation: The server retrieves user behavior history data through a messaging platform interface. The retrieved data is aggregated and analyzed using big data analytics tools (e.g., Apache Spark) to analyze user preferences and trends. For example, data on products the user has purchased or searched for in the past six months is analyzed.

[1563] Output: The server stores the user's behavioral history, and analysis results of preferences and trends based on that history are obtained.

[1564] Step 4: Creating the outfit

[1565] Input: User criteria (budget, style, preferences), product data, behavioral history data

[1566] Operation: The user enters criteria such as "casual style" and "budget of 50,000 yen" in a dedicated app or website. The device sends this information to the server. The server queries the database and selects products that match the user's criteria. It combines the selected products to generate the optimal outfit and creates an outfit image using the Adobe Photoshop API, etc.

[1567] Output: The generated outfit and its image are created on the server and sent to the terminal.

[1568] Step 5: Obtain feedback and make revisions

[1569] Input: User feedback

[1570] Operation: The device displays the generated outfit image to the user. When the user enters feedback such as "I want the sneakers to be a different brand," the device sends that feedback to the server. The server receives the feedback, selects new items, and generates the outfit again.

[1571] Output: The newly generated outfit image is created on the server, sent back to the terminal, and presented to the user.

[1572] Through the processing steps described above, this system can efficiently generate and provide fashion coordinates that match the user's preferences and budget.

[1573] (Application Example 1)

[1574] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1575] Currently, many users spend time and effort finding the perfect outfit that suits their preferences and budget when purchasing fashion items online. Furthermore, the inability to try on items online leads to concerns about their appearance and fit, resulting in low post-purchase satisfaction. Therefore, there is a need for a system that suggests optimal outfits based on user preferences and budget, and also provides a virtual try-on experience.

[1576] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1577] In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple brands; and means for the server to learn the acquired product data. This makes it possible not only to generate an optimal outfit based on the user's preferences and budget, but also to confirm the outfit and receive feedback through virtual try-on.

[1578] A "user" is someone who uses the system to input their profile information, preferences, and budget, and then receives suggestions for the most suitable outfit.

[1579] "Profile information" refers to basic personal information entered by the user, including, for example, name, age, and gender.

[1580] "Preferences" refer to a user's personal tastes in fashion, such as their favorite styles, colors, and brands.

[1581] "Budget" refers to the maximum amount of money a user can spend on coordinating an outfit.

[1582] A "server" is a central computer system that handles tasks such as acquiring product data, learning, analyzing user behavior history, and generating coordinated product combinations.

[1583] A "brand" is a group of products offered by a specific company or designer.

[1584] "Product data" refers to information about a product, including images, price, category, and features.

[1585] "Learning" refers to the process where the server uses machine learning algorithms to extract features from product data and generate outfits that match the user's preferences.

[1586] "Behavioral history" refers to data that includes the user's search history and purchase history when using the system.

[1587] "Analysis" is the process of understanding user preferences and trends based on behavioral history data acquired by the server.

[1588] A "coordinate" is a combination of multiple fashion items selected based on the user's preferences and budget.

[1589] An "image" is a visual representation of an outfit and is presented to the user.

[1590] A "terminal" is a device that allows users to access the system, receive coordination suggestions, and send feedback.

[1591] "Feedback" refers to the evaluations and comments that users make regarding the suggested outfits.

[1592] "Virtual try-on" is a feature that allows users to try out outfits in a virtual space, and check the appearance and fit in real time.

[1593] A "communication interface" is a means of connection that allows a server to obtain a user's behavior history from an external system.

[1594] This invention relates to a system that allows users to input their profile information, preferences, and budget using a smartphone or smart glasses, and virtually try on the optimal outfit. This system provides the optimal outfit based on the user's preferences and budget, and further allows them to try on the actual look and fit through virtual try-on.

[1595] The server performs the following processes: First, it receives the user's profile information, preferences, and budget, and uses this information to retrieve product data from each brand's API. Next, it uses a machine learning algorithm to learn from the retrieved product data and extract the characteristics of each product. This learning process uses the Python library scikit-learn. Furthermore, it obtains the user's behavioral history (search history, purchase history, etc.) through the communication interface and uses this to analyze the user's preferences and tendencies.

[1596] In the outfit generation process, the system selects the most suitable items from a database based on the user's entered budget and style, and then combines them to generate an outfit. The server creates an image of the generated outfit and sends it to the user's device. The device then displays the generated outfit to the user and provides a virtual try-on function. The ability to check the appearance and fit of the outfit in real time operates on smart glasses or smartphones.

[1597] If a user is dissatisfied with the outfit after a virtual try-on, they can submit feedback. The device sends the feedback to the server, which then regenerates the outfit based on the feedback and provides an optimized new suggestion.

[1598] As a concrete example, consider a case where a user desires a casual style and has a budget of 50,000 yen or less. In this case, the user opens the application and enters their profile information, preferences, and budget. The server then retrieves appropriate product data from the database and learns from it using a machine learning algorithm. Based on the user's past behavior history, an optimal outfit is generated. The generated outfit is provided to the user using a virtual try-on function, allowing the user to check the outfit as if they were actually trying it on.

[1599] Example of a prompt:

[1600] "Based on the user's profile information, please suggest the best casual outfit within a budget of 50,000 yen. Please also consider their past search and purchase history."

[1601] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1602] Step 1: User Registration

[1603] Users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget using a dedicated application or website. The entered information is sent from the device to the server. This registers the user's basic information and purchase conditions on the server.

[1604] Input: Profile information, preferences, budget

[1605] Output: User ID, registration completion confirmation

[1606] Step 2: Acquire and learn product data

[1607] The server uses each brand's API to retrieve product data (images, price, category, features, etc.) and stores it in a database. Next, a machine learning algorithm (e.g., scikit-learn's RandomForestClassifier) ​​is used to train the server on the product data and extract the features of each product.

[1608] Input: Product data from brand API

[1609] Output: Trained model, feature data

[1610] Step 3: Acquisition and analysis of behavioral history

[1611] The server obtains user behavior history (search history, purchase history, etc.) through a communication interface. Based on this data, user preferences and trends are analyzed using machine learning algorithms.

[1612] Input: Behavioral history data

[1613] Output: Analysis results of preferences and tendencies

[1614] Step 4: Creating the outfit

[1615] When a user enters their budget and desired style through a dedicated app or website, that information is sent to the server. Based on the user's conditions (budget, style, and preferences), the server uses a trained model to select the most suitable items from its database and generate an outfit.

[1616] Input: Budget, Style (user input)

[1617] Output: Optimal coordination suggestion

[1618] Step 5: Virtual fitting provided

[1619] The server creates an image of the outfit and sends it to the device. The device then provides the user with a virtual try-on experience by displaying the outfit in real time via smart glasses or a smartphone.

[1620] Input: Generated coordinate data

[1621] Output: Virtual try-on images and feedback screen

[1622] Step 6: Collecting User Feedback

[1623] Users review suggested outfits through virtual try-on and submit feedback if they are dissatisfied. The device then forwards this feedback to the server.

[1624] Input: User Feedback

[1625] Output: Receiving and transferring feedback data

[1626] Step 7: Generating a re-coordinate based on feedback

[1627] The server regenerates the coordination based on the user's feedback data. This generates new suggestions that meet the user's requests and sends them to the terminal.

[1628] Input: Feedback data

[1629] Output: Newly optimized outfit suggestions

[1630] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1631] This invention is a system that generates and proposes optimal outfits based on user-entered profile information, preferences, and budget, and further analyzes the user's emotional data using an emotion engine to provide even more personalized suggestions. Specifically, the server acquires product data from multiple brands, learns from it using a machine learning algorithm, and analyzes it based on the user's behavior history and feedback data. Furthermore, by analyzing the user's emotional data using an emotion engine, the system generates outfits that are optimal for the user's conditions and emotional state. The processing of this system's program is described below in natural language.

[1632] Program processing flow

[1633] 1. User Registration

[1634] Device: Users create an account using a dedicated application or website. Here, users enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[1635] User: Follow the appropriate prompts to enter your information and complete the registration process.

[1636] Terminal: Sends the entered information to the server.

[1637] 2. Acquisition of product data

[1638] Server: Calls APIs from multiple brands to retrieve the latest product data (images, prices, categories, features, etc.).

[1639] Server: Stores the retrieved product data in the database.

[1640] 3. Learning from product data

[1641] Server: Starts a machine learning algorithm and learns from product data stored in the database.

[1642] Server: Extracts the characteristics of each product and updates the model to accommodate user preferences.

[1643] 4. Integration with the LINE interface

[1644] Server: Obtains user activity history (search history, purchase history, etc.) via the LINE interface.

[1645] Server: Integrates acquired behavioral history into a database and analyzes user preferences and trends.

[1646] 5. Coordination generation

[1647] User: Enter your budget and desired style using the dedicated app or website.

[1648] Terminal: Sends information about the budget and style entered by the user to the server.

[1649] Server: Selects the most suitable product based on the user's conditions (budget, style, preferences).

[1650] Server: Combines selected products to generate outfit combinations.

[1651] Server: Creates an image of the generated outfit and sends it to the terminal.

[1652] 6. Acquisition of emotional data

[1653] Terminal: When suggesting outfits, the emotion engine acquires the user's emotional data (e.g., facial recognition and voice analysis).

[1654] Terminal: Sends acquired emotion data to the server.

[1655] 7. Suggestions for Users

[1656] Terminal: Displays the generated outfit image to the user.

[1657] User: Review the outfit and provide feedback as needed.

[1658] Terminal: Sends emotional data acquired when suggesting outfits to the server.

[1659] 8. Processing Feedback

[1660] Terminal: Sends user feedback to the server.

[1661] Server: Analyzes feedback and sentiment data to generate new outfits that reflect the user's new requests and areas for improvement.

[1662] Server: Sends the newly generated outfit image to the terminal.

[1663] 9. Final confirmation and proposal

[1664] Terminal: Presents a new outfit to the user and allows for final confirmation.

[1665] User: Review the new outfit and proceed to purchase if satisfied.

[1666] Specific example

[1667] For example, let's say a user who has just created an account wants a casual outfit. The user creates an account using the app and enters profile information, preferences, and budget. In this case, the budget is under 50,000 yen, and the favorite items are a denim jacket and sneakers. This information is sent to the server.

[1668] The server retrieves the latest product data from each brand and stores it in a database. Then, it uses machine learning algorithms to learn from the product data and understand the characteristics of each product. Furthermore, it uses the LINE interface to retrieve products that users have previously searched for and purchased within LINE, and analyzes user preferences.

[1669] Based on the user's budget and preferences, the server selects appropriate items from a database and combines them to generate the optimal outfit. The generated outfit image is sent to the device and displayed to the user. At this time, the emotion engine acquires emotion data from the user's facial expressions and voice and sends it to the server. If the user is not satisfied with the outfit, they can send feedback from their device.

[1670] The server analyzes emotional data and feedback, generates a new outfit reflecting the user's new requests and suggestions for improvement, and sends the image to the device. The device then presents the new outfit to the user, and if the user is satisfied, it proceeds to the purchase process.

[1671] Thus, the present invention is a system that provides highly personalized coordination based on the user's emotional state by combining an emotion engine, thereby further improving user satisfaction.

[1672] The following describes the processing flow.

[1673] Step 1:

[1674] User Registration

[1675] Device: The user launches the app or website and accesses the account registration screen.

[1676] User: Enter profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget.

[1677] Terminal: Sends the entered information to the server.

[1678] Step 2:

[1679] Product data acquisition

[1680] Server: Calls APIs from multiple brands to retrieve product data (images, prices, categories, features, etc.).

[1681] Server: Stores the retrieved product data in the database.

[1682] Step 3:

[1683] Learning about product specifications

[1684] Server: Starts a machine learning algorithm and learns from product data stored in the database.

[1685] Server: Extracts the characteristics of each product and updates the model.

[1686] Step 4:

[1687] Integration with LINE interface

[1688] Server: Obtains user activity history (search history, purchase history, etc.) via the LINE interface.

[1689] Server: Integrates acquired behavioral history into a database and analyzes user preferences and trends.

[1690] Step 5:

[1691] Acquisition of emotional data

[1692] Terminal: When displaying suggested outfits, the emotion engine acquires the user's emotion data (such as facial recognition and voice analysis).

[1693] Terminal: Sends acquired emotion data to the server.

[1694] Step 6:

[1695] Coordination generation

[1696] User: Enter your budget and desired style using the dedicated app or website.

[1697] Terminal: Sends information about the budget and style entered by the user to the server.

[1698] Server: Selects the optimal product based on user criteria (budget, style, preferences) and sentiment data.

[1699] Server: Combines selected products to generate outfit combinations.

[1700] Server: Creates an image of the generated outfit and sends it to the terminal.

[1701] Step 7:

[1702] Suggestions for users

[1703] Terminal: Displays the generated outfit image to the user.

[1704] User: Review the outfit and provide feedback as needed.

[1705] Terminal: The emotion engine retrieves the user's emotion data regarding the suggestion and sends it to the server.

[1706] Step 8:

[1707] Feedback processing

[1708] Terminal: Sends user feedback to the server.

[1709] Server: Analyzes feedback and sentiment data to generate new outfits that reflect the user's new requests and areas for improvement.

[1710] Server: Sends the newly generated outfit image to the terminal.

[1711] Step 9:

[1712] Final confirmation and proposal

[1713] Terminal: Presents a new outfit to the user and allows for final confirmation.

[1714] User: Review the new outfit and proceed to purchase if satisfied.

[1715] This allows us to comprehensively utilize user profile information, preferences, budget, behavioral history, and even emotional data to provide users with the most suitable outfits. As a result, highly personalized suggestions become possible, improving user satisfaction.

[1716] (Example 2)

[1717] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1718] Conventional outfit suggestion systems generate outfits based on the user's basic information and behavioral history, but they do not provide personalized suggestions that take into account the user's emotional state. As a result, user satisfaction does not improve sufficiently, and user engagement in the process leading to a purchase decision sometimes decreases.

[1719] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple data sources; means for the server to learn the acquired product data; means for the server to acquire the user's behavior history; means for the server to analyze preferences and tendencies based on the user's behavior history; means for the server to generate outfits based on the user's conditions; means for the server to create images of the generated outfits; means for the terminal to present the generated outfits to the user; means for the terminal to receive feedback from the user; means for the terminal to acquire the user's emotional data; means for the server to analyze the user's emotional data using an emotional engine; and means for the server to generate outfits again based on the feedback and emotional data. This makes it possible to provide highly personalized outfits based on emotional states.

[1720] A "user" refers to an individual who uses this system to receive styling suggestions.

[1721] "Profile information" refers to basic personal information that users enter, such as their name, age, and gender.

[1722] "Preferences" refer to the user's input regarding style, color, brand, and other tastes.

[1723] "Budget" refers to the range of money users are willing to spend on an outfit.

[1724] A "server" refers to a central computer that controls the entire system and processes, stores, and analyzes data.

[1725] A "data source" refers to a source of information from which product data is obtained from multiple brands and other information providers.

[1726] "Product data" refers to information about each product, including images, price, category, and features.

[1727] "Learning" refers to the process of analyzing product data acquired using machine learning algorithms, extracting features, and updating the model.

[1728] "Behavioral history" refers to data such as search history and purchase history that a user has performed within the system.

[1729] "Preferences and tendencies" refer to the user's interests and preferences, which are analyzed based on the user's behavioral history.

[1730] "Conditions" refer to constraints and requests such as budget, style, and preferences specified by the user.

[1731] "Coordination" refers to a set of fashion items created by combining multiple products selected based on the user's criteria.

[1732] "Emotional data" refers to data about the user's emotional state obtained through methods such as facial recognition and voice analysis.

[1733] An "emotion engine" refers to a system or algorithm used to analyze user emotional data.

[1734] "Feedback" refers to the user's opinion and evaluation of the suggested outfit.

[1735] "Terminal" refers to a device that a user interacts with, including applications and websites.

[1736] This invention is a system that generates and proposes optimal outfits to users based on their profile information, preferences, and budget. Furthermore, it can analyze the user's emotional data using an emotion engine to provide personalized suggestions. The specific configuration and operation procedure of this system are described below.

[1737] System Configuration

[1738] This system consists of the following main components:

[1739] 1. User terminal:

[1740] A dedicated application or website is used. Users enter their profile information, preferences, and budget here.

[1741] It is equipped with cameras and microphones to acquire emotional data.

[1742] 2. Server:

[1743] A server for acquiring and learning from product data. It collects product data from multiple data sources.

[1744] The system stores and analyzes user behavior history and feedback data.

[1745] The emotion engine is activated to analyze the user's emotional data.

[1746] Data processing flow

[1747] 1. User registration:

[1748] Device: Users enter their profile information (name, age, gender, etc.), preferences (style, color, brand, etc.), and budget using a dedicated application or website. This information is then sent to the server.

[1749] Server: Stores the transmitted information in the database.

[1750] 2. Acquisition of product data:

[1751] Server: Retrieves the latest product data (images, prices, categories, features, etc.) from multiple brands via API. This data is stored in a database.

[1752] 3. Learning from product data:

[1753] Server: Applies machine learning algorithms and trains the model on product data. The Python library Scikit-learn can be used here. Extracts product features and updates the model.

[1754] 4. Acquisition and integration of user behavior history:

[1755] Server: Obtains user activity history (search history, purchase history, etc.). This data is obtained through messaging interfaces such as the LINE interface.

[1756] Server: Integrates and analyzes behavioral history data into a database. This allows for the understanding of user preferences and trends.

[1757] 5. Coordination generation:

[1758] User: Enter your desired style and budget using the dedicated app or website.

[1759] Terminal: Sends this information to the server.

[1760] Server: Based on the input conditions, the server selects the most suitable products and generates a coordinated outfit. Then, it uses an image processing library (e.g., OpenCV) to create an image of the generated outfit. The image is sent to the terminal and presented to the user.

[1761] 6. Acquisition and analysis of emotional data:

[1762] Device: Uses a camera and microphone to acquire user emotion data (facial expressions, voice, etc.) when suggesting outfit combinations.

[1763] Terminal: Sends emotional data to the server.

[1764] Server: Analyzes sentiment data using an emotion engine (e.g., Amazon Rekognition or Google Cloud Speech-to-Text).

[1765] 7. Processing feedback:

[1766] Terminal: Receives user feedback and sends it to the server. If the user is not satisfied with the coordination, they can enter that information.

[1767] Server: Analyzes feedback and emotion data to generate a new outfit. The generated new outfit is then sent back to the terminal.

[1768] 8. Final confirmation and purchase:

[1769] User: Review the new outfit and proceed to purchase if satisfied. This cycle is repeated until the user is satisfied.

[1770] Specific example

[1771] For example, let's say a user wants a casual outfit. The user enters information such as their name, age, gender, style preferences, and budget on the app. The user then enters a prompt message such as, "I'd like a casual outfit. My budget is under 50,000 yen, and I like denim jackets and sneakers."

[1772] The server retrieves the latest product data from multiple data sources and stores it in a database. It uses machine learning algorithms to learn from the data and understand the characteristics of each product. When a user enters their desired style and budget in the app, that information is sent to the server. Based on these conditions, the server selects the most suitable products and generates outfit ideas.

[1773] The generated outfit is sent as an image to the device and presented to the user. During this process, the device uses its camera and microphone to capture the user's emotional data and send it to the server. If the user is not satisfied with the suggested outfit, they provide feedback. The server analyzes the feedback and emotional data to generate a new outfit. This process is repeated until the user is satisfied.

[1774] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1775] Step 1: User Registration

[1776] Terminal: Users enter profile information such as name, age, gender, style preferences, and budget using a dedicated application or website. The entered information is organized according to a database structure and sent to the server.

[1777] Server: Stores the submitted user profile information in the database and creates records to manage information for each user. The input is the user's profile information, and the output is the user information stored in the database.

[1778] Step 2: Obtain product data

[1779] Server: The server calls APIs from multiple data sources to retrieve the latest product data. The retrieved product data includes images, prices, categories, features, etc. The input is product data retrieved from the APIs, and the output is product data stored in the database in a standardized format.

[1780] Step 3: Learn product data

[1781] Server: The server launches a machine learning algorithm and learns from product data stored in the database. Specifically, it uses libraries such as Python's Scikit-learn to extract product features. The input is product data, and the output is updated information for the model that has grasped the features.

[1782] Step 4: Acquisition and integration of behavioral history

[1783] Server: The server uses a messaging interface (e.g., LINE interface) to retrieve user activity history. This activity history includes search history and purchase history. The input is activity history data from the interface, and the output is activity history information integrated into a database.

[1784] Step 5: Creating the outfit

[1785] User: Enter your desired style and budget using the dedicated app or website.

[1786] Terminal: Sends this information to the server.

[1787] Server: The server selects the most suitable products based on the input conditions. After selection, it creates an image of the coordinated outfit using an image processing library such as OpenCV. The input is the user's condition information, and the output is the generated image of the coordinated outfit.

[1788] Step 6: Acquisition and analysis of emotional data

[1789] Device: When suggesting outfit combinations, the camera and microphone are used to acquire user emotion data (facial expressions, voice, etc.).

[1790] Terminal: Sends acquired emotion data to the server. Input is emotion data acquired from the camera and microphone, and output is emotion data sent to the server.

[1791] Server: Analyzes sentiment data using an emotion engine. For example, Amazon Rekognition or Google Cloud Speech-to-Text are used. The analysis results are stored as analysis data.

[1792] Step 7: Proposal to the user

[1793] Terminal: Displays the generated outfit image to the user.

[1794] User: Review the outfit and enter feedback. Input is the outfit image and user feedback, and output is the user's feedback information.

[1795] Terminal: Sends feedback information to the server. Input is user feedback information, and output is the transmission of feedback to the server.

[1796] Step 8: Processing Feedback

[1797] Server: The server analyzes feedback and sentiment data and generates new outfits that reflect the user's new requests and areas for improvement. The input is user feedback information and sentiment data, and the output is new outfit information.

[1798] Server: Sends the newly generated outfit image to the terminal. The input is the new outfit information, and the output is the outfit image sent to the terminal.

[1799] Step 9: Final Review and Proposal

[1800] Terminal: Presents a new outfit to the user and allows for final confirmation.

[1801] User: Review the new outfit and proceed to purchase if satisfied. The input is the image of the new outfit, and the output is the user's final confirmation information.

[1802] Terminal: Sends the user's final confirmation information to the server to proceed with the purchase process. The input is the final confirmation information, and the output is the progress of the purchase process.

[1803] (Application Example 2)

[1804] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1805] Traditional styling suggestion systems, while based on user profile information, preferences, and budget, had the challenge of not being able to provide personalized suggestions that took into account the user's instantaneous emotions and reactions. As a result, it was difficult to provide suggestions that truly satisfied users and to fully stimulate their purchasing intent. Furthermore, it was difficult to efficiently learn from product data of multiple brands and to conduct detailed analysis of user behavior history.

[1806] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1807] In this invention, the server includes means for the user to input at least profile information, preferences, and budget; means for the server to acquire product data from multiple brands; means for the server to learn the acquired product data; means for the server to acquire the user's behavior history; means for the server to analyze preferences and tendencies based on the user's behavior history; means for the server to generate outfits based on the user's conditions; means for the server to create images of the generated outfits; means for the terminal to present the generated outfits to the user; means for the terminal to receive feedback from the user; means for the server to generate outfits again based on the feedback; means for using an emotion engine to acquire the user's emotional data when suggesting outfits; and means for the server to analyze the acquired emotional data and make suggestions that are optimal for the user's emotional state. This makes it possible to provide more personalized outfit suggestions that take into account the user's instantaneous emotional state.

[1808] "A means for users to input at least profile information, preferences, and budget" refers to a system consisting of an interface for users to input their basic information, clothing preferences, and the amount of money they are willing to spend.

[1809] "A means for a server to acquire product data from multiple brands" refers to a system in which a server has the function of collecting the latest product information from multiple brands via APIs, etc., and storing it in a database.

[1810] "Means for a server to learn from acquired product data" refers to a system that has the function of analyzing product data collected by the server using machine learning algorithms, and understanding and learning its characteristics.

[1811] "Means by which a server obtains a user's behavioral history" refers to a system that includes interfaces and functions for a server to collect a user's past behavioral history, such as search history and purchase history.

[1812] "Means by which a server analyzes user preferences and trends based on user behavior history" refers to a system that includes algorithms and analytical functions for analyzing the behavior history collected by the server and deriving user preferences and purchasing trends.

[1813] "A means by which a server generates outfits based on user conditions" refers to a system equipped with an algorithm that generates the optimal outfit based on the conditions (profile information, preferences, budget) entered by the user.

[1814] "Means for creating images of server-generated outfits" refers to a system that has the function of converting server-generated outfits into images so that they can be presented to the user in a visual format.

[1815] "Means for the terminal to present the generated coordinate to the user" refers to a system that includes an interface for the user's terminal to display the coordinate image sent from the server.

[1816] "A means for a device to receive user feedback" refers to a system that includes an interface for users to input their opinions and impressions of the proposed outfits.

[1817] "Means for the server to regenerate the coordination based on feedback" refers to a system equipped with an algorithm that allows the server to analyze user feedback and regenerate the optimal coordination based on the results.

[1818] "Methods for using an emotion engine to acquire user emotion data when suggesting outfits" refers to a system that includes an engine that recognizes the user's emotions towards the suggested outfits from their facial expressions and voice, and collects that data.

[1819] "A means of analyzing emotional data acquired by a server to provide optimal suggestions for the user's emotional state" refers to a system that includes an algorithm and analysis function for analyzing emotional data collected by a server and providing optimal coordination suggestions for the user's emotional state.

[1820] This invention is a system that suggests the optimal outfit based on the user's profile information, preferences, budget, behavioral history, and emotional data. The system has the following configuration:

[1821] First, users create an account using a dedicated application or website. They enter profile information such as their name, age, gender, style preferences, and budget. This sends the user's basic data to the server.

[1822] The server retrieves product data (images, prices, categories, features, etc.) from multiple brands via APIs. This product data is stored in a database. The server then uses machine learning algorithms to learn from this data and extract the features of each product. Specifically, the Python programming language and the Keras library are used in this process.

[1823] Next, the server retrieves the user's activity history via the API interface. This activity history includes past search and purchase history. This information is also stored in the database and used to analyze the user's preferences and trends.

[1824] To generate an outfit, the user enters their desired style and budget via a dedicated app or website. The server receives this information and selects the most suitable items based on the user's criteria. The selected items are combined into an outfit, and an image of the outfit is generated. The generated outfit image is sent to the device and presented to the user.

[1825] When an outfit is presented to the user, the emotion engine acquires the user's emotion data. This emotion data is analyzed from the user's facial expressions and voice. This includes analyzing facial expressions from camera frames using the OpenCV library. The acquired emotion data is sent to the server for analysis.

[1826] The server analyzes user feedback and sentiment data to generate new outfits that reflect the user's new requests and reactions. This allows for personalized suggestions to be made to the user. This process is repeated until an outfit that the user is ultimately satisfied with is generated.

[1827] To give a concrete example, a user creates an account and enters profile information, preferences, and budget. For example, if the budget is under 50,000 yen and the user's favorite items are denim jackets and sneakers, the server retrieves the latest product data from each brand and stores it in a database. A machine learning algorithm is used to learn from the product data and understand its characteristics. The server also analyzes the user's preferences using their past search and purchase history. Then, it generates the optimal outfit and presents the image to the user. At this time, the emotion engine captures the user's reaction and sends it to the server. If the user is not satisfied with the outfit, they send feedback from their device. The server analyzes the emotion data and feedback, generates a new outfit, and presents it.

[1828] An example of an input prompt for a generative AI model is as follows:

[1829] "User profile information: name, age, gender, style preferences, budget. Product data: brand, price, category, features. Sentiment data: emotional state, confidence level."

[1830] This makes it possible to offer more personalized outfit suggestions that take into account the user's emotional state at that moment.

[1831] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1832] Step 1:

[1833] User Registration

[1834] Users create an account using a dedicated application or website. They enter profile information such as their name, age, gender, style preferences, and budget.

[1835] (Input): User profile information

[1836] (Processing): Send the entered information to the server.

[1837] (Output): User information stored on the server

[1838] Step 2:

[1839] Product data acquisition

[1840] The server retrieves product data from multiple brands via APIs. This product data includes images, prices, categories, and features.

[1841] (Input): API endpoint, brand information

[1842] (Processing): Retrieve data via API call and store it in the database.

[1843] (Output): Product data stored in the database

[1844] Step 3:

[1845] Learning about product specifications

[1846] The server learns from acquired product data using machine learning algorithms. This allows it to understand and classify the characteristics of each product.

[1847] (Input): Product data

[1848] (Processing): Analyze and learn data using machine learning algorithms (using Python and Keras).

[1849] (Output): Product Feature Model

[1850] Step 4:

[1851] Acquisition of user behavior history

[1852] The server retrieves the user's behavioral history (past search history and purchase history) via an API interface. This information is also stored in the database.

[1853] (Input): User ID, API endpoint

[1854] (Processing): Obtain behavioral history via API and store it in the database.

[1855] (Output): Behavioral history data stored in the database

[1856] Step 5:

[1857] Analysis of preferences and tendencies

[1858] The server analyzes user preferences and tendencies based on the acquired behavioral history. This reveals the user's basic preferences and purchasing trends.

[1859] (Input): Behavioral history data

[1860] (Processing): Analyze user preferences and trends using data analysis algorithms.

[1861] (Output): User-preferred model

[1862] Step 6:

[1863] Coordination generation

[1864] The server selects the most suitable products based on the user's conditions (profile information, preferences, budget) and generates outfit combinations using a generative AI model.

[1865] (Input): User information, product feature model, user preference model

[1866] (Processing): The AI ​​model generates the outfits and produces an image.

[1867] (Output): Image of the generated coordinate

[1868] Step 7:

[1869] Coordination suggestions

[1870] The terminal presents the generated outfit to the user. During this process, the emotion engine acquires the user's emotion data. This emotion data is sent to the server.

[1871] (Input): Coordinated image, camera frame

[1872] (Processing): Display the coordinated image, acquire emotion data using the emotion engine, and send it to the server (using OpenCV).

[1873] (Output): User sentiment data

[1874] Step 8:

[1875] Obtaining feedback

[1876] The device receives feedback from the user. The user enters their opinions and impressions of the suggested outfit. The feedback data is also sent to the server.

[1877] (Input): User Feedback

[1878] (Processing): Send feedback to the server

[1879] (Output): Feedback data stored on the server

[1880] Step 9:

[1881] Generating a new outfit

[1882] The server analyzes feedback and sentiment data and generates new outfits that reflect the user's new requests and reactions.

[1883] (Input): Sentiment data, feedback data

[1884] (Processing): Analyze this data using a data analysis algorithm and generate a new coordinate.

[1885] (Output): Image of the new outfit

[1886] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1887] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1888] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1889] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1890] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1891] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1892] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1893] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1894] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1895] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1896] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1897] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1898] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1899] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1900] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1901] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1902] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1903] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1904] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1905] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1906] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1907] The following is further disclosed regarding the embodiments described above.

[1908] (Claim 1)

[1909] A means for users to input at least profile information, preferences, and budget,

[1910] A means by which the server retrieves product data from multiple brands,

[1911] A means for the server to learn from the product data it has acquired,

[1912] A means by which the server obtains the user's behavior history,

[1913] A means by which the server analyzes user preferences and tendencies based on user behavior history,

[1914] A means by which the server generates a coordinate based on user conditions,

[1915] A means for creating an image of the generated outfit on the server,

[1916] A means by which the terminal presents the generated coordinate to the user,

[1917] The means by which the device receives user feedback,

[1918] A system that includes means for the server to regenerate the coordination based on feedback.

[1919] (Claim 2)

[1920] The system according to claim 1, wherein the server has means for obtaining the user's behavior history through the LINE interface.

[1921] (Claim 3)

[1922] The system according to claim 1, further comprising means for the terminal to send feedback to a server after receiving feedback, and for the server to generate a new coordinate and send it to the terminal.

[1923] "Example 1"

[1924] (Claim 1)

[1925] A means for users to input at least profile information, preferences, and budget,

[1926] A means for the server to retrieve product data from multiple online shops,

[1927] A means of analyzing product data acquired by the server,

[1928] A means by which the server obtains the user's behavior history,

[1929] A means by which the server analyzes user preferences and tendencies based on user behavior history,

[1930] A means by which the server generates a coordinate based on user conditions,

[1931] A means for creating an image of the generated outfit on the server,

[1932] A means by which the terminal presents the generated coordinate to the user,

[1933] The means by which the device receives user feedback,

[1934] A system that includes means for the server to regenerate the coordination based on feedback.

[1935] (Claim 2)

[1936] The system according to claim 1, wherein the server has means for obtaining the user's behavior history through a messaging platform interface.

[1937] (Claim 3)

[1938] The system according to claim 1, further comprising means for the terminal to send feedback to a server after receiving feedback, and for the server to generate a new coordinate and send it to the terminal.

[1939] "Application Example 1"

[1940] (Claim 1)

[1941] A means for users to input at least profile information, preferences, and budget,

[1942] A means by which the server retrieves product data from multiple brands,

[1943] A means for the server to learn from the product data it has acquired,

[1944] A means by which the server obtains the user's behavior history,

[1945] A means by which the server analyzes user preferences and tendencies based on user behavior history,

[1946] A means by which the server generates a coordinate based on user conditions,

[1947] A means for creating an image of the generated outfit on the server,

[1948] A means by which the terminal presents the generated coordinate to the user,

[1949] The means by which the device receives user feedback,

[1950] A means by which the server generates a new coordination based on the feedback,

[1951] A means by which the terminal provides the user with a virtual try-on,

[1952] A system that includes a means for users to check and provide feedback on outfits through virtual try-on.

[1953] (Claim 2)

[1954] The system according to claim 1, wherein the server has means for acquiring the user's behavior history through a communication interface.

[1955] (Claim 3)

[1956] The system according to claim 1, further comprising means for the terminal to send feedback to a server after receiving feedback, and for the server to generate a new coordinate and send it to the terminal.

[1957] "Example 2 of combining an emotion engine"

[1958] (Claim 1)

[1959] A means for users to input at least profile information, preferences, and budget,

[1960] A means by which the server obtains product data from multiple data sources,

[1961] A means for the server to learn from the product data it has acquired,

[1962] A means by which the server obtains the user's behavior history,

[1963] A means by which the server analyzes user preferences and tendencies based on user behavior history,

[1964] A means by which the server generates a coordinate based on user conditions,

[1965] A means for creating an image of the generated outfit on the server,

[1966] A means by which the terminal presents the generated coordinate to the user,

[1967] The means by which the device receives user feedback,

[1968] A means by which the device acquires user sentiment data,

[1969] A means by which the server analyzes user emotion data using an emotion engine,

[1970] A system including means for a server to regenerate coordinates based on feedback and sentiment data.

[1971] (Claim 2)

[1972] The system according to claim 1, wherein the server has means for obtaining the user's behavior history through a messaging interface.

[1973] (Claim 3)

[1974] The system according to claim 1, further comprising means for the terminal to send feedback and emotion data to a server after it has received such data, and for the server to generate a new coordinate and send it to the terminal.

[1975] "Application example 2 when combining with an emotional engine"

[1976] (Claim 1)

[1977] A means for users to input at least profile information, preferences, and budget,

[1978] A means by which the server retrieves product data from multiple brands,

[1979] A means for the server to learn from the product data it has acquired,

[1980] A means by which the server obtains the user's behavior history,

[1981] A means by which the server analyzes user preferences and tendencies based on user behavior history,

[1982] A means by which the server generates a coordinate based on user conditions,

[1983] A means for creating an image of the generated outfit on the server,

[1984] A means by which the terminal presents the generated coordinate to the user,

[1985] The means by which the device receives user feedback,

[1986] A means by which the server generates a new coordination based on the feedback,

[1987] A method of using an emotion engine to acquire user emotion data when suggesting outfits,

[1988] A system that includes a means of analyzing emotional data acquired by a server to provide suggestions best suited to the user's emotional state.

[1989] (Claim 2)

[1990] The system according to claim 1, wherein the server has means for acquiring the user's behavior history through a communication interface.

[1991] (Claim 3)

[1992] The system according to claim 1, further comprising means for the terminal to receive feedback and emotion data and send it to a server, and for the server to generate a new coordination and send it to the terminal. [Explanation of Symbols]

[1993] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to input at least profile information, preferences, and budget, A means by which the server retrieves product data from multiple brands, A means for the server to learn from the product data it has acquired, A means by which the server obtains the user's behavior history, A means by which the server analyzes user preferences and tendencies based on user behavior history, A means by which the server generates a coordinate based on user conditions, A means for creating an image of the generated outfit on the server, A means by which the terminal presents the generated coordinate to the user, The means by which the device receives user feedback, A system that includes means for the server to regenerate the coordination based on feedback.

2. The system according to claim 1, wherein the server has means for acquiring the user's behavior history through the LINE interface.

3. The system according to claim 1, further comprising means for the terminal to send feedback to a server after receiving feedback, and for the server to generate a new coordinate and send it to the terminal.

Images