System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving an image selected and uploaded by a user; means for analyzing the received image and extracting features of fashion items in the image; means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and generating a recommendation list; means for customizing the recommendation list in view of the user's past purchase history and input information; and means for providing the generated recommendation list to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, it takes time and effort for users to find fashion items that suit their tastes. Even if they want to emulate the style of fashion influencers or celebrities, they often have trouble knowing which items to buy and where to buy them. Furthermore, there is a lack of systems that provide specialized recommendations based on users' past purchasing data and preferences. Given this current situation, there is a need for a system that allows users to quickly find the right items while intuitively incorporating their preferred style. [Means for solving the problem]

[0005] The present invention provides a system including: means for receiving images selected and uploaded by a user; means for analyzing the received images and extracting features of fashion items in the images; means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and generating a recommendation list; means for customizing the recommendation list in consideration of the user's past purchase history and input information; and means for providing the generated recommendation list to the user. This system enables users to intuitively and efficiently find fashion items that suit their preferences and receive recommendations based on their characteristics.

[0006] "User" refers to an individual who uses the system, primarily a consumer who searches for and purchases fashion items.

[0007] "Means for receiving images" refers to the functions and processes by which the server obtains image data uploaded by the user from the terminal.

[0008] "Means for analyzing images" refers to the technology or algorithms used to process the received image data and identify the characteristics of the fashion items in the images.

[0009] "Fashion items" refers to items such as clothing, accessories, and shoes worn by people in a user's image.

[0010] "Means of extracting features" refers to the process of identifying the attributes of fashion items (color, shape, brand, etc.) from the analyzed images and extracting them as information.

[0011] "Online Shopping Platform" means a web service for e-commerce that enables the purchase of fashion items via the Internet.

[0012] "Recommendation list" refers to a list of fashion items selected based on extracted features, and is a list to suggest to users.

[0013] "Purchase History" refers to the record of fashion items a User has previously purchased through online shopping platforms or other channels.

[0014] "Input Information" refers to additional data provided by the user to the system (e.g., preferences, size, budget, etc.).

[0015] "Means for customization" refers to the process or algorithm used to personalize recommendation lists by taking into account a user's past purchase history and input information.

[0016] "System" means a comprehensive configuration that integrates the above means and provides consistent services to users. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] To implement the present invention, the following describes a specific system and its program processing. This system allows users to upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the items.

[0039] System configuration

[0040] 1. Device:

[0041] The system allows users to select and upload photos on their smartphones, computers, or other devices, and includes an interface for displaying recommendation lists.

[0042] 2. Server:

[0043] It has the ability to receive images, analyze them, extract features, generate and customize recommendation lists, and also includes a database that manages user purchase history and input information.

[0044] 3. Online shopping platform:

[0045] It is an e-commerce site where users can actually purchase recommended items, and provides product data in response to queries from the server.

[0046] Program processing

[0047] 1. Upload an image

[0048] Users use their own devices to select photos of their favorite celebrities or influencers and upload them to the system.

[0049] The device sends the selected photo to the server as an HTTP request.

[0050] 2. Image Analysis

[0051] The server passes the received photo data to the AI ​​image analysis module and begins analysis.

[0052] The AI ​​image analysis module uses object detection technology to detect fashion items in photos (e.g., a red dress, black heels, gold earrings).

[0053] 3. Feature Extraction

[0054] The server extracts the item's meta information based on the analysis results, including color, shape, brand name, etc.

[0055] For example, "red dress" includes information such as color: red, item: dress, brand name: (if known).

[0056] 4. Recommendation Generation

[0057] The server uses the extracted meta information to access the online shopping platform's database and search for similar items.

[0058] The server generates a recommendation list based on the items obtained from the search results.

[0059] 5. Customization

[0060] The server compares the user's past purchase history with the information they provide, including their preferred colors, styles, budget, and other information.

[0061] The server uses this information to customize the recommendation list and arrange the items in the optimal order.

[0062] 6. Providing Recommendations

[0063] The server sends the customized recommendation list to the user's device.

[0064] The device displays the received recommendation list on the user interface so that the user can easily check it.

[0065] Specific examples

[0066] Example 1: When a user uploads a photo of celebrity A

[0067] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[0068] 2. The server passes the photo to the AI, which then detects the "red dress," "black heels," and "gold earrings."

[0069] 3. The server extracts meta information for each item and searches for similar items on online shopping platforms.

[0070] 4. The server generates a recommendation list from the search results, customizing it based on the user's past preference for red dresses.

[0071] 5. The customized list is provided to the user, who can then view the list on their device, select and purchase the items they like.

[0072] This allows the system to easily find fashion items that match the user's preferred style.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] Users can use their own devices to select photos of their favorite celebrities or influencers and upload them to the system by clicking the upload button.

[0076] Step 2:

[0077] The device sends the selected photo to the server, typically using an HTTP request.

[0078] Step 3:

[0079] The server receives the received photo data and passes it to the AI ​​image analysis module, which then starts the AI ​​image analysis process.

[0080] Step 4:

[0081] The AI ​​image analysis module analyzes the received photos and detects fashion items in the images, using object detection technology to identify items such as a red dress, black heels, and gold earrings.

[0082] Step 5:

[0083] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[0084] Step 6:

[0085] The server uses the extracted meta information to access the online shopping platform's database to search for similar items, and calls the platform's API to retrieve the required information.

[0086] Step 7:

[0087] The server generates a recommendation list based on search results obtained from the online shopping platform, and the recommendation list includes items to be suggested to the user.

[0088] Step 8:

[0089] The server retrieves and analyzes the user's past purchase history and information entered into the system, such as preferences, size, and budget, from a database.

[0090] Step 9:

[0091] The server customizes the recommendation list based on the acquired user information, taking into account past purchase history and input information to determine the most appropriate item order.

[0092] Step 10:

[0093] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[0094] Step 11:

[0095] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[0096] This series of steps allows users to easily find fashion items that suit their tastes and style.

[0097] Example 1

[0098] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] Conventional fashion item recommendation systems often fail to fully reflect user preferences, and the items they suggest often do not meet user expectations. Another issue is the low accuracy of image analysis, which makes it difficult to accurately extract item features.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0101] In this invention, the server includes means for receiving images selected and uploaded by a user, means for passing the received images to an AI image analysis module and starting analysis, means for extracting features of fashion items in the images using the AI ​​image analysis module, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items, means for matching the user's past purchase history and input information to customize a recommendation list, and means for providing the generated recommendation list to the user, thereby enabling highly accurate recommendations that reflect the user's preferences.

[0102] "User" refers to an individual who uses this system to receive fashion item recommendations.

[0103] "Terminal" refers to a device used by a user, such as a smartphone or computer, that selects and uploads images.

[0104] "Server" refers to the central computer system that receives, analyzes, extracts features from, and generates and customizes recommendation lists for images.

[0105] The "AI Image Analysis Module" is a software component that uses artificial intelligence technology to analyze received images and detect fashion items within the images.

[0106] "Fashion items" refers to items such as clothing and accessories for which users request recommendations via photos.

[0107] "Features" refers to information necessary to identify a fashion item, such as its color, shape, brand name, etc.

[0108] A "recommendation list" refers to a list of fashion items suggested to users.

[0109] "Online shopping platform" refers to an e-commerce site where users can physically purchase fashion items.

[0110] "Purchase history" refers to information about fashion items a user has purchased in the past.

[0111] "Input information" refers to information such as preferences and budget that a user provides to the system.

[0112] An "API (Application Program Interface)" refers to the conventions and tools that allow different software systems to communicate with each other and exchange data.

[0113] System Overview

[0114] This invention is a system in which users upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the uploaded photos. The main components of the system are the user device, a server, and the online shopping platform.

[0115] Hardware and software used

[0116] 1. Device:

[0117] Hardware: Devices used by users, such as smartphones and computers.

[0118] Software: Web browser and dedicated applications.

[0119] Features: Image selection, upload, and display recommendation list.

[0120] 2. Server:

[0121] Hardware: High-performance cloud servers.

[0122] Software: AI image analysis module using libraries such as Python, TensorFlow, and OpenCV.

[0123] Functions: Image reception, analysis, feature extraction, recommendation list generation, and customization.

[0124] 3. Online shopping platform:

[0125] Hardware: The servers that run your e-commerce site.

[0126] Software: Product database and API.

[0127] Features: Find and offer similar items.

[0128] Detailed program description

[0129] 1. Upload an image

[0130] Users can select photos of their favorite celebrities or influencers using their smartphones or computers and upload them to the system. The images selected on the device are sent to the server as HTTP requests.

[0131] 2. Image Analysis

[0132] The server temporarily stores the received image data and passes it to the AI ​​image analysis module, which uses libraries such as TensorFlow and OpenCV to apply object detection technology to detect fashion items in the image.

[0133] 3. Feature Extraction

[0134] The server extracts meta information for each item from the analysis results. This meta information includes the item's color, shape, brand name, etc. For example, for a "red dress," the server extracts the following information: "Color: Red, Item: Dress, Brand: Unknown."

[0135] 4. Recommendation Generation

[0136] The server then uses the extracted meta information to access the online shopping platform's database and search for similar fashion items, using SQL queries and APIs to retrieve related items.

[0137] 5. Customization

[0138] The server compares the user's past purchase history and input information to customize the recommendation list, taking into account the user's preferences, budget, and other information to create a list of the most suitable items.

[0139] 6. Providing Recommendations

[0140] The server sends the customized recommendation list in JSON format to the user's device, which interprets the received data and displays it in a formatted user interface.

[0141] Specific examples

[0142] Example: When a user uploads a photo of celebrity A

[0143] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[0144] 2. The server passes the photo to an AI image analysis module, which then detects the "red dress," "black heels," and "gold earrings."

[0145] 3. The server extracts meta information for each item and searches for similar items on online shopping platforms.

[0146] 4. The server generates a recommendation list from the search results, customizing it based on the user's past preference for red dresses.

[0147] 5. A customized list is provided to the user, who can then view the list on their device, select and purchase the items they like.

[0148] Prompt Sentence Examples

[0149] For generative AI models, use the following prompt:

[0150] "Write a program that analyzes fashion items in images uploaded by users, extracts their features, and generates a recommendation list based on the user's preferences."

[0151] This prompt statement allows the model to generate the appropriate code.

[0152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0153] Step 1: Select and upload an image

[0154] Users open the gallery app or file browser on their smartphone or PC and select photos of celebrities or influencers they like. The images selected by the user are sent to the system by clicking the upload button. The device then sends the selected images to the server as an HTTP request.

[0155] Input: An image file selected by the user on their device.

[0156] Output: Image data included in the HTTP request sent to the server.

[0157] Step 2: Receiving the image

[0158] The server receives the image data sent from the device, stores it in temporary storage, and then passes it to the AI ​​image analysis module for image analysis.

[0159] Input: Image data sent from the device.

[0160] Output: Image data saved in temporary storage.

[0161] Step 3: Begin image analysis

[0162] The server passes the image data stored in temporary storage to the AI ​​image analysis module, which then uses libraries such as TensorFlow and OpenCV to perform image analysis.

[0163] Input: Image data stored in temporary storage.

[0164] Output: A list of identified fashion items in the image.

[0165] Step 4: Extracting fashion item features

[0166] The server extracts meta information (color, shape, brand name, etc.) for each item based on the analysis results received from the AI ​​image analysis module. For example, if a red dress is identified, the meta information is extracted as "Color: Red, Item: Dress, Brand Name: Unknown."

[0167] Input: Analysis results of the AI ​​image analysis module.

[0168] Output: Extracted fashion item meta information.

[0169] Step 5: Search for recommended items

[0170] The server then accesses the online shopping platform's database based on the extracted fashion item meta information and searches for similar items using SQL queries or APIs, such as "SELECT FROM Items WHERE color="red" AND type="dress".

[0171] Input: Extracted fashion item meta information.

[0172] Output: A list of similar items retrieved from an online shopping platform.

[0173] Step 6: Generate a recommendation list

[0174] The server generates a recommendation list based on a list of similar items retrieved from an online shopping platform, and the list is customized based on the user's preferences and purchasing history.

[0175] Input: A list of similar items obtained from an online shopping platform.

[0176] Output: The generated recommendation list.

[0177] Step 7: Customize your recommendation list

[0178] The server customizes the generated recommendation list based on the user's past purchase history and input information. For example, if the user has previously purchased a favorite "red dress," the server will adjust the ranking of the list to reflect that information.

[0179] Input: User's past purchase history and input information, generated recommendation list.

[0180] Output: A customized recommendation list.

[0181] Step 8: Providing a recommendation list

[0182] The server sends the customized recommendation list in JSON format to the user's device, which interprets the received data, formats it, and displays it in a user interface. The user can then select and purchase the items they like from the list.

[0183] Input: A customized recommendation list.

[0184] Output: The recommendation list displayed on the user's device.

[0185] (Application example 1)

[0186] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0187] On conventional online shopping platforms, users had to spend a lot of time and effort finding fashion items that matched their tastes and style. Furthermore, they had to enter their payment information each time they made a purchase, which posed security risks during the payment process. To solve these problems, a system was needed that would allow users to easily upload images from their devices, analyze their features to recommend similar items, and even handle electronic payment.

[0188] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0189] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of objects in the images, means for searching for similar items from an online trading platform based on the extracted features of the objects and generating a recommendation list, means for customizing the recommendation list taking into account the user's past transaction history and input information, means for providing the generated recommendation list to the user, and means for purchasing items selected from the recommendation list via an electronic transaction payment gateway. This allows users to easily find fashion items that suit their tastes and style and to purchase them safely and quickly.

[0190] The term "user" refers to a person who uses a particular service or product, and in the present invention particularly refers to a person who uploads photos of fashion items and purchases recommended items.

[0191] "Image" refers to visual data selected from a terminal and uploaded to the system, which is the subject of analysis of the characteristics of fashion items.

[0192] "Particular object" refers to an object present in the received image, and in the present invention particularly refers to a fashion item (e.g., a dress, shoes, accessories).

[0193] "Feature" refers to an identifiable attribute or property of an object, and in the present invention includes color, shape, brand name, and the like.

[0194] A "recommendation list" refers to a list of recommended items generated based on extracted features, which users can browse and select according to their preferences.

[0195] "Online trading platform" refers to a website or application that offers products through e-commerce and allows users to search, browse, and purchase products.

[0196] "Transaction history" refers to a record of a user's past purchases and transactions, and is information used to customize recommendation lists.

[0197] "Payment Gateway" refers to the payment processing system used in a transaction, which functions to enable secure and fast payments.

[0198] A "server" refers to a computer system that receives requests from clients (user devices) and performs data processing and communication. In this invention, it is responsible for image analysis, feature extraction, recommendation generation, payment processing, etc.

[0199] "API" is an abbreviation for Application Programming Interface, which means an interface that enables communication between software programs. In the present invention, it is used to access the online trading platform.

[0200] The present invention relates to a system for analyzing an image, providing a user with recommended items, and then purchasing the items through electronic payment. A specific embodiment of the present invention will be described below.

[0201] The overall system configuration is as follows: It includes the devices used by users (smartphones and computers), a server that receives and analyzes data sent from these devices, and an online trading platform (website or application).

[0202] Program processing

[0203] Device:

[0204] Users use their own devices (e.g., smartphones) to take pictures of their favorite fashion items or select and upload existing images. The device then sends this image data to the server using an HTTP request.

[0205] server:

[0206] The server does the following:

[0207] Image analysis: The received image is passed to an AI image analysis module, which analyzes the features of specific objects. This uses image processing libraries such as OpenCV and an AI model to extract features such as color, shape, and brand name.

[0208] Recommendation generation: Based on the extracted features, queries are sent to online trading platforms to search for similar fashion items, using APIs.

[0209] Customization: The recommendation list is customized based on the user's past transaction history and input information, resulting in a recommendation list optimized for the user's individual preferences.

[0210] Recommendation provision: The generated recommendation list is sent to the user's device and displayed on the device.

[0211] Online trading platform:

[0212] The online trading platform provides product data in response to search queries from the server, which then lists the items desired by the user.

[0213] Electronic Payment:

[0214] Users can select the desired items from the recommendation list and make an electronic payment. The server processes the payment securely and quickly via a payment gateway (e.g., Stripe or PayPal).

[0215] Specific examples

[0216] For example, if User A uploads a photo of Celebrity B from his / her smartphone, the photo may contain a "red dress" and "black high heels." The server uses image analysis technology to extract these features and sends a query to an online trading platform. Similar items are retrieved from the platform, and a customized recommendation list is generated taking into account User A's past purchase history (e.g., a preference for red dresses). Finally, the list is provided to User A, who can select the items they like and easily purchase them.

[0217] To comprehensively support this process, the following prompts could be fed to the generative AI model:

[0218] Example prompt sentence:

[0219] A user takes a photo of a photogenic fashion and uploads it to the app. The photo includes a blue dress and white heels. The app extracts the features of the blue dress and searches for and recommends multiple similar items, including blue dresses and white heels. For each item, the user can easily pay electronically through the app.

[0220] The above is a specific embodiment for carrying out the present invention. This system allows users to easily find and safely purchase fashion items that suit their style.

[0221] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0222] Step 1:

[0223] Input: The user takes or selects a photo of a fashion item on their smartphone.

[0224] Specific operation: The user acquires an image using the device's camera / gallery function, selects the image from the system's application screen, and presses the upload button.

[0225] Output: The uploaded image data is sent to the server as an HTTP request.

[0226] Step 2:

[0227] Input: The server receives the image data sent from the terminal.

[0228] Specific operation: The server receives the request, obtains the image data contained therein, checks the integrity of the data using the HTTP protocol, and passes it to the analysis module.

[0229] Output: The image data is passed to the image analysis module.

[0230] Step 3:

[0231] Input: The server passes the image data to the image analysis module.

[0232] How it works: The server uses the OpenCV library to read an image and input it into the generative AI model. The AI ​​model then analyzes the input image and extracts the features of the object. During this process, attribute information such as color, shape, and brand name is identified.

[0233] Output: Feature data of objects extracted from the image (e.g. color: blue, shape: dress, brand name: XYZ).

[0234] Step 4:

[0235] Input: Based on the extracted feature data, the server sends a query to the online trading platform.

[0236] How it works: The server uses an API to access the database of an online trading platform and search for items with similar characteristics. The query statement includes characteristics such as color, shape, and brand name.

[0237] Output: A data list of the retrieved similar items.

[0238] Step 5:

[0239] Input: The server generates a recommendation list based on this acquired data list.

[0240] How it works: The server customizes the recommendation list based on the user's past transaction history and input information. For example, if the user has previously purchased blue dresses, the server uses that information to prioritize items in the list.

[0241] Output: A customized recommendation list.

[0242] Step 6:

[0243] Input: The server sends the generated recommendation list to the terminal.

[0244] Specific operation: The server sends the generated recommendation list to the user's device and displays it on the application's user interface.

[0245] Output: The recommendation list displayed on the user's device.

[0246] Step 7:

[0247] Input: The user selects the desired item from the recommendation list.

[0248] Specific operation: The user operates the terminal interface, selects a specific item from the list, and performs the purchase operation. After the purchase operation, the information of the selected item and payment information are sent to the server.

[0249] Output: Selected item information and payment information are sent to the server.

[0250] Step 8:

[0251] Input: The server processes the received payment information and executes the transaction via the payment gateway.

[0252] What happens: The server passes the payment information to a payment gateway (e.g., Stripe, PayPal) to process the secure transaction. After the transaction is complete, the user is notified of the transaction status.

[0253] Output: A notification of the payment processing result and purchase confirmation is sent to the user.

[0254] This allows users to easily find fashion items that suit their tastes and style and purchase them quickly and safely.

[0255] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0256] To implement the present invention, the following specific system and its program processing will be described. In this system, users upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the items and the user's emotions.

[0257] System configuration

[0258] 1. Device:

[0259] The system allows users to select and upload photos on their smartphones, computers, or other devices, and includes an interface for displaying recommendation lists.

[0260] 2. Server:

[0261] It has the ability to receive images, analyze them, extract features, recognize emotions, generate and customize recommendation lists, and also includes a database that manages user purchase history, input information, and an emotion recognition engine.

[0262] 3. Online shopping platform:

[0263] It is an e-commerce site where users can actually purchase recommended items, and provides product data in response to queries from the server.

[0264] Program processing

[0265] 1. Upload an image

[0266] Users can select photos of their favorite celebrities or influencers using their own devices and upload them to the system by clicking the upload button.

[0267] The device sends the selected photo to the server as an HTTP request.

[0268] 2. Image Analysis

[0269] The server receives the received photo data, passes it to the AI ​​image analysis module, and begins analysis.

[0270] The AI ​​image analysis module uses object detection technology to detect fashion items in photos (e.g., a red dress, black heels, gold earrings).

[0271] 3. Feature Extraction

[0272] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[0273] 4. Emotion recognition

[0274] The server runs an emotion recognition engine based on the uploaded photo and the user's facial expressions, which classifies the user's emotional state (e.g., joy, sadness, surprise, fear).

[0275] 5. Recommendation Generation

[0276] The server accesses the database of the online shopping platform based on the extracted meta information and emotion recognition data to search for similar items.

[0277] The server generates a recommendation list based on the items obtained from the search results, prioritizing appropriate items based on the user's sentiment.

[0278] 6. Customization

[0279] The server retrieves the user's past purchase history and input information from the database and analyzes it.

[0280] The server customizes the recommendation list based on the acquired user information and emotional information, and determines the most appropriate item order, taking into account past purchase history and input information.

[0281] 7. Providing Recommendations

[0282] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[0283] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[0284] Specific examples

[0285] Example 1: When a user uploads a photo of celebrity A

[0286] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[0287] 2. The server passes the photo to the AI, which then detects the "red dress," "black heels," and "gold earrings."

[0288] 3. The server extracts meta information for each item and uses an emotion recognition engine to determine the user's emotion as "joy."

[0289] 4. The server searches for similar items on the online shopping platform and generates a recommendation list based on the user's sentiment.

[0290] 5. The server customizes the list based on past purchase history and provides it to the user.

[0291] 6. The device will display a customized list, allowing the user to select and purchase the items they like.

[0292] This allows the system to easily find fashion items that suit users' preferences and style, and also provides optimal recommendations based on their emotional state.

[0293] The processing flow will be explained below.

[0294] Step 1:

[0295] Users select photos of their favorite celebrities or influencers using their own devices and upload them to the system. The user then clicks the upload button.

[0296] Step 2:

[0297] The device sends the selected photo to the server, typically using an HTTP request.

[0298] Step 3:

[0299] The server receives the received photo data and passes it to the AI ​​image analysis module, which then starts the AI ​​image analysis process.

[0300] Step 4:

[0301] The AI ​​image analysis module analyzes the received photos and detects fashion items in the images, using object detection technology to identify items such as a red dress, black heels, and gold earrings.

[0302] Step 5:

[0303] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[0304] Step 6:

[0305] The server runs an emotion recognition engine based on the uploaded photo and the user's facial expressions, which classifies the user's emotional state (e.g., joy, sadness, surprise, fear).

[0306] Step 7:

[0307] The server accesses the online shopping platform's database based on the extracted meta information and emotion recognition data to search for similar items, and calls the platform's API to retrieve the required information.

[0308] Step 8:

[0309] The server generates a recommendation list based on search results obtained from the online shopping platform, prioritizing appropriate items based on the user's sentiment.

[0310] Step 9:

[0311] The server retrieves and analyzes the user's past purchase history and information entered into the system, such as preferences, size, and budget, from a database.

[0312] Step 10:

[0313] The server customizes the recommendation list based on the acquired user information and sentiment information, and determines the most appropriate item order, taking into account past purchase history and input information.

[0314] Step 11:

[0315] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[0316] Step 12:

[0317] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[0318] This series of steps allows users to easily find fashion items that suit their tastes and style, and also provides appropriate recommendations based on their emotional state at the time.

[0319] Example 2

[0320] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0321] Conventional fashion item recommendation systems simply make recommendations based on a user's past purchase history and simple feature information, making it difficult to take into account the user's emotional state or detailed item features.It also makes it difficult for users to easily find fashion items that suit their preferences and emotions.

[0322] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0323] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of fashion items in the images, means for analyzing the user's facial expression data and classifying emotions, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and the classified emotion data, means for customizing the recommendation list in consideration of the user's past purchase history and input information, and means for providing the generated recommendation list to the user, thereby enabling the user to efficiently find optimal fashion items based on their preferences and emotions.

[0324] A "user" is someone who uses the system to receive fashion item recommendations.

[0325] "Terminal" means the device used by a User to access the System and upload Images.

[0326] A "server" is a computer system that performs central processing such as image analysis, emotion recognition, database management, and generation of recommendation lists.

[0327] "Image upload" is the act of a user using a device to send a selected image to a server.

[0328] "Image analysis" is the process of using AI technology to extract the characteristics of fashion items based on received image data.

[0329] "Feature extraction" refers to obtaining attribute information such as color, shape, and brand of fashion items identified through image analysis.

[0330] "Emotion recognition" is the process of analyzing a user's facial expressions in an image to determine the user's emotional state.

[0331] An "online shopping platform" is an e-commerce system that allows users to purchase recommended fashion items.

[0332] A "recommendation list" is a list of fashion items suggested to a user.

[0333] "Customization" is the process of optimizing the recommendation list based on the user's past purchase history and input information.

[0334] An "HTTP request" is a form of data request sent from a user's device to a server.

[0335] An "HTTP response" is a data response sent from a server to a terminal.

[0336] The present invention provides a system in which a user uploads photos of fashion items using their own terminal, and an online shopping platform provides recommended items based on the features of the items and the user's feelings.

[0337] System configuration

[0338] Hardware and Software

[0339] Device: A device used by a user (e.g., smartphone, tablet, PC, etc.). The device has the functionality to select and upload photos, includes an interface to display the recommendation list, and is responsible for sending HTTP requests to the server.

[0340] Server: Has the ability to analyze received image data, extract features, recognize emotions, generate recommendation lists, and customize them. It also includes a database that manages user purchase history and input information. AI image analysis uses software such as TensorFlow, and emotion recognition uses the Microsoft Azure Emotion API.

[0341] Online shopping platform: An e-commerce system that provides product data, where the server accesses and retrieves product information via API. Users can purchase the items provided.

[0342] Program processing explanation

[0343] The program of the system of the present invention is processed in the following procedure.

[0344] Image upload

[0345] A user uses a device to select a photo of their favorite celebrity or influencer and upload it to the system. Specifically, the user selects a photo from the device's photo gallery and presses the upload button. The device then sends the selected photo to the server as an HTTP POST request. This request includes the photo data and the user's ID.

[0346] Image analysis

[0347] The server receives the photo data from the device. The server then passes this data to an AI image analysis module (e.g., TensorFlow), which uses an object detection algorithm to detect fashion items in the photo (e.g., "red dress," "black heels," "gold earrings").

[0348] Feature extraction

[0349] The server extracts meta information about fashion items based on the detection results returned by the image analysis module, specifically, color information (e.g., red), item type (e.g., dress), and brand (if identifiable) for each item.

[0350] emotion recognition

[0351] The server inputs the photo and the user's facial expression data into an emotion recognition engine (for example, Microsoft Azure Emotion API), which then classifies the user's emotional state from the photo into "joy," "sadness," "surprise," "fear," etc.

[0352] Recommendation generation

[0353] The server then queries the online shopping platform's database to search for similar items based on the extracted item meta information and sentiment data, generating a recommendation list and prioritizing items based on user sentiment.

[0354] Customization

[0355] The server retrieves the user's past purchase history and input information from a database, and based on the retrieved data, customizes the recommendation list and determines the most appropriate item order.

[0356] Providing recommendations

[0357] The server finally generates a customized recommendation list and sends it to the device as an HTTP response. The device displays the received recommendation list on a user interface, allowing the user to browse the list and purchase items.

[0358] Specific examples

[0359] Example 1: When a user uploads a photo of celebrity A

[0360] The user selects a photo of celebrity A from their smartphone and uploads it to the system. Specifically, they select a photo from their photo gallery and tap the "Upload" button.

[0361] The server receives the photo data and passes it to the AI ​​image analysis module, which then detects the "red dress," "black heels," and "gold earrings."

[0362] The server extracts meta information for each item based on the analysis results, and an emotion recognition engine determines the user's emotion as "joy."

[0363] The server searches for similar items from online shopping platforms based on meta information and emotion data, and lists the items obtained, prioritizing them based on the emotion of "joy."

[0364] The server customizes the list taking into account purchase history and sends it to the terminal as an HTTP response.

[0365] The device will display a customized list, allowing users to select and purchase their preferred items.

[0366] Prompt Sentence Examples

[0367] A user uploads a photo of celebrity A wearing a red dress using their smartphone. The AI ​​analyzes the image, detects items, and extracts their features. It then analyzes the user's emotion as "joy" and recommends similar items from online shops based on the emotion. Finally, the customized list is displayed to the user, ready for purchase.

[0368] This system allows users to efficiently find the perfect fashion items based on their preferences and feelings.

[0369] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0370] Program processing flow

[0371] Step 1:

[0372] Image upload

[0373] Users can select a photo of their favorite celebrity or influencer from their device. Specifically, they select a photo from their device's photo gallery and tap the "Upload" button.

[0374] Input: An image file selected by the user.

[0375] The device sends the selected photo to the server as an HTTP POST request, which includes the image data and the user ID.

[0376] Output: Image data and user ID transferred from the device to the server.

[0377] Step 2:

[0378] Image analysis

[0379] The server receives the image data from the device and passes it to the AI ​​image analysis module.

[0380] Input: Image data received from the device and user ID.

[0381] The server starts analyzing the photo using an AI image analysis module (e.g. TensorFlow) and uses object detection algorithms to detect fashion items in the image (e.g. red dress, black heels, gold earrings).

[0382] Output: A list of parsed fashion items.

[0383] Step 3:

[0384] Feature extraction

[0385] The server extracts meta information about the fashion item based on the detection results returned by the image analysis module.

[0386] Input: A list of parsed fashion items.

[0387] The server retrieves information about each item, such as color (red), item type (dress), and brand (if identifiable).

[0388] Output: Extracted fashion item features (color, type, brand).

[0389] Step 4:

[0390] emotion recognition

[0391] The server uses an emotion recognition engine (e.g., Microsoft Azure Emotion API) to identify the user's emotional state and classify emotions based on the photo and the user's facial expression data.

[0392] Input: Uploaded image data, user face detection information.

[0393] The server uses an emotion recognition engine to extract emotional states such as "happiness," "sadness," "surprise," and "fear."

[0394] Output: User sentiment classification data.

[0395] Step 5:

[0396] Recommendation generation

[0397] The server then queries the online shopping platform's database based on the extracted item features and sentiment data.

[0398] Input: Item feature data, emotion data.

[0399] The server searches for similar items and generates a recommendation list that prioritizes appropriate items based on the user's sentiment.

[0400] Output: The generated recommendation list.

[0401] Step 6:

[0402] Customization

[0403] The server retrieves the user's past purchase history and input information from a database.

[0404] Input: Past purchase history, input information.

[0405] The server then customizes the recommendation list based on the information it has obtained, determining the optimal order of items based on past purchase history.

[0406] Output: A customized recommendation list.

[0407] Step 7:

[0408] Providing recommendations

[0409] The server then sends the final customized recommendation list to the device as an HTTP response.

[0410] Input: A customized recommendation list.

[0411] The device displays the received recommendation list on the user interface.

[0412] Output: A list of recommendations displayed in a user interface. The user can browse the list, select items, and purchase them.

[0413] (Application example 2)

[0414] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0415] Conventional online shopping systems do not take into account the user's emotional state when making recommendations, making it difficult for users to find the perfect fashion item that best suits their current mood. Furthermore, technology for accurately extracting the characteristics of fashion items from images uploaded by users and generating recommendation lists based on those characteristics is also inadequate. This leads to issues such as lower user satisfaction and a decrease in purchasing motivation.

[0416] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0417] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of fashion items in the images, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and the user's emotional state and generating a recommendation list, means for customizing the recommendation list taking into account the user's past purchase history and input information, and means for providing the generated recommendation list to the user. This enables optimal fashion item recommendations that take into account the user's emotional state. Furthermore, users can easily find products that match their emotional state, which is expected to increase their purchasing motivation.

[0418] "User" refers to a person who uses the system, primarily someone who uploads images of fashion items and receives recommendations.

[0419] "Image receiving means" refers to a device or software that has the function of receiving image data uploaded by a user and sending it to a server.

[0420] "Image analysis means" refers to a device or software that has the function of analyzing received image data and identifying and extracting the characteristics of fashion items in the image.

[0421] The "feature extraction means" refers to a device or software that has the function of extracting the features of a fashion item, such as color, shape, or type, from the analyzed image.

[0422] "Emotion recognition means" refers to a device or software that has the function of recognizing and determining the emotional state of a user.

[0423] "Online shopping platform" means an e-commerce site that allows users to purchase fashion items via the Internet.

[0424] "Recommendation list generation means" refers to a device or software that has the function of searching for similar items from an online shopping platform based on the extracted features and the user's emotional state and generating a list to present to the user.

[0425] The "customization means" refers to a device or software that has the function of individually adjusting the generated recommendation list, taking into account the user's past purchase history and input information.

[0426] "Recommendation list providing means" refers to a device or software that has the function of displaying and providing a final customized recommendation list to a user.

[0427] To implement the present invention, it is necessary to build a system that allows users to upload images of fashion items using their own devices and provides recommended items based on the features of the items and the user's emotions. Specifically, this is implemented in the following way.

[0428] Details of the hardware and software used

[0429] Hardware: Smartphones, servers, database servers

[0430] Software: Flask (web framework), OpenCV (image processing library), Keras (deep learning framework), Pandas (data analysis library)

[0431] System configuration

[0432] 1. Device:

[0433] Users use their smartphones to take or select their favorite fashion items and upload the images to the system, which then sends the images to the server as HTTP requests.

[0434] 2. Server:

[0435] It analyzes the received images. First, it preprocesses the images using OpenCV, then extracts the features of the fashion items using an AI image analysis module built with Keras. It also uses another Keras model to recognize the user's emotions.

[0436] 3. Database Server:

[0437] Based on the extracted features and sentiment data, similar items are searched for using Pandas. Product data is obtained through APIs in collaboration with the database of an online shopping platform.

[0438] Data processing and calculation

[0439] Image upload:

[0440] When a user uploads an image from a device, the device sends the image data to the server, where it is passed as request data via the HTTP protocol.

[0441] Image analysis:

[0442] The server processes the received image data using OpenCV and converts it into an appropriate format, then inputs it into an image analysis model built with Keras to extract the main features of the fashion item (color, shape, type).

[0443] Emotion recognition:

[0444] The server analyzes the user's facial expressions and recognizes their emotional state (e.g., happy, surprised, sad), again using Keras' emotion recognition model.

[0445] Recommendation generation and customization:

[0446] The database server searches for similar items from online shopping platforms based on the extracted features and the recognized emotions, then uses Pandas to generate a recommendation list, which is further customized by taking into account the user's past purchase history and input information.

[0447] Providing a recommendation list:

[0448] The server sends the generated customized recommendation list to the device, which then displays the list on the user interface, allowing the user to select and purchase the items they like.

[0449] Specific examples

[0450] Example 1: A user uploads a photo wearing a black coat

[0451] Users select a photo from their smartphone of themselves wearing a black coat and upload it to the system.

[0452] The server passes the photo to the AI, which then detects the "black coat."

[0453] The server extracts the characteristics of the item and uses an emotion recognition engine to determine the user's emotion as "joy."

[0454] The server searches for similar items on the online shopping platform and generates a recommendation list based on the user's sentiment.

[0455] The server takes into account past purchase history to customize the list and provides it to the user.

[0456] The device will display a customized list, allowing users to select and purchase their preferred items.

[0457] In this way, the system allows users to easily find fashion items that suit their preferences and style, and provides optimal recommendations based on their emotional state.

[0458] Example prompts to input to a generative AI model:

[0459] plaintext

[0460] Generate recommended items when a user uploads an image of themselves wearing a black coat. The user's emotion is recognized as "joy." The item characteristics obtained from the image are "black, coat." Recommend similar items.

[0461] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0462] Step 1:

[0463] A user selects and uploads an image of a fashion item using a device. At this time, the image file is sent to the server as an HTTP request. The input is the image file on the device, and the output is an HTTP request containing this image file.

[0464] Step 2:

[0465] The server analyzes the received HTTP request and obtains image data. The obtained image data is preprocessed using OpenCV and converted into a format that can be input to the AI ​​image analysis model. The input is the image data in the HTTP request, and the output is the preprocessed image data.

[0466] Step 3:

[0467] The server inputs the preprocessed image data into a Keras AI image analysis model to extract the features of the fashion items. The AI ​​model analyzes the image and obtains features such as color, shape, and type. The input is the preprocessed image data, and the output is the obtained feature data.

[0468] Step 4:

[0469] The server uses another Keras model to recognize the user's emotion based on the extracted feature data. The user's emotional state (e.g., joy, surprise, sadness, etc.) is obtained along with the feature data. The input is the feature data, and the output is the emotion data.

[0470] Step 5:

[0471] The server sends a search query for similar items to the database server based on the feature data and emotion data. Pandas is used to search for similar items from the database of the online shopping platform and obtain a list. The input is the feature data and emotion data, and the output is a search result list of similar items.

[0472] Step 6:

[0473] The server customizes the search result list by taking into account the user's past purchase history and input information. It uses Pandas to analyze the list and generate an individually tailored recommendation list. The input is the search result list and the user's purchase history and input information, and the output is a customized recommendation list.

[0474] Step 7:

[0475] The server sends the generated customized recommendation list to the terminal as an HTTP response. The terminal displays the received recommendation list on the user interface. The input is the customized recommendation list, and the output is the list displayed on the user interface.

[0476] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0477] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0478] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0479] [Second embodiment]

[0480] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0481] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0482] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0483] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0484] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0485] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0486] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0487] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0488] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0489] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0490] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0491] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0492] To implement the present invention, the following describes a specific system and its program processing. This system allows users to upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the items.

[0493] System configuration

[0494] 1. Device:

[0495] The system allows users to select and upload photos on their smartphones, computers, or other devices, and includes an interface for displaying recommendation lists.

[0496] 2. Server:

[0497] It has the ability to receive images, analyze them, extract features, generate and customize recommendation lists, and also includes a database that manages user purchase history and input information.

[0498] 3. Online shopping platform:

[0499] It is an e-commerce site where users can actually purchase recommended items, and provides product data in response to queries from the server.

[0500] Program processing

[0501] 1. Upload an image

[0502] Users use their own devices to select photos of their favorite celebrities or influencers and upload them to the system.

[0503] The device sends the selected photo to the server as an HTTP request.

[0504] 2. Image Analysis

[0505] The server passes the received photo data to the AI ​​image analysis module and begins analysis.

[0506] The AI ​​image analysis module uses object detection technology to detect fashion items in photos (e.g., a red dress, black heels, gold earrings).

[0507] 3. Feature Extraction

[0508] The server extracts the item's meta information based on the analysis results, including color, shape, brand name, etc.

[0509] For example, "red dress" includes information such as color: red, item: dress, brand name: (if known).

[0510] 4. Recommendation Generation

[0511] The server uses the extracted meta information to access the online shopping platform's database and search for similar items.

[0512] The server generates a recommendation list based on the items obtained from the search results.

[0513] 5. Customization

[0514] The server compares the user's past purchase history with the information they provide, including their preferred colors, styles, budget, and other information.

[0515] The server uses this information to customize the recommendation list and arrange the items in the optimal order.

[0516] 6. Providing Recommendations

[0517] The server sends the customized recommendation list to the user's device.

[0518] The device displays the received recommendation list on the user interface so that the user can easily check it.

[0519] Specific examples

[0520] Example 1: When a user uploads a photo of celebrity A

[0521] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[0522] 2. The server passes the photo to the AI, which then detects the "red dress," "black heels," and "gold earrings."

[0523] 3. The server extracts meta information for each item and searches for similar items on online shopping platforms.

[0524] 4. The server generates a recommendation list from the search results, customizing it based on the user's past preference for red dresses.

[0525] 5. The customized list is provided to the user, who can then view the list on their device, select and purchase the items they like.

[0526] This allows the system to easily find fashion items that match the user's preferred style.

[0527] The processing flow will be explained below.

[0528] Step 1:

[0529] Users can use their own devices to select photos of their favorite celebrities or influencers and upload them to the system by clicking the upload button.

[0530] Step 2:

[0531] The device sends the selected photo to the server, typically using an HTTP request.

[0532] Step 3:

[0533] The server receives the received photo data and passes it to the AI ​​image analysis module, which then starts the AI ​​image analysis process.

[0534] Step 4:

[0535] The AI ​​image analysis module analyzes the received photos and detects fashion items in the images, using object detection technology to identify items such as a red dress, black heels, and gold earrings.

[0536] Step 5:

[0537] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[0538] Step 6:

[0539] The server uses the extracted meta information to access the online shopping platform's database to search for similar items, and calls the platform's API to retrieve the required information.

[0540] Step 7:

[0541] The server generates a recommendation list based on search results obtained from the online shopping platform, and the recommendation list includes items to be suggested to the user.

[0542] Step 8:

[0543] The server retrieves and analyzes the user's past purchase history and information entered into the system, such as preferences, size, and budget, from a database.

[0544] Step 9:

[0545] The server customizes the recommendation list based on the acquired user information, taking into account past purchase history and input information to determine the most appropriate item order.

[0546] Step 10:

[0547] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[0548] Step 11:

[0549] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[0550] This series of steps allows users to easily find fashion items that suit their tastes and style.

[0551] Example 1

[0552] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0553] Conventional fashion item recommendation systems often fail to fully reflect user preferences, and the items they suggest often do not meet user expectations. Another issue is the low accuracy of image analysis, which makes it difficult to accurately extract item features.

[0554] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0555] In this invention, the server includes means for receiving images selected and uploaded by a user, means for passing the received images to an AI image analysis module and starting analysis, means for extracting features of fashion items in the images using the AI ​​image analysis module, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items, means for matching the user's past purchase history and input information to customize a recommendation list, and means for providing the generated recommendation list to the user, thereby enabling highly accurate recommendations that reflect the user's preferences.

[0556] "User" refers to an individual who uses this system to receive fashion item recommendations.

[0557] "Terminal" refers to a device used by a user, such as a smartphone or computer, that selects and uploads images.

[0558] "Server" refers to the central computer system that receives, analyzes, extracts features from, and generates and customizes recommendation lists for images.

[0559] The "AI Image Analysis Module" is a software component that uses artificial intelligence technology to analyze received images and detect fashion items within the images.

[0560] "Fashion items" refers to items such as clothing and accessories for which users request recommendations via photos.

[0561] "Features" refers to information necessary to identify a fashion item, such as its color, shape, brand name, etc.

[0562] A "recommendation list" refers to a list of fashion items suggested to users.

[0563] "Online shopping platform" refers to an e-commerce site where users can physically purchase fashion items.

[0564] "Purchase history" refers to information about fashion items a user has purchased in the past.

[0565] "Input information" refers to information such as preferences and budget that a user provides to the system.

[0566] An "API (Application Program Interface)" refers to the conventions and tools that allow different software systems to communicate with each other and exchange data.

[0567] System Overview

[0568] This invention is a system in which users upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the uploaded photos. The main components of the system are the user device, a server, and the online shopping platform.

[0569] Hardware and software used

[0570] 1. Device:

[0571] Hardware: Devices used by users, such as smartphones and computers.

[0572] Software: Web browser and dedicated applications.

[0573] Features: Image selection, upload, and display recommendation list.

[0574] 2. Server:

[0575] Hardware: High-performance cloud servers.

[0576] Software: AI image analysis module using libraries such as Python, TensorFlow, and OpenCV.

[0577] Functions: Image reception, analysis, feature extraction, recommendation list generation, and customization.

[0578] 3. Online shopping platform:

[0579] Hardware: The servers that run your e-commerce site.

[0580] Software: Product database and API.

[0581] Features: Find and offer similar items.

[0582] Detailed program description

[0583] 1. Upload an image

[0584] Users can select photos of their favorite celebrities or influencers using their smartphones or computers and upload them to the system. The images selected on the device are sent to the server as HTTP requests.

[0585] 2. Image Analysis

[0586] The server temporarily stores the received image data and passes it to the AI ​​image analysis module, which uses libraries such as TensorFlow and OpenCV to apply object detection technology to detect fashion items in the image.

[0587] 3. Feature Extraction

[0588] The server extracts meta information for each item from the analysis results. This meta information includes the item's color, shape, brand name, etc. For example, for a "red dress," the server extracts the following information: "Color: Red, Item: Dress, Brand: Unknown."

[0589] 4. Recommendation Generation

[0590] The server then uses the extracted meta information to access the online shopping platform's database and search for similar fashion items, using SQL queries and APIs to retrieve related items.

[0591] 5. Customization

[0592] The server compares the user's past purchase history and input information to customize the recommendation list, taking into account the user's preferences, budget, and other information to create a list of the most suitable items.

[0593] 6. Providing Recommendations

[0594] The server sends the customized recommendation list in JSON format to the user's device, which interprets the received data and displays it in a formatted user interface.

[0595] Specific examples

[0596] Example: When a user uploads a photo of celebrity A

[0597] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[0598] 2. The server passes the photo to an AI image analysis module, which then detects the "red dress," "black heels," and "gold earrings."

[0599] 3. The server extracts meta information for each item and searches for similar items on online shopping platforms.

[0600] 4. The server generates a recommendation list from the search results, customizing it based on the user's past preference for red dresses.

[0601] 5. A customized list is provided to the user, who can then view the list on their device, select and purchase the items they like.

[0602] Prompt Sentence Examples

[0603] For generative AI models, use the following prompt:

[0604] "Write a program that analyzes fashion items in images uploaded by users, extracts their features, and generates a recommendation list based on the user's preferences."

[0605] This prompt statement allows the model to generate the appropriate code.

[0606] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0607] Step 1: Select and upload an image

[0608] Users open the gallery app or file browser on their smartphone or PC and select photos of celebrities or influencers they like. The images selected by the user are sent to the system by clicking the upload button. The device then sends the selected images to the server as an HTTP request.

[0609] Input: An image file selected by the user on their device.

[0610] Output: Image data included in the HTTP request sent to the server.

[0611] Step 2: Receiving the image

[0612] The server receives the image data sent from the device, stores it in temporary storage, and then passes it to the AI ​​image analysis module for image analysis.

[0613] Input: Image data sent from the device.

[0614] Output: Image data saved in temporary storage.

[0615] Step 3: Begin image analysis

[0616] The server passes the image data stored in temporary storage to the AI ​​image analysis module, which then uses libraries such as TensorFlow and OpenCV to perform image analysis.

[0617] Input: Image data stored in temporary storage.

[0618] Output: A list of identified fashion items in the image.

[0619] Step 4: Extracting fashion item features

[0620] The server extracts meta information (color, shape, brand name, etc.) for each item based on the analysis results received from the AI ​​image analysis module. For example, if a red dress is identified, the meta information is extracted as "Color: Red, Item: Dress, Brand Name: Unknown."

[0621] Input: Analysis results of the AI ​​image analysis module.

[0622] Output: Extracted fashion item meta information.

[0623] Step 5: Search for recommended items

[0624] The server then accesses the online shopping platform's database based on the extracted fashion item meta information and searches for similar items using SQL queries or APIs, such as "SELECT FROM Items WHERE color="red" AND type="dress".

[0625] Input: Extracted fashion item meta information.

[0626] Output: A list of similar items retrieved from an online shopping platform.

[0627] Step 6: Generate a recommendation list

[0628] The server generates a recommendation list based on a list of similar items retrieved from an online shopping platform, and the list is customized based on the user's preferences and purchasing history.

[0629] Input: A list of similar items obtained from an online shopping platform.

[0630] Output: The generated recommendation list.

[0631] Step 7: Customize your recommendation list

[0632] The server customizes the generated recommendation list based on the user's past purchase history and input information. For example, if the user has previously purchased a favorite "red dress," the server will adjust the ranking of the list to reflect that information.

[0633] Input: User's past purchase history and input information, generated recommendation list.

[0634] Output: A customized recommendation list.

[0635] Step 8: Providing a recommendation list

[0636] The server sends the customized recommendation list in JSON format to the user's device, which interprets the received data, formats it, and displays it in a user interface. The user can then select and purchase the items they like from the list.

[0637] Input: A customized recommendation list.

[0638] Output: The recommendation list displayed on the user's device.

[0639] (Application example 1)

[0640] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0641] On conventional online shopping platforms, users had to spend a lot of time and effort finding fashion items that matched their tastes and style. Furthermore, they had to enter their payment information each time they made a purchase, which posed security risks during the payment process. To solve these problems, a system was needed that would allow users to easily upload images from their devices, analyze their features to recommend similar items, and even handle electronic payment.

[0642] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0643] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of objects in the images, means for searching for similar items from an online trading platform based on the extracted features of the objects and generating a recommendation list, means for customizing the recommendation list taking into account the user's past transaction history and input information, means for providing the generated recommendation list to the user, and means for purchasing items selected from the recommendation list via an electronic transaction payment gateway. This allows users to easily find fashion items that suit their tastes and style and to purchase them safely and quickly.

[0644] The term "user" refers to a person who uses a particular service or product, and in the present invention particularly refers to a person who uploads photos of fashion items and purchases recommended items.

[0645] "Image" refers to visual data selected from a terminal and uploaded to the system, which is the subject of analysis of the characteristics of fashion items.

[0646] "Particular object" refers to an object present in the received image, and in the present invention particularly refers to a fashion item (e.g., a dress, shoes, accessories).

[0647] "Feature" refers to an identifiable attribute or property of an object, and in the present invention includes color, shape, brand name, and the like.

[0648] A "recommendation list" refers to a list of recommended items generated based on extracted features, which users can browse and select according to their preferences.

[0649] "Online trading platform" refers to a website or application that offers products through e-commerce and allows users to search, browse, and purchase products.

[0650] "Transaction history" refers to a record of a user's past purchases and transactions, and is information used to customize recommendation lists.

[0651] "Payment Gateway" refers to the payment processing system used in a transaction, which functions to enable secure and fast payments.

[0652] A "server" refers to a computer system that receives requests from clients (user devices) and performs data processing and communication. In this invention, it is responsible for image analysis, feature extraction, recommendation generation, payment processing, etc.

[0653] "API" is an abbreviation for Application Programming Interface, which means an interface that enables communication between software programs. In the present invention, it is used to access the online trading platform.

[0654] The present invention relates to a system for analyzing an image, providing a user with recommended items, and then purchasing the items through electronic payment. A specific embodiment of the present invention will be described below.

[0655] The overall system configuration is as follows: It includes the devices used by users (smartphones and computers), a server that receives and analyzes data sent from these devices, and an online trading platform (website or application).

[0656] Program processing

[0657] Device:

[0658] Users use their own devices (e.g., smartphones) to take pictures of their favorite fashion items or select and upload existing images. The device then sends this image data to the server using an HTTP request.

[0659] server:

[0660] The server does the following:

[0661] Image analysis: The received image is passed to an AI image analysis module, which analyzes the features of specific objects. This uses image processing libraries such as OpenCV and an AI model to extract features such as color, shape, and brand name.

[0662] Recommendation generation: Based on the extracted features, queries are sent to online trading platforms to search for similar fashion items, using APIs.

[0663] Customization: The recommendation list is customized based on the user's past transaction history and input information, resulting in a recommendation list optimized for the user's individual preferences.

[0664] Recommendation provision: The generated recommendation list is sent to the user's device and displayed on the device.

[0665] Online trading platform:

[0666] The online trading platform provides product data in response to search queries from the server, which then lists the items desired by the user.

[0667] Electronic Payment:

[0668] Users can select the desired items from the recommendation list and make an electronic payment. The server processes the payment securely and quickly via a payment gateway (e.g., Stripe or PayPal).

[0669] Specific examples

[0670] For example, if User A uploads a photo of Celebrity B from his / her smartphone, the photo may contain a "red dress" and "black high heels." The server uses image analysis technology to extract these features and sends a query to an online trading platform. Similar items are retrieved from the platform, and a customized recommendation list is generated taking into account User A's past purchase history (e.g., a preference for red dresses). Finally, the list is provided to User A, who can select the items they like and easily purchase them.

[0671] To comprehensively support this process, the following prompts could be fed to the generative AI model:

[0672] Example prompt sentence:

[0673] A user takes a photo of a photogenic fashion and uploads it to the app. The photo includes a blue dress and white heels. The app extracts the features of the blue dress and searches for and recommends multiple similar items, including blue dresses and white heels. For each item, the user can easily pay electronically through the app.

[0674] The above is a specific embodiment for carrying out the present invention. This system allows users to easily find and safely purchase fashion items that suit their style.

[0675] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0676] Step 1:

[0677] Input: The user takes or selects a photo of a fashion item on their smartphone.

[0678] Specific operation: The user acquires an image using the device's camera / gallery function, selects the image from the system's application screen, and presses the upload button.

[0679] Output: The uploaded image data is sent to the server as an HTTP request.

[0680] Step 2:

[0681] Input: The server receives the image data sent from the terminal.

[0682] Specific operation: The server receives the request, obtains the image data contained therein, checks the integrity of the data using the HTTP protocol, and passes it to the analysis module.

[0683] Output: The image data is passed to the image analysis module.

[0684] Step 3:

[0685] Input: The server passes the image data to the image analysis module.

[0686] How it works: The server uses the OpenCV library to read an image and input it into the generative AI model. The AI ​​model then analyzes the input image and extracts the features of the object. During this process, attribute information such as color, shape, and brand name is identified.

[0687] Output: Feature data of objects extracted from the image (e.g. color: blue, shape: dress, brand name: XYZ).

[0688] Step 4:

[0689] Input: Based on the extracted feature data, the server sends a query to the online trading platform.

[0690] How it works: The server uses an API to access the database of an online trading platform and search for items with similar characteristics. The query statement includes characteristics such as color, shape, and brand name.

[0691] Output: A data list of the retrieved similar items.

[0692] Step 5:

[0693] Input: The server generates a recommendation list based on this acquired data list.

[0694] How it works: The server customizes the recommendation list based on the user's past transaction history and input information. For example, if the user has previously purchased blue dresses, the server uses that information to prioritize items in the list.

[0695] Output: A customized recommendation list.

[0696] Step 6:

[0697] Input: The server sends the generated recommendation list to the terminal.

[0698] Specific operation: The server sends the generated recommendation list to the user's device and displays it on the application's user interface.

[0699] Output: The recommendation list displayed on the user's device.

[0700] Step 7:

[0701] Input: The user selects the desired item from the recommendation list.

[0702] Specific operation: The user operates the terminal interface, selects a specific item from the list, and performs the purchase operation. After the purchase operation, the information of the selected item and payment information are sent to the server.

[0703] Output: Selected item information and payment information are sent to the server.

[0704] Step 8:

[0705] Input: The server processes the received payment information and executes the transaction via the payment gateway.

[0706] What happens: The server passes the payment information to a payment gateway (e.g., Stripe, PayPal) to process the secure transaction. After the transaction is complete, the user is notified of the transaction status.

[0707] Output: A notification of the payment processing result and purchase confirmation is sent to the user.

[0708] This allows users to easily find fashion items that suit their tastes and style and purchase them quickly and safely.

[0709] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0710] To implement the present invention, the following specific system and its program processing will be described. In this system, users upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the items and the user's emotions.

[0711] System configuration

[0712] 1. Device:

[0713] The system allows users to select and upload photos on their smartphones, computers, or other devices, and includes an interface for displaying recommendation lists.

[0714] 2. Server:

[0715] It has the ability to receive images, analyze them, extract features, recognize emotions, generate and customize recommendation lists, and also includes a database that manages user purchase history, input information, and an emotion recognition engine.

[0716] 3. Online shopping platform:

[0717] It is an e-commerce site where users can actually purchase recommended items, and provides product data in response to queries from the server.

[0718] Program processing

[0719] 1. Upload an image

[0720] Users can select photos of their favorite celebrities or influencers using their own devices and upload them to the system by clicking the upload button.

[0721] The device sends the selected photo to the server as an HTTP request.

[0722] 2. Image Analysis

[0723] The server receives the received photo data, passes it to the AI ​​image analysis module, and begins analysis.

[0724] The AI ​​image analysis module uses object detection technology to detect fashion items in photos (e.g., a red dress, black heels, gold earrings).

[0725] 3. Feature Extraction

[0726] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[0727] 4. Emotion recognition

[0728] The server runs an emotion recognition engine based on the uploaded photo and the user's facial expressions, which classifies the user's emotional state (e.g., joy, sadness, surprise, fear).

[0729] 5. Recommendation Generation

[0730] The server accesses the database of the online shopping platform based on the extracted meta information and emotion recognition data to search for similar items.

[0731] The server generates a recommendation list based on the items obtained from the search results, prioritizing appropriate items based on the user's sentiment.

[0732] 6. Customization

[0733] The server retrieves the user's past purchase history and input information from the database and analyzes it.

[0734] The server customizes the recommendation list based on the acquired user information and emotional information, and determines the most appropriate item order, taking into account past purchase history and input information.

[0735] 7. Providing Recommendations

[0736] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[0737] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[0738] Specific examples

[0739] Example 1: When a user uploads a photo of celebrity A

[0740] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[0741] 2. The server passes the photo to the AI, which then detects the "red dress," "black heels," and "gold earrings."

[0742] 3. The server extracts meta information for each item and uses an emotion recognition engine to determine the user's emotion as "joy."

[0743] 4. The server searches for similar items on the online shopping platform and generates a recommendation list based on the user's sentiment.

[0744] 5. The server customizes the list based on past purchase history and provides it to the user.

[0745] 6. The device will display a customized list, allowing the user to select and purchase the items they like.

[0746] This allows the system to easily find fashion items that suit users' preferences and style, and also provides optimal recommendations based on their emotional state.

[0747] The processing flow will be explained below.

[0748] Step 1:

[0749] Users select photos of their favorite celebrities or influencers using their own devices and upload them to the system. The user then clicks the upload button.

[0750] Step 2:

[0751] The device sends the selected photo to the server, typically using an HTTP request.

[0752] Step 3:

[0753] The server receives the received photo data and passes it to the AI ​​image analysis module, which then starts the AI ​​image analysis process.

[0754] Step 4:

[0755] The AI ​​image analysis module analyzes the received photos and detects fashion items in the images, using object detection technology to identify items such as a red dress, black heels, and gold earrings.

[0756] Step 5:

[0757] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[0758] Step 6:

[0759] The server runs an emotion recognition engine based on the uploaded photo and the user's facial expressions, which classifies the user's emotional state (e.g., joy, sadness, surprise, fear).

[0760] Step 7:

[0761] The server accesses the online shopping platform's database based on the extracted meta information and emotion recognition data to search for similar items, and calls the platform's API to retrieve the required information.

[0762] Step 8:

[0763] The server generates a recommendation list based on search results obtained from the online shopping platform, prioritizing appropriate items based on the user's sentiment.

[0764] Step 9:

[0765] The server retrieves and analyzes the user's past purchase history and information entered into the system, such as preferences, size, and budget, from a database.

[0766] Step 10:

[0767] The server customizes the recommendation list based on the acquired user information and sentiment information, and determines the most appropriate item order, taking into account past purchase history and input information.

[0768] Step 11:

[0769] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[0770] Step 12:

[0771] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[0772] This series of steps allows users to easily find fashion items that suit their tastes and style, and also provides appropriate recommendations based on their emotional state at the time.

[0773] Example 2

[0774] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0775] Conventional fashion item recommendation systems simply make recommendations based on a user's past purchase history and simple feature information, making it difficult to take into account the user's emotional state or detailed item features.It also makes it difficult for users to easily find fashion items that suit their preferences and emotions.

[0776] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0777] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of fashion items in the images, means for analyzing the user's facial expression data and classifying emotions, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and the classified emotion data, means for customizing the recommendation list in consideration of the user's past purchase history and input information, and means for providing the generated recommendation list to the user, thereby enabling the user to efficiently find optimal fashion items based on their preferences and emotions.

[0778] A "user" is someone who uses the system to receive fashion item recommendations.

[0779] "Terminal" means the device used by a User to access the System and upload Images.

[0780] A "server" is a computer system that performs central processing such as image analysis, emotion recognition, database management, and generation of recommendation lists.

[0781] "Image upload" is the act of a user using a device to send a selected image to a server.

[0782] "Image analysis" is the process of using AI technology to extract the characteristics of fashion items based on received image data.

[0783] "Feature extraction" refers to obtaining attribute information such as color, shape, and brand of fashion items identified through image analysis.

[0784] "Emotion recognition" is the process of analyzing a user's facial expressions in an image to determine the user's emotional state.

[0785] An "online shopping platform" is an e-commerce system that allows users to purchase recommended fashion items.

[0786] A "recommendation list" is a list of fashion items suggested to a user.

[0787] "Customization" is the process of optimizing the recommendation list based on the user's past purchase history and input information.

[0788] An "HTTP request" is a form of data request sent from a user's device to a server.

[0789] An "HTTP response" is a data response sent from a server to a terminal.

[0790] The present invention provides a system in which a user uploads photos of fashion items using their own terminal, and an online shopping platform provides recommended items based on the features of the items and the user's feelings.

[0791] System configuration

[0792] Hardware and Software

[0793] Device: A device used by a user (e.g., smartphone, tablet, PC, etc.). The device has the functionality to select and upload photos, includes an interface to display the recommendation list, and is responsible for sending HTTP requests to the server.

[0794] Server: Has the ability to analyze received image data, extract features, recognize emotions, generate recommendation lists, and customize them. It also includes a database that manages user purchase history and input information. AI image analysis uses software such as TensorFlow, and emotion recognition uses the Microsoft Azure Emotion API.

[0795] Online shopping platform: An e-commerce system that provides product data, where the server accesses and retrieves product information via API. Users can purchase the items provided.

[0796] Program processing explanation

[0797] The program of the system of the present invention is processed in the following procedure.

[0798] Image upload

[0799] A user uses a device to select a photo of their favorite celebrity or influencer and upload it to the system. Specifically, the user selects a photo from the device's photo gallery and presses the upload button. The device then sends the selected photo to the server as an HTTP POST request. This request includes the photo data and the user's ID.

[0800] Image analysis

[0801] The server receives the photo data from the device. The server then passes this data to an AI image analysis module (e.g., TensorFlow), which uses an object detection algorithm to detect fashion items in the photo (e.g., "red dress," "black heels," "gold earrings").

[0802] Feature extraction

[0803] The server extracts meta information about fashion items based on the detection results returned by the image analysis module, specifically, color information (e.g., red), item type (e.g., dress), and brand (if identifiable) for each item.

[0804] emotion recognition

[0805] The server inputs the photo and the user's facial expression data into an emotion recognition engine (for example, Microsoft Azure Emotion API), which then classifies the user's emotional state from the photo into "joy," "sadness," "surprise," "fear," etc.

[0806] Recommendation generation

[0807] The server then queries the online shopping platform's database to search for similar items based on the extracted item meta information and sentiment data, generating a recommendation list and prioritizing items based on user sentiment.

[0808] Customization

[0809] The server retrieves the user's past purchase history and input information from a database, and based on the retrieved data, customizes the recommendation list and determines the most appropriate item order.

[0810] Providing recommendations

[0811] The server finally generates a customized recommendation list and sends it to the device as an HTTP response. The device displays the received recommendation list on a user interface, allowing the user to browse the list and purchase items.

[0812] Specific examples

[0813] Example 1: When a user uploads a photo of celebrity A

[0814] The user selects a photo of celebrity A from their smartphone and uploads it to the system. Specifically, they select a photo from their photo gallery and tap the "Upload" button.

[0815] The server receives the photo data and passes it to the AI ​​image analysis module, which then detects the "red dress," "black heels," and "gold earrings."

[0816] The server extracts meta information for each item based on the analysis results, and an emotion recognition engine determines the user's emotion as "joy."

[0817] The server searches for similar items from online shopping platforms based on meta information and emotion data, and lists the items obtained, prioritizing them based on the emotion of "joy."

[0818] The server customizes the list taking into account purchase history and sends it to the terminal as an HTTP response.

[0819] The device will display a customized list, allowing users to select and purchase their preferred items.

[0820] Prompt Sentence Examples

[0821] A user uploads a photo of celebrity A wearing a red dress using their smartphone. The AI ​​analyzes the image, detects items, and extracts their features. It then analyzes the user's emotion as "joy" and recommends similar items from online shops based on the emotion. Finally, the customized list is displayed to the user, ready for purchase.

[0822] This system allows users to efficiently find the perfect fashion items based on their preferences and feelings.

[0823] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0824] Program processing flow

[0825] Step 1:

[0826] Image upload

[0827] Users can select a photo of their favorite celebrity or influencer from their device. Specifically, they select a photo from their device's photo gallery and tap the "Upload" button.

[0828] Input: An image file selected by the user.

[0829] The device sends the selected photo to the server as an HTTP POST request, which includes the image data and the user ID.

[0830] Output: Image data and user ID transferred from the device to the server.

[0831] Step 2:

[0832] Image analysis

[0833] The server receives the image data from the device and passes it to the AI ​​image analysis module.

[0834] Input: Image data received from the device and user ID.

[0835] The server starts analyzing the photo using an AI image analysis module (e.g. TensorFlow) and uses object detection algorithms to detect fashion items in the image (e.g. red dress, black heels, gold earrings).

[0836] Output: A list of parsed fashion items.

[0837] Step 3:

[0838] Feature extraction

[0839] The server extracts meta information about the fashion item based on the detection results returned by the image analysis module.

[0840] Input: A list of parsed fashion items.

[0841] The server retrieves information about each item, such as color (red), item type (dress), and brand (if identifiable).

[0842] Output: Extracted fashion item features (color, type, brand).

[0843] Step 4:

[0844] emotion recognition

[0845] The server uses an emotion recognition engine (e.g., Microsoft Azure Emotion API) to identify the user's emotional state and classify emotions based on the photo and the user's facial expression data.

[0846] Input: Uploaded image data, user face detection information.

[0847] The server uses an emotion recognition engine to extract emotional states such as "happiness," "sadness," "surprise," and "fear."

[0848] Output: User sentiment classification data.

[0849] Step 5:

[0850] Recommendation generation

[0851] The server then queries the online shopping platform's database based on the extracted item features and sentiment data.

[0852] Input: Item feature data, emotion data.

[0853] The server searches for similar items and generates a recommendation list that prioritizes appropriate items based on the user's sentiment.

[0854] Output: The generated recommendation list.

[0855] Step 6:

[0856] Customization

[0857] The server retrieves the user's past purchase history and input information from a database.

[0858] Input: Past purchase history, input information.

[0859] The server then customizes the recommendation list based on the information it has obtained, determining the optimal order of items based on past purchase history.

[0860] Output: A customized recommendation list.

[0861] Step 7:

[0862] Providing recommendations

[0863] The server then sends the final customized recommendation list to the device as an HTTP response.

[0864] Input: A customized recommendation list.

[0865] The device displays the received recommendation list on the user interface.

[0866] Output: A list of recommendations displayed in a user interface. The user can browse the list, select items, and purchase them.

[0867] (Application example 2)

[0868] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0869] Conventional online shopping systems do not take into account the user's emotional state when making recommendations, making it difficult for users to find the perfect fashion item that best suits their current mood. Furthermore, technology for accurately extracting the characteristics of fashion items from images uploaded by users and generating recommendation lists based on those characteristics is also inadequate. This leads to issues such as lower user satisfaction and a decrease in purchasing motivation.

[0870] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0871] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of fashion items in the images, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and the user's emotional state and generating a recommendation list, means for customizing the recommendation list taking into account the user's past purchase history and input information, and means for providing the generated recommendation list to the user. This enables optimal fashion item recommendations that take into account the user's emotional state. Furthermore, users can easily find products that match their emotional state, which is expected to increase their purchasing motivation.

[0872] "User" refers to a person who uses the system, primarily someone who uploads images of fashion items and receives recommendations.

[0873] "Image receiving means" refers to a device or software that has the function of receiving image data uploaded by a user and sending it to a server.

[0874] "Image analysis means" refers to a device or software that has the function of analyzing received image data and identifying and extracting the characteristics of fashion items in the image.

[0875] The "feature extraction means" refers to a device or software that has the function of extracting the features of a fashion item, such as color, shape, or type, from the analyzed image.

[0876] "Emotion recognition means" refers to a device or software that has the function of recognizing and determining the emotional state of a user.

[0877] "Online shopping platform" means an e-commerce site that allows users to purchase fashion items via the Internet.

[0878] "Recommendation list generation means" refers to a device or software that has the function of searching for similar items from an online shopping platform based on the extracted features and the user's emotional state and generating a list to present to the user.

[0879] The "customization means" refers to a device or software that has the function of individually adjusting the generated recommendation list, taking into account the user's past purchase history and input information.

[0880] "Recommendation list providing means" refers to a device or software that has the function of displaying and providing a final customized recommendation list to a user.

[0881] To implement the present invention, it is necessary to build a system that allows users to upload images of fashion items using their own devices and provides recommended items based on the features of the items and the user's emotions. Specifically, this is implemented in the following way.

[0882] Details of the hardware and software used

[0883] Hardware: Smartphones, servers, database servers

[0884] Software: Flask (web framework), OpenCV (image processing library), Keras (deep learning framework), Pandas (data analysis library)

[0885] System configuration

[0886] 1. Device:

[0887] Users use their smartphones to take or select their favorite fashion items and upload the images to the system, which then sends the images to the server as HTTP requests.

[0888] 2. Server:

[0889] It analyzes the received images. First, it preprocesses the images using OpenCV, then extracts the features of the fashion items using an AI image analysis module built with Keras. It also uses another Keras model to recognize the user's emotions.

[0890] 3. Database Server:

[0891] Based on the extracted features and sentiment data, similar items are searched for using Pandas. Product data is obtained through APIs in collaboration with the database of an online shopping platform.

[0892] Data processing and calculation

[0893] Image upload:

[0894] When a user uploads an image from a device, the device sends the image data to the server, where it is passed as request data via the HTTP protocol.

[0895] Image analysis:

[0896] The server processes the received image data using OpenCV and converts it into an appropriate format, then inputs it into an image analysis model built with Keras to extract the main features of the fashion item (color, shape, type).

[0897] Emotion recognition:

[0898] The server analyzes the user's facial expressions and recognizes their emotional state (e.g., happy, surprised, sad), again using Keras' emotion recognition model.

[0899] Recommendation generation and customization:

[0900] The database server searches for similar items from online shopping platforms based on the extracted features and the recognized emotions, then uses Pandas to generate a recommendation list, which is further customized by taking into account the user's past purchase history and input information.

[0901] Providing a recommendation list:

[0902] The server sends the generated customized recommendation list to the device, which then displays the list on the user interface, allowing the user to select and purchase the items they like.

[0903] Specific examples

[0904] Example 1: A user uploads a photo wearing a black coat

[0905] Users select a photo from their smartphone of themselves wearing a black coat and upload it to the system.

[0906] The server passes the photo to the AI, which then detects the "black coat."

[0907] The server extracts the characteristics of the item and uses an emotion recognition engine to determine the user's emotion as "joy."

[0908] The server searches for similar items on the online shopping platform and generates a recommendation list based on the user's sentiment.

[0909] The server takes into account past purchase history to customize the list and provides it to the user.

[0910] The device will display a customized list, allowing users to select and purchase their preferred items.

[0911] In this way, the system allows users to easily find fashion items that suit their preferences and style, and provides optimal recommendations based on their emotional state.

[0912] Example prompts to input to a generative AI model:

[0913] plaintext

[0914] Generate recommended items when a user uploads an image of themselves wearing a black coat. The user's emotion is recognized as "joy." The item characteristics obtained from the image are "black, coat." Recommend similar items.

[0915] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0916] Step 1:

[0917] A user selects and uploads an image of a fashion item using a device. At this time, the image file is sent to the server as an HTTP request. The input is the image file on the device, and the output is an HTTP request containing this image file.

[0918] Step 2:

[0919] The server analyzes the received HTTP request and obtains image data. The obtained image data is preprocessed using OpenCV and converted into a format that can be input to the AI ​​image analysis model. The input is the image data in the HTTP request, and the output is the preprocessed image data.

[0920] Step 3:

[0921] The server inputs the preprocessed image data into a Keras AI image analysis model to extract the features of the fashion items. The AI ​​model analyzes the image and obtains features such as color, shape, and type. The input is the preprocessed image data, and the output is the obtained feature data.

[0922] Step 4:

[0923] The server uses another Keras model to recognize the user's emotion based on the extracted feature data. The user's emotional state (e.g., joy, surprise, sadness, etc.) is obtained along with the feature data. The input is the feature data, and the output is the emotion data.

[0924] Step 5:

[0925] The server sends a search query for similar items to the database server based on the feature data and emotion data. Pandas is used to search for similar items from the database of the online shopping platform and obtain a list. The input is the feature data and emotion data, and the output is a search result list of similar items.

[0926] Step 6:

[0927] The server customizes the search result list by taking into account the user's past purchase history and input information. It uses Pandas to analyze the list and generate an individually tailored recommendation list. The input is the search result list and the user's purchase history and input information, and the output is a customized recommendation list.

[0928] Step 7:

[0929] The server sends the generated customized recommendation list to the terminal as an HTTP response. The terminal displays the received recommendation list on the user interface. The input is the customized recommendation list, and the output is the list displayed on the user interface.

[0930] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0931] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0932] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0933] [Third embodiment]

[0934] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0935] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0936] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0937] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0938] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0939] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0940] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0941] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0942] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0943] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0944] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0945] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0946] To implement the present invention, the following describes a specific system and its program processing. This system allows users to upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the items.

[0947] System configuration

[0948] 1. Device:

[0949] The system allows users to select and upload photos on their smartphones, computers, or other devices, and includes an interface for displaying recommendation lists.

[0950] 2. Server:

[0951] It has the ability to receive images, analyze them, extract features, generate and customize recommendation lists, and also includes a database that manages user purchase history and input information.

[0952] 3. Online shopping platform:

[0953] It is an e-commerce site where users can actually purchase recommended items, and provides product data in response to queries from the server.

[0954] Program processing

[0955] 1. Upload an image

[0956] Users use their own devices to select photos of their favorite celebrities or influencers and upload them to the system.

[0957] The device sends the selected photo to the server as an HTTP request.

[0958] 2. Image Analysis

[0959] The server passes the received photo data to the AI ​​image analysis module and begins analysis.

[0960] The AI ​​image analysis module uses object detection technology to detect fashion items in photos (e.g., a red dress, black heels, gold earrings).

[0961] 3. Feature Extraction

[0962] The server extracts the item's meta information based on the analysis results, including color, shape, brand name, etc.

[0963] For example, "red dress" includes information such as color: red, item: dress, brand name: (if known).

[0964] 4. Recommendation Generation

[0965] The server uses the extracted meta information to access the online shopping platform's database and search for similar items.

[0966] The server generates a recommendation list based on the items obtained from the search results.

[0967] 5. Customization

[0968] The server compares the user's past purchase history with the information they provide, including their preferred colors, styles, budget, and other information.

[0969] The server uses this information to customize the recommendation list and arrange the items in the optimal order.

[0970] 6. Providing Recommendations

[0971] The server sends the customized recommendation list to the user's device.

[0972] The device displays the received recommendation list on the user interface so that the user can easily check it.

[0973] Specific examples

[0974] Example 1: When a user uploads a photo of celebrity A

[0975] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[0976] 2. The server passes the photo to the AI, which then detects the "red dress," "black heels," and "gold earrings."

[0977] 3. The server extracts meta information for each item and searches for similar items on online shopping platforms.

[0978] 4. The server generates a recommendation list from the search results, customizing it based on the user's past preference for red dresses.

[0979] 5. The customized list is provided to the user, who can then view the list on their device, select and purchase the items they like.

[0980] This allows the system to easily find fashion items that match the user's preferred style.

[0981] The processing flow will be explained below.

[0982] Step 1:

[0983] Users can use their own devices to select photos of their favorite celebrities or influencers and upload them to the system by clicking the upload button.

[0984] Step 2:

[0985] The device sends the selected photo to the server, typically using an HTTP request.

[0986] Step 3:

[0987] The server receives the received photo data and passes it to the AI ​​image analysis module, which then starts the AI ​​image analysis process.

[0988] Step 4:

[0989] The AI ​​image analysis module analyzes the received photos and detects fashion items in the images, using object detection technology to identify items such as a red dress, black heels, and gold earrings.

[0990] Step 5:

[0991] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[0992] Step 6:

[0993] The server uses the extracted meta information to access the online shopping platform's database to search for similar items, and calls the platform's API to retrieve the required information.

[0994] Step 7:

[0995] The server generates a recommendation list based on search results obtained from the online shopping platform, and the recommendation list includes items to be suggested to the user.

[0996] Step 8:

[0997] The server retrieves and analyzes the user's past purchase history and information entered into the system, such as preferences, size, and budget, from a database.

[0998] Step 9:

[0999] The server customizes the recommendation list based on the acquired user information, taking into account past purchase history and input information to determine the most appropriate item order.

[1000] Step 10:

[1001] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[1002] Step 11:

[1003] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[1004] This series of steps allows users to easily find fashion items that suit their tastes and style.

[1005] Example 1

[1006] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1007] Conventional fashion item recommendation systems often fail to fully reflect user preferences, and the items they suggest often do not meet user expectations. Another issue is the low accuracy of image analysis, which makes it difficult to accurately extract item features.

[1008] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1009] In this invention, the server includes means for receiving images selected and uploaded by a user, means for passing the received images to an AI image analysis module and starting analysis, means for extracting features of fashion items in the images using the AI ​​image analysis module, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items, means for matching the user's past purchase history and input information to customize a recommendation list, and means for providing the generated recommendation list to the user, thereby enabling highly accurate recommendations that reflect the user's preferences.

[1010] "User" refers to an individual who uses this system to receive fashion item recommendations.

[1011] "Terminal" refers to a device used by a user, such as a smartphone or computer, that selects and uploads images.

[1012] "Server" refers to the central computer system that receives, analyzes, extracts features from, and generates and customizes recommendation lists for images.

[1013] The "AI Image Analysis Module" is a software component that uses artificial intelligence technology to analyze received images and detect fashion items within the images.

[1014] "Fashion items" refers to items such as clothing and accessories for which users request recommendations via photos.

[1015] "Features" refers to information necessary to identify a fashion item, such as its color, shape, brand name, etc.

[1016] A "recommendation list" refers to a list of fashion items suggested to users.

[1017] "Online shopping platform" refers to an e-commerce site where users can physically purchase fashion items.

[1018] "Purchase history" refers to information about fashion items a user has purchased in the past.

[1019] "Input information" refers to information such as preferences and budget that a user provides to the system.

[1020] An "API (Application Program Interface)" refers to the conventions and tools that allow different software systems to communicate with each other and exchange data.

[1021] System Overview

[1022] This invention is a system in which users upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the uploaded photos. The main components of the system are the user device, a server, and the online shopping platform.

[1023] Hardware and software used

[1024] 1. Device:

[1025] Hardware: Devices used by users, such as smartphones and computers.

[1026] Software: Web browser and dedicated applications.

[1027] Features: Image selection, upload, and display recommendation list.

[1028] 2. Server:

[1029] Hardware: High-performance cloud servers.

[1030] Software: AI image analysis module using libraries such as Python, TensorFlow, and OpenCV.

[1031] Functions: Image reception, analysis, feature extraction, recommendation list generation, and customization.

[1032] 3. Online shopping platform:

[1033] Hardware: The servers that run your e-commerce site.

[1034] Software: Product database and API.

[1035] Features: Find and offer similar items.

[1036] Detailed program description

[1037] 1. Upload an image

[1038] Users can select photos of their favorite celebrities or influencers using their smartphones or computers and upload them to the system. The images selected on the device are sent to the server as HTTP requests.

[1039] 2. Image Analysis

[1040] The server temporarily stores the received image data and passes it to the AI ​​image analysis module, which uses libraries such as TensorFlow and OpenCV to apply object detection technology to detect fashion items in the image.

[1041] 3. Feature Extraction

[1042] The server extracts meta information for each item from the analysis results. This meta information includes the item's color, shape, brand name, etc. For example, for a "red dress," the server extracts the following information: "Color: Red, Item: Dress, Brand: Unknown."

[1043] 4. Recommendation Generation

[1044] The server then uses the extracted meta information to access the online shopping platform's database and search for similar fashion items, using SQL queries and APIs to retrieve related items.

[1045] 5. Customization

[1046] The server compares the user's past purchase history and input information to customize the recommendation list, taking into account the user's preferences, budget, and other information to create a list of the most suitable items.

[1047] 6. Providing Recommendations

[1048] The server sends the customized recommendation list in JSON format to the user's device, which interprets the received data and displays it in a formatted user interface.

[1049] Specific examples

[1050] Example: When a user uploads a photo of celebrity A

[1051] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[1052] 2. The server passes the photo to an AI image analysis module, which then detects the "red dress," "black heels," and "gold earrings."

[1053] 3. The server extracts meta information for each item and searches for similar items on online shopping platforms.

[1054] 4. The server generates a recommendation list from the search results, customizing it based on the user's past preference for red dresses.

[1055] 5. A customized list is provided to the user, who can then view the list on their device, select and purchase the items they like.

[1056] Prompt Sentence Examples

[1057] For generative AI models, use the following prompt:

[1058] "Write a program that analyzes fashion items in images uploaded by users, extracts their features, and generates a recommendation list based on the user's preferences."

[1059] This prompt statement allows the model to generate the appropriate code.

[1060] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1061] Step 1: Select and upload an image

[1062] Users open the gallery app or file browser on their smartphone or PC and select photos of celebrities or influencers they like. The images selected by the user are sent to the system by clicking the upload button. The device then sends the selected images to the server as an HTTP request.

[1063] Input: An image file selected by the user on their device.

[1064] Output: Image data included in the HTTP request sent to the server.

[1065] Step 2: Receiving the image

[1066] The server receives the image data sent from the device, stores it in temporary storage, and then passes it to the AI ​​image analysis module for image analysis.

[1067] Input: Image data sent from the device.

[1068] Output: Image data saved in temporary storage.

[1069] Step 3: Begin image analysis

[1070] The server passes the image data stored in temporary storage to the AI ​​image analysis module, which then uses libraries such as TensorFlow and OpenCV to perform image analysis.

[1071] Input: Image data stored in temporary storage.

[1072] Output: A list of identified fashion items in the image.

[1073] Step 4: Extracting fashion item features

[1074] The server extracts meta information (color, shape, brand name, etc.) for each item based on the analysis results received from the AI ​​image analysis module. For example, if a red dress is identified, the meta information is extracted as "Color: Red, Item: Dress, Brand Name: Unknown."

[1075] Input: Analysis results of the AI ​​image analysis module.

[1076] Output: Extracted fashion item meta information.

[1077] Step 5: Search for recommended items

[1078] The server then accesses the online shopping platform's database based on the extracted fashion item meta information and searches for similar items using SQL queries or APIs, such as "SELECT FROM Items WHERE color="red" AND type="dress".

[1079] Input: Extracted fashion item meta information.

[1080] Output: A list of similar items retrieved from an online shopping platform.

[1081] Step 6: Generate a recommendation list

[1082] The server generates a recommendation list based on a list of similar items retrieved from an online shopping platform, and the list is customized based on the user's preferences and purchasing history.

[1083] Input: A list of similar items obtained from an online shopping platform.

[1084] Output: The generated recommendation list.

[1085] Step 7: Customize your recommendation list

[1086] The server customizes the generated recommendation list based on the user's past purchase history and input information. For example, if the user has previously purchased a favorite "red dress," the server will adjust the ranking of the list to reflect that information.

[1087] Input: User's past purchase history and input information, generated recommendation list.

[1088] Output: A customized recommendation list.

[1089] Step 8: Providing a recommendation list

[1090] The server sends the customized recommendation list in JSON format to the user's device, which interprets the received data, formats it, and displays it in a user interface. The user can then select and purchase the items they like from the list.

[1091] Input: A customized recommendation list.

[1092] Output: The recommendation list displayed on the user's device.

[1093] (Application example 1)

[1094] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1095] On conventional online shopping platforms, users had to spend a lot of time and effort finding fashion items that matched their tastes and style. Furthermore, they had to enter their payment information each time they made a purchase, which posed security risks during the payment process. To solve these problems, a system was needed that would allow users to easily upload images from their devices, analyze their features to recommend similar items, and even handle electronic payment.

[1096] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1097] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of objects in the images, means for searching for similar items from an online trading platform based on the extracted features of the objects and generating a recommendation list, means for customizing the recommendation list taking into account the user's past transaction history and input information, means for providing the generated recommendation list to the user, and means for purchasing items selected from the recommendation list via an electronic transaction payment gateway. This allows users to easily find fashion items that suit their tastes and style and to purchase them safely and quickly.

[1098] The term "user" refers to a person who uses a particular service or product, and in the present invention particularly refers to a person who uploads photos of fashion items and purchases recommended items.

[1099] "Image" refers to visual data selected from a terminal and uploaded to the system, which is the subject of analysis of the characteristics of fashion items.

[1100] "Particular object" refers to an object present in the received image, and in the present invention particularly refers to a fashion item (e.g., a dress, shoes, accessories).

[1101] "Feature" refers to an identifiable attribute or property of an object, and in the present invention includes color, shape, brand name, and the like.

[1102] A "recommendation list" refers to a list of recommended items generated based on extracted features, which users can browse and select according to their preferences.

[1103] "Online trading platform" refers to a website or application that offers products through e-commerce and allows users to search, browse, and purchase products.

[1104] "Transaction history" refers to a record of a user's past purchases and transactions, and is information used to customize recommendation lists.

[1105] "Payment Gateway" refers to the payment processing system used in a transaction, which functions to enable secure and fast payments.

[1106] A "server" refers to a computer system that receives requests from clients (user devices) and performs data processing and communication. In this invention, it is responsible for image analysis, feature extraction, recommendation generation, payment processing, etc.

[1107] "API" is an abbreviation for Application Programming Interface, which means an interface that enables communication between software programs. In the present invention, it is used to access the online trading platform.

[1108] The present invention relates to a system for analyzing an image, providing a user with recommended items, and then purchasing the items through electronic payment. A specific embodiment of the present invention will be described below.

[1109] The overall system configuration is as follows: It includes the devices used by users (smartphones and computers), a server that receives and analyzes data sent from these devices, and an online trading platform (website or application).

[1110] Program processing

[1111] Device:

[1112] Users use their own devices (e.g., smartphones) to take pictures of their favorite fashion items or select and upload existing images. The device then sends this image data to the server using an HTTP request.

[1113] server:

[1114] The server does the following:

[1115] Image analysis: The received image is passed to an AI image analysis module, which analyzes the features of specific objects. This uses image processing libraries such as OpenCV and an AI model to extract features such as color, shape, and brand name.

[1116] Recommendation generation: Based on the extracted features, queries are sent to online trading platforms to search for similar fashion items, using APIs.

[1117] Customization: The recommendation list is customized based on the user's past transaction history and input information, resulting in a recommendation list optimized for the user's individual preferences.

[1118] Recommendation provision: The generated recommendation list is sent to the user's device and displayed on the device.

[1119] Online trading platform:

[1120] The online trading platform provides product data in response to search queries from the server, which then lists the items desired by the user.

[1121] Electronic Payment:

[1122] Users can select the desired items from the recommendation list and make an electronic payment. The server processes the payment securely and quickly via a payment gateway (e.g., Stripe or PayPal).

[1123] Specific examples

[1124] For example, if User A uploads a photo of Celebrity B from his / her smartphone, the photo may contain a "red dress" and "black high heels." The server uses image analysis technology to extract these features and sends a query to an online trading platform. Similar items are retrieved from the platform, and a customized recommendation list is generated taking into account User A's past purchase history (e.g., a preference for red dresses). Finally, the list is provided to User A, who can select the items they like and easily purchase them.

[1125] To comprehensively support this process, the following prompts could be fed to the generative AI model:

[1126] Example prompt sentence:

[1127] A user takes a photo of a photogenic fashion and uploads it to the app. The photo includes a blue dress and white heels. The app extracts the features of the blue dress and searches for and recommends multiple similar items, including blue dresses and white heels. For each item, the user can easily pay electronically through the app.

[1128] The above is a specific embodiment for carrying out the present invention. This system allows users to easily find and safely purchase fashion items that suit their style.

[1129] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1130] Step 1:

[1131] Input: The user takes or selects a photo of a fashion item on their smartphone.

[1132] Specific operation: The user acquires an image using the device's camera / gallery function, selects the image from the system's application screen, and presses the upload button.

[1133] Output: The uploaded image data is sent to the server as an HTTP request.

[1134] Step 2:

[1135] Input: The server receives the image data sent from the terminal.

[1136] Specific operation: The server receives the request, obtains the image data contained therein, checks the integrity of the data using the HTTP protocol, and passes it to the analysis module.

[1137] Output: The image data is passed to the image analysis module.

[1138] Step 3:

[1139] Input: The server passes the image data to the image analysis module.

[1140] How it works: The server uses the OpenCV library to read an image and input it into the generative AI model. The AI ​​model then analyzes the input image and extracts the features of the object. During this process, attribute information such as color, shape, and brand name is identified.

[1141] Output: Feature data of objects extracted from the image (e.g. color: blue, shape: dress, brand name: XYZ).

[1142] Step 4:

[1143] Input: Based on the extracted feature data, the server sends a query to the online trading platform.

[1144] How it works: The server uses an API to access the database of an online trading platform and search for items with similar characteristics. The query statement includes characteristics such as color, shape, and brand name.

[1145] Output: A data list of the retrieved similar items.

[1146] Step 5:

[1147] Input: The server generates a recommendation list based on this acquired data list.

[1148] How it works: The server customizes the recommendation list based on the user's past transaction history and input information. For example, if the user has previously purchased blue dresses, the server uses that information to prioritize items in the list.

[1149] Output: A customized recommendation list.

[1150] Step 6:

[1151] Input: The server sends the generated recommendation list to the terminal.

[1152] Specific operation: The server sends the generated recommendation list to the user's device and displays it on the application's user interface.

[1153] Output: The recommendation list displayed on the user's device.

[1154] Step 7:

[1155] Input: The user selects the desired item from the recommendation list.

[1156] Specific operation: The user operates the terminal interface, selects a specific item from the list, and performs the purchase operation. After the purchase operation, the information of the selected item and payment information are sent to the server.

[1157] Output: Selected item information and payment information are sent to the server.

[1158] Step 8:

[1159] Input: The server processes the received payment information and executes the transaction via the payment gateway.

[1160] What happens: The server passes the payment information to a payment gateway (e.g., Stripe, PayPal) to process the secure transaction. After the transaction is complete, the user is notified of the transaction status.

[1161] Output: A notification of the payment processing result and purchase confirmation is sent to the user.

[1162] This allows users to easily find fashion items that suit their tastes and style and purchase them quickly and safely.

[1163] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1164] To implement the present invention, the following specific system and its program processing will be described. In this system, users upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the items and the user's emotions.

[1165] System configuration

[1166] 1. Device:

[1167] The system allows users to select and upload photos on their smartphones, computers, or other devices, and includes an interface for displaying recommendation lists.

[1168] 2. Server:

[1169] It has the ability to receive images, analyze them, extract features, recognize emotions, generate and customize recommendation lists, and also includes a database that manages user purchase history, input information, and an emotion recognition engine.

[1170] 3. Online shopping platform:

[1171] It is an e-commerce site where users can actually purchase recommended items, and provides product data in response to queries from the server.

[1172] Program processing

[1173] 1. Upload an image

[1174] Users can select photos of their favorite celebrities or influencers using their own devices and upload them to the system by clicking the upload button.

[1175] The device sends the selected photo to the server as an HTTP request.

[1176] 2. Image Analysis

[1177] The server receives the received photo data, passes it to the AI ​​image analysis module, and begins analysis.

[1178] The AI ​​image analysis module uses object detection technology to detect fashion items in photos (e.g., a red dress, black heels, gold earrings).

[1179] 3. Feature Extraction

[1180] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[1181] 4. Emotion recognition

[1182] The server runs an emotion recognition engine based on the uploaded photo and the user's facial expressions, which classifies the user's emotional state (e.g., joy, sadness, surprise, fear).

[1183] 5. Recommendation Generation

[1184] The server accesses the database of the online shopping platform based on the extracted meta information and emotion recognition data to search for similar items.

[1185] The server generates a recommendation list based on the items obtained from the search results, prioritizing appropriate items based on the user's sentiment.

[1186] 6. Customization

[1187] The server retrieves the user's past purchase history and input information from the database and analyzes it.

[1188] The server customizes the recommendation list based on the acquired user information and emotional information, and determines the most appropriate item order, taking into account past purchase history and input information.

[1189] 7. Providing Recommendations

[1190] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[1191] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[1192] Specific examples

[1193] Example 1: When a user uploads a photo of celebrity A

[1194] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[1195] 2. The server passes the photo to the AI, which then detects the "red dress," "black heels," and "gold earrings."

[1196] 3. The server extracts meta information for each item and uses an emotion recognition engine to determine the user's emotion as "joy."

[1197] 4. The server searches for similar items on the online shopping platform and generates a recommendation list based on the user's sentiment.

[1198] 5. The server customizes the list based on past purchase history and provides it to the user.

[1199] 6. The device will display a customized list, allowing the user to select and purchase the items they like.

[1200] This allows the system to easily find fashion items that suit users' preferences and style, and also provides optimal recommendations based on their emotional state.

[1201] The processing flow will be explained below.

[1202] Step 1:

[1203] Users select photos of their favorite celebrities or influencers using their own devices and upload them to the system. The user then clicks the upload button.

[1204] Step 2:

[1205] The device sends the selected photo to the server, typically using an HTTP request.

[1206] Step 3:

[1207] The server receives the received photo data and passes it to the AI ​​image analysis module, which then starts the AI ​​image analysis process.

[1208] Step 4:

[1209] The AI ​​image analysis module analyzes the received photos and detects fashion items in the images, using object detection technology to identify items such as a red dress, black heels, and gold earrings.

[1210] Step 5:

[1211] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[1212] Step 6:

[1213] The server runs an emotion recognition engine based on the uploaded photo and the user's facial expressions, which classifies the user's emotional state (e.g., joy, sadness, surprise, fear).

[1214] Step 7:

[1215] The server accesses the online shopping platform's database based on the extracted meta information and emotion recognition data to search for similar items, and calls the platform's API to retrieve the required information.

[1216] Step 8:

[1217] The server generates a recommendation list based on search results obtained from the online shopping platform, prioritizing appropriate items based on the user's sentiment.

[1218] Step 9:

[1219] The server retrieves and analyzes the user's past purchase history and information entered into the system, such as preferences, size, and budget, from a database.

[1220] Step 10:

[1221] The server customizes the recommendation list based on the acquired user information and sentiment information, and determines the most appropriate item order, taking into account past purchase history and input information.

[1222] Step 11:

[1223] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[1224] Step 12:

[1225] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[1226] This series of steps allows users to easily find fashion items that suit their tastes and style, and also provides appropriate recommendations based on their emotional state at the time.

[1227] Example 2

[1228] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1229] Conventional fashion item recommendation systems simply make recommendations based on a user's past purchase history and simple feature information, making it difficult to take into account the user's emotional state or detailed item features.It also makes it difficult for users to easily find fashion items that suit their preferences and emotions.

[1230] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1231] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of fashion items in the images, means for analyzing the user's facial expression data and classifying emotions, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and the classified emotion data, means for customizing the recommendation list in consideration of the user's past purchase history and input information, and means for providing the generated recommendation list to the user, thereby enabling the user to efficiently find optimal fashion items based on their preferences and emotions.

[1232] A "user" is someone who uses the system to receive fashion item recommendations.

[1233] "Terminal" means the device used by a User to access the System and upload Images.

[1234] A "server" is a computer system that performs central processing such as image analysis, emotion recognition, database management, and generation of recommendation lists.

[1235] "Image upload" is the act of a user using a device to send a selected image to a server.

[1236] "Image analysis" is the process of using AI technology to extract the characteristics of fashion items based on received image data.

[1237] "Feature extraction" refers to obtaining attribute information such as color, shape, and brand of fashion items identified through image analysis.

[1238] "Emotion recognition" is the process of analyzing a user's facial expressions in an image to determine the user's emotional state.

[1239] An "online shopping platform" is an e-commerce system that allows users to purchase recommended fashion items.

[1240] A "recommendation list" is a list of fashion items suggested to a user.

[1241] "Customization" is the process of optimizing the recommendation list based on the user's past purchase history and input information.

[1242] An "HTTP request" is a form of data request sent from a user's device to a server.

[1243] An "HTTP response" is a data response sent from a server to a terminal.

[1244] The present invention provides a system in which a user uploads photos of fashion items using their own terminal, and an online shopping platform provides recommended items based on the features of the items and the user's feelings.

[1245] System configuration

[1246] Hardware and Software

[1247] Device: A device used by a user (e.g., smartphone, tablet, PC, etc.). The device has the functionality to select and upload photos, includes an interface to display the recommendation list, and is responsible for sending HTTP requests to the server.

[1248] Server: Has the ability to analyze received image data, extract features, recognize emotions, generate recommendation lists, and customize them. It also includes a database that manages user purchase history and input information. AI image analysis uses software such as TensorFlow, and emotion recognition uses the Microsoft Azure Emotion API.

[1249] Online shopping platform: An e-commerce system that provides product data, where the server accesses and retrieves product information via API. Users can purchase the items provided.

[1250] Program processing explanation

[1251] The program of the system of the present invention is processed in the following procedure.

[1252] Image upload

[1253] A user uses a device to select a photo of their favorite celebrity or influencer and upload it to the system. Specifically, the user selects a photo from the device's photo gallery and presses the upload button. The device then sends the selected photo to the server as an HTTP POST request. This request includes the photo data and the user's ID.

[1254] Image analysis

[1255] The server receives the photo data from the device. The server then passes this data to an AI image analysis module (e.g., TensorFlow), which uses an object detection algorithm to detect fashion items in the photo (e.g., "red dress," "black heels," "gold earrings").

[1256] Feature extraction

[1257] The server extracts meta information about fashion items based on the detection results returned by the image analysis module, specifically, color information (e.g., red), item type (e.g., dress), and brand (if identifiable) for each item.

[1258] emotion recognition

[1259] The server inputs the photo and the user's facial expression data into an emotion recognition engine (for example, Microsoft Azure Emotion API), which then classifies the user's emotional state from the photo into "joy," "sadness," "surprise," "fear," etc.

[1260] Recommendation generation

[1261] The server then queries the online shopping platform's database to search for similar items based on the extracted item meta information and sentiment data, generating a recommendation list and prioritizing items based on user sentiment.

[1262] Customization

[1263] The server retrieves the user's past purchase history and input information from a database, and based on the retrieved data, customizes the recommendation list and determines the most appropriate item order.

[1264] Providing recommendations

[1265] The server finally generates a customized recommendation list and sends it to the device as an HTTP response. The device displays the received recommendation list on a user interface, allowing the user to browse the list and purchase items.

[1266] Specific examples

[1267] Example 1: When a user uploads a photo of celebrity A

[1268] The user selects a photo of celebrity A from their smartphone and uploads it to the system. Specifically, they select a photo from their photo gallery and tap the "Upload" button.

[1269] The server receives the photo data and passes it to the AI ​​image analysis module, which then detects the "red dress," "black heels," and "gold earrings."

[1270] The server extracts meta information for each item based on the analysis results, and an emotion recognition engine determines the user's emotion as "joy."

[1271] The server searches for similar items from online shopping platforms based on meta information and emotion data, and lists the items obtained, prioritizing them based on the emotion of "joy."

[1272] The server customizes the list taking into account purchase history and sends it to the terminal as an HTTP response.

[1273] The device will display a customized list, allowing users to select and purchase their preferred items.

[1274] Prompt Sentence Examples

[1275] A user uploads a photo of celebrity A wearing a red dress using their smartphone. The AI ​​analyzes the image, detects items, and extracts their features. It then analyzes the user's emotion as "joy" and recommends similar items from online shops based on the emotion. Finally, the customized list is displayed to the user, ready for purchase.

[1276] This system allows users to efficiently find the perfect fashion items based on their preferences and feelings.

[1277] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1278] Program processing flow

[1279] Step 1:

[1280] Image upload

[1281] Users can select a photo of their favorite celebrity or influencer from their device. Specifically, they select a photo from their device's photo gallery and tap the "Upload" button.

[1282] Input: An image file selected by the user.

[1283] The device sends the selected photo to the server as an HTTP POST request, which includes the image data and the user ID.

[1284] Output: Image data and user ID transferred from the device to the server.

[1285] Step 2:

[1286] Image analysis

[1287] The server receives the image data from the device and passes it to the AI ​​image analysis module.

[1288] Input: Image data received from the device and user ID.

[1289] The server starts analyzing the photo using an AI image analysis module (e.g. TensorFlow) and uses object detection algorithms to detect fashion items in the image (e.g. red dress, black heels, gold earrings).

[1290] Output: A list of parsed fashion items.

[1291] Step 3:

[1292] Feature extraction

[1293] The server extracts meta information about the fashion item based on the detection results returned by the image analysis module.

[1294] Input: A list of parsed fashion items.

[1295] The server retrieves information about each item, such as color (red), item type (dress), and brand (if identifiable).

[1296] Output: Extracted fashion item features (color, type, brand).

[1297] Step 4:

[1298] emotion recognition

[1299] The server uses an emotion recognition engine (e.g., Microsoft Azure Emotion API) to identify the user's emotional state and classify emotions based on the photo and the user's facial expression data.

[1300] Input: Uploaded image data, user face detection information.

[1301] The server uses an emotion recognition engine to extract emotional states such as "happiness," "sadness," "surprise," and "fear."

[1302] Output: User sentiment classification data.

[1303] Step 5:

[1304] Recommendation generation

[1305] The server then queries the online shopping platform's database based on the extracted item features and sentiment data.

[1306] Input: Item feature data, emotion data.

[1307] The server searches for similar items and generates a recommendation list that prioritizes appropriate items based on the user's sentiment.

[1308] Output: The generated recommendation list.

[1309] Step 6:

[1310] Customization

[1311] The server retrieves the user's past purchase history and input information from a database.

[1312] Input: Past purchase history, input information.

[1313] The server then customizes the recommendation list based on the information it has obtained, determining the optimal order of items based on past purchase history.

[1314] Output: A customized recommendation list.

[1315] Step 7:

[1316] Providing recommendations

[1317] The server then sends the final customized recommendation list to the device as an HTTP response.

[1318] Input: A customized recommendation list.

[1319] The device displays the received recommendation list on the user interface.

[1320] Output: A list of recommendations displayed in a user interface. The user can browse the list, select items, and purchase them.

[1321] (Application example 2)

[1322] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1323] Conventional online shopping systems do not take into account the user's emotional state when making recommendations, making it difficult for users to find the perfect fashion item that best suits their current mood. Furthermore, technology for accurately extracting the characteristics of fashion items from images uploaded by users and generating recommendation lists based on those characteristics is also inadequate. This leads to issues such as lower user satisfaction and a decrease in purchasing motivation.

[1324] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1325] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of fashion items in the images, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and the user's emotional state and generating a recommendation list, means for customizing the recommendation list taking into account the user's past purchase history and input information, and means for providing the generated recommendation list to the user. This enables optimal fashion item recommendations that take into account the user's emotional state. Furthermore, users can easily find products that match their emotional state, which is expected to increase their purchasing motivation.

[1326] "User" refers to a person who uses the system, primarily someone who uploads images of fashion items and receives recommendations.

[1327] "Image receiving means" refers to a device or software that has the function of receiving image data uploaded by a user and sending it to a server.

[1328] "Image analysis means" refers to a device or software that has the function of analyzing received image data and identifying and extracting the characteristics of fashion items in the image.

[1329] The "feature extraction means" refers to a device or software that has the function of extracting the features of a fashion item, such as color, shape, or type, from the analyzed image.

[1330] "Emotion recognition means" refers to a device or software that has the function of recognizing and determining the emotional state of a user.

[1331] "Online shopping platform" means an e-commerce site that allows users to purchase fashion items via the Internet.

[1332] "Recommendation list generation means" refers to a device or software that has the function of searching for similar items from an online shopping platform based on the extracted features and the user's emotional state and generating a list to present to the user.

[1333] The "customization means" refers to a device or software that has the function of individually adjusting the generated recommendation list, taking into account the user's past purchase history and input information.

[1334] "Recommendation list providing means" refers to a device or software that has the function of displaying and providing a final customized recommendation list to a user.

[1335] To implement the present invention, it is necessary to build a system that allows users to upload images of fashion items using their own devices and provides recommended items based on the features of the items and the user's emotions. Specifically, this is implemented in the following way.

[1336] Details of the hardware and software used

[1337] Hardware: Smartphones, servers, database servers

[1338] Software: Flask (web framework), OpenCV (image processing library), Keras (deep learning framework), Pandas (data analysis library)

[1339] System configuration

[1340] 1. Device:

[1341] Users use their smartphones to take or select their favorite fashion items and upload the images to the system, which then sends the images to the server as HTTP requests.

[1342] 2. Server:

[1343] It analyzes the received images. First, it preprocesses the images using OpenCV, then extracts the features of the fashion items using an AI image analysis module built with Keras. It also uses another Keras model to recognize the user's emotions.

[1344] 3. Database Server:

[1345] Based on the extracted features and sentiment data, similar items are searched for using Pandas. Product data is obtained through APIs in collaboration with the database of an online shopping platform.

[1346] Data processing and calculation

[1347] Image upload:

[1348] When a user uploads an image from a device, the device sends the image data to the server, where it is passed as request data via the HTTP protocol.

[1349] Image analysis:

[1350] The server processes the received image data using OpenCV and converts it into an appropriate format, then inputs it into an image analysis model built with Keras to extract the main features of the fashion item (color, shape, type).

[1351] Emotion recognition:

[1352] The server analyzes the user's facial expressions and recognizes their emotional state (e.g., happy, surprised, sad), again using Keras' emotion recognition model.

[1353] Recommendation generation and customization:

[1354] The database server searches for similar items from online shopping platforms based on the extracted features and the recognized emotions, then uses Pandas to generate a recommendation list, which is further customized by taking into account the user's past purchase history and input information.

[1355] Providing a recommendation list:

[1356] The server sends the generated customized recommendation list to the device, which then displays the list on the user interface, allowing the user to select and purchase the items they like.

[1357] Specific examples

[1358] Example 1: A user uploads a photo wearing a black coat

[1359] Users select a photo from their smartphone of themselves wearing a black coat and upload it to the system.

[1360] The server passes the photo to the AI, which then detects the "black coat."

[1361] The server extracts the characteristics of the item and uses an emotion recognition engine to determine the user's emotion as "joy."

[1362] The server searches for similar items on the online shopping platform and generates a recommendation list based on the user's sentiment.

[1363] The server takes into account past purchase history to customize the list and provides it to the user.

[1364] The device will display a customized list, allowing users to select and purchase their preferred items.

[1365] In this way, the system allows users to easily find fashion items that suit their preferences and style, and provides optimal recommendations based on their emotional state.

[1366] Example prompts to input to a generative AI model:

[1367] plaintext

[1368] Generate recommended items when a user uploads an image of themselves wearing a black coat. The user's emotion is recognized as "joy." The item characteristics obtained from the image are "black, coat." Recommend similar items.

[1369] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1370] Step 1:

[1371] A user selects and uploads an image of a fashion item using a device. At this time, the image file is sent to the server as an HTTP request. The input is the image file on the device, and the output is an HTTP request containing this image file.

[1372] Step 2:

[1373] The server analyzes the received HTTP request and obtains image data. The obtained image data is preprocessed using OpenCV and converted into a format that can be input to the AI ​​image analysis model. The input is the image data in the HTTP request, and the output is the preprocessed image data.

[1374] Step 3:

[1375] The server inputs the preprocessed image data into a Keras AI image analysis model to extract the features of the fashion items. The AI ​​model analyzes the image and obtains features such as color, shape, and type. The input is the preprocessed image data, and the output is the obtained feature data.

[1376] Step 4:

[1377] The server uses another Keras model to recognize the user's emotion based on the extracted feature data. The user's emotional state (e.g., joy, surprise, sadness, etc.) is obtained along with the feature data. The input is the feature data, and the output is the emotion data.

[1378] Step 5:

[1379] The server sends a search query for similar items to the database server based on the feature data and emotion data. Pandas is used to search for similar items from the database of the online shopping platform and obtain a list. The input is the feature data and emotion data, and the output is a search result list of similar items.

[1380] Step 6:

[1381] The server customizes the search result list by taking into account the user's past purchase history and input information. It uses Pandas to analyze the list and generate an individually tailored recommendation list. The input is the search result list and the user's purchase history and input information, and the output is a customized recommendation list.

[1382] Step 7:

[1383] The server sends the generated customized recommendation list to the terminal as an HTTP response. The terminal displays the received recommendation list on the user interface. The input is the customized recommendation list, and the output is the list displayed on the user interface.

[1384] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1385] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1386] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1387] [Fourth embodiment]

[1388] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1389] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1390] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1391] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1392] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1393] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1394] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1395] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1396] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1397] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1398] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1399] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1400] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1401] To implement the present invention, the following describes a specific system and its program processing. This system allows users to upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the items.

[1402] System configuration

[1403] 1. Device:

[1404] The system allows users to select and upload photos on their smartphones, computers, or other devices, and includes an interface for displaying recommendation lists.

[1405] 2. Server:

[1406] It has the ability to receive images, analyze them, extract features, generate and customize recommendation lists, and also includes a database that manages user purchase history and input information.

[1407] 3. Online shopping platform:

[1408] It is an e-commerce site where users can actually purchase recommended items, and provides product data in response to queries from the server.

[1409] Program processing

[1410] 1. Upload an image

[1411] Users use their own devices to select photos of their favorite celebrities or influencers and upload them to the system.

[1412] The device sends the selected photo to the server as an HTTP request.

[1413] 2. Image Analysis

[1414] The server passes the received photo data to the AI ​​image analysis module and begins analysis.

[1415] The AI ​​image analysis module uses object detection technology to detect fashion items in photos (e.g., a red dress, black heels, gold earrings).

[1416] 3. Feature Extraction

[1417] The server extracts the item's meta information based on the analysis results, including color, shape, brand name, etc.

[1418] For example, "red dress" includes information such as color: red, item: dress, brand name: (if known).

[1419] 4. Recommendation Generation

[1420] The server uses the extracted meta information to access the online shopping platform's database and search for similar items.

[1421] The server generates a recommendation list based on the items obtained from the search results.

[1422] 5. Customization

[1423] The server compares the user's past purchase history with the information they provide, including their preferred colors, styles, budget, and other information.

[1424] The server uses this information to customize the recommendation list and arrange the items in the optimal order.

[1425] 6. Providing Recommendations

[1426] The server sends the customized recommendation list to the user's device.

[1427] The device displays the received recommendation list on the user interface so that the user can easily check it.

[1428] Specific examples

[1429] Example 1: When a user uploads a photo of celebrity A

[1430] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[1431] 2. The server passes the photo to the AI, which then detects the "red dress," "black heels," and "gold earrings."

[1432] 3. The server extracts meta information for each item and searches for similar items on online shopping platforms.

[1433] 4. The server generates a recommendation list from the search results, customizing it based on the user's past preference for red dresses.

[1434] 5. The customized list is provided to the user, who can then view the list on their device, select and purchase the items they like.

[1435] This allows the system to easily find fashion items that match the user's preferred style.

[1436] The processing flow will be explained below.

[1437] Step 1:

[1438] Users can use their own devices to select photos of their favorite celebrities or influencers and upload them to the system by clicking the upload button.

[1439] Step 2:

[1440] The device sends the selected photo to the server, typically using an HTTP request.

[1441] Step 3:

[1442] The server receives the received photo data and passes it to the AI ​​image analysis module, which then starts the AI ​​image analysis process.

[1443] Step 4:

[1444] The AI ​​image analysis module analyzes the received photos and detects fashion items in the images, using object detection technology to identify items such as a red dress, black heels, and gold earrings.

[1445] Step 5:

[1446] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[1447] Step 6:

[1448] The server uses the extracted meta information to access the online shopping platform's database to search for similar items, and calls the platform's API to retrieve the required information.

[1449] Step 7:

[1450] The server generates a recommendation list based on search results obtained from the online shopping platform, and the recommendation list includes items to be suggested to the user.

[1451] Step 8:

[1452] The server retrieves and analyzes the user's past purchase history and information entered into the system, such as preferences, size, and budget, from a database.

[1453] Step 9:

[1454] The server customizes the recommendation list based on the acquired user information, taking into account past purchase history and input information to determine the most appropriate item order.

[1455] Step 10:

[1456] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[1457] Step 11:

[1458] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[1459] This series of steps allows users to easily find fashion items that suit their tastes and style.

[1460] Example 1

[1461] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1462] Conventional fashion item recommendation systems often fail to fully reflect user preferences, and the items they suggest often do not meet user expectations. Another issue is the low accuracy of image analysis, which makes it difficult to accurately extract item features.

[1463] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1464] In this invention, the server includes means for receiving images selected and uploaded by a user, means for passing the received images to an AI image analysis module and starting analysis, means for extracting features of fashion items in the images using the AI ​​image analysis module, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items, means for matching the user's past purchase history and input information to customize a recommendation list, and means for providing the generated recommendation list to the user, thereby enabling highly accurate recommendations that reflect the user's preferences.

[1465] "User" refers to an individual who uses this system to receive fashion item recommendations.

[1466] "Terminal" refers to a device used by a user, such as a smartphone or computer, that selects and uploads images.

[1467] "Server" refers to the central computer system that receives, analyzes, extracts features from, and generates and customizes recommendation lists for images.

[1468] The "AI Image Analysis Module" is a software component that uses artificial intelligence technology to analyze received images and detect fashion items within the images.

[1469] "Fashion items" refers to items such as clothing and accessories for which users request recommendations via photos.

[1470] "Features" refers to information necessary to identify a fashion item, such as its color, shape, brand name, etc.

[1471] A "recommendation list" refers to a list of fashion items suggested to users.

[1472] "Online shopping platform" refers to an e-commerce site where users can physically purchase fashion items.

[1473] "Purchase history" refers to information about fashion items a user has purchased in the past.

[1474] "Input information" refers to information such as preferences and budget that a user provides to the system.

[1475] An "API (Application Program Interface)" refers to the conventions and tools that allow different software systems to communicate with each other and exchange data.

[1476] System Overview

[1477] This invention is a system in which users upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the uploaded photos. The main components of the system are the user device, a server, and the online shopping platform.

[1478] Hardware and software used

[1479] 1. Device:

[1480] Hardware: Devices used by users, such as smartphones and computers.

[1481] Software: Web browser and dedicated applications.

[1482] Features: Image selection, upload, and display recommendation list.

[1483] 2. Server:

[1484] Hardware: High-performance cloud servers.

[1485] Software: AI image analysis module using libraries such as Python, TensorFlow, and OpenCV.

[1486] Functions: Image reception, analysis, feature extraction, recommendation list generation, and customization.

[1487] 3. Online shopping platform:

[1488] Hardware: The servers that run your e-commerce site.

[1489] Software: Product database and API.

[1490] Features: Find and offer similar items.

[1491] Detailed program description

[1492] 1. Upload an image

[1493] Users can select photos of their favorite celebrities or influencers using their smartphones or computers and upload them to the system. The images selected on the device are sent to the server as HTTP requests.

[1494] 2. Image Analysis

[1495] The server temporarily stores the received image data and passes it to the AI ​​image analysis module, which uses libraries such as TensorFlow and OpenCV to apply object detection technology to detect fashion items in the image.

[1496] 3. Feature Extraction

[1497] The server extracts meta information for each item from the analysis results. This meta information includes the item's color, shape, brand name, etc. For example, for a "red dress," the server extracts the following information: "Color: Red, Item: Dress, Brand: Unknown."

[1498] 4. Recommendation Generation

[1499] The server then uses the extracted meta information to access the online shopping platform's database and search for similar fashion items, using SQL queries and APIs to retrieve related items.

[1500] 5. Customization

[1501] The server compares the user's past purchase history and input information to customize the recommendation list, taking into account the user's preferences, budget, and other information to create a list of the most suitable items.

[1502] 6. Providing Recommendations

[1503] The server sends the customized recommendation list in JSON format to the user's device, which interprets the received data and displays it in a formatted user interface.

[1504] Specific examples

[1505] Example: When a user uploads a photo of celebrity A

[1506] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[1507] 2. The server passes the photo to an AI image analysis module, which then detects the "red dress," "black heels," and "gold earrings."

[1508] 3. The server extracts meta information for each item and searches for similar items on online shopping platforms.

[1509] 4. The server generates a recommendation list from the search results, customizing it based on the user's past preference for red dresses.

[1510] 5. A customized list is provided to the user, who can then view the list on their device, select and purchase the items they like.

[1511] Prompt Sentence Examples

[1512] For generative AI models, use the following prompt:

[1513] "Write a program that analyzes fashion items in images uploaded by users, extracts their features, and generates a recommendation list based on the user's preferences."

[1514] This prompt statement allows the model to generate the appropriate code.

[1515] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1516] Step 1: Select and upload an image

[1517] Users open the gallery app or file browser on their smartphone or PC and select photos of celebrities or influencers they like. The images selected by the user are sent to the system by clicking the upload button. The device then sends the selected images to the server as an HTTP request.

[1518] Input: An image file selected by the user on their device.

[1519] Output: Image data included in the HTTP request sent to the server.

[1520] Step 2: Receiving the image

[1521] The server receives the image data sent from the device, stores it in temporary storage, and then passes it to the AI ​​image analysis module for image analysis.

[1522] Input: Image data sent from the device.

[1523] Output: Image data saved in temporary storage.

[1524] Step 3: Begin image analysis

[1525] The server passes the image data stored in temporary storage to the AI ​​image analysis module, which then uses libraries such as TensorFlow and OpenCV to perform image analysis.

[1526] Input: Image data stored in temporary storage.

[1527] Output: A list of identified fashion items in the image.

[1528] Step 4: Extracting fashion item features

[1529] The server extracts meta information (color, shape, brand name, etc.) for each item based on the analysis results received from the AI ​​image analysis module. For example, if a red dress is identified, the meta information is extracted as "Color: Red, Item: Dress, Brand Name: Unknown."

[1530] Input: Analysis results of the AI ​​image analysis module.

[1531] Output: Extracted fashion item meta information.

[1532] Step 5: Search for recommended items

[1533] The server then accesses the online shopping platform's database based on the extracted fashion item meta information and searches for similar items using SQL queries or APIs, such as "SELECT FROM Items WHERE color="red" AND type="dress".

[1534] Input: Extracted fashion item meta information.

[1535] Output: A list of similar items retrieved from an online shopping platform.

[1536] Step 6: Generate a recommendation list

[1537] The server generates a recommendation list based on a list of similar items retrieved from an online shopping platform, and the list is customized based on the user's preferences and purchasing history.

[1538] Input: A list of similar items obtained from an online shopping platform.

[1539] Output: The generated recommendation list.

[1540] Step 7: Customize your recommendation list

[1541] The server customizes the generated recommendation list based on the user's past purchase history and input information. For example, if the user has previously purchased a favorite "red dress," the server will adjust the ranking of the list to reflect that information.

[1542] Input: User's past purchase history and input information, generated recommendation list.

[1543] Output: A customized recommendation list.

[1544] Step 8: Providing a recommendation list

[1545] The server sends the customized recommendation list in JSON format to the user's device, which interprets the received data, formats it, and displays it in a user interface. The user can then select and purchase the items they like from the list.

[1546] Input: A customized recommendation list.

[1547] Output: The recommendation list displayed on the user's device.

[1548] (Application example 1)

[1549] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1550] On conventional online shopping platforms, users had to spend a lot of time and effort finding fashion items that matched their tastes and style. Furthermore, they had to enter their payment information each time they made a purchase, which posed security risks during the payment process. To solve these problems, a system was needed that would allow users to easily upload images from their devices, analyze their features to recommend similar items, and even handle electronic payment.

[1551] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1552] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of objects in the images, means for searching for similar items from an online trading platform based on the extracted features of the objects and generating a recommendation list, means for customizing the recommendation list taking into account the user's past transaction history and input information, means for providing the generated recommendation list to the user, and means for purchasing items selected from the recommendation list via an electronic transaction payment gateway. This allows users to easily find fashion items that suit their tastes and style and to purchase them safely and quickly.

[1553] The term "user" refers to a person who uses a particular service or product, and in the present invention particularly refers to a person who uploads photos of fashion items and purchases recommended items.

[1554] "Image" refers to visual data selected from a terminal and uploaded to the system, which is the subject of analysis of the characteristics of fashion items.

[1555] "Particular object" refers to an object present in the received image, and in the present invention particularly refers to a fashion item (e.g., a dress, shoes, accessories).

[1556] "Feature" refers to an identifiable attribute or property of an object, and in the present invention includes color, shape, brand name, and the like.

[1557] A "recommendation list" refers to a list of recommended items generated based on extracted features, which users can browse and select according to their preferences.

[1558] "Online trading platform" refers to a website or application that offers products through e-commerce and allows users to search, browse, and purchase products.

[1559] "Transaction history" refers to a record of a user's past purchases and transactions, and is information used to customize recommendation lists.

[1560] "Payment Gateway" refers to the payment processing system used in a transaction, which functions to enable secure and fast payments.

[1561] A "server" refers to a computer system that receives requests from clients (user devices) and performs data processing and communication. In this invention, it is responsible for image analysis, feature extraction, recommendation generation, payment processing, etc.

[1562] "API" is an abbreviation for Application Programming Interface, which means an interface that enables communication between software programs. In the present invention, it is used to access the online trading platform.

[1563] The present invention relates to a system for analyzing an image, providing a user with recommended items, and then purchasing the items through electronic payment. A specific embodiment of the present invention will be described below.

[1564] The overall system configuration is as follows: It includes the devices used by users (smartphones and computers), a server that receives and analyzes data sent from these devices, and an online trading platform (website or application).

[1565] Program processing

[1566] Device:

[1567] Users use their own devices (e.g., smartphones) to take pictures of their favorite fashion items or select and upload existing images. The device then sends this image data to the server using an HTTP request.

[1568] server:

[1569] The server does the following:

[1570] Image analysis: The received image is passed to an AI image analysis module, which analyzes the features of specific objects. This uses image processing libraries such as OpenCV and an AI model to extract features such as color, shape, and brand name.

[1571] Recommendation generation: Based on the extracted features, queries are sent to online trading platforms to search for similar fashion items, using APIs.

[1572] Customization: The recommendation list is customized based on the user's past transaction history and input information, resulting in a recommendation list optimized for the user's individual preferences.

[1573] Recommendation provision: The generated recommendation list is sent to the user's device and displayed on the device.

[1574] Online trading platform:

[1575] The online trading platform provides product data in response to search queries from the server, which then lists the items desired by the user.

[1576] Electronic Payment:

[1577] Users can select the desired items from the recommendation list and make an electronic payment. The server processes the payment securely and quickly via a payment gateway (e.g., Stripe or PayPal).

[1578] Specific examples

[1579] For example, if User A uploads a photo of Celebrity B from his / her smartphone, the photo may contain a "red dress" and "black high heels." The server uses image analysis technology to extract these features and sends a query to an online trading platform. Similar items are retrieved from the platform, and a customized recommendation list is generated taking into account User A's past purchase history (e.g., a preference for red dresses). Finally, the list is provided to User A, who can select the items they like and easily purchase them.

[1580] To comprehensively support this process, the following prompts could be fed to the generative AI model:

[1581] Example prompt sentence:

[1582] A user takes a photo of a photogenic fashion and uploads it to the app. The photo includes a blue dress and white heels. The app extracts the features of the blue dress and searches for and recommends multiple similar items, including blue dresses and white heels. For each item, the user can easily pay electronically through the app.

[1583] The above is a specific embodiment for carrying out the present invention. This system allows users to easily find and safely purchase fashion items that suit their style.

[1584] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1585] Step 1:

[1586] Input: The user takes or selects a photo of a fashion item on their smartphone.

[1587] Specific operation: The user acquires an image using the device's camera / gallery function, selects the image from the system's application screen, and presses the upload button.

[1588] Output: The uploaded image data is sent to the server as an HTTP request.

[1589] Step 2:

[1590] Input: The server receives the image data sent from the terminal.

[1591] Specific operation: The server receives the request, obtains the image data contained therein, checks the integrity of the data using the HTTP protocol, and passes it to the analysis module.

[1592] Output: The image data is passed to the image analysis module.

[1593] Step 3:

[1594] Input: The server passes the image data to the image analysis module.

[1595] How it works: The server uses the OpenCV library to read an image and input it into the generative AI model. The AI ​​model then analyzes the input image and extracts the features of the object. During this process, attribute information such as color, shape, and brand name is identified.

[1596] Output: Feature data of objects extracted from the image (e.g. color: blue, shape: dress, brand name: XYZ).

[1597] Step 4:

[1598] Input: Based on the extracted feature data, the server sends a query to the online trading platform.

[1599] How it works: The server uses an API to access the database of an online trading platform and search for items with similar characteristics. The query statement includes characteristics such as color, shape, and brand name.

[1600] Output: A data list of the retrieved similar items.

[1601] Step 5:

[1602] Input: The server generates a recommendation list based on this acquired data list.

[1603] How it works: The server customizes the recommendation list based on the user's past transaction history and input information. For example, if the user has previously purchased blue dresses, the server uses that information to prioritize items in the list.

[1604] Output: A customized recommendation list.

[1605] Step 6:

[1606] Input: The server sends the generated recommendation list to the terminal.

[1607] Specific operation: The server sends the generated recommendation list to the user's device and displays it on the application's user interface.

[1608] Output: The recommendation list displayed on the user's device.

[1609] Step 7:

[1610] Input: The user selects the desired item from the recommendation list.

[1611] Specific operation: The user operates the terminal interface, selects a specific item from the list, and performs the purchase operation. After the purchase operation, the information of the selected item and payment information are sent to the server.

[1612] Output: Selected item information and payment information are sent to the server.

[1613] Step 8:

[1614] Input: The server processes the received payment information and executes the transaction via the payment gateway.

[1615] What happens: The server passes the payment information to a payment gateway (e.g., Stripe, PayPal) to process the secure transaction. After the transaction is complete, the user is notified of the transaction status.

[1616] Output: A notification of the payment processing result and purchase confirmation is sent to the user.

[1617] This allows users to easily find fashion items that suit their tastes and style and purchase them quickly and safely.

[1618] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1619] To implement the present invention, the following specific system and its program processing will be described. In this system, users upload photos of fashion items using their own devices, and an online shopping platform provides recommended items based on the features of the items and the user's emotions.

[1620] System configuration

[1621] 1. Device:

[1622] The system allows users to select and upload photos on their smartphones, computers, or other devices, and includes an interface for displaying recommendation lists.

[1623] 2. Server:

[1624] It has the ability to receive images, analyze them, extract features, recognize emotions, generate and customize recommendation lists, and also includes a database that manages user purchase history, input information, and an emotion recognition engine.

[1625] 3. Online shopping platform:

[1626] It is an e-commerce site where users can actually purchase recommended items, and provides product data in response to queries from the server.

[1627] Program processing

[1628] 1. Upload an image

[1629] Users can select photos of their favorite celebrities or influencers using their own devices and upload them to the system by clicking the upload button.

[1630] The device sends the selected photo to the server as an HTTP request.

[1631] 2. Image Analysis

[1632] The server receives the received photo data, passes it to the AI ​​image analysis module, and begins analysis.

[1633] The AI ​​image analysis module uses object detection technology to detect fashion items in photos (e.g., a red dress, black heels, gold earrings).

[1634] 3. Feature Extraction

[1635] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[1636] 4. Emotion recognition

[1637] The server runs an emotion recognition engine based on the uploaded photo and the user's facial expressions, which classifies the user's emotional state (e.g., joy, sadness, surprise, fear).

[1638] 5. Recommendation Generation

[1639] The server accesses the database of the online shopping platform based on the extracted meta information and emotion recognition data to search for similar items.

[1640] The server generates a recommendation list based on the items obtained from the search results, prioritizing appropriate items based on the user's sentiment.

[1641] 6. Customization

[1642] The server retrieves the user's past purchase history and input information from the database and analyzes it.

[1643] The server customizes the recommendation list based on the acquired user information and emotional information, and determines the most appropriate item order, taking into account past purchase history and input information.

[1644] 7. Providing Recommendations

[1645] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[1646] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[1647] Specific examples

[1648] Example 1: When a user uploads a photo of celebrity A

[1649] 1. The user selects a photo of celebrity A from their smartphone and uploads it to the system.

[1650] 2. The server passes the photo to the AI, which then detects the "red dress," "black heels," and "gold earrings."

[1651] 3. The server extracts meta information for each item and uses an emotion recognition engine to determine the user's emotion as "joy."

[1652] 4. The server searches for similar items on the online shopping platform and generates a recommendation list based on the user's sentiment.

[1653] 5. The server customizes the list based on past purchase history and provides it to the user.

[1654] 6. The device will display a customized list, allowing the user to select and purchase the items they like.

[1655] This allows the system to easily find fashion items that suit users' preferences and style, and also provides optimal recommendations based on their emotional state.

[1656] The processing flow will be explained below.

[1657] Step 1:

[1658] Users select photos of their favorite celebrities or influencers using their own devices and upload them to the system. The user then clicks the upload button.

[1659] Step 2:

[1660] The device sends the selected photo to the server, typically using an HTTP request.

[1661] Step 3:

[1662] The server receives the received photo data and passes it to the AI ​​image analysis module, which then starts the AI ​​image analysis process.

[1663] Step 4:

[1664] The AI ​​image analysis module analyzes the received photos and detects fashion items in the images, using object detection technology to identify items such as a red dress, black heels, and gold earrings.

[1665] Step 5:

[1666] The server extracts meta information about the fashion item based on the analysis results, such as color information (red), item type (dress), and brand (if known).

[1667] Step 6:

[1668] The server runs an emotion recognition engine based on the uploaded photo and the user's facial expressions, which classifies the user's emotional state (e.g., joy, sadness, surprise, fear).

[1669] Step 7:

[1670] The server accesses the online shopping platform's database based on the extracted meta information and emotion recognition data to search for similar items, and calls the platform's API to retrieve the required information.

[1671] Step 8:

[1672] The server generates a recommendation list based on search results obtained from the online shopping platform, prioritizing appropriate items based on the user's sentiment.

[1673] Step 9:

[1674] The server retrieves and analyzes the user's past purchase history and information entered into the system, such as preferences, size, and budget, from a database.

[1675] Step 10:

[1676] The server customizes the recommendation list based on the acquired user information and sentiment information, and determines the most appropriate item order, taking into account past purchase history and input information.

[1677] Step 11:

[1678] The server generates a customized recommendation list and sends it to the terminal as an HTTP response.

[1679] Step 12:

[1680] The device displays the received recommendation list on the user interface, allowing the user to browse through it, select the items they like, and purchase them.

[1681] This series of steps allows users to easily find fashion items that suit their tastes and style, and also provides appropriate recommendations based on their emotional state at the time.

[1682] Example 2

[1683] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1684] Conventional fashion item recommendation systems simply make recommendations based on a user's past purchase history and simple feature information, making it difficult to take into account the user's emotional state or detailed item features.It also makes it difficult for users to easily find fashion items that suit their preferences and emotions.

[1685] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1686] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of fashion items in the images, means for analyzing the user's facial expression data and classifying emotions, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and the classified emotion data, means for customizing the recommendation list in consideration of the user's past purchase history and input information, and means for providing the generated recommendation list to the user, thereby enabling the user to efficiently find optimal fashion items based on their preferences and emotions.

[1687] A "user" is someone who uses the system to receive fashion item recommendations.

[1688] "Terminal" means the device used by a User to access the System and upload Images.

[1689] A "server" is a computer system that performs central processing such as image analysis, emotion recognition, database management, and generation of recommendation lists.

[1690] "Image upload" is the act of a user using a device to send a selected image to a server.

[1691] "Image analysis" is the process of using AI technology to extract the characteristics of fashion items based on received image data.

[1692] "Feature extraction" refers to obtaining attribute information such as color, shape, and brand of fashion items identified through image analysis.

[1693] "Emotion recognition" is the process of analyzing a user's facial expressions in an image to determine the user's emotional state.

[1694] An "online shopping platform" is an e-commerce system that allows users to purchase recommended fashion items.

[1695] A "recommendation list" is a list of fashion items suggested to a user.

[1696] "Customization" is the process of optimizing the recommendation list based on the user's past purchase history and input information.

[1697] An "HTTP request" is a form of data request sent from a user's device to a server.

[1698] An "HTTP response" is a data response sent from a server to a terminal.

[1699] The present invention provides a system in which a user uploads photos of fashion items using their own terminal, and an online shopping platform provides recommended items based on the features of the items and the user's feelings.

[1700] System configuration

[1701] Hardware and Software

[1702] Device: A device used by a user (e.g., smartphone, tablet, PC, etc.). The device has the functionality to select and upload photos, includes an interface to display the recommendation list, and is responsible for sending HTTP requests to the server.

[1703] Server: Has the ability to analyze received image data, extract features, recognize emotions, generate recommendation lists, and customize them. It also includes a database that manages user purchase history and input information. AI image analysis uses software such as TensorFlow, and emotion recognition uses the Microsoft Azure Emotion API.

[1704] Online shopping platform: An e-commerce system that provides product data, where the server accesses and retrieves product information via API. Users can purchase the items provided.

[1705] Program processing explanation

[1706] The program of the system of the present invention is processed in the following procedure.

[1707] Image upload

[1708] A user uses a device to select a photo of their favorite celebrity or influencer and upload it to the system. Specifically, the user selects a photo from the device's photo gallery and presses the upload button. The device then sends the selected photo to the server as an HTTP POST request. This request includes the photo data and the user's ID.

[1709] Image analysis

[1710] The server receives the photo data from the device. The server then passes this data to an AI image analysis module (e.g., TensorFlow), which uses an object detection algorithm to detect fashion items in the photo (e.g., "red dress," "black heels," "gold earrings").

[1711] Feature extraction

[1712] The server extracts meta information about fashion items based on the detection results returned by the image analysis module, specifically, color information (e.g., red), item type (e.g., dress), and brand (if identifiable) for each item.

[1713] emotion recognition

[1714] The server inputs the photo and the user's facial expression data into an emotion recognition engine (for example, Microsoft Azure Emotion API), which then classifies the user's emotional state from the photo into "joy," "sadness," "surprise," "fear," etc.

[1715] Recommendation generation

[1716] The server then queries the online shopping platform's database to search for similar items based on the extracted item meta information and sentiment data, generating a recommendation list and prioritizing items based on user sentiment.

[1717] Customization

[1718] The server retrieves the user's past purchase history and input information from a database, and based on the retrieved data, customizes the recommendation list and determines the most appropriate item order.

[1719] Providing recommendations

[1720] The server finally generates a customized recommendation list and sends it to the device as an HTTP response. The device displays the received recommendation list on a user interface, allowing the user to browse the list and purchase items.

[1721] Specific examples

[1722] Example 1: When a user uploads a photo of celebrity A

[1723] The user selects a photo of celebrity A from their smartphone and uploads it to the system. Specifically, they select a photo from their photo gallery and tap the "Upload" button.

[1724] The server receives the photo data and passes it to the AI ​​image analysis module, which then detects the "red dress," "black heels," and "gold earrings."

[1725] The server extracts meta information for each item based on the analysis results, and an emotion recognition engine determines the user's emotion as "joy."

[1726] The server searches for similar items from online shopping platforms based on meta information and emotion data, and lists the items obtained, prioritizing them based on the emotion of "joy."

[1727] The server customizes the list taking into account purchase history and sends it to the terminal as an HTTP response.

[1728] The device will display a customized list, allowing users to select and purchase their preferred items.

[1729] Prompt Sentence Examples

[1730] A user uploads a photo of celebrity A wearing a red dress using their smartphone. The AI ​​analyzes the image, detects items, and extracts their features. It then analyzes the user's emotion as "joy" and recommends similar items from online shops based on the emotion. Finally, the customized list is displayed to the user, ready for purchase.

[1731] This system allows users to efficiently find the perfect fashion items based on their preferences and feelings.

[1732] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1733] Program processing flow

[1734] Step 1:

[1735] Image upload

[1736] Users can select a photo of their favorite celebrity or influencer from their device. Specifically, they select a photo from their device's photo gallery and tap the "Upload" button.

[1737] Input: An image file selected by the user.

[1738] The device sends the selected photo to the server as an HTTP POST request, which includes the image data and the user ID.

[1739] Output: Image data and user ID transferred from the device to the server.

[1740] Step 2:

[1741] Image analysis

[1742] The server receives the image data from the device and passes it to the AI ​​image analysis module.

[1743] Input: Image data received from the device and user ID.

[1744] The server starts analyzing the photo using an AI image analysis module (e.g. TensorFlow) and uses object detection algorithms to detect fashion items in the image (e.g. red dress, black heels, gold earrings).

[1745] Output: A list of parsed fashion items.

[1746] Step 3:

[1747] Feature extraction

[1748] The server extracts meta information about the fashion item based on the detection results returned by the image analysis module.

[1749] Input: A list of parsed fashion items.

[1750] The server retrieves information about each item, such as color (red), item type (dress), and brand (if identifiable).

[1751] Output: Extracted fashion item features (color, type, brand).

[1752] Step 4:

[1753] emotion recognition

[1754] The server uses an emotion recognition engine (e.g., Microsoft Azure Emotion API) to identify the user's emotional state and classify emotions based on the photo and the user's facial expression data.

[1755] Input: Uploaded image data, user face detection information.

[1756] The server uses an emotion recognition engine to extract emotional states such as "happiness," "sadness," "surprise," and "fear."

[1757] Output: User sentiment classification data.

[1758] Step 5:

[1759] Recommendation generation

[1760] The server then queries the online shopping platform's database based on the extracted item features and sentiment data.

[1761] Input: Item feature data, emotion data.

[1762] The server searches for similar items and generates a recommendation list that prioritizes appropriate items based on the user's sentiment.

[1763] Output: The generated recommendation list.

[1764] Step 6:

[1765] Customization

[1766] The server retrieves the user's past purchase history and input information from a database.

[1767] Input: Past purchase history, input information.

[1768] The server then customizes the recommendation list based on the information it has obtained, determining the optimal order of items based on past purchase history.

[1769] Output: A customized recommendation list.

[1770] Step 7:

[1771] Providing recommendations

[1772] The server then sends the final customized recommendation list to the device as an HTTP response.

[1773] Input: A customized recommendation list.

[1774] The device displays the received recommendation list on the user interface.

[1775] Output: A list of recommendations displayed in a user interface. The user can browse the list, select items, and purchase them.

[1776] (Application example 2)

[1777] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1778] Conventional online shopping systems do not take into account the user's emotional state when making recommendations, making it difficult for users to find the perfect fashion item that best suits their current mood. Furthermore, technology for accurately extracting the characteristics of fashion items from images uploaded by users and generating recommendation lists based on those characteristics is also inadequate. This leads to issues such as lower user satisfaction and a decrease in purchasing motivation.

[1779] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1780] In this invention, the server includes means for receiving images selected and uploaded by a user, means for analyzing the received images and extracting features of fashion items in the images, means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and the user's emotional state and generating a recommendation list, means for customizing the recommendation list taking into account the user's past purchase history and input information, and means for providing the generated recommendation list to the user. This enables optimal fashion item recommendations that take into account the user's emotional state. Furthermore, users can easily find products that match their emotional state, which is expected to increase their purchasing motivation.

[1781] "User" refers to a person who uses the system, primarily someone who uploads images of fashion items and receives recommendations.

[1782] "Image receiving means" refers to a device or software that has the function of receiving image data uploaded by a user and sending it to a server.

[1783] "Image analysis means" refers to a device or software that has the function of analyzing received image data and identifying and extracting the characteristics of fashion items in the image.

[1784] The "feature extraction means" refers to a device or software that has the function of extracting the features of a fashion item, such as color, shape, or type, from the analyzed image.

[1785] "Emotion recognition means" refers to a device or software that has the function of recognizing and determining the emotional state of a user.

[1786] "Online shopping platform" means an e-commerce site that allows users to purchase fashion items via the Internet.

[1787] "Recommendation list generation means" refers to a device or software that has the function of searching for similar items from an online shopping platform based on the extracted features and the user's emotional state and generating a list to present to the user.

[1788] The "customization means" refers to a device or software that has the function of individually adjusting the generated recommendation list, taking into account the user's past purchase history and input information.

[1789] "Recommendation list providing means" refers to a device or software that has the function of displaying and providing a final customized recommendation list to a user.

[1790] To implement the present invention, it is necessary to build a system that allows users to upload images of fashion items using their own devices and provides recommended items based on the features of the items and the user's emotions. Specifically, this is implemented in the following way.

[1791] Details of the hardware and software used

[1792] Hardware: Smartphones, servers, database servers

[1793] Software: Flask (web framework), OpenCV (image processing library), Keras (deep learning framework), Pandas (data analysis library)

[1794] System configuration

[1795] 1. Device:

[1796] Users use their smartphones to take or select their favorite fashion items and upload the images to the system, which then sends the images to the server as HTTP requests.

[1797] 2. Server:

[1798] It analyzes the received images. First, it preprocesses the images using OpenCV, then extracts the features of the fashion items using an AI image analysis module built with Keras. It also uses another Keras model to recognize the user's emotions.

[1799] 3. Database Server:

[1800] Based on the extracted features and sentiment data, similar items are searched for using Pandas. Product data is obtained through APIs in collaboration with the database of an online shopping platform.

[1801] Data processing and calculation

[1802] Image upload:

[1803] When a user uploads an image from a device, the device sends the image data to the server, where it is passed as request data via the HTTP protocol.

[1804] Image analysis:

[1805] The server processes the received image data using OpenCV and converts it into an appropriate format, then inputs it into an image analysis model built with Keras to extract the main features of the fashion item (color, shape, type).

[1806] Emotion recognition:

[1807] The server analyzes the user's facial expressions and recognizes their emotional state (e.g., happy, surprised, sad), again using Keras' emotion recognition model.

[1808] Recommendation generation and customization:

[1809] The database server searches for similar items from online shopping platforms based on the extracted features and the recognized emotions, then uses Pandas to generate a recommendation list, which is further customized by taking into account the user's past purchase history and input information.

[1810] Providing a recommendation list:

[1811] The server sends the generated customized recommendation list to the device, which then displays the list on the user interface, allowing the user to select and purchase the items they like.

[1812] Specific examples

[1813] Example 1: A user uploads a photo wearing a black coat

[1814] Users select a photo from their smartphone of themselves wearing a black coat and upload it to the system.

[1815] The server passes the photo to the AI, which then detects the "black coat."

[1816] The server extracts the characteristics of the item and uses an emotion recognition engine to determine the user's emotion as "joy."

[1817] The server searches for similar items on the online shopping platform and generates a recommendation list based on the user's sentiment.

[1818] The server takes into account past purchase history to customize the list and provides it to the user.

[1819] The device will display a customized list, allowing users to select and purchase their preferred items.

[1820] In this way, the system allows users to easily find fashion items that suit their preferences and style, and provides optimal recommendations based on their emotional state.

[1821] Example prompts to input to a generative AI model:

[1822] plaintext

[1823] Generate recommended items when a user uploads an image of themselves wearing a black coat. The user's emotion is recognized as "joy." The item characteristics obtained from the image are "black, coat." Recommend similar items.

[1824] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1825] Step 1:

[1826] A user selects and uploads an image of a fashion item using a device. At this time, the image file is sent to the server as an HTTP request. The input is the image file on the device, and the output is an HTTP request containing this image file.

[1827] Step 2:

[1828] The server analyzes the received HTTP request and obtains image data. The obtained image data is preprocessed using OpenCV and converted into a format that can be input to the AI ​​image analysis model. The input is the image data in the HTTP request, and the output is the preprocessed image data.

[1829] Step 3:

[1830] The server inputs the preprocessed image data into a Keras AI image analysis model to extract the features of the fashion items. The AI ​​model analyzes the image and obtains features such as color, shape, and type. The input is the preprocessed image data, and the output is the obtained feature data.

[1831] Step 4:

[1832] The server uses another Keras model to recognize the user's emotion based on the extracted feature data. The user's emotional state (e.g., joy, surprise, sadness, etc.) is obtained along with the feature data. The input is the feature data, and the output is the emotion data.

[1833] Step 5:

[1834] The server sends a search query for similar items to the database server based on the feature data and emotion data. Pandas is used to search for similar items from the database of the online shopping platform and obtain a list. The input is the feature data and emotion data, and the output is a search result list of similar items.

[1835] Step 6:

[1836] The server customizes the search result list by taking into account the user's past purchase history and input information. It uses Pandas to analyze the list and generate an individually tailored recommendation list. The input is the search result list and the user's purchase history and input information, and the output is a customized recommendation list.

[1837] Step 7:

[1838] The server sends the generated customized recommendation list to the terminal as an HTTP response. The terminal displays the received recommendation list on the user interface. The input is the customized recommendation list, and the output is the list displayed on the user interface.

[1839] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1840] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1841] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1842] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1843] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1844] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1845] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1846] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1847] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1848] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1849] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1850] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1851] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1852] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1853] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1854] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1855] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1856] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1857] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1858] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1859] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1860] The following is further disclosed regarding the above embodiment.

[1861] (Claim 1)

[1862] means for receiving images selected and uploaded by a user;

[1863] means for analyzing the received image and extracting features of the fashion items in the image;

[1864] A means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and generating a recommendation list;

[1865] A means for customizing the recommendation list taking into account the user's past purchase history and input information;

[1866] A means for providing the generated recommendation list to a user;

[1867] A system including:

[1868] (Claim 2)

[1869] 10. The system of claim 1, wherein the extracted features include color, shape, and brand name.

[1870] (Claim 3)

[1871] 10. The system of claim 1, wherein the online shopping platform is accessible via an API.

[1872] "Example 1"

[1873] (Claim 1)

[1874] means for receiving images selected and uploaded by a user;

[1875] A means for passing the received image to an AI image analysis module and starting the analysis;

[1876] A means for extracting features of fashion items in an image using an AI image analysis module;

[1877] A means for searching for similar items from an online shopping platform based on the extracted fashion item features;

[1878] A means for matching a user's past purchase history and input information to customize the recommendation list;

[1879] A means for providing the generated recommendation list to a user;

[1880] A system including:

[1881] (Claim 2)

[1882] 10. The system of claim 1, wherein the extracted features include color, shape, and brand name.

[1883] (Claim 3)

[1884] 10. The system of claim 1, wherein the online shopping platform is accessible via an application program interface (API).

[1885] "Application Example 1"

[1886] (Claim 1)

[1887] means for receiving images selected and uploaded by a user;

[1888] means for analyzing the received image and extracting features of objects in the image;

[1889] A means for searching for similar items from an online trading platform based on the extracted object features and generating a recommendation list;

[1890] A means for customizing the recommendation list taking into account the user's past transaction history and input information;

[1891] A means for providing the generated recommendation list to a user;

[1892] means for purchasing the selected items from the recommendation list via a payment gateway for electronic transactions;

[1893] A system including:

[1894] (Claim 2)

[1895] 10. The system of claim 1, wherein the extracted features include color, shape, and brand name.

[1896] (Claim 3)

[1897] 10. The system of claim 1, wherein the online trading platform is accessible via an API.

[1898] "Example 2: Combining Emotion Engines"

[1899] (Claim 1)

[1900] means for receiving images selected and uploaded by a user;

[1901] means for analyzing the received image and extracting features of the fashion items in the image;

[1902] A means for analyzing a user's facial expression data and classifying emotions;

[1903] A means for searching for similar items from an online shopping platform and generating a recommendation list based on the extracted fashion item features and the classified emotion data;

[1904] A means for customizing the recommendation list taking into account the user's past purchase history and input information;

[1905] A means for providing the generated recommendation list to a user;

[1906] A system including:

[1907] (Claim 2)

[1908] 10. The system of claim 1, wherein the extracted features include color, shape, and brand name.

[1909] (Claim 3)

[1910] 10. The system of claim 1, wherein the online shopping platform is accessible via an API.

[1911] "Application example 2 when combining emotion engines"

[1912] (Claim 1)

[1913] means for receiving images selected and uploaded by a user;

[1914] means for analyzing the received image and extracting features of the fashion items in the image;

[1915] A means for searching for similar items from an online shopping platform and generating a recommendation list based on the extracted fashion item features and the user's emotional state;

[1916] A means for customizing the recommendation list taking into account the user's past purchase history and input information;

[1917] A means for providing the generated recommendation list to a user;

[1918] A system including:

[1919] (Claim 2)

[1920] 10. The system of claim 1, wherein the extracted features include color, shape, and type.

[1921] (Claim 3)

[1922] 10. The system of claim 1, wherein the online shopping platform is accessible via an API. [Explanation of symbols]

[1923] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving images selected and uploaded by a user; means for analyzing the received image and extracting features of the fashion items in the image; A means for searching for similar items from an online shopping platform based on the extracted features of the fashion items and generating a recommendation list; A means for customizing the recommendation list taking into account the user's past purchase history and input information; A means for providing the generated recommendation list to a user; A system including:

2. The system of claim 1 , wherein the extracted features include color, shape, and brand name.

3. The system of claim 1 , wherein the online shopping platform is accessible via an API.

Images