System

Abstract

A system is provided.SOLUTION: A means for collecting a past purchase history and feedback data of a user, a means for preprocessing the collected data and extracting a feature amount, and a means for learning a preference of the user by using the preprocessed data; A system includes means for generating a prediction model, means for creating a personalized suggestion list for each user based on the prediction model, means for transmitting the suggestion list and 3D model data to a terminal, means for capturing a user's body using a camera in the terminal and performing virtual try-on using captured data and 3D model data, and means for collecting feedback of the user and transmitting the feedback to the server.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's online shopping world, users face several problems when selecting products from the comfort of their own homes. For example, it can be difficult to select the right product from the vast amount of information available, and users often feel uneasy about purchasing because they are unable to visualize the product's size or appearance. As a result, users hesitate to make purchases, reducing the convenience of online shopping. The objective of this invention is to provide a system that solves these problems and allows users to easily and efficiently try on and select products from the comfort of their own homes. [Means for solving the problem]

[0005] The present invention provides a system that solves the above-mentioned problems by the following means. It includes a means for collecting a user's past purchase history and feedback data, and a means for preprocessing the data and extracting features. It also includes a means for using the preprocessed data to learn the user's preferences using generative AI and generate a predictive model. It then provides a means for creating a personalized recommendation list for each user based on the predictive model, and sending the recommendation list and 3D model data to the user's device. It also includes a means for capturing the user's body using a camera on the device and performing a virtual try-on using augmented reality technology using the captured data and 3D model data. It also provides a means for collecting user feedback and sending it to a server. By combining these means, users can check the size and appearance of products at home and receive recommendations tailored to their individual preferences.

[0006] "User's past purchase history" refers to products that the user has previously purchased through mail order sales, etc., and related information.

[0007] "Feedback data" refers to data including information such as impressions, ratings, and improvements provided by users regarding products they have purchased in the past.

[0008] "Preprocessing" refers to the process of processing raw data, filling in missing values, removing inappropriate data, extracting features, etc.

[0009] "Features" refer to important attributes or elements used to train models in data analysis and machine learning.

[0010] "Generative AI" is a type of artificial intelligence technology that refers to algorithms and models that have the ability to learn patterns from data and generate new data and suggestions.

[0011] "Predictive Model" means a statistical or machine learning model used to forecast future trends or outcomes based on collected and pre-processed data.

[0012] A "personalized suggestion list" refers to a product suggestion list that is individually optimized based on the user's preferences and purchasing history.

[0013] "3D model data" refers to data depicting the three-dimensional shape of a product, and is used for virtual try-on.

[0014] "Virtual try-on" refers to the experience of using augmented reality technology to allow users to try on products on an image of themselves.

[0015] "Augmented reality technology" refers to the technology of overlaying computer-generated information onto images of the real world.

[0016] "Capture data" refers to data such as the user's body dimensions and pose obtained using the device's camera or sensors.

[0017] "Server" refers to a computing system that manages and processes data over a network and provides services to other computers and devices.

[0018] A "terminal" is an electronic device that is directly operated by a user, such as a smartphone or tablet. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention provides a system that collects users' past purchase history and feedback data, learns their preferences using generative AI, and provides personalized fashion suggestions that enable virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and a user.

[0041] Server-side behavior

[0042] 1. Data Collection and Preprocessing

[0043] The server collects the user's past purchase history and feedback data from a database.

[0044] Clean the collected data, impute missing values, and extract important features.

[0045] 2. Model learning using generative AI

[0046] The server uses the preprocessed data to train a generative AI model, specifically, to learn user preference patterns using a deep learning algorithm.

[0047] 3. Create a personalized suggestion list

[0048] Based on the trained model, the server generates an optimized list of fashion suggestions for each user.

[0049] The suggestion list includes 3D model data for each product item.

[0050] 4. Data transmission

[0051] The server sends the proposal list and 3D model data to the user's device.

[0052] Operation on the terminal side

[0053] 1. Data Reception

[0054] The terminal receives the proposal list and the 3D model data received from the server.

[0055] 2. Performing camera capture

[0056] The device uses the user's camera to capture the user's body measurements and pose.

[0057] The captured data is stored and used for virtual try-on.

[0058] 3. Virtual try-on using augmented reality

[0059] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using AR.

[0060] Virtual clothing is superimposed on a real-time video image of the user.

[0061] 4. Feedback Collection and Submission

[0062] The terminal provides an interface that allows the user to input feedback on the items they try on.

[0063] Sends the user-entered feedback to the server.

[0064] User behavior

[0065] 1. Login and item selection

[0066] Users log in to the app using a smartphone or tablet and choose the items they want to try on from a list of suggestions.

[0067] 2. Start the virtual try-on

[0068] The user captures their own body using the device's camera and begins the virtual try-on.

[0069] During the virtual try-on, users can check the appearance and size of the product on the screen.

[0070] 3. Providing Feedback

[0071] Users enter feedback about the items they try on (e.g., whether the size fits or whether they like the color) and submit it within the app.

[0072] Specific examples

[0073] For example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. The user checks how the dress looks and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server. The server collects this feedback and reflects it in future suggestions.

[0074] The present invention allows users to efficiently try on and select products from the comfort of their own homes, significantly improving the convenience and satisfaction of online shopping.

[0075] The processing flow will be explained below.

[0076] Server-side processing steps

[0077] Step 1:

[0078] The server collects the user's past purchase history and feedback data from a database.

[0079] Step 2:

[0080] The server preprocesses the collected data, specifically cleaning the data (removing inappropriate data and filling in missing values) and extracting features.

[0081] Step 3:

[0082] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[0083] Step 4:

[0084] The server uses the trained model to generate an optimized fashion suggestion list for each user.

[0085] Step 5:

[0086] The server transmits the proposal list and the 3D model data to the user's terminal.

[0087] Terminal processing steps

[0088] Step 1:

[0089] The terminal receives the proposal list and the 3D model data from the server.

[0090] Step 2:

[0091] The device activates the user's camera and prepares to capture body measurements and poses.

[0092] Step 3:

[0093] The user stands in front of the camera, and the device captures the user's entire body and saves the data.

[0094] Step 4:

[0095] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[0096] Step 5:

[0097] The terminal displays an interface that allows the user to provide feedback on the results of the try-on.

[0098] Step 6:

[0099] The feedback entered by the user is stored on the device and transmitted to the server.

[0100] User processing steps

[0101] Step 1:

[0102] Users log in to the app using their smartphone or tablet.

[0103] Step 2:

[0104] The user selects the product they wish to try on from the list of items suggested by the server.

[0105] Step 3:

[0106] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[0107] Step 4:

[0108] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[0109] Step 5:

[0110] Users enter feedback on the items they try on into a form within the app.

[0111] Step 6:

[0112] The user presses the "Submit" button to send the feedback to the server.

[0113] Through these steps, the StyleSync+ AR Edition system will provide personalized fashion suggestions based on the user's past data and offer a virtual try-on experience at home.

[0114] Example 1

[0115] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0116] Conventional online fashion sales systems do not allow users to actually try on products, which can lead to problems such as the size not fitting or the color or design being different from what was imagined after purchase. Furthermore, it is difficult to provide personalized suggestions that fully take into account the user's preferences, which can result in low user satisfaction. This leads to an increase in returns and exchanges, causing inconvenience to both sellers and consumers. It is also difficult to effectively utilize user feedback and incorporate it into future suggestions.

[0117] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0118] In this invention, the server includes: means for collecting a user's past purchase history and feedback data; means for preprocessing the collected data and extracting features; means for learning the user's preferences using the preprocessed data and generating a predictive model; means for creating a personalized proposal list for each user based on the predictive model; means for transmitting the proposal list and three-dimensional model data to a terminal; means for capturing the user's body using a camera on the terminal and performing a virtual try-on using the captured data and the three-dimensional model data; and means for collecting user feedback and transmitting it to the server. This allows the user to virtually try on products at home and check the fit and design before purchasing. Furthermore, future proposals are improved based on the collected feedback, resulting in personalized fashion proposals.

[0119] A "user" is an entity that utilizes the system to browse, select, try on, and purchase products.

[0120] "Past purchase history" is data including information on products purchased by the user in the past.

[0121] "Feedback data" refers to data including ratings, comments, and opinions provided by users regarding products.

[0122] "Data collection means" is a mechanism for collecting users' past purchasing history and feedback data.

[0123] "Preprocessing means" is a mechanism for preparing collected data, complementing missing values, and extracting features.

[0124] "Features" are important elements extracted from data for analysis and prediction.

[0125] A "predictive model" is a computational model constructed to predict future behavior and preferences based on a user's preferences and behavior.

[0126] A "personalized suggestion list" is a list of product suggestions optimized for each user based on a predictive model.

[0127] "Three-dimensional model data" refers to three-dimensional digital representation data of a product used for virtual try-on.

[0128] A "terminal" is a device operated by a user, and is an electronic device such as a smartphone or tablet that includes a camera function and a display.

[0129] The "camera capture means" is a mechanism that uses the device's camera to capture the user's body measurements and pose.

[0130] "Virtual try-on" refers to the act of virtually trying on a product using user capture data and three-dimensional model data.

[0131] "Augmented reality technology" is a technology that displays digital information overlaid on images of the real world.

[0132] The "feedback collection means" is a mechanism by which users input their ratings and opinions about products and transmit that information to a server.

[0133] MODE FOR CARRYING OUT THE INVENTION

[0134] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI to learn their preferences, thereby providing personalized fashion suggestions and enabling virtual try-on sessions at home. This system is realized through the cooperation of a server, a device, and a user.

[0135] Server-side behavior

[0136] The server first collects the user's past purchase history and feedback data from a database. This collection is done using a database query method such as SQL. The collected data is then preprocessed, missing values ​​are filled in, and features are extracted. This is often done using data analysis libraries such as Python's Pandas or Numpy.

[0137] Using the preprocessed data, the server trains a generative AI model using a deep learning framework such as TENSORFLOW® or PyTorch. For example, the model is applied to learn the user's preference patterns, and a personalized recommendation list is generated based on the trained model. This recommendation list includes 3D model data for each product item.

[0138] The proposal list and 3D model data are sent from the server to the user's device using a REST API, with data exchanged in JSON format.

[0139] Operation on the terminal side

[0140] The device receives and stores the proposal list and 3D model data from the server. The device then uses the user's camera to capture the user's body dimensions and pose. The camera capture uses a computer vision library such as OpenCV. The captured data is used for virtual try-on and is then stored.

[0141] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using augmented reality technology. Using frameworks such as ARKit (iOS) and ARCore (ANDROID (registered trademark)), the device overlays the virtual clothing on the real-time video.

[0142] After the user has completed the virtual try-on, the device provides an interface for collecting user feedback, which is then sent to the server and used to improve future proposals.

[0143] User behavior

[0144] Users log in to the app using a smartphone or tablet and select the items they want to try on from a list of suggestions. Standard authentication methods are used for login. Next, users capture their entire body using the device's camera and begin the virtual try-on. They can view the virtual garments in real-time AR footage and evaluate the product's appearance and fit.

[0145] After the virtual try-on, the user enters feedback on the item they tried on in the app. For example, they rate it on whether the size fits or whether they like the color, and this information is sent to the server. The server can use this feedback to improve its suggestions for future purchases.

[0146] Specific examples

[0147] For example, suppose a user logs in to their smartphone and selects a red dress from a list of items suggested by the server. The user uses a camera to capture their body and uses augmented reality technology to virtually try on the red dress. The user checks how the dress looks on them and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server and reflected in future suggestions.

[0148] Prompt Sentence Examples

[0149] "Build a generative AI model that provides personalized fashion recommendations based on users' past purchase history and feedback data."

[0150] This invention allows users to efficiently try on and select products from the comfort of their own homes, significantly improving the convenience and satisfaction of online shopping.

[0151] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0152] Program processing flow

[0153] Server-side processing

[0154] Step 1: Data collection

[0155] The server collects the user's past purchase history and feedback data from a database.

[0156] Input: SQL query (e.g. SELECT FROM purchase_history WHERE user_id = ?)

[0157] Output: User purchase history and feedback data

[0158] What happens: The server connects to the database and executes an SQL query to retrieve the data.

[0159] Step 2: Data Preprocessing

[0160] The server cleans the collected data, imputes missing values, and extracts features.

[0161] Input: Purchase history and feedback data

[0162] Output: Preprocessed data

[0163] Specific operation: Create a data frame using the Pandas library, fill in missing values ​​with df.fillna(method='ffill'), and extract the necessary features.

[0164] Step 3: Training the generative AI model

[0165] The server uses the preprocessed data to train a generative AI model.

[0166] Input: Preprocessed data

[0167] Output: Trained model

[0168] Specific operation: Build a deep learning model using TensorFlow or PyTorch and train it with model.fit(data, labels).

[0169] Step 4: Create a list of suggestions

[0170] The server generates a list of suggestions optimized for each user based on the trained model.

[0171] Input: trained model, user data

[0172] Output: Proposal list and 3D model data

[0173] Specific operation: Create a suggestion list with predictions = model.predict(user_data) and load the related 3D model data from a file.

[0174] Step 5: Sending data

[0175] The server transmits the proposal list and the three-dimensional model data to the user's terminal.

[0176] Input: Proposal list and 3D model data

[0177] Output: None (transmission completion status)

[0178] Specific operation: Using the REST API, send data with requests.post('http: / / terminal URL / api', json=data).

[0179] Terminal side processing

[0180] Step 1: Receiving data

[0181] The terminal receives the proposal list and the three-dimensional model data received from the server.

[0182] Input: Data from the server

[0183] Output: Save the proposal list and 3D model data

[0184] Specific operation: Using the Flask framework, receive and save data using request.get_json().

[0185] Step 2: Performing a camera capture

[0186] The device uses the user's camera to capture the user's body measurements and pose.

[0187] Input: Device camera image

[0188] Output: Captured image or video data

[0189] Specific operation: Using OpenCV, capture camera footage with cv2.VideoCapture() and save the image with cv2.imwrite('user_image.jpg', frame).

[0190] Step 3: Virtual try-on with augmented reality

[0191] The device integrates the captured data with the three-dimensional model data received from the server and performs a virtual try-on using AR.

[0192] Input: Captured images and 3D model data

[0193] Output: Virtual try-on video

[0194] Specific operation: Using ARKit or ARCore, virtual clothing is overlaid on real-time video (e.g., ar_session.add(anchor)).

[0195] Step 4: Collect and submit feedback

[0196] The terminal provides an interface that allows the user to input feedback on the items they try on.

[0197] Input: User ratings and comments

[0198] Output: Sending feedback data

[0199] What it does: Collect feedback via an HTML form or in-app interface and send it to the server with requests.post('http: / / server_url / api / feedback', json=feedback_data).

[0200] User processing

[0201] Step 1: Log in and select an item

[0202] Users log in to the app using a smartphone or tablet and choose the items they want to try on from a list of suggestions.

[0203] Input: User ID and password

[0204] Output: Selected items

[0205] Specific behavior: Authenticate on the login screen and select an item from the suggestion list.

[0206] Step 2: Start the virtual try-on

[0207] The user captures their own body using the device's camera and begins the virtual try-on.

[0208] Input: Camera image

[0209] Output: Virtual try-on footage and feedback

[0210] Specific operations: While checking the video, select virtual clothing and try it on.

[0211] Step 3: Provide feedback

[0212] Users can enter and submit feedback on the items they try on within the app.

[0213] Input: Feedback data (e.g., size and color ratings)

[0214] Output: Feedback data sent to the server

[0215] Specific behavior: Enter feedback in the evaluation form within the app and press the submit button to send it to the server.

[0216] (Application example 1)

[0217] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0218] Traditional online shopping has the problem that users cannot actually try on products, which leads to mismatches in size and appearance, resulting in a high return rate. Furthermore, the lack of personalized suggestions makes it difficult to improve the user experience. To solve these issues, there is a need for a system that utilizes users' past purchase history and feedback data to make individually optimized fashion suggestions and enable virtual try-on sessions at home.

[0219] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0220] In this invention, the server includes means for collecting a user's past purchase history and feedback data, means for preprocessing the collected data and extracting features, means for learning user preferences using the preprocessed data and generating a predictive model, means for creating a personalized suggestion list for each user based on the predictive model, means for transmitting the suggestion list and 3D model data to the terminal, means for capturing the user's body using a camera in the terminal and performing a virtual try-on using the captured data and the 3D model data, means for collecting user feedback and transmitting it to the server, means for integrating the captured user's body dimension data and the 3D model data and performing a virtual try-on using augmented reality, and means for improving the suggestion list for future purchases using the user's feedback data. This allows users to efficiently try on and select products based on personalized suggestions, significantly improving the convenience and satisfaction of online shopping.

[0221] "User's past purchase history" is information about products purchased by the user in the past, and is data including purchase date and time, product name, category, price, quantity, and the like.

[0222] "Feedback data" is data collected by users providing ratings and comments about products they have purchased or tried on.

[0223] "Preprocessing" is the process of cleaning up the collected data, filling in missing values, and extracting important features.

[0224] "Features" are useful information extracted from data, and are variables or attributes used to learn user preferences and purchasing patterns.

[0225] A "predictive model" is a mathematical or algorithmic model built to learn user preferences and predict future purchasing behavior and tastes based on collected and preprocessed data.

[0226] A "suggestion list" is a list of products personalized for each user, generated based on a predictive model.

[0227] "3D Model Data" means data containing a three-dimensional representation of a product that is used to virtually try on the product.

[0228] A "terminal" is a computing device used by a user, such as a smartphone, tablet, or PC.

[0229] "Capturing the user's body using a camera" means capturing the user's body dimensions and pose using a camera mounted on the device and capturing them as data.

[0230] "Virtual try-on" is the integration of captured data and 3D model data, and the use of augmented reality technology to provide users with the experience of virtually trying on products.

[0231] "Augmented reality" is a technology that overlays digital elements onto real-world images to provide an interactive experience.

[0232] The "means for collecting feedback and transmitting it to the server" is a function for collecting data on users' try-on experiences and product evaluations, and transmitting the data to the server via a network.

[0233] "Means for improving the suggestion list from next time onwards" is a function that improves the algorithm for generating future suggestion lists based on feedback data collected from users, thereby achieving more advanced personalization.

[0234] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI to learn their preferences, thereby providing personalized fashion suggestions and enabling virtual try-on sessions at home. This system is realized through the cooperation of a server, a device, and a user.

[0235] Server-side behavior

[0236] The server first collects the user's past purchase history and feedback data from a database. This data is then preprocessed to fill in missing values ​​and extract important features. The preprocessed data is then used to train a generative AI model. Specifically, a deep learning algorithm is used to learn the user's preference patterns and generate a predictive model. Based on the trained model, a list of fashion suggestions optimized for each user is then generated and sent to the device, along with 3D model data for each product item.

[0237] Operation on the terminal side

[0238] The device receives the suggestion list and 3D model data from the server. When the user selects an item they want to try on from the suggestion list, the device uses its camera to capture the user's body measurements and pose. The captured data is processed in real time and used for the virtual try-on. Augmented reality (AR) technology is then used to integrate the captured data and the 3D model data, and the virtual garment is displayed superimposed on the user's real-time video. The device also provides an interface that collects feedback on the user's virtual try-on and sends it to the server.

[0239] User behavior

[0240] Users log in to the application using a smartphone or tablet and select the item they want to try on from a personalized list of suggestions provided by the server. Next, they use the device's camera to capture a photo of their body and begin the virtual try-on. During the virtual try-on, users can check the look and size of the product on the screen. They can then enter feedback on the item they tried on (e.g., whether the size fits, whether they like the color), and submit it within the app.

[0241] Specific examples

[0242] For example, suppose a user logs in and selects a blue shirt from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the blue shirt. The user checks how the shirt looks and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server. The server collects this feedback and reflects it in future suggestions.

[0243] Example prompts for generative AI models

[0244] Analyze users' past purchase history and feedback data to generate a personalized list of fashion suggestions. Provide 3D model data for each product.

[0245] This system allows users to efficiently try on and select products from the comfort of their own home, significantly improving the convenience and satisfaction of online shopping. Hardware used includes smartphones, tablets, and servers. Software used includes Python, OpenCV, Keras, REST API, and generative AI models.

[0246] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0247] Step 1:

[0248] The server collects users' past purchase history and feedback data from a database. The input is the purchase history data and feedback data in the database, and the output is the collected data. This data collection provides basic data for understanding users' preferences and behavioral patterns. Specifically, it executes queries to the database to retrieve related data.

[0249] Step 2:

[0250] The server preprocesses the collected data and extracts features. The input is the collected purchase history and feedback data, and the output is the preprocessed data and extracted features. Specific operations include data cleansing (filling in missing values ​​and removing outliers), normalization, and feature engineering. This converts the data into a format suitable for learning generative AI models.

[0251] Step 3:

[0252] The server uses the preprocessed data to train a generative AI model. The input is the preprocessed data and the generative AI model, and the output is a trained predictive model. Specifically, it uses a deep learning algorithm to learn the user's preference patterns and generate a predictive model. This builds a model that predicts the user's future purchasing behavior.

[0253] Step 4:

[0254] The server generates a personalized fashion suggestion list for each user based on the trained model and sends it to the device, including 3D model data for each product item. The input is the trained model and product database, and the output is a personalized suggestion list and 3D model data. Specifically, the server uses the trained model to evaluate the suitability of products for each user and generates an optimal product list.

[0255] Step 5:

[0256] The terminal receives the proposal list and 3D model data from the server. The input is the proposal list and 3D model data sent from the server, and the output is the received proposal list and 3D model data. Specifically, the terminal receives the data via the network and stores it in the terminal.

[0257] Step 6:

[0258] The user selects an item they want to try on from the suggestion list. The input is the suggestion list, and the output is the selected item. Specifically, the user selects an item using the interface within the application.

[0259] Step 7:

[0260] The device uses a camera to capture the user's body dimensions and pose. The input is the camera image, and the output is the captured body dimension data. Specifically, the device starts the camera and captures the user's entire body.

[0261] Step 8:

[0262] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology. The input is the captured data and 3D model data, and the output is a real-time video of the virtual try-on. Specifically, the device uses an AR engine to overlay the 3D model on the user's video and display it on the screen.

[0263] Step 9:

[0264] During the virtual try-on, the user checks the appearance and size of the product and provides feedback. The input is the user's confirmation result, and the output is feedback data. Specifically, the user checks the virtual try-on results on the screen and enters their rating and comments in the app's interface.

[0265] Step 10:

[0266] The terminal collects user feedback and transmits it to the server. The input is the user feedback data and the output is the transmitted feedback data. The specific operation is to collect the feedback data and transmit it to the server through the network.

[0267] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0268] This invention is a system that collects users' past purchase history and feedback data, and uses a generative AI and emotion engine to learn the user's preferences and emotions based on that data, thereby providing personalized fashion suggestions and enabling virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and the user.

[0269] Server-side behavior

[0270] 1. Data Collection and Preprocessing

[0271] The server collects the user's past purchase history and feedback data from a database.

[0272] The collected data is preprocessed, and data cleaning (removal of inappropriate data, completion of missing values) and feature extraction are performed.

[0273] 2. Model learning using generative AI

[0274] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[0275] 3. Emotion analysis using an emotion engine

[0276] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotion patterns into a model.

[0277] 4. Create a personalized suggestion list

[0278] Based on the learned preference model and emotion model, the server generates an optimized fashion suggestion list for each user.

[0279] The suggestion list includes 3D model data for each product item.

[0280] 5. Data transmission

[0281] The server sends the proposal list and 3D model data to the user's device.

[0282] Operation on the terminal side

[0283] 1. Data Reception

[0284] The terminal receives the proposal list and the 3D model data from the server.

[0285] 2. Performing camera capture

[0286] The device activates the user's camera and prepares to capture body measurements and poses.

[0287] 3. Emotion analysis

[0288] Using the device's camera and microphone, the emotion engine analyzes the user's facial expressions and voice to recognize the user's emotions in real time.

[0289] 4. Virtual try-on using augmented reality

[0290] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[0291] 5. Feedback Collection and Submission

[0292] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time.

[0293] Sends the user-entered feedback to the server.

[0294] User behavior

[0295] 1. Login and item selection

[0296] Users log in to the app using their smartphone or tablet and choose the items they want to try on from a list of suggestions.

[0297] 2. Start the virtual try-on

[0298] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[0299] 3. Emotion presentation

[0300] Users are also shown emotions (e.g., happy, excited) that the device recognizes in real time during the virtual try-on.

[0301] 4. Check the fitting results

[0302] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[0303] 5. Providing Feedback

[0304] Users enter feedback into an in-app form based on the items they try on and how they feel about them.

[0305] 6. Send Feedback

[0306] The user presses the "Submit" button to send the feedback to the server.

[0307] Specific examples

[0308] For example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. While trying it on, the device's camera and microphone analyze the user's facial expressions and voice using an emotion engine, which recognizes that the user has a positive emotional reaction to the dress. The user then enters their try-on experience (whether the size and color suit them) and their emotions (happy, excited, etc.) into the app and presses the "Send" button. This feedback is then sent to the server, which receives it as new data. The server learns from this feedback and reflects it in future suggestions.

[0309] The present invention not only allows users to efficiently try on and select products from the comfort of their own homes, but also allows them to receive personalized suggestions based on their emotional responses, further improving the convenience and satisfaction of online shopping.

[0310] The processing flow will be explained below.

[0311] Server-side processing steps

[0312] Step 1:

[0313] The server collects the user's past purchase history and feedback data from a database.

[0314] Step 2:

[0315] The server preprocesses the collected data, specifically cleaning the data (removing inappropriate data and filling in missing values) and extracting features.

[0316] Step 3:

[0317] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[0318] Step 4:

[0319] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotion patterns into a model.

[0320] Step 5:

[0321] The server generates a list of fashion suggestions optimized for each user based on the learned preference model and emotion model.

[0322] Step 6:

[0323] The server transmits the proposal list and the 3D model data to the user's terminal.

[0324] Terminal processing steps

[0325] Step 1:

[0326] The terminal receives the proposal list and the 3D model data from the server.

[0327] Step 2:

[0328] The device activates the user's camera and prepares to capture body measurements and poses.

[0329] Step 3:

[0330] The user stands in front of the camera, and the device captures the user's entire body and saves the data.

[0331] Step 4:

[0332] The device's emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotions.

[0333] Step 5:

[0334] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[0335] Step 6:

[0336] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time.

[0337] Step 7:

[0338] The feedback entered by the user is stored on the device and transmitted to the server.

[0339] User processing steps

[0340] Step 1:

[0341] Users log in to the app using their smartphone or tablet.

[0342] Step 2:

[0343] The user selects the product they wish to try on from the list of items suggested by the server.

[0344] Step 3:

[0345] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[0346] Step 4:

[0347] The user's facial expressions and voice are analyzed by the emotion engine, and emotions are recognized in real time.

[0348] Step 5:

[0349] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[0350] Step 6:

[0351] Users enter feedback into an in-app form based on the items they try on and how they feel about them.

[0352] Step 7:

[0353] The user presses the "Submit" button to send the feedback to the server.

[0354] Through these steps, the StyleSync+ AR Edition system provides personalized fashion suggestions based on the user's past data and real-time recognized emotions, and offers a virtual try-on experience at home.

[0355] Example 2

[0356] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0357] While online shopping has become commonplace in modern society, there are limited ways to check the actual fit and appearance of products before purchasing. This has led to an increase in returns and exchanges after purchase, resulting in a decline in user satisfaction. Furthermore, traditional methods are not sufficient to provide personalized product recommendations that take into account individual user preferences and emotions. This makes it difficult to improve the user experience and reduces the convenience of online shopping.

[0358] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0359] In this invention, the server includes means for collecting a user's past purchase history and feedback data, means for preprocessing the collected data and extracting features, means for learning the user's preferences using a deep learning algorithm with the preprocessed data and generating a predictive model, means for analyzing the user's past feedback data and identifying emotions, means for creating a personalized suggestion list for each user based on the generated predictive model and the results of the emotion analysis, and means for transmitting the suggestion list and 3D model data to a terminal. This allows users to efficiently try on and select products at home, as well as receive personalized suggestions based on their emotional reactions.

[0360] "User" refers to an individual who utilizes the system to virtually try on fashion items and provide feedback.

[0361] "Past purchase history" refers to a record of purchases made by the user up to now, and includes information such as product names, purchase dates, and quantities.

[0362] "Feedback data" refers to data including evaluations, impressions, and emotional expressions provided by users regarding products.

[0363] "Preprocessing" refers to a series of processes that convert collected data into a format suitable for analysis and learning, and includes data cleaning and feature extraction.

[0364] "Features" refer to important attributes or information extracted from data in data analysis and machine learning.

[0365] "Deep learning algorithm" refers to a type of machine learning that uses artificial neural networks to learn patterns from large amounts of data.

[0366] "Predictive Model" refers to a mathematical model trained to predict future preferences based on user tastes and behavior.

[0367] "Emotion engine" refers to technology that analyzes emotions from user feedback data and real-time facial expressions and voice.

[0368] A "personalized suggestion list" refers to a recommended list of specific fashion items created based on the preferences and emotional patterns of an individual user.

[0369] "3D model data" refers to digital data used to display fashion items in three dimensions, and includes formats such as glTF and OBJ.

[0370] "Terminal" refers to the device that a user uses to access the system and perform virtual try-on and emotion analysis, and primarily includes smartphones and tablets.

[0371] "Virtual try-on" refers to the process of virtually trying on clothes by combining captured data of the user's body with 3D model data.

[0372] "Augmented reality technology" refers to a technique that overlays digital information onto real-world images, providing users with an experience that makes them feel as if they are trying on clothes in real time.

[0373] "Feedback" refers to the ratings and impressions provided by users based on their fitting results and emotions, and includes data used to train the system.

[0374] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI and an emotion engine to learn the user's preferences and emotions based on that data, and then makes personalized fashion suggestions. This system is realized through the cooperation of a server, a terminal, and the user.

[0375] First, the server collects the user's past purchase history and feedback data from a database (e.g., MySQL (registered trademark), PostgreSQL). The collected data is preprocessed to remove inappropriate data, fill in missing values, and extract features. During this preprocessing stage, the user's purchasing trends and product category information are analyzed.

[0376] The server then uses deep learning frameworks such as TensorFlow and PyTorch to train a generative AI model from the preprocessed data, which learns user preference patterns based on algorithms such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs).

[0377] Furthermore, the server utilizes an emotion engine (e.g., IBM Watson®, Microsoft® Azure® Emotion API) to analyze emotions from the user's feedback data. This emotion analysis result is then combined with a generative AI model to create a personalized recommendation list based on emotions.

[0378] The generated recommendation list includes 3D model data (e.g., glTF, OBJ format) for each product item, and this data is sent to the user's device using a secure protocol (e.g., HTTPS).

[0379] The device receives the proposal list and 3D model data from the server. Then, it activates its camera to capture the user's body dimensions and pose, and acquires this data using a depth camera (e.g., Microsoft Kinect) or a standard camera. At the same time, it uses an emotion engine (e.g., OpenCV, Azure Cognitive Services) to analyze the user's facial expressions and voice and recognize emotions in real time.

[0380] The collected capture data and 3D model data are integrated, and the device uses augmented reality technology (e.g., ARKit, ARCore) to perform a virtual try-on experience, allowing users to see virtual clothing overlaid on real-world footage.

[0381] Users can use an interface on their device to provide feedback based on their fitting experience and their feelings. The feedback they provide is then sent back to the server and recorded as new data. This feedback is used in the system's overall learning process and is reflected in future recommendations.

[0382] As a concrete example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. While trying it on, the device's camera and microphone analyze the user's facial expressions and voice using an emotion engine, which recognizes that the user has a positive emotional reaction to the dress. The user then enters their try-on experience (whether the size and color suit them) and their emotions (happy, excited, etc.) into the app and presses the "Submit" button. This sends the feedback to the server and receives it as new data. The server learns from this data and reflects it in future suggestions.

[0383] An example prompt is, "The user chooses a red dress, performs a virtual try-on, and records the positive emotion analyzed by the emotion engine."

[0384] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0385] Processing flow

[0386] Server-side behavior

[0387] Step 1:

[0388] Data collection and preprocessing

[0389] The server collects users' past purchase history and feedback data from a database as input. Since the collected data may contain inaccurate data, it performs data cleaning. It removes inappropriate data and fills in missing values ​​to organize the information. It also extracts features and tags each product item with the category and user purchasing trends. The preprocessed dataset is obtained as output.

[0390] Step 2:

[0391] Model learning using generative AI models

[0392] The server uses the preprocessed dataset as input to train a generative AI model using a deep learning framework (e.g., TensorFlow or PyTorch). It uses a recurrent neural network (RNN) or convolutional neural network (CNN) to learn user preference patterns. The output is a predictive model that reflects user preferences.

[0393] Step 3:

[0394] Emotion analysis using an emotion engine

[0395] The server takes the user's past feedback data as input and analyzes the emotions using an emotion engine (e.g., IBM Watson or Microsoft Azure Emotion API). It uses natural language processing (NLP) technology to analyze the feedback text and assign emotion labels such as positive or negative. The output is emotion-labeled feedback data.

[0396] Step 4:

[0397] Creating a personalized suggestion list

[0398] The server uses the generative AI model and the sentiment analysis results as inputs to generate a personalized fashion suggestion list. This suggestion list also includes 3D model data (e.g., glTF, OBJ format) for each product item. The output is the personalized suggestion list and 3D model data.

[0399] Step 5:

[0400] Sending data

[0401] The server takes the generated proposal list and 3D model data as input and sends them to the user's device. The data is securely transmitted using a secure protocol (e.g., HTTPS). As output, the proposal list and 3D model data sent to the device are obtained.

[0402] Operation on the terminal side

[0403] Step 1:

[0404] Data reception

[0405] The terminal receives as input the suggestion list and 3D model data sent from the server, prepares the received data for display, and obtains as output the suggestion list and 3D model data prepared in a displayable format.

[0406] Step 2:

[0407] Performing a camera capture

[0408] The device accepts user operations as input, activates a camera to capture the user's body dimensions and pose, and obtains body capture data using a depth camera (e.g., Microsoft Kinect) or a standard camera. The device obtains the capture data as output.

[0409] Step 3:

[0410] Emotion analysis

[0411] The device uses a camera and microphone as input, and analyzes the user's facial expressions and voice using an emotion engine (e.g., OpenCV, Azure Cognitive Services). The device recognizes the user's emotions in real time, and obtains real-time analyzed emotion data as output.

[0412] Step 4:

[0413] Augmented reality virtual try-on

[0414] The device integrates the captured data and 3D model data as input, and performs a virtual try-on using augmented reality technology (e.g., ARKit, ARCore). The device overlays the suggested clothing on a real-time video of the user. The output is a video of the virtual try-on using augmented reality technology.

[0415] Step 5:

[0416] Feedback collection and submission

[0417] The terminal accepts feedback based on the fitting results and emotions entered by the user and sends it to the server. It displays a feedback collection interface, prompts the user to enter information, and sends the entered data to the server using a secure protocol (e.g., HTTPS). The feedback data sent to the server is obtained as output.

[0418] User behavior

[0419] Step 1:

[0420] Login and item selection

[0421] As input, users log in to the app using their smartphone or tablet and select the items they want to try on from a list of suggestions. As output, they receive the selected items.

[0422] Step 2:

[0423] Start a virtual try-on

[0424] The user uses the device's camera as input to capture their body and begin the virtual try-on. The user stands in front of the camera, which captures and saves their entire body. The captured data is obtained as output.

[0425] Step 3:

[0426] emotion presentation

[0427] As input, the user confirms the emotions recognized by the device in real time during the virtual try-on. As output, real-time emotion data is obtained.

[0428] Step 4:

[0429] Check the fitting results

[0430] The user checks the appearance and size of the virtually tried-on clothes while looking at the screen of the device as input, and receives the confirmed fitting results as output.

[0431] Step 5:

[0432] Providing Feedback

[0433] The user inputs the items they tried on and their feedback based on the emotions they felt at the time into a form within the app. The input feedback data is obtained as output.

[0434] Step 6:

[0435] Send Feedback

[0436] As input, the user presses the "Send" button to send the feedback data to the server. The data is sent using a secure protocol (e.g., HTTPS). As output, the user gets the feedback data sent to the server.

[0437] (Application example 2)

[0438] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0439] In conventional online shopping environments, it is difficult for users to check the actual feel and fit of products before purchasing, which often leads to mistaken purchases and returns. Furthermore, product suggestions based on user preferences and emotions are rarely made, limiting the provision of a personalized shopping experience. This leads to issues such as lower user satisfaction and the inability to fully utilize the convenience of online shopping.

[0440] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0441] In this invention, the server includes: means for collecting a user's past purchase history and feedback data; means for preprocessing the collected data and extracting features; means for learning the user's preferences using the preprocessed data and generating a predictive model using a generative AI model; means for creating a personalized recommendation list for each user based on the predictive model; and means for transmitting the recommendation list and 3D model data to the terminal. This allows the user to not only receive personalized product recommendations but also virtually try on products at home using augmented reality technology. The system also includes means for analyzing the user's emotions in real time using a camera and microphone to improve the accuracy of the personalized recommendations, and means for collecting feedback based on the try-on results and emotions and transmitting the results to the server. This improves user satisfaction and convenience in online shopping.

[0442] "User's past purchase history" refers to information about products and services purchased by the user in the past.

[0443] "Feedback data" refers to data on evaluations and opinions regarding satisfaction and usability collected from users.

[0444] "Preprocessing" refers to the process of converting collected data into a form suitable for analysis.

[0445] "Features" refer to important attributes or characteristics of the data input into a machine learning model.

[0446] A "generative AI model" refers to an artificial intelligence model that uses technologies such as deep learning to learn and predict user preferences and behavior.

[0447] "Predictive model" refers to an algorithm created by a generative AI model to predict user behavior and preferences.

[0448] A "personalized suggestion list" refers to a list of products and services optimized based on a user's individual preferences and past purchasing history.

[0449] "3D model data" refers to the digital representation of the three-dimensional shape of a product.

[0450] "Augmented reality technology" refers to the technology of overlaying digital information onto images of the real world.

[0451] "Camera" refers to a device that captures images or videos.

[0452] "Capture data" refers to image and video data captured using a camera.

[0453] "Analyzing user emotions in real time" refers to the process of instantly analyzing data such as the user's facial expressions and voice to evaluate their current emotional state.

[0454] "Feedback" refers to opinions and ratings provided by users regarding their experience and satisfaction with the product.

[0455] This invention is a system that collects users' past purchase history and feedback data, and uses a generative AI model and emotion engine to learn the user's preferences and emotions based on that data, thereby providing personalized fashion suggestions and enabling virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and the user.

[0456] Server-side behavior

[0457] Data collection and preprocessing

[0458] The server collects users' past purchase history and feedback data from a database. It preprocesses the collected data, cleaning it (removing inappropriate data and filling in missing values) and extracting features. This is done using Python, with libraries such as Pandas and Numpy used for data processing.

[0459] Model learning using generative AI models

[0460] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms and frameworks such as TensorFlow and Keras to learn user preference patterns.

[0461] Emotion analysis

[0462] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotional patterns into a model using a deep learning model specialized for emotion analysis.

[0463] Creating a personalized suggestion list

[0464] Based on the trained preference and emotion models, the server generates a personalized fashion suggestion list, including 3D model data for each product item, which is updated in real time.

[0465] Sending data

[0466] The server communicates data using, for example, a REST API to send the proposal list and 3D model data to the user's device.

[0467] Operation on the terminal side

[0468] Data reception

[0469] The device receives the proposal list and 3D model data from the server using HTTP requests.

[0470] Camera and microphone activation and capture

[0471] The device activates the user's camera and microphone, capturing body measurements and poses, and preparing to analyze emotions in real time, using libraries such as OpenCV and MediaPipe.

[0472] Performing a virtual try-on

[0473] The device integrates the captured data and 3D model data, and performs a virtual try-on using augmented reality technology. Using Unity or ARKit / ARCore, the device overlays suggested clothing on the user's real-time video.

[0474] Feedback collection and submission

[0475] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time. An HTTP POST request is used to send the user-entered feedback to the server.

[0476] User behavior

[0477] Log in and receive personalized offers

[0478] The user logs in to the app using their smartphone or tablet and receives a list of suggested items from the server. This operation requires the user's account information.

[0479] Start a virtual try-on

[0480] Users can capture their body using the device's camera and begin the virtual try-on experience, giving them the experience of trying on products in real life, without leaving the comfort of their own home.

[0481] Real-time emotion display

[0482] The device uses a camera and microphone to analyze the user's emotions in real time while trying on clothes, and displays that information on the screen, allowing the user to check their own emotions.

[0483] Checking fitting results and providing feedback

[0484] The user checks the appearance and size of the clothes they try on on the device screen and inputs the results as feedback into the device. This feedback is sent to the server and reflected in future suggestions.

[0485] Examples of prompt statements

[0486] When a user logs in to receive personalized suggestions, the prompt text looks like this:

[0487] Text format

[0488] Get personalized fashion suggestions based on user ID 12345's past purchase history and feedback data.

[0489] In this way, the present invention provides personalized fashion suggestions based on the user's tastes and emotions, providing an enhanced online shopping experience.

[0490] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0491] Step 1:

[0492] The server collects the user's past purchase history and feedback data from the database. The input is the user ID, and the output is the past purchase history and feedback data. Specifically, it uses SQL queries to extract relevant data from the database.

[0493] Step 2:

[0494] The server preprocesses the collected data and extracts features. The input is the collected purchase history and feedback data, and the output is a cleaned dataset and features. Specific operations include cleaning the data (filling in missing values ​​and removing inappropriate data) and extracting features using Pandas.

[0495] Step 3:

[0496] The server uses the preprocessed data to train a generative AI model. The input is the preprocessed data and features, and the output is a trained model. Specifically, it learns user preference patterns using deep learning algorithms using TensorFlow and Keras.

[0497] Step 4:

[0498] The server uses an emotion engine to analyze emotions from the user's past feedback data. The input is the feedback data, and the output is emotion features. Specifically, it extracts emotions from text data using natural language processing technology (e.g., an emotion analysis model).

[0499] Step 5:

[0500] The server generates a personalized fashion suggestion list for each user based on the trained preference and emotion models. The input is the trained model, and the output is the suggestion list and 3D model data. Specifically, it selects items that match the user's preferences and creates a list including the 3D model data.

[0501] Step 6:

[0502] The server sends the proposal list and 3D model data to the device. The input is the proposal list and 3D model data, and the output is the successful data transmission to the device. Specifically, the data is sent using the REST API.

[0503] Step 7:

[0504] The device receives the proposal list and 3D model data from the server. The input is the proposal list sent from the server, and the output is the proposal list displayed on the device. Specifically, the data is received using an HTTP GET request.

[0505] Step 8:

[0506] The device activates the camera and microphone, and prepares to capture the user's body and analyze emotions in real time. The input is real-time data from the camera and microphone, and the output is captured data and emotion data. Specifically, the device uses OpenCV to capture camera images and perform emotion analysis in real time.

[0507] Step 9:

[0508] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology. The input is the captured data and 3D model data, and the output is an image of the virtual try-on. Specifically, using Unity or ARKit / ARCore, the suggested clothing is overlaid on the user's real-time video.

[0509] Step 10:

[0510] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their emotions. The input is the user's feedback, and the output is the collected feedback data. Specifically, the terminal displays a GUI form to receive user input.

[0511] Step 11:

[0512] The terminal sends the feedback input by the user to the server. The input is the feedback data, and the output is the successful data transmission to the server. Specifically, the feedback data is sent using an HTTP POST request.

[0513] Step 12:

[0514] The server stores the received feedback data in a database and updates the data to reflect future proposals. The input is the feedback data, and the output is the updated database. Specifically, it inserts new data and updates existing data using SQL queries.

[0515] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0516] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0517] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0518] [Second embodiment]

[0519] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0520] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0521] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0522] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0523] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0524] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0525] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0526] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0527] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0528] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0529] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0530] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0531] This invention provides a system that collects users' past purchase history and feedback data, learns their preferences using generative AI, and provides personalized fashion suggestions that enable virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and a user.

[0532] Server-side behavior

[0533] 1. Data Collection and Preprocessing

[0534] The server collects the user's past purchase history and feedback data from a database.

[0535] Clean the collected data, impute missing values, and extract important features.

[0536] 2. Model learning using generative AI

[0537] The server uses the preprocessed data to train a generative AI model, specifically, to learn user preference patterns using a deep learning algorithm.

[0538] 3. Create a personalized suggestion list

[0539] Based on the trained model, the server generates an optimized list of fashion suggestions for each user.

[0540] The suggestion list includes 3D model data for each product item.

[0541] 4. Data transmission

[0542] The server sends the proposal list and 3D model data to the user's device.

[0543] Operation on the terminal side

[0544] 1. Data Reception

[0545] The terminal receives the proposal list and the 3D model data received from the server.

[0546] 2. Performing camera capture

[0547] The device uses the user's camera to capture the user's body measurements and pose.

[0548] The captured data is stored and used for virtual try-on.

[0549] 3. Virtual try-on using augmented reality

[0550] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using AR.

[0551] Virtual clothing is superimposed on a real-time video image of the user.

[0552] 4. Feedback Collection and Submission

[0553] The terminal provides an interface that allows the user to input feedback on the items they try on.

[0554] Sends the user-entered feedback to the server.

[0555] User behavior

[0556] 1. Login and item selection

[0557] Users log in to the app using a smartphone or tablet and choose the items they want to try on from a list of suggestions.

[0558] 2. Start the virtual try-on

[0559] The user captures their own body using the device's camera and begins the virtual try-on.

[0560] During the virtual try-on, users can check the appearance and size of the product on the screen.

[0561] 3. Providing Feedback

[0562] Users enter feedback about the items they try on (e.g., whether the size fits or whether they like the color) and submit it within the app.

[0563] Specific examples

[0564] For example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. The user checks how the dress looks and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server. The server collects this feedback and reflects it in future suggestions.

[0565] The present invention allows users to efficiently try on and select products from the comfort of their own homes, significantly improving the convenience and satisfaction of online shopping.

[0566] The processing flow will be explained below.

[0567] Server-side processing steps

[0568] Step 1:

[0569] The server collects the user's past purchase history and feedback data from a database.

[0570] Step 2:

[0571] The server preprocesses the collected data, specifically cleaning the data (removing inappropriate data and filling in missing values) and extracting features.

[0572] Step 3:

[0573] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[0574] Step 4:

[0575] The server uses the trained model to generate an optimized fashion suggestion list for each user.

[0576] Step 5:

[0577] The server transmits the proposal list and the 3D model data to the user's terminal.

[0578] Terminal processing steps

[0579] Step 1:

[0580] The terminal receives the proposal list and the 3D model data from the server.

[0581] Step 2:

[0582] The device activates the user's camera and prepares to capture body measurements and poses.

[0583] Step 3:

[0584] The user stands in front of the camera, and the device captures the user's entire body and saves the data.

[0585] Step 4:

[0586] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[0587] Step 5:

[0588] The terminal displays an interface that allows the user to provide feedback on the results of the try-on.

[0589] Step 6:

[0590] The feedback entered by the user is stored on the device and transmitted to the server.

[0591] User processing steps

[0592] Step 1:

[0593] Users log in to the app using their smartphone or tablet.

[0594] Step 2:

[0595] The user selects the product they wish to try on from the list of items suggested by the server.

[0596] Step 3:

[0597] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[0598] Step 4:

[0599] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[0600] Step 5:

[0601] Users enter feedback on the items they try on into a form within the app.

[0602] Step 6:

[0603] The user presses the "Submit" button to send the feedback to the server.

[0604] Through these steps, the StyleSync+ AR Edition system will provide personalized fashion suggestions based on the user's past data and offer a virtual try-on experience at home.

[0605] Example 1

[0606] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0607] Conventional online fashion sales systems do not allow users to actually try on products, which can lead to problems such as the size not fitting or the color or design being different from what was imagined after purchase. Furthermore, it is difficult to provide personalized suggestions that fully take into account the user's preferences, which can result in low user satisfaction. This leads to an increase in returns and exchanges, causing inconvenience to both sellers and consumers. It is also difficult to effectively utilize user feedback and incorporate it into future suggestions.

[0608] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0609] In this invention, the server includes: means for collecting a user's past purchase history and feedback data; means for preprocessing the collected data and extracting features; means for learning the user's preferences using the preprocessed data and generating a predictive model; means for creating a personalized proposal list for each user based on the predictive model; means for transmitting the proposal list and three-dimensional model data to a terminal; means for capturing the user's body using a camera on the terminal and performing a virtual try-on using the captured data and the three-dimensional model data; and means for collecting user feedback and transmitting it to the server. This allows the user to virtually try on products at home and check the fit and design before purchasing. Furthermore, future proposals are improved based on the collected feedback, resulting in personalized fashion proposals.

[0610] A "user" is an entity that utilizes the system to browse, select, try on, and purchase products.

[0611] "Past purchase history" is data including information on products purchased by the user in the past.

[0612] "Feedback data" refers to data including ratings, comments, and opinions provided by users regarding products.

[0613] "Data collection means" is a mechanism for collecting users' past purchasing history and feedback data.

[0614] "Preprocessing means" is a mechanism for preparing collected data, complementing missing values, and extracting features.

[0615] "Features" are important elements extracted from data for analysis and prediction.

[0616] A "predictive model" is a computational model constructed to predict future behavior and preferences based on a user's preferences and behavior.

[0617] A "personalized suggestion list" is a list of product suggestions optimized for each user based on a predictive model.

[0618] "Three-dimensional model data" refers to three-dimensional digital representation data of a product used for virtual try-on.

[0619] A "terminal" is a device operated by a user, and is an electronic device such as a smartphone or tablet that includes a camera function and a display.

[0620] The "camera capture means" is a mechanism that uses the device's camera to capture the user's body measurements and pose.

[0621] "Virtual try-on" refers to the act of virtually trying on a product using user capture data and three-dimensional model data.

[0622] "Augmented reality technology" is a technology that displays digital information overlaid on images of the real world.

[0623] The "feedback collection means" is a mechanism by which users input their ratings and opinions about products and transmit that information to a server.

[0624] MODE FOR CARRYING OUT THE INVENTION

[0625] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI to learn their preferences, thereby providing personalized fashion suggestions and enabling virtual try-on sessions at home. This system is realized through the cooperation of a server, a device, and a user.

[0626] Server-side behavior

[0627] The server first collects the user's past purchase history and feedback data from a database. This collection is done using a database query method such as SQL. The collected data is then preprocessed, missing values ​​are filled in, and features are extracted. This is often done using data analysis libraries such as Python's Pandas or Numpy.

[0628] Using the preprocessed data, the server trains a generative AI model using deep learning frameworks such as TensorFlow and PyTorch. For example, the model is applied to learn the user's preference patterns, and a personalized recommendation list is generated based on the trained model. This recommendation list includes 3D model data for each product item.

[0629] The proposal list and 3D model data are sent from the server to the user's device using a REST API, with data exchanged in JSON format.

[0630] Operation on the terminal side

[0631] The device receives and stores the proposal list and 3D model data from the server. The device then uses the user's camera to capture the user's body dimensions and pose. The camera capture uses a computer vision library such as OpenCV. The captured data is used for virtual try-on and is then stored.

[0632] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using augmented reality technology. Using frameworks such as ARKit (iOS) and ARCore (Android), the device overlays the virtual clothing on a real-time video image.

[0633] After the user has completed the virtual try-on, the device provides an interface for collecting user feedback, which is then sent to the server and used to improve future proposals.

[0634] User behavior

[0635] Users log in to the app using a smartphone or tablet and select the items they want to try on from a list of suggestions. Standard authentication methods are used for login. Next, users capture their entire body using the device's camera and begin the virtual try-on. They can view the virtual garments in real-time AR footage and evaluate the product's appearance and fit.

[0636] After the virtual try-on, the user enters feedback on the item they tried on in the app. For example, they rate it on whether the size fits or whether they like the color, and this information is sent to the server. The server can use this feedback to improve its suggestions for future purchases.

[0637] Specific examples

[0638] For example, suppose a user logs in to their smartphone and selects a red dress from a list of items suggested by the server. The user uses a camera to capture their body and uses augmented reality technology to virtually try on the red dress. The user checks how the dress looks on them and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server and reflected in future suggestions.

[0639] Prompt Sentence Examples

[0640] "Build a generative AI model that provides personalized fashion recommendations based on users' past purchase history and feedback data."

[0641] This invention allows users to efficiently try on and select products from the comfort of their own homes, significantly improving the convenience and satisfaction of online shopping.

[0642] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0643] Program processing flow

[0644] Server-side processing

[0645] Step 1: Data collection

[0646] The server collects the user's past purchase history and feedback data from a database.

[0647] Input: SQL query (e.g. SELECT FROM purchase_history WHERE user_id = ?)

[0648] Output: User purchase history and feedback data

[0649] What happens: The server connects to the database and executes an SQL query to retrieve the data.

[0650] Step 2: Data Preprocessing

[0651] The server cleans the collected data, imputes missing values, and extracts features.

[0652] Input: Purchase history and feedback data

[0653] Output: Preprocessed data

[0654] Specific operation: Create a data frame using the Pandas library, fill in missing values ​​with df.fillna(method='ffill'), and extract the necessary features.

[0655] Step 3: Training the generative AI model

[0656] The server uses the preprocessed data to train a generative AI model.

[0657] Input: Preprocessed data

[0658] Output: Trained model

[0659] Specific operation: Build a deep learning model using TensorFlow or PyTorch and train it with model.fit(data, labels).

[0660] Step 4: Create a list of suggestions

[0661] The server generates a list of suggestions optimized for each user based on the trained model.

[0662] Input: trained model, user data

[0663] Output: Proposal list and 3D model data

[0664] Specific operation: Create a suggestion list with predictions = model.predict(user_data) and load the related 3D model data from a file.

[0665] Step 5: Sending data

[0666] The server transmits the proposal list and the three-dimensional model data to the user's terminal.

[0667] Input: Proposal list and 3D model data

[0668] Output: None (transmission completion status)

[0669] Specific operation: Using the REST API, send data with requests.post('http: / / terminal URL / api', json=data).

[0670] Terminal side processing

[0671] Step 1: Receiving data

[0672] The terminal receives the proposal list and the three-dimensional model data received from the server.

[0673] Input: Data from the server

[0674] Output: Save the proposal list and 3D model data

[0675] Specific operation: Using the Flask framework, receive and save data using request.get_json().

[0676] Step 2: Performing a camera capture

[0677] The device uses the user's camera to capture the user's body measurements and pose.

[0678] Input: Device camera image

[0679] Output: Captured image or video data

[0680] Specific operation: Using OpenCV, capture camera footage with cv2.VideoCapture() and save the image with cv2.imwrite('user_image.jpg', frame).

[0681] Step 3: Virtual try-on with augmented reality

[0682] The device integrates the captured data with the three-dimensional model data received from the server and performs a virtual try-on using AR.

[0683] Input: Captured images and 3D model data

[0684] Output: Virtual try-on video

[0685] Specific operation: Using ARKit or ARCore, virtual clothing is overlaid on real-time video (e.g., ar_session.add(anchor)).

[0686] Step 4: Collect and submit feedback

[0687] The terminal provides an interface that allows the user to input feedback on the items they try on.

[0688] Input: User ratings and comments

[0689] Output: Sending feedback data

[0690] What it does: Collect feedback via an HTML form or in-app interface and send it to the server with requests.post('http: / / server_url / api / feedback', json=feedback_data).

[0691] User processing

[0692] Step 1: Log in and select an item

[0693] Users log in to the app using a smartphone or tablet and choose the items they want to try on from a list of suggestions.

[0694] Input: User ID and password

[0695] Output: Selected items

[0696] Specific behavior: Authenticate on the login screen and select an item from the suggestion list.

[0697] Step 2: Start the virtual try-on

[0698] The user captures their own body using the device's camera and begins the virtual try-on.

[0699] Input: Camera image

[0700] Output: Virtual try-on footage and feedback

[0701] Specific operations: While checking the video, select virtual clothing and try it on.

[0702] Step 3: Provide feedback

[0703] Users can enter and submit feedback on the items they try on within the app.

[0704] Input: Feedback data (e.g., size and color ratings)

[0705] Output: Feedback data sent to the server

[0706] Specific behavior: Enter feedback in the evaluation form within the app and press the submit button to send it to the server.

[0707] (Application example 1)

[0708] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0709] Traditional online shopping has the problem that users cannot actually try on products, which leads to mismatches in size and appearance, resulting in a high return rate. Furthermore, the lack of personalized suggestions makes it difficult to improve the user experience. To solve these issues, there is a need for a system that utilizes users' past purchase history and feedback data to make individually optimized fashion suggestions and enable virtual try-on sessions at home.

[0710] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0711] In this invention, the server includes means for collecting a user's past purchase history and feedback data, means for preprocessing the collected data and extracting features, means for learning user preferences using the preprocessed data and generating a predictive model, means for creating a personalized suggestion list for each user based on the predictive model, means for transmitting the suggestion list and 3D model data to the terminal, means for capturing the user's body using a camera in the terminal and performing a virtual try-on using the captured data and the 3D model data, means for collecting user feedback and transmitting it to the server, means for integrating the captured user's body dimension data and the 3D model data and performing a virtual try-on using augmented reality, and means for improving the suggestion list for future purchases using the user's feedback data. This allows users to efficiently try on and select products based on personalized suggestions, significantly improving the convenience and satisfaction of online shopping.

[0712] "User's past purchase history" is information about products purchased by the user in the past, and is data including purchase date and time, product name, category, price, quantity, and the like.

[0713] "Feedback data" is data collected by users providing ratings and comments about products they have purchased or tried on.

[0714] "Preprocessing" is the process of cleaning up the collected data, filling in missing values, and extracting important features.

[0715] "Features" are useful information extracted from data, and are variables or attributes used to learn user preferences and purchasing patterns.

[0716] A "predictive model" is a mathematical or algorithmic model built to learn user preferences and predict future purchasing behavior and tastes based on collected and preprocessed data.

[0717] A "suggestion list" is a list of products personalized for each user, generated based on a predictive model.

[0718] "3D Model Data" means data containing a three-dimensional representation of a product that is used to virtually try on the product.

[0719] A "terminal" is a computing device used by a user, such as a smartphone, tablet, or PC.

[0720] "Capturing the user's body using a camera" means capturing the user's body dimensions and pose using a camera mounted on the device and capturing them as data.

[0721] "Virtual try-on" is the integration of captured data and 3D model data, and the use of augmented reality technology to provide users with the experience of virtually trying on products.

[0722] "Augmented reality" is a technology that overlays digital elements onto real-world images to provide an interactive experience.

[0723] The "means for collecting feedback and transmitting it to the server" is a function for collecting data on users' try-on experiences and product evaluations, and transmitting the data to the server via a network.

[0724] "Means for improving the suggestion list from next time onwards" is a function that improves the algorithm for generating future suggestion lists based on feedback data collected from users, thereby achieving more advanced personalization.

[0725] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI to learn their preferences, thereby providing personalized fashion suggestions and enabling virtual try-on sessions at home. This system is realized through the cooperation of a server, a device, and a user.

[0726] Server-side behavior

[0727] The server first collects the user's past purchase history and feedback data from a database. This data is then preprocessed to fill in missing values ​​and extract important features. The preprocessed data is then used to train a generative AI model. Specifically, a deep learning algorithm is used to learn the user's preference patterns and generate a predictive model. Based on the trained model, a list of fashion suggestions optimized for each user is then generated and sent to the device, along with 3D model data for each product item.

[0728] Operation on the terminal side

[0729] The device receives the suggestion list and 3D model data from the server. When the user selects an item they want to try on from the suggestion list, the device uses its camera to capture the user's body measurements and pose. The captured data is processed in real time and used for the virtual try-on. Augmented reality (AR) technology is then used to integrate the captured data and the 3D model data, and the virtual garment is displayed superimposed on the user's real-time video. The device also provides an interface that collects feedback on the user's virtual try-on and sends it to the server.

[0730] User behavior

[0731] Users log in to the application using a smartphone or tablet and select the item they want to try on from a personalized list of suggestions provided by the server. Next, they use the device's camera to capture a photo of their body and begin the virtual try-on. During the virtual try-on, users can check the look and size of the product on the screen. They can then enter feedback on the item they tried on (e.g., whether the size fits, whether they like the color), and submit it within the app.

[0732] Specific examples

[0733] For example, suppose a user logs in and selects a blue shirt from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the blue shirt. The user checks how the shirt looks and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server. The server collects this feedback and reflects it in future suggestions.

[0734] Example prompts for generative AI models

[0735] Analyze users' past purchase history and feedback data to generate a personalized list of fashion suggestions. Provide 3D model data for each product.

[0736] This system allows users to efficiently try on and select products from the comfort of their own home, significantly improving the convenience and satisfaction of online shopping. Hardware used includes smartphones, tablets, and servers. Software used includes Python, OpenCV, Keras, REST API, and generative AI models.

[0737] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0738] Step 1:

[0739] The server collects users' past purchase history and feedback data from a database. The input is the purchase history data and feedback data in the database, and the output is the collected data. This data collection provides basic data for understanding users' preferences and behavioral patterns. Specifically, it executes queries to the database to retrieve related data.

[0740] Step 2:

[0741] The server preprocesses the collected data and extracts features. The input is the collected purchase history and feedback data, and the output is the preprocessed data and extracted features. Specific operations include data cleansing (filling in missing values ​​and removing outliers), normalization, and feature engineering. This converts the data into a format suitable for learning generative AI models.

[0742] Step 3:

[0743] The server uses the preprocessed data to train a generative AI model. The input is the preprocessed data and the generative AI model, and the output is a trained predictive model. Specifically, it uses a deep learning algorithm to learn the user's preference patterns and generate a predictive model. This builds a model that predicts the user's future purchasing behavior.

[0744] Step 4:

[0745] The server generates a personalized fashion suggestion list for each user based on the trained model and sends it to the device, including 3D model data for each product item. The input is the trained model and product database, and the output is a personalized suggestion list and 3D model data. Specifically, the server uses the trained model to evaluate the suitability of products for each user and generates an optimal product list.

[0746] Step 5:

[0747] The terminal receives the proposal list and 3D model data from the server. The input is the proposal list and 3D model data sent from the server, and the output is the received proposal list and 3D model data. Specifically, the terminal receives the data via the network and stores it in the terminal.

[0748] Step 6:

[0749] The user selects an item they want to try on from the suggestion list. The input is the suggestion list, and the output is the selected item. Specifically, the user selects an item using the interface within the application.

[0750] Step 7:

[0751] The device uses a camera to capture the user's body dimensions and pose. The input is the camera image, and the output is the captured body dimension data. Specifically, the device starts the camera and captures the user's entire body.

[0752] Step 8:

[0753] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology. The input is the captured data and 3D model data, and the output is a real-time video of the virtual try-on. Specifically, the device uses an AR engine to overlay the 3D model on the user's video and display it on the screen.

[0754] Step 9:

[0755] During the virtual try-on, the user checks the appearance and size of the product and provides feedback. The input is the user's confirmation result, and the output is feedback data. Specifically, the user checks the virtual try-on results on the screen and enters their rating and comments in the app's interface.

[0756] Step 10:

[0757] The terminal collects user feedback and transmits it to the server. The input is the user feedback data and the output is the transmitted feedback data. The specific operation is to collect the feedback data and transmit it to the server through the network.

[0758] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0759] This invention is a system that collects users' past purchase history and feedback data, and uses a generative AI and emotion engine to learn the user's preferences and emotions based on that data, thereby providing personalized fashion suggestions and enabling virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and the user.

[0760] Server-side behavior

[0761] 1. Data Collection and Preprocessing

[0762] The server collects the user's past purchase history and feedback data from a database.

[0763] The collected data is preprocessed, and data cleaning (removal of inappropriate data, completion of missing values) and feature extraction are performed.

[0764] 2. Model learning using generative AI

[0765] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[0766] 3. Emotion analysis using an emotion engine

[0767] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotion patterns into a model.

[0768] 4. Create a personalized suggestion list

[0769] Based on the learned preference model and emotion model, the server generates an optimized fashion suggestion list for each user.

[0770] The suggestion list includes 3D model data for each product item.

[0771] 5. Data transmission

[0772] The server sends the proposal list and 3D model data to the user's device.

[0773] Operation on the terminal side

[0774] 1. Data Reception

[0775] The terminal receives the proposal list and the 3D model data from the server.

[0776] 2. Performing camera capture

[0777] The device activates the user's camera and prepares to capture body measurements and poses.

[0778] 3. Emotion analysis

[0779] Using the device's camera and microphone, the emotion engine analyzes the user's facial expressions and voice to recognize the user's emotions in real time.

[0780] 4. Virtual try-on using augmented reality

[0781] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[0782] 5. Feedback Collection and Submission

[0783] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time.

[0784] Sends the user-entered feedback to the server.

[0785] User behavior

[0786] 1. Login and item selection

[0787] Users log in to the app using their smartphone or tablet and choose the items they want to try on from a list of suggestions.

[0788] 2. Start the virtual try-on

[0789] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[0790] 3. Emotion presentation

[0791] Users are also shown emotions (e.g., happy, excited) that the device recognizes in real time during the virtual try-on.

[0792] 4. Check the fitting results

[0793] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[0794] 5. Providing Feedback

[0795] Users enter feedback into an in-app form based on the items they try on and how they feel about them.

[0796] 6. Send Feedback

[0797] The user presses the "Submit" button to send the feedback to the server.

[0798] Specific examples

[0799] For example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. While trying it on, the device's camera and microphone analyze the user's facial expressions and voice using an emotion engine, which recognizes that the user has a positive emotional reaction to the dress. The user then enters their try-on experience (whether the size and color suit them) and their emotions (happy, excited, etc.) into the app and presses the "Send" button. This feedback is then sent to the server, which receives it as new data. The server learns from this feedback and reflects it in future suggestions.

[0800] The present invention not only allows users to efficiently try on and select products from the comfort of their own homes, but also allows them to receive personalized suggestions based on their emotional responses, further improving the convenience and satisfaction of online shopping.

[0801] The processing flow will be explained below.

[0802] Server-side processing steps

[0803] Step 1:

[0804] The server collects the user's past purchase history and feedback data from a database.

[0805] Step 2:

[0806] The server preprocesses the collected data, specifically cleaning the data (removing inappropriate data and filling in missing values) and extracting features.

[0807] Step 3:

[0808] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[0809] Step 4:

[0810] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotion patterns into a model.

[0811] Step 5:

[0812] The server generates a list of fashion suggestions optimized for each user based on the learned preference model and emotion model.

[0813] Step 6:

[0814] The server transmits the proposal list and the 3D model data to the user's terminal.

[0815] Terminal processing steps

[0816] Step 1:

[0817] The terminal receives the proposal list and the 3D model data from the server.

[0818] Step 2:

[0819] The device activates the user's camera and prepares to capture body measurements and poses.

[0820] Step 3:

[0821] The user stands in front of the camera, and the device captures the user's entire body and saves the data.

[0822] Step 4:

[0823] The device's emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotions.

[0824] Step 5:

[0825] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[0826] Step 6:

[0827] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time.

[0828] Step 7:

[0829] The feedback entered by the user is stored on the device and transmitted to the server.

[0830] User processing steps

[0831] Step 1:

[0832] Users log in to the app using their smartphone or tablet.

[0833] Step 2:

[0834] The user selects the product they wish to try on from the list of items suggested by the server.

[0835] Step 3:

[0836] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[0837] Step 4:

[0838] The user's facial expressions and voice are analyzed by the emotion engine, and emotions are recognized in real time.

[0839] Step 5:

[0840] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[0841] Step 6:

[0842] Users enter feedback into an in-app form based on the items they try on and how they feel about them.

[0843] Step 7:

[0844] The user presses the "Submit" button to send the feedback to the server.

[0845] Through these steps, the StyleSync+ AR Edition system provides personalized fashion suggestions based on the user's past data and real-time recognized emotions, and offers a virtual try-on experience at home.

[0846] Example 2

[0847] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0848] While online shopping has become commonplace in modern society, there are limited ways to check the actual fit and appearance of products before purchasing. This has led to an increase in returns and exchanges after purchase, resulting in a decline in user satisfaction. Furthermore, traditional methods are not sufficient to provide personalized product recommendations that take into account individual user preferences and emotions. This makes it difficult to improve the user experience and reduces the convenience of online shopping.

[0849] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0850] In this invention, the server includes means for collecting a user's past purchase history and feedback data, means for preprocessing the collected data and extracting features, means for learning the user's preferences using a deep learning algorithm with the preprocessed data and generating a predictive model, means for analyzing the user's past feedback data and identifying emotions, means for creating a personalized suggestion list for each user based on the generated predictive model and the results of the emotion analysis, and means for transmitting the suggestion list and 3D model data to a terminal. This allows users to efficiently try on and select products at home, as well as receive personalized suggestions based on their emotional reactions.

[0851] "User" refers to an individual who utilizes the system to virtually try on fashion items and provide feedback.

[0852] "Past purchase history" refers to a record of purchases made by the user up to now, and includes information such as product names, purchase dates, and quantities.

[0853] "Feedback data" refers to data including evaluations, impressions, and emotional expressions provided by users regarding products.

[0854] "Preprocessing" refers to a series of processes that convert collected data into a format suitable for analysis and learning, and includes data cleaning and feature extraction.

[0855] "Features" refer to important attributes or information extracted from data in data analysis and machine learning.

[0856] "Deep learning algorithm" refers to a type of machine learning that uses artificial neural networks to learn patterns from large amounts of data.

[0857] "Predictive Model" refers to a mathematical model trained to predict future preferences based on user tastes and behavior.

[0858] "Emotion engine" refers to technology that analyzes emotions from user feedback data and real-time facial expressions and voice.

[0859] A "personalized suggestion list" refers to a recommended list of specific fashion items created based on the preferences and emotional patterns of an individual user.

[0860] "3D model data" refers to digital data used to display fashion items in three dimensions, and includes formats such as glTF and OBJ.

[0861] "Terminal" refers to the device that a user uses to access the system and perform virtual try-on and emotion analysis, and primarily includes smartphones and tablets.

[0862] "Virtual try-on" refers to the process of virtually trying on clothes by combining captured data of the user's body with 3D model data.

[0863] "Augmented reality technology" refers to a technique that overlays digital information onto real-world images, providing users with an experience that makes them feel as if they are trying on clothes in real time.

[0864] "Feedback" refers to the ratings and impressions provided by users based on their fitting results and emotions, and includes data used to train the system.

[0865] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI and an emotion engine to learn the user's preferences and emotions based on that data, and then makes personalized fashion suggestions. This system is realized through the cooperation of a server, a terminal, and the user.

[0866] First, the server collects the user's past purchase history and feedback data from a database (e.g., MySQL, PostgreSQL). The collected data is preprocessed to remove inappropriate data, fill in missing values, and extract features. During this preprocessing stage, the user's purchasing trends and product category information are analyzed.

[0867] The server then uses deep learning frameworks such as TensorFlow and PyTorch to train a generative AI model from the preprocessed data, which learns user preference patterns based on algorithms such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs).

[0868] Furthermore, the server utilizes an emotion engine (e.g., IBM Watson, Microsoft Azure Emotion API) to analyze emotions from user feedback data. This emotion analysis result is then combined with a generative AI model to create a personalized recommendation list based on emotions.

[0869] The generated recommendation list includes 3D model data (e.g., glTF, OBJ format) for each product item, and this data is sent to the user's device using a secure protocol (e.g., HTTPS).

[0870] The device receives the proposal list and 3D model data from the server. Then, it activates its camera to capture the user's body dimensions and pose, and acquires this data using a depth camera (e.g., Microsoft Kinect) or a standard camera. At the same time, it uses an emotion engine (e.g., OpenCV, Azure Cognitive Services) to analyze the user's facial expressions and voice and recognize emotions in real time.

[0871] The collected capture data and 3D model data are integrated, and the device uses augmented reality technology (e.g., ARKit, ARCore) to perform a virtual try-on experience, allowing users to see virtual clothing overlaid on real-world footage.

[0872] Users can use an interface on their device to provide feedback based on their fitting experience and their feelings. The feedback they provide is then sent back to the server and recorded as new data. This feedback is used in the system's overall learning process and is reflected in future recommendations.

[0873] As a concrete example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. While trying it on, the device's camera and microphone analyze the user's facial expressions and voice using an emotion engine, which recognizes that the user has a positive emotional reaction to the dress. The user then enters their try-on experience (whether the size and color suit them) and their emotions (happy, excited, etc.) into the app and presses the "Submit" button. This sends the feedback to the server and receives it as new data. The server learns from this data and reflects it in future suggestions.

[0874] An example prompt is, "The user chooses a red dress, performs a virtual try-on, and records the positive emotion analyzed by the emotion engine."

[0875] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0876] Processing flow

[0877] Server-side behavior

[0878] Step 1:

[0879] Data collection and preprocessing

[0880] The server collects users' past purchase history and feedback data from a database as input. Since the collected data may contain inaccurate data, it performs data cleaning. It removes inappropriate data and fills in missing values ​​to organize the information. It also extracts features and tags each product item with the category and user purchasing trends. The preprocessed dataset is obtained as output.

[0881] Step 2:

[0882] Model learning using generative AI models

[0883] The server uses the preprocessed dataset as input to train a generative AI model using a deep learning framework (e.g., TensorFlow or PyTorch). It uses a recurrent neural network (RNN) or convolutional neural network (CNN) to learn user preference patterns. The output is a predictive model that reflects user preferences.

[0884] Step 3:

[0885] Emotion analysis using an emotion engine

[0886] The server takes the user's past feedback data as input and analyzes the emotions using an emotion engine (e.g., IBM Watson or Microsoft Azure Emotion API). It uses natural language processing (NLP) technology to analyze the feedback text and assign emotion labels such as positive or negative. The output is emotion-labeled feedback data.

[0887] Step 4:

[0888] Creating a personalized suggestion list

[0889] The server uses the generative AI model and the sentiment analysis results as inputs to generate a personalized fashion suggestion list. This suggestion list also includes 3D model data (e.g., glTF, OBJ format) for each product item. The output is the personalized suggestion list and 3D model data.

[0890] Step 5:

[0891] Sending data

[0892] The server takes the generated proposal list and 3D model data as input and sends them to the user's device. The data is securely transmitted using a secure protocol (e.g., HTTPS). As output, the proposal list and 3D model data sent to the device are obtained.

[0893] Operation on the terminal side

[0894] Step 1:

[0895] Data reception

[0896] The terminal receives as input the suggestion list and 3D model data sent from the server, prepares the received data for display, and obtains as output the suggestion list and 3D model data prepared in a displayable format.

[0897] Step 2:

[0898] Performing a camera capture

[0899] The device accepts user operations as input, activates a camera to capture the user's body dimensions and pose, and obtains body capture data using a depth camera (e.g., Microsoft Kinect) or a standard camera. The device obtains the capture data as output.

[0900] Step 3:

[0901] Emotion analysis

[0902] The device uses a camera and microphone as input, and analyzes the user's facial expressions and voice using an emotion engine (e.g., OpenCV, Azure Cognitive Services). The device recognizes the user's emotions in real time, and obtains real-time analyzed emotion data as output.

[0903] Step 4:

[0904] Augmented reality virtual try-on

[0905] The device integrates the captured data and 3D model data as input, and performs a virtual try-on using augmented reality technology (e.g., ARKit, ARCore). The device overlays the suggested clothing on a real-time video of the user. The output is a video of the virtual try-on using augmented reality technology.

[0906] Step 5:

[0907] Feedback collection and submission

[0908] The terminal accepts feedback based on the fitting results and emotions entered by the user and sends it to the server. It displays a feedback collection interface, prompts the user to enter information, and sends the entered data to the server using a secure protocol (e.g., HTTPS). The feedback data sent to the server is obtained as output.

[0909] User behavior

[0910] Step 1:

[0911] Login and item selection

[0912] As input, users log in to the app using their smartphone or tablet and select the items they want to try on from a list of suggestions. As output, they receive the selected items.

[0913] Step 2:

[0914] Start a virtual try-on

[0915] The user uses the device's camera as input to capture their body and begin the virtual try-on. The user stands in front of the camera, which captures and saves their entire body. The captured data is obtained as output.

[0916] Step 3:

[0917] emotion presentation

[0918] As input, the user confirms the emotions recognized by the device in real time during the virtual try-on. As output, real-time emotion data is obtained.

[0919] Step 4:

[0920] Check the fitting results

[0921] The user checks the appearance and size of the virtually tried-on clothes while looking at the screen of the device as input, and receives the confirmed fitting results as output.

[0922] Step 5:

[0923] Providing Feedback

[0924] The user inputs the items they tried on and their feedback based on the emotions they felt at the time into a form within the app. The input feedback data is obtained as output.

[0925] Step 6:

[0926] Send Feedback

[0927] As input, the user presses the "Send" button to send the feedback data to the server. The data is sent using a secure protocol (e.g., HTTPS). As output, the user gets the feedback data sent to the server.

[0928] (Application example 2)

[0929] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0930] In conventional online shopping environments, it is difficult for users to check the actual feel and fit of products before purchasing, which often leads to mistaken purchases and returns. Furthermore, product suggestions based on user preferences and emotions are rarely made, limiting the provision of a personalized shopping experience. This leads to issues such as lower user satisfaction and the inability to fully utilize the convenience of online shopping.

[0931] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0932] In this invention, the server includes: means for collecting a user's past purchase history and feedback data; means for preprocessing the collected data and extracting features; means for learning the user's preferences using the preprocessed data and generating a predictive model using a generative AI model; means for creating a personalized recommendation list for each user based on the predictive model; and means for transmitting the recommendation list and 3D model data to the terminal. This allows the user to not only receive personalized product recommendations but also virtually try on products at home using augmented reality technology. The system also includes means for analyzing the user's emotions in real time using a camera and microphone to improve the accuracy of the personalized recommendations, and means for collecting feedback based on the try-on results and emotions and transmitting the results to the server. This improves user satisfaction and convenience in online shopping.

[0933] "User's past purchase history" refers to information about products and services purchased by the user in the past.

[0934] "Feedback data" refers to data on evaluations and opinions regarding satisfaction and usability collected from users.

[0935] "Preprocessing" refers to the process of converting collected data into a form suitable for analysis.

[0936] "Features" refer to important attributes or characteristics of the data input into a machine learning model.

[0937] A "generative AI model" refers to an artificial intelligence model that uses technologies such as deep learning to learn and predict user preferences and behavior.

[0938] "Predictive model" refers to an algorithm created by a generative AI model to predict user behavior and preferences.

[0939] A "personalized suggestion list" refers to a list of products and services optimized based on a user's individual preferences and past purchasing history.

[0940] "3D model data" refers to the digital representation of the three-dimensional shape of a product.

[0941] "Augmented reality technology" refers to the technology of overlaying digital information onto images of the real world.

[0942] "Camera" refers to a device that captures images or videos.

[0943] "Capture data" refers to image and video data captured using a camera.

[0944] "Analyzing user emotions in real time" refers to the process of instantly analyzing data such as the user's facial expressions and voice to evaluate their current emotional state.

[0945] "Feedback" refers to opinions and ratings provided by users regarding their experience and satisfaction with the product.

[0946] This invention is a system that collects users' past purchase history and feedback data, and uses a generative AI model and emotion engine to learn the user's preferences and emotions based on that data, thereby providing personalized fashion suggestions and enabling virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and the user.

[0947] Server-side behavior

[0948] Data collection and preprocessing

[0949] The server collects users' past purchase history and feedback data from a database. It preprocesses the collected data, cleaning it (removing inappropriate data and filling in missing values) and extracting features. This is done using Python, with libraries such as Pandas and Numpy used for data processing.

[0950] Model learning using generative AI models

[0951] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms and frameworks such as TensorFlow and Keras to learn user preference patterns.

[0952] Emotion analysis

[0953] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotional patterns into a model using a deep learning model specialized for emotion analysis.

[0954] Creating a personalized suggestion list

[0955] Based on the trained preference and emotion models, the server generates a personalized fashion suggestion list, including 3D model data for each product item, which is updated in real time.

[0956] Sending data

[0957] The server communicates data using, for example, a REST API to send the proposal list and 3D model data to the user's device.

[0958] Operation on the terminal side

[0959] Data reception

[0960] The device receives the proposal list and 3D model data from the server using HTTP requests.

[0961] Camera and microphone activation and capture

[0962] The device activates the user's camera and microphone, capturing body measurements and poses, and preparing to analyze emotions in real time, using libraries such as OpenCV and MediaPipe.

[0963] Performing a virtual try-on

[0964] The device integrates the captured data and 3D model data, and performs a virtual try-on using augmented reality technology. Using Unity or ARKit / ARCore, the device overlays suggested clothing on the user's real-time video.

[0965] Feedback collection and submission

[0966] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time. An HTTP POST request is used to send the user-entered feedback to the server.

[0967] User behavior

[0968] Log in and receive personalized offers

[0969] The user logs in to the app using their smartphone or tablet and receives a list of suggested items from the server. This operation requires the user's account information.

[0970] Start a virtual try-on

[0971] Users can capture their body using the device's camera and begin the virtual try-on experience, giving them the experience of trying on products in real life, without leaving the comfort of their own home.

[0972] Real-time emotion display

[0973] The device uses a camera and microphone to analyze the user's emotions in real time while trying on clothes, and displays that information on the screen, allowing the user to check their own emotions.

[0974] Checking fitting results and providing feedback

[0975] The user checks the appearance and size of the clothes they try on on the device screen and inputs the results as feedback into the device. This feedback is sent to the server and reflected in future suggestions.

[0976] Examples of prompt statements

[0977] When a user logs in to receive personalized suggestions, the prompt text looks like this:

[0978] Text format

[0979] Get personalized fashion suggestions based on user ID 12345's past purchase history and feedback data.

[0980] In this way, the present invention provides personalized fashion suggestions based on the user's tastes and emotions, providing an enhanced online shopping experience.

[0981] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0982] Step 1:

[0983] The server collects the user's past purchase history and feedback data from the database. The input is the user ID, and the output is the past purchase history and feedback data. Specifically, it uses SQL queries to extract relevant data from the database.

[0984] Step 2:

[0985] The server preprocesses the collected data and extracts features. The input is the collected purchase history and feedback data, and the output is a cleaned dataset and features. Specific operations include cleaning the data (filling in missing values ​​and removing inappropriate data) and extracting features using Pandas.

[0986] Step 3:

[0987] The server uses the preprocessed data to train a generative AI model. The input is the preprocessed data and features, and the output is a trained model. Specifically, it learns user preference patterns using deep learning algorithms using TensorFlow and Keras.

[0988] Step 4:

[0989] The server uses an emotion engine to analyze emotions from the user's past feedback data. The input is the feedback data, and the output is emotion features. Specifically, it extracts emotions from text data using natural language processing technology (e.g., an emotion analysis model).

[0990] Step 5:

[0991] The server generates a personalized fashion suggestion list for each user based on the trained preference and emotion models. The input is the trained model, and the output is the suggestion list and 3D model data. Specifically, it selects items that match the user's preferences and creates a list including the 3D model data.

[0992] Step 6:

[0993] The server sends the proposal list and 3D model data to the device. The input is the proposal list and 3D model data, and the output is the successful data transmission to the device. Specifically, the data is sent using the REST API.

[0994] Step 7:

[0995] The device receives the proposal list and 3D model data from the server. The input is the proposal list sent from the server, and the output is the proposal list displayed on the device. Specifically, the data is received using an HTTP GET request.

[0996] Step 8:

[0997] The device activates the camera and microphone, and prepares to capture the user's body and analyze emotions in real time. The input is real-time data from the camera and microphone, and the output is captured data and emotion data. Specifically, the device uses OpenCV to capture camera images and perform emotion analysis in real time.

[0998] Step 9:

[0999] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology. The input is the captured data and 3D model data, and the output is an image of the virtual try-on. Specifically, using Unity or ARKit / ARCore, the suggested clothing is overlaid on the user's real-time video.

[1000] Step 10:

[1001] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their emotions. The input is the user's feedback, and the output is the collected feedback data. Specifically, the terminal displays a GUI form to receive user input.

[1002] Step 11:

[1003] The terminal sends the feedback input by the user to the server. The input is the feedback data, and the output is the successful data transmission to the server. Specifically, the feedback data is sent using an HTTP POST request.

[1004] Step 12:

[1005] The server stores the received feedback data in a database and updates the data to reflect future proposals. The input is the feedback data, and the output is the updated database. Specifically, it inserts new data and updates existing data using SQL queries.

[1006] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1007] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1008] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1009] [Third embodiment]

[1010] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1011] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1012] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1013] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1014] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1015] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1016] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1017] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1018] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1019] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1020] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1021] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1022] This invention provides a system that collects users' past purchase history and feedback data, learns their preferences using generative AI, and provides personalized fashion suggestions that enable virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and a user.

[1023] Server-side behavior

[1024] 1. Data Collection and Preprocessing

[1025] The server collects the user's past purchase history and feedback data from a database.

[1026] Clean the collected data, impute missing values, and extract important features.

[1027] 2. Model learning using generative AI

[1028] The server uses the preprocessed data to train a generative AI model, specifically, to learn user preference patterns using a deep learning algorithm.

[1029] 3. Create a personalized suggestion list

[1030] Based on the trained model, the server generates an optimized list of fashion suggestions for each user.

[1031] The suggestion list includes 3D model data for each product item.

[1032] 4. Data transmission

[1033] The server sends the proposal list and 3D model data to the user's device.

[1034] Operation on the terminal side

[1035] 1. Data Reception

[1036] The terminal receives the proposal list and the 3D model data received from the server.

[1037] 2. Performing camera capture

[1038] The device uses the user's camera to capture the user's body measurements and pose.

[1039] The captured data is stored and used for virtual try-on.

[1040] 3. Virtual try-on using augmented reality

[1041] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using AR.

[1042] Virtual clothing is superimposed on a real-time video image of the user.

[1043] 4. Feedback Collection and Submission

[1044] The terminal provides an interface that allows the user to input feedback on the items they try on.

[1045] Sends the user-entered feedback to the server.

[1046] User behavior

[1047] 1. Login and item selection

[1048] Users log in to the app using a smartphone or tablet and choose the items they want to try on from a list of suggestions.

[1049] 2. Start the virtual try-on

[1050] The user captures their own body using the device's camera and begins the virtual try-on.

[1051] During the virtual try-on, users can check the appearance and size of the product on the screen.

[1052] 3. Providing Feedback

[1053] Users enter feedback about the items they try on (e.g., whether the size fits or whether they like the color) and submit it within the app.

[1054] Specific examples

[1055] For example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. The user checks how the dress looks and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server. The server collects this feedback and reflects it in future suggestions.

[1056] The present invention allows users to efficiently try on and select products from the comfort of their own homes, significantly improving the convenience and satisfaction of online shopping.

[1057] The processing flow will be explained below.

[1058] Server-side processing steps

[1059] Step 1:

[1060] The server collects the user's past purchase history and feedback data from a database.

[1061] Step 2:

[1062] The server preprocesses the collected data, specifically cleaning the data (removing inappropriate data and filling in missing values) and extracting features.

[1063] Step 3:

[1064] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[1065] Step 4:

[1066] The server uses the trained model to generate an optimized fashion suggestion list for each user.

[1067] Step 5:

[1068] The server transmits the proposal list and the 3D model data to the user's terminal.

[1069] Terminal processing steps

[1070] Step 1:

[1071] The terminal receives the proposal list and the 3D model data from the server.

[1072] Step 2:

[1073] The device activates the user's camera and prepares to capture body measurements and poses.

[1074] Step 3:

[1075] The user stands in front of the camera, and the device captures the user's entire body and saves the data.

[1076] Step 4:

[1077] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[1078] Step 5:

[1079] The terminal displays an interface that allows the user to provide feedback on the results of the try-on.

[1080] Step 6:

[1081] The feedback entered by the user is stored on the device and transmitted to the server.

[1082] User processing steps

[1083] Step 1:

[1084] Users log in to the app using their smartphone or tablet.

[1085] Step 2:

[1086] The user selects the product they wish to try on from the list of items suggested by the server.

[1087] Step 3:

[1088] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[1089] Step 4:

[1090] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[1091] Step 5:

[1092] Users enter feedback on the items they try on into a form within the app.

[1093] Step 6:

[1094] The user presses the "Submit" button to send the feedback to the server.

[1095] Through these steps, the StyleSync+ AR Edition system will provide personalized fashion suggestions based on the user's past data and offer a virtual try-on experience at home.

[1096] Example 1

[1097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1098] Conventional online fashion sales systems do not allow users to actually try on products, which can lead to problems such as the size not fitting or the color or design being different from what was imagined after purchase. Furthermore, it is difficult to provide personalized suggestions that fully take into account the user's preferences, which can result in low user satisfaction. This leads to an increase in returns and exchanges, causing inconvenience to both sellers and consumers. It is also difficult to effectively utilize user feedback and incorporate it into future suggestions.

[1099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1100] In this invention, the server includes: means for collecting a user's past purchase history and feedback data; means for preprocessing the collected data and extracting features; means for learning the user's preferences using the preprocessed data and generating a predictive model; means for creating a personalized proposal list for each user based on the predictive model; means for transmitting the proposal list and three-dimensional model data to a terminal; means for capturing the user's body using a camera on the terminal and performing a virtual try-on using the captured data and the three-dimensional model data; and means for collecting user feedback and transmitting it to the server. This allows the user to virtually try on products at home and check the fit and design before purchasing. Furthermore, future proposals are improved based on the collected feedback, resulting in personalized fashion proposals.

[1101] A "user" is an entity that utilizes the system to browse, select, try on, and purchase products.

[1102] "Past purchase history" is data including information on products purchased by the user in the past.

[1103] "Feedback data" refers to data including ratings, comments, and opinions provided by users regarding products.

[1104] "Data collection means" is a mechanism for collecting users' past purchasing history and feedback data.

[1105] "Preprocessing means" is a mechanism for preparing collected data, complementing missing values, and extracting features.

[1106] "Features" are important elements extracted from data for analysis and prediction.

[1107] A "predictive model" is a computational model constructed to predict future behavior and preferences based on a user's preferences and behavior.

[1108] A "personalized suggestion list" is a list of product suggestions optimized for each user based on a predictive model.

[1109] "Three-dimensional model data" refers to three-dimensional digital representation data of a product used for virtual try-on.

[1110] A "terminal" is a device operated by a user, and is an electronic device such as a smartphone or tablet that includes a camera function and a display.

[1111] The "camera capture means" is a mechanism that uses the device's camera to capture the user's body measurements and pose.

[1112] "Virtual try-on" refers to the act of virtually trying on a product using user capture data and three-dimensional model data.

[1113] "Augmented reality technology" is a technology that displays digital information overlaid on images of the real world.

[1114] The "feedback collection means" is a mechanism by which users input their ratings and opinions about products and transmit that information to a server.

[1115] MODE FOR CARRYING OUT THE INVENTION

[1116] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI to learn their preferences, thereby providing personalized fashion suggestions and enabling virtual try-on sessions at home. This system is realized through the cooperation of a server, a device, and a user.

[1117] Server-side behavior

[1118] The server first collects the user's past purchase history and feedback data from a database. This collection is done using a database query method such as SQL. The collected data is then preprocessed, missing values ​​are filled in, and features are extracted. This is often done using data analysis libraries such as Python's Pandas or Numpy.

[1119] Using the preprocessed data, the server trains a generative AI model using deep learning frameworks such as TensorFlow and PyTorch. For example, the model is applied to learn the user's preference patterns, and a personalized recommendation list is generated based on the trained model. This recommendation list includes 3D model data for each product item.

[1120] The proposal list and 3D model data are sent from the server to the user's device using a REST API, with data exchanged in JSON format.

[1121] Operation on the terminal side

[1122] The device receives and stores the proposal list and 3D model data from the server. The device then uses the user's camera to capture the user's body dimensions and pose. The camera capture uses a computer vision library such as OpenCV. The captured data is used for virtual try-on and is then stored.

[1123] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using augmented reality technology. Using frameworks such as ARKit (iOS) and ARCore (Android), the device overlays the virtual clothing on a real-time video image.

[1124] After the user has completed the virtual try-on, the device provides an interface for collecting user feedback, which is then sent to the server and used to improve future proposals.

[1125] User behavior

[1126] Users log in to the app using a smartphone or tablet and select the items they want to try on from a list of suggestions. Standard authentication methods are used for login. Next, users capture their entire body using the device's camera and begin the virtual try-on. They can view the virtual garments in real-time AR footage and evaluate the product's appearance and fit.

[1127] After the virtual try-on, the user enters feedback on the item they tried on in the app. For example, they rate it on whether the size fits or whether they like the color, and this information is sent to the server. The server can use this feedback to improve its suggestions for future purchases.

[1128] Specific examples

[1129] For example, suppose a user logs in to their smartphone and selects a red dress from a list of items suggested by the server. The user uses a camera to capture their body and uses augmented reality technology to virtually try on the red dress. The user checks how the dress looks on them and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server and reflected in future suggestions.

[1130] Prompt Sentence Examples

[1131] "Build a generative AI model that provides personalized fashion recommendations based on users' past purchase history and feedback data."

[1132] This invention allows users to efficiently try on and select products from the comfort of their own homes, significantly improving the convenience and satisfaction of online shopping.

[1133] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1134] Program processing flow

[1135] Server-side processing

[1136] Step 1: Data collection

[1137] The server collects the user's past purchase history and feedback data from a database.

[1138] Input: SQL query (e.g. SELECT FROM purchase_history WHERE user_id = ?)

[1139] Output: User purchase history and feedback data

[1140] What happens: The server connects to the database and executes an SQL query to retrieve the data.

[1141] Step 2: Data Preprocessing

[1142] The server cleans the collected data, imputes missing values, and extracts features.

[1143] Input: Purchase history and feedback data

[1144] Output: Preprocessed data

[1145] Specific operation: Create a data frame using the Pandas library, fill in missing values ​​with df.fillna(method='ffill'), and extract the necessary features.

[1146] Step 3: Training the generative AI model

[1147] The server uses the preprocessed data to train a generative AI model.

[1148] Input: Preprocessed data

[1149] Output: Trained model

[1150] Specific operation: Build a deep learning model using TensorFlow or PyTorch and train it with model.fit(data, labels).

[1151] Step 4: Create a list of suggestions

[1152] The server generates a list of suggestions optimized for each user based on the trained model.

[1153] Input: trained model, user data

[1154] Output: Proposal list and 3D model data

[1155] Specific operation: Create a suggestion list with predictions = model.predict(user_data) and load the related 3D model data from a file.

[1156] Step 5: Sending data

[1157] The server transmits the proposal list and the three-dimensional model data to the user's terminal.

[1158] Input: Proposal list and 3D model data

[1159] Output: None (transmission completion status)

[1160] Specific operation: Using the REST API, send data with requests.post('http: / / terminal URL / api', json=data).

[1161] Terminal side processing

[1162] Step 1: Receiving data

[1163] The terminal receives the proposal list and the three-dimensional model data received from the server.

[1164] Input: Data from the server

[1165] Output: Save the proposal list and 3D model data

[1166] Specific operation: Using the Flask framework, receive and save data using request.get_json().

[1167] Step 2: Performing a camera capture

[1168] The device uses the user's camera to capture the user's body measurements and pose.

[1169] Input: Device camera image

[1170] Output: Captured image or video data

[1171] Specific operation: Using OpenCV, capture camera footage with cv2.VideoCapture() and save the image with cv2.imwrite('user_image.jpg', frame).

[1172] Step 3: Virtual try-on with augmented reality

[1173] The device integrates the captured data with the three-dimensional model data received from the server and performs a virtual try-on using AR.

[1174] Input: Captured images and 3D model data

[1175] Output: Virtual try-on video

[1176] Specific operation: Using ARKit or ARCore, virtual clothing is overlaid on real-time video (e.g., ar_session.add(anchor)).

[1177] Step 4: Collect and submit feedback

[1178] The terminal provides an interface that allows the user to input feedback on the items they try on.

[1179] Input: User ratings and comments

[1180] Output: Sending feedback data

[1181] What it does: Collect feedback via an HTML form or in-app interface and send it to the server with requests.post('http: / / server_url / api / feedback', json=feedback_data).

[1182] User processing

[1183] Step 1: Log in and select an item

[1184] Users log in to the app using a smartphone or tablet and choose the items they want to try on from a list of suggestions.

[1185] Input: User ID and password

[1186] Output: Selected items

[1187] Specific behavior: Authenticate on the login screen and select an item from the suggestion list.

[1188] Step 2: Start the virtual try-on

[1189] The user captures their own body using the device's camera and begins the virtual try-on.

[1190] Input: Camera image

[1191] Output: Virtual try-on footage and feedback

[1192] Specific operations: While checking the video, select virtual clothing and try it on.

[1193] Step 3: Provide feedback

[1194] Users can enter and submit feedback on the items they try on within the app.

[1195] Input: Feedback data (e.g., size and color ratings)

[1196] Output: Feedback data sent to the server

[1197] Specific behavior: Enter feedback in the evaluation form within the app and press the submit button to send it to the server.

[1198] (Application example 1)

[1199] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1200] Traditional online shopping has the problem that users cannot actually try on products, which leads to mismatches in size and appearance, resulting in a high return rate. Furthermore, the lack of personalized suggestions makes it difficult to improve the user experience. To solve these issues, there is a need for a system that utilizes users' past purchase history and feedback data to make individually optimized fashion suggestions and enable virtual try-on sessions at home.

[1201] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1202] In this invention, the server includes means for collecting a user's past purchase history and feedback data, means for preprocessing the collected data and extracting features, means for learning user preferences using the preprocessed data and generating a predictive model, means for creating a personalized suggestion list for each user based on the predictive model, means for transmitting the suggestion list and 3D model data to the terminal, means for capturing the user's body using a camera in the terminal and performing a virtual try-on using the captured data and the 3D model data, means for collecting user feedback and transmitting it to the server, means for integrating the captured user's body dimension data and the 3D model data and performing a virtual try-on using augmented reality, and means for improving the suggestion list for future purchases using the user's feedback data. This allows users to efficiently try on and select products based on personalized suggestions, significantly improving the convenience and satisfaction of online shopping.

[1203] "User's past purchase history" is information about products purchased by the user in the past, and is data including purchase date and time, product name, category, price, quantity, and the like.

[1204] "Feedback data" is data collected by users providing ratings and comments about products they have purchased or tried on.

[1205] "Preprocessing" is the process of cleaning up the collected data, filling in missing values, and extracting important features.

[1206] "Features" are useful information extracted from data, and are variables or attributes used to learn user preferences and purchasing patterns.

[1207] A "predictive model" is a mathematical or algorithmic model built to learn user preferences and predict future purchasing behavior and tastes based on collected and preprocessed data.

[1208] A "suggestion list" is a list of products personalized for each user, generated based on a predictive model.

[1209] "3D Model Data" means data containing a three-dimensional representation of a product that is used to virtually try on the product.

[1210] A "terminal" is a computing device used by a user, such as a smartphone, tablet, or PC.

[1211] "Capturing the user's body using a camera" means capturing the user's body dimensions and pose using a camera mounted on the device and capturing them as data.

[1212] "Virtual try-on" is the integration of captured data and 3D model data, and the use of augmented reality technology to provide users with the experience of virtually trying on products.

[1213] "Augmented reality" is a technology that overlays digital elements onto real-world images to provide an interactive experience.

[1214] The "means for collecting feedback and transmitting it to the server" is a function for collecting data on users' try-on experiences and product evaluations, and transmitting the data to the server via a network.

[1215] "Means for improving the suggestion list from next time onwards" is a function that improves the algorithm for generating future suggestion lists based on feedback data collected from users, thereby achieving more advanced personalization.

[1216] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI to learn their preferences, thereby providing personalized fashion suggestions and enabling virtual try-on sessions at home. This system is realized through the cooperation of a server, a device, and a user.

[1217] Server-side behavior

[1218] The server first collects the user's past purchase history and feedback data from a database. This data is then preprocessed to fill in missing values ​​and extract important features. The preprocessed data is then used to train a generative AI model. Specifically, a deep learning algorithm is used to learn the user's preference patterns and generate a predictive model. Based on the trained model, a list of fashion suggestions optimized for each user is then generated and sent to the device, along with 3D model data for each product item.

[1219] Operation on the terminal side

[1220] The device receives the suggestion list and 3D model data from the server. When the user selects an item they want to try on from the suggestion list, the device uses its camera to capture the user's body measurements and pose. The captured data is processed in real time and used for the virtual try-on. Augmented reality (AR) technology is then used to integrate the captured data and the 3D model data, and the virtual garment is displayed superimposed on the user's real-time video. The device also provides an interface that collects feedback on the user's virtual try-on and sends it to the server.

[1221] User behavior

[1222] Users log in to the application using a smartphone or tablet and select the item they want to try on from a personalized list of suggestions provided by the server. Next, they use the device's camera to capture a photo of their body and begin the virtual try-on. During the virtual try-on, users can check the look and size of the product on the screen. They can then enter feedback on the item they tried on (e.g., whether the size fits, whether they like the color), and submit it within the app.

[1223] Specific examples

[1224] For example, suppose a user logs in and selects a blue shirt from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the blue shirt. The user checks how the shirt looks and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server. The server collects this feedback and reflects it in future suggestions.

[1225] Example prompts for generative AI models

[1226] Analyze users' past purchase history and feedback data to generate a personalized list of fashion suggestions. Provide 3D model data for each product.

[1227] This system allows users to efficiently try on and select products from the comfort of their own home, significantly improving the convenience and satisfaction of online shopping. Hardware used includes smartphones, tablets, and servers. Software used includes Python, OpenCV, Keras, REST API, and generative AI models.

[1228] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1229] Step 1:

[1230] The server collects users' past purchase history and feedback data from a database. The input is the purchase history data and feedback data in the database, and the output is the collected data. This data collection provides basic data for understanding users' preferences and behavioral patterns. Specifically, it executes queries to the database to retrieve related data.

[1231] Step 2:

[1232] The server preprocesses the collected data and extracts features. The input is the collected purchase history and feedback data, and the output is the preprocessed data and extracted features. Specific operations include data cleansing (filling in missing values ​​and removing outliers), normalization, and feature engineering. This converts the data into a format suitable for learning generative AI models.

[1233] Step 3:

[1234] The server uses the preprocessed data to train a generative AI model. The input is the preprocessed data and the generative AI model, and the output is a trained predictive model. Specifically, it uses a deep learning algorithm to learn the user's preference patterns and generate a predictive model. This builds a model that predicts the user's future purchasing behavior.

[1235] Step 4:

[1236] The server generates a personalized fashion suggestion list for each user based on the trained model and sends it to the device, including 3D model data for each product item. The input is the trained model and product database, and the output is a personalized suggestion list and 3D model data. Specifically, the server uses the trained model to evaluate the suitability of products for each user and generates an optimal product list.

[1237] Step 5:

[1238] The terminal receives the proposal list and 3D model data from the server. The input is the proposal list and 3D model data sent from the server, and the output is the received proposal list and 3D model data. Specifically, the terminal receives the data via the network and stores it in the terminal.

[1239] Step 6:

[1240] The user selects an item they want to try on from the suggestion list. The input is the suggestion list, and the output is the selected item. Specifically, the user selects an item using the interface within the application.

[1241] Step 7:

[1242] The device uses a camera to capture the user's body dimensions and pose. The input is the camera image, and the output is the captured body dimension data. Specifically, the device starts the camera and captures the user's entire body.

[1243] Step 8:

[1244] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology. The input is the captured data and 3D model data, and the output is a real-time video of the virtual try-on. Specifically, the device uses an AR engine to overlay the 3D model on the user's video and display it on the screen.

[1245] Step 9:

[1246] During the virtual try-on, the user checks the appearance and size of the product and provides feedback. The input is the user's confirmation result, and the output is feedback data. Specifically, the user checks the virtual try-on results on the screen and enters their rating and comments in the app's interface.

[1247] Step 10:

[1248] The terminal collects user feedback and transmits it to the server. The input is the user feedback data and the output is the transmitted feedback data. The specific operation is to collect the feedback data and transmit it to the server through the network.

[1249] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1250] This invention is a system that collects users' past purchase history and feedback data, and uses a generative AI and emotion engine to learn the user's preferences and emotions based on that data, thereby providing personalized fashion suggestions and enabling virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and the user.

[1251] Server-side behavior

[1252] 1. Data Collection and Preprocessing

[1253] The server collects the user's past purchase history and feedback data from a database.

[1254] The collected data is preprocessed, and data cleaning (removal of inappropriate data, completion of missing values) and feature extraction are performed.

[1255] 2. Model learning using generative AI

[1256] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[1257] 3. Emotion analysis using an emotion engine

[1258] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotion patterns into a model.

[1259] 4. Create a personalized suggestion list

[1260] Based on the learned preference model and emotion model, the server generates an optimized fashion suggestion list for each user.

[1261] The suggestion list includes 3D model data for each product item.

[1262] 5. Data transmission

[1263] The server sends the proposal list and 3D model data to the user's device.

[1264] Operation on the terminal side

[1265] 1. Data Reception

[1266] The terminal receives the proposal list and the 3D model data from the server.

[1267] 2. Performing camera capture

[1268] The device activates the user's camera and prepares to capture body measurements and poses.

[1269] 3. Emotion analysis

[1270] Using the device's camera and microphone, the emotion engine analyzes the user's facial expressions and voice to recognize the user's emotions in real time.

[1271] 4. Virtual try-on using augmented reality

[1272] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[1273] 5. Feedback Collection and Submission

[1274] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time.

[1275] Sends the user-entered feedback to the server.

[1276] User behavior

[1277] 1. Login and item selection

[1278] Users log in to the app using their smartphone or tablet and choose the items they want to try on from a list of suggestions.

[1279] 2. Start the virtual try-on

[1280] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[1281] 3. Emotion presentation

[1282] Users are also shown emotions (e.g., happy, excited) that the device recognizes in real time during the virtual try-on.

[1283] 4. Check the fitting results

[1284] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[1285] 5. Providing Feedback

[1286] Users enter feedback into an in-app form based on the items they try on and how they feel about them.

[1287] 6. Send Feedback

[1288] The user presses the "Submit" button to send the feedback to the server.

[1289] Specific examples

[1290] For example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. While trying it on, the device's camera and microphone analyze the user's facial expressions and voice using an emotion engine, which recognizes that the user has a positive emotional reaction to the dress. The user then enters their try-on experience (whether the size and color suit them) and their emotions (happy, excited, etc.) into the app and presses the "Send" button. This feedback is then sent to the server, which receives it as new data. The server learns from this feedback and reflects it in future suggestions.

[1291] The present invention not only allows users to efficiently try on and select products from the comfort of their own homes, but also allows them to receive personalized suggestions based on their emotional responses, further improving the convenience and satisfaction of online shopping.

[1292] The processing flow will be explained below.

[1293] Server-side processing steps

[1294] Step 1:

[1295] The server collects the user's past purchase history and feedback data from a database.

[1296] Step 2:

[1297] The server preprocesses the collected data, specifically cleaning the data (removing inappropriate data and filling in missing values) and extracting features.

[1298] Step 3:

[1299] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[1300] Step 4:

[1301] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotion patterns into a model.

[1302] Step 5:

[1303] The server generates a list of fashion suggestions optimized for each user based on the learned preference model and emotion model.

[1304] Step 6:

[1305] The server transmits the proposal list and the 3D model data to the user's terminal.

[1306] Terminal processing steps

[1307] Step 1:

[1308] The terminal receives the proposal list and the 3D model data from the server.

[1309] Step 2:

[1310] The device activates the user's camera and prepares to capture body measurements and poses.

[1311] Step 3:

[1312] The user stands in front of the camera, and the device captures the user's entire body and saves the data.

[1313] Step 4:

[1314] The device's emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotions.

[1315] Step 5:

[1316] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[1317] Step 6:

[1318] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time.

[1319] Step 7:

[1320] The feedback entered by the user is stored on the device and transmitted to the server.

[1321] User processing steps

[1322] Step 1:

[1323] Users log in to the app using their smartphone or tablet.

[1324] Step 2:

[1325] The user selects the product they wish to try on from the list of items suggested by the server.

[1326] Step 3:

[1327] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[1328] Step 4:

[1329] The user's facial expressions and voice are analyzed by the emotion engine, and emotions are recognized in real time.

[1330] Step 5:

[1331] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[1332] Step 6:

[1333] Users enter feedback into an in-app form based on the items they try on and how they feel about them.

[1334] Step 7:

[1335] The user presses the "Submit" button to send the feedback to the server.

[1336] Through these steps, the StyleSync+ AR Edition system provides personalized fashion suggestions based on the user's past data and real-time recognized emotions, and offers a virtual try-on experience at home.

[1337] Example 2

[1338] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1339] While online shopping has become commonplace in modern society, there are limited ways to check the actual fit and appearance of products before purchasing. This has led to an increase in returns and exchanges after purchase, resulting in a decline in user satisfaction. Furthermore, traditional methods are not sufficient to provide personalized product recommendations that take into account individual user preferences and emotions. This makes it difficult to improve the user experience and reduces the convenience of online shopping.

[1340] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1341] In this invention, the server includes means for collecting a user's past purchase history and feedback data, means for preprocessing the collected data and extracting features, means for learning the user's preferences using a deep learning algorithm with the preprocessed data and generating a predictive model, means for analyzing the user's past feedback data and identifying emotions, means for creating a personalized suggestion list for each user based on the generated predictive model and the results of the emotion analysis, and means for transmitting the suggestion list and 3D model data to a terminal. This allows users to efficiently try on and select products at home, as well as receive personalized suggestions based on their emotional reactions.

[1342] "User" refers to an individual who utilizes the system to virtually try on fashion items and provide feedback.

[1343] "Past purchase history" refers to a record of purchases made by the user up to now, and includes information such as product names, purchase dates, and quantities.

[1344] "Feedback data" refers to data including evaluations, impressions, and emotional expressions provided by users regarding products.

[1345] "Preprocessing" refers to a series of processes that convert collected data into a format suitable for analysis and learning, and includes data cleaning and feature extraction.

[1346] "Features" refer to important attributes or information extracted from data in data analysis and machine learning.

[1347] "Deep learning algorithm" refers to a type of machine learning that uses artificial neural networks to learn patterns from large amounts of data.

[1348] "Predictive Model" refers to a mathematical model trained to predict future preferences based on user tastes and behavior.

[1349] "Emotion engine" refers to technology that analyzes emotions from user feedback data and real-time facial expressions and voice.

[1350] A "personalized suggestion list" refers to a recommended list of specific fashion items created based on the preferences and emotional patterns of an individual user.

[1351] "3D model data" refers to digital data used to display fashion items in three dimensions, and includes formats such as glTF and OBJ.

[1352] "Terminal" refers to the device that a user uses to access the system and perform virtual try-on and emotion analysis, and primarily includes smartphones and tablets.

[1353] "Virtual try-on" refers to the process of virtually trying on clothes by combining captured data of the user's body with 3D model data.

[1354] "Augmented reality technology" refers to a technique that overlays digital information onto real-world images, providing users with an experience that makes them feel as if they are trying on clothes in real time.

[1355] "Feedback" refers to the ratings and impressions provided by users based on their fitting results and emotions, and includes data used to train the system.

[1356] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI and an emotion engine to learn the user's preferences and emotions based on that data, and then makes personalized fashion suggestions. This system is realized through the cooperation of a server, a terminal, and the user.

[1357] First, the server collects the user's past purchase history and feedback data from a database (e.g., MySQL, PostgreSQL). The collected data is preprocessed to remove inappropriate data, fill in missing values, and extract features. During this preprocessing stage, the user's purchasing trends and product category information are analyzed.

[1358] The server then uses deep learning frameworks such as TensorFlow and PyTorch to train a generative AI model from the preprocessed data, which learns user preference patterns based on algorithms such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs).

[1359] Furthermore, the server utilizes an emotion engine (e.g., IBM Watson, Microsoft Azure Emotion API) to analyze emotions from user feedback data. This emotion analysis result is then combined with a generative AI model to create a personalized recommendation list based on emotions.

[1360] The generated recommendation list includes 3D model data (e.g., glTF, OBJ format) for each product item, and this data is sent to the user's device using a secure protocol (e.g., HTTPS).

[1361] The device receives the proposal list and 3D model data from the server. Then, it activates its camera to capture the user's body dimensions and pose, and acquires this data using a depth camera (e.g., Microsoft Kinect) or a standard camera. At the same time, it uses an emotion engine (e.g., OpenCV, Azure Cognitive Services) to analyze the user's facial expressions and voice and recognize emotions in real time.

[1362] The collected capture data and 3D model data are integrated, and the device uses augmented reality technology (e.g., ARKit, ARCore) to perform a virtual try-on experience, allowing users to see virtual clothing overlaid on real-world footage.

[1363] Users can use an interface on their device to provide feedback based on their fitting experience and their feelings. The feedback they provide is then sent back to the server and recorded as new data. This feedback is used in the system's overall learning process and is reflected in future recommendations.

[1364] As a concrete example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. While trying it on, the device's camera and microphone analyze the user's facial expressions and voice using an emotion engine, which recognizes that the user has a positive emotional reaction to the dress. The user then enters their try-on experience (whether the size and color suit them) and their emotions (happy, excited, etc.) into the app and presses the "Submit" button. This sends the feedback to the server and receives it as new data. The server learns from this data and reflects it in future suggestions.

[1365] An example prompt is, "The user chooses a red dress, performs a virtual try-on, and records the positive emotion analyzed by the emotion engine."

[1366] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1367] Processing flow

[1368] Server-side behavior

[1369] Step 1:

[1370] Data collection and preprocessing

[1371] The server collects users' past purchase history and feedback data from a database as input. Since the collected data may contain inaccurate data, it performs data cleaning. It removes inappropriate data and fills in missing values ​​to organize the information. It also extracts features and tags each product item with the category and user purchasing trends. The preprocessed dataset is obtained as output.

[1372] Step 2:

[1373] Model learning using generative AI models

[1374] The server uses the preprocessed dataset as input to train a generative AI model using a deep learning framework (e.g., TensorFlow or PyTorch). It uses a recurrent neural network (RNN) or convolutional neural network (CNN) to learn user preference patterns. The output is a predictive model that reflects user preferences.

[1375] Step 3:

[1376] Emotion analysis using an emotion engine

[1377] The server takes the user's past feedback data as input and analyzes the emotions using an emotion engine (e.g., IBM Watson or Microsoft Azure Emotion API). It uses natural language processing (NLP) technology to analyze the feedback text and assign emotion labels such as positive or negative. The output is emotion-labeled feedback data.

[1378] Step 4:

[1379] Creating a personalized suggestion list

[1380] The server uses the generative AI model and the sentiment analysis results as inputs to generate a personalized fashion suggestion list. This suggestion list also includes 3D model data (e.g., glTF, OBJ format) for each product item. The output is the personalized suggestion list and 3D model data.

[1381] Step 5:

[1382] Sending data

[1383] The server takes the generated proposal list and 3D model data as input and sends them to the user's device. The data is securely transmitted using a secure protocol (e.g., HTTPS). As output, the proposal list and 3D model data sent to the device are obtained.

[1384] Operation on the terminal side

[1385] Step 1:

[1386] Data reception

[1387] The terminal receives as input the suggestion list and 3D model data sent from the server, prepares the received data for display, and obtains as output the suggestion list and 3D model data prepared in a displayable format.

[1388] Step 2:

[1389] Performing a camera capture

[1390] The device accepts user operations as input, activates a camera to capture the user's body dimensions and pose, and obtains body capture data using a depth camera (e.g., Microsoft Kinect) or a standard camera. The device obtains the capture data as output.

[1391] Step 3:

[1392] Emotion analysis

[1393] The device uses a camera and microphone as input, and analyzes the user's facial expressions and voice using an emotion engine (e.g., OpenCV, Azure Cognitive Services). The device recognizes the user's emotions in real time, and obtains real-time analyzed emotion data as output.

[1394] Step 4:

[1395] Augmented reality virtual try-on

[1396] The device integrates the captured data and 3D model data as input, and performs a virtual try-on using augmented reality technology (e.g., ARKit, ARCore). The device overlays the suggested clothing on a real-time video of the user. The output is a video of the virtual try-on using augmented reality technology.

[1397] Step 5:

[1398] Feedback collection and submission

[1399] The terminal accepts feedback based on the fitting results and emotions entered by the user and sends it to the server. It displays a feedback collection interface, prompts the user to enter information, and sends the entered data to the server using a secure protocol (e.g., HTTPS). The feedback data sent to the server is obtained as output.

[1400] User behavior

[1401] Step 1:

[1402] Login and item selection

[1403] As input, users log in to the app using their smartphone or tablet and select the items they want to try on from a list of suggestions. As output, they receive the selected items.

[1404] Step 2:

[1405] Start a virtual try-on

[1406] The user uses the device's camera as input to capture their body and begin the virtual try-on. The user stands in front of the camera, which captures and saves their entire body. The captured data is obtained as output.

[1407] Step 3:

[1408] emotion presentation

[1409] As input, the user confirms the emotions recognized by the device in real time during the virtual try-on. As output, real-time emotion data is obtained.

[1410] Step 4:

[1411] Check the fitting results

[1412] The user checks the appearance and size of the virtually tried-on clothes while looking at the screen of the device as input, and receives the confirmed fitting results as output.

[1413] Step 5:

[1414] Providing Feedback

[1415] The user inputs the items they tried on and their feedback based on the emotions they felt at the time into a form within the app. The input feedback data is obtained as output.

[1416] Step 6:

[1417] Send Feedback

[1418] As input, the user presses the "Send" button to send the feedback data to the server. The data is sent using a secure protocol (e.g., HTTPS). As output, the user gets the feedback data sent to the server.

[1419] (Application example 2)

[1420] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1421] In conventional online shopping environments, it is difficult for users to check the actual feel and fit of products before purchasing, which often leads to mistaken purchases and returns. Furthermore, product suggestions based on user preferences and emotions are rarely made, limiting the provision of a personalized shopping experience. This leads to issues such as lower user satisfaction and the inability to fully utilize the convenience of online shopping.

[1422] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1423] In this invention, the server includes: means for collecting a user's past purchase history and feedback data; means for preprocessing the collected data and extracting features; means for learning the user's preferences using the preprocessed data and generating a predictive model using a generative AI model; means for creating a personalized recommendation list for each user based on the predictive model; and means for transmitting the recommendation list and 3D model data to the terminal. This allows the user to not only receive personalized product recommendations but also virtually try on products at home using augmented reality technology. The system also includes means for analyzing the user's emotions in real time using a camera and microphone to improve the accuracy of the personalized recommendations, and means for collecting feedback based on the try-on results and emotions and transmitting the results to the server. This improves user satisfaction and convenience in online shopping.

[1424] "User's past purchase history" refers to information about products and services purchased by the user in the past.

[1425] "Feedback data" refers to data on evaluations and opinions regarding satisfaction and usability collected from users.

[1426] "Preprocessing" refers to the process of converting collected data into a form suitable for analysis.

[1427] "Features" refer to important attributes or characteristics of the data input into a machine learning model.

[1428] A "generative AI model" refers to an artificial intelligence model that uses technologies such as deep learning to learn and predict user preferences and behavior.

[1429] "Predictive model" refers to an algorithm created by a generative AI model to predict user behavior and preferences.

[1430] A "personalized suggestion list" refers to a list of products and services optimized based on a user's individual preferences and past purchasing history.

[1431] "3D model data" refers to the digital representation of the three-dimensional shape of a product.

[1432] "Augmented reality technology" refers to the technology of overlaying digital information onto images of the real world.

[1433] "Camera" refers to a device that captures images or videos.

[1434] "Capture data" refers to image and video data captured using a camera.

[1435] "Analyzing user emotions in real time" refers to the process of instantly analyzing data such as the user's facial expressions and voice to evaluate their current emotional state.

[1436] "Feedback" refers to opinions and ratings provided by users regarding their experience and satisfaction with the product.

[1437] This invention is a system that collects users' past purchase history and feedback data, and uses a generative AI model and emotion engine to learn the user's preferences and emotions based on that data, thereby providing personalized fashion suggestions and enabling virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and the user.

[1438] Server-side behavior

[1439] Data collection and preprocessing

[1440] The server collects users' past purchase history and feedback data from a database. It preprocesses the collected data, cleaning it (removing inappropriate data and filling in missing values) and extracting features. This is done using Python, with libraries such as Pandas and Numpy used for data processing.

[1441] Model learning using generative AI models

[1442] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms and frameworks such as TensorFlow and Keras to learn user preference patterns.

[1443] Emotion analysis

[1444] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotional patterns into a model using a deep learning model specialized for emotion analysis.

[1445] Creating a personalized suggestion list

[1446] Based on the trained preference and emotion models, the server generates a personalized fashion suggestion list, including 3D model data for each product item, which is updated in real time.

[1447] Sending data

[1448] The server communicates data using, for example, a REST API to send the proposal list and 3D model data to the user's device.

[1449] Operation on the terminal side

[1450] Data reception

[1451] The device receives the proposal list and 3D model data from the server using HTTP requests.

[1452] Camera and microphone activation and capture

[1453] The device activates the user's camera and microphone, capturing body measurements and poses, and preparing to analyze emotions in real time, using libraries such as OpenCV and MediaPipe.

[1454] Performing a virtual try-on

[1455] The device integrates the captured data and 3D model data, and performs a virtual try-on using augmented reality technology. Using Unity or ARKit / ARCore, the device overlays suggested clothing on the user's real-time video.

[1456] Feedback collection and submission

[1457] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time. An HTTP POST request is used to send the user-entered feedback to the server.

[1458] User behavior

[1459] Log in and receive personalized offers

[1460] The user logs in to the app using their smartphone or tablet and receives a list of suggested items from the server. This operation requires the user's account information.

[1461] Start a virtual try-on

[1462] Users can capture their body using the device's camera and begin the virtual try-on experience, giving them the experience of trying on products in real life, without leaving the comfort of their own home.

[1463] Real-time emotion display

[1464] The device uses a camera and microphone to analyze the user's emotions in real time while trying on clothes, and displays that information on the screen, allowing the user to check their own emotions.

[1465] Checking fitting results and providing feedback

[1466] The user checks the appearance and size of the clothes they try on on the device screen and inputs the results as feedback into the device. This feedback is sent to the server and reflected in future suggestions.

[1467] Examples of prompt statements

[1468] When a user logs in to receive personalized suggestions, the prompt text looks like this:

[1469] Text format

[1470] Get personalized fashion suggestions based on user ID 12345's past purchase history and feedback data.

[1471] In this way, the present invention provides personalized fashion suggestions based on the user's tastes and emotions, providing an enhanced online shopping experience.

[1472] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1473] Step 1:

[1474] The server collects the user's past purchase history and feedback data from the database. The input is the user ID, and the output is the past purchase history and feedback data. Specifically, it uses SQL queries to extract relevant data from the database.

[1475] Step 2:

[1476] The server preprocesses the collected data and extracts features. The input is the collected purchase history and feedback data, and the output is a cleaned dataset and features. Specific operations include cleaning the data (filling in missing values ​​and removing inappropriate data) and extracting features using Pandas.

[1477] Step 3:

[1478] The server uses the preprocessed data to train a generative AI model. The input is the preprocessed data and features, and the output is a trained model. Specifically, it learns user preference patterns using deep learning algorithms using TensorFlow and Keras.

[1479] Step 4:

[1480] The server uses an emotion engine to analyze emotions from the user's past feedback data. The input is the feedback data, and the output is emotion features. Specifically, it extracts emotions from text data using natural language processing technology (e.g., an emotion analysis model).

[1481] Step 5:

[1482] The server generates a personalized fashion suggestion list for each user based on the trained preference and emotion models. The input is the trained model, and the output is the suggestion list and 3D model data. Specifically, it selects items that match the user's preferences and creates a list including the 3D model data.

[1483] Step 6:

[1484] The server sends the proposal list and 3D model data to the device. The input is the proposal list and 3D model data, and the output is the successful data transmission to the device. Specifically, the data is sent using the REST API.

[1485] Step 7:

[1486] The device receives the proposal list and 3D model data from the server. The input is the proposal list sent from the server, and the output is the proposal list displayed on the device. Specifically, the data is received using an HTTP GET request.

[1487] Step 8:

[1488] The device activates the camera and microphone, and prepares to capture the user's body and analyze emotions in real time. The input is real-time data from the camera and microphone, and the output is captured data and emotion data. Specifically, the device uses OpenCV to capture camera images and perform emotion analysis in real time.

[1489] Step 9:

[1490] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology. The input is the captured data and 3D model data, and the output is an image of the virtual try-on. Specifically, using Unity or ARKit / ARCore, the suggested clothing is overlaid on the user's real-time video.

[1491] Step 10:

[1492] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their emotions. The input is the user's feedback, and the output is the collected feedback data. Specifically, the terminal displays a GUI form to receive user input.

[1493] Step 11:

[1494] The terminal sends the feedback input by the user to the server. The input is the feedback data, and the output is the successful data transmission to the server. Specifically, the feedback data is sent using an HTTP POST request.

[1495] Step 12:

[1496] The server stores the received feedback data in a database and updates the data to reflect future proposals. The input is the feedback data, and the output is the updated database. Specifically, it inserts new data and updates existing data using SQL queries.

[1497] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1498] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1499] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1500] [Fourth embodiment]

[1501] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1502] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1503] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1504] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1505] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1506] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1507] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1508] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1509] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1510] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1511] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1512] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1513] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1514] This invention provides a system that collects users' past purchase history and feedback data, learns their preferences using generative AI, and provides personalized fashion suggestions that enable virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and a user.

[1515] Server-side behavior

[1516] 1. Data Collection and Preprocessing

[1517] The server collects the user's past purchase history and feedback data from a database.

[1518] Clean the collected data, impute missing values, and extract important features.

[1519] 2. Model learning using generative AI

[1520] The server uses the preprocessed data to train a generative AI model, specifically, to learn user preference patterns using a deep learning algorithm.

[1521] 3. Create a personalized suggestion list

[1522] Based on the trained model, the server generates an optimized list of fashion suggestions for each user.

[1523] The suggestion list includes 3D model data for each product item.

[1524] 4. Data transmission

[1525] The server sends the proposal list and 3D model data to the user's device.

[1526] Operation on the terminal side

[1527] 1. Data Reception

[1528] The terminal receives the proposal list and the 3D model data received from the server.

[1529] 2. Performing camera capture

[1530] The device uses the user's camera to capture the user's body measurements and pose.

[1531] The captured data is stored and used for virtual try-on.

[1532] 3. Virtual try-on using augmented reality

[1533] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using AR.

[1534] Virtual clothing is superimposed on a real-time video image of the user.

[1535] 4. Feedback Collection and Submission

[1536] The terminal provides an interface that allows the user to input feedback on the items they try on.

[1537] Sends the user-entered feedback to the server.

[1538] User behavior

[1539] 1. Login and item selection

[1540] Users log in to the app using a smartphone or tablet and choose the items they want to try on from a list of suggestions.

[1541] 2. Start the virtual try-on

[1542] The user captures their own body using the device's camera and begins the virtual try-on.

[1543] During the virtual try-on, users can check the appearance and size of the product on the screen.

[1544] 3. Providing Feedback

[1545] Users enter feedback about the items they try on (e.g., whether the size fits or whether they like the color) and submit it within the app.

[1546] Specific examples

[1547] For example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. The user checks how the dress looks and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server. The server collects this feedback and reflects it in future suggestions.

[1548] The present invention allows users to efficiently try on and select products from the comfort of their own homes, significantly improving the convenience and satisfaction of online shopping.

[1549] The processing flow will be explained below.

[1550] Server-side processing steps

[1551] Step 1:

[1552] The server collects the user's past purchase history and feedback data from a database.

[1553] Step 2:

[1554] The server preprocesses the collected data, specifically cleaning the data (removing inappropriate data and filling in missing values) and extracting features.

[1555] Step 3:

[1556] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[1557] Step 4:

[1558] The server uses the trained model to generate an optimized fashion suggestion list for each user.

[1559] Step 5:

[1560] The server transmits the proposal list and the 3D model data to the user's terminal.

[1561] Terminal processing steps

[1562] Step 1:

[1563] The terminal receives the proposal list and the 3D model data from the server.

[1564] Step 2:

[1565] The device activates the user's camera and prepares to capture body measurements and poses.

[1566] Step 3:

[1567] The user stands in front of the camera, and the device captures the user's entire body and saves the data.

[1568] Step 4:

[1569] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[1570] Step 5:

[1571] The terminal displays an interface that allows the user to provide feedback on the results of the try-on.

[1572] Step 6:

[1573] The feedback entered by the user is stored on the device and transmitted to the server.

[1574] User processing steps

[1575] Step 1:

[1576] Users log in to the app using their smartphone or tablet.

[1577] Step 2:

[1578] The user selects the product they wish to try on from the list of items suggested by the server.

[1579] Step 3:

[1580] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[1581] Step 4:

[1582] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[1583] Step 5:

[1584] Users enter feedback on the items they try on into a form within the app.

[1585] Step 6:

[1586] The user presses the "Submit" button to send the feedback to the server.

[1587] Through these steps, the StyleSync+ AR Edition system will provide personalized fashion suggestions based on the user's past data and offer a virtual try-on experience at home.

[1588] Example 1

[1589] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1590] Conventional online fashion sales systems do not allow users to actually try on products, which can lead to problems such as the size not fitting or the color or design being different from what was imagined after purchase. Furthermore, it is difficult to provide personalized suggestions that fully take into account the user's preferences, which can result in low user satisfaction. This leads to an increase in returns and exchanges, causing inconvenience to both sellers and consumers. It is also difficult to effectively utilize user feedback and incorporate it into future suggestions.

[1591] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1592] In this invention, the server includes: means for collecting a user's past purchase history and feedback data; means for preprocessing the collected data and extracting features; means for learning the user's preferences using the preprocessed data and generating a predictive model; means for creating a personalized proposal list for each user based on the predictive model; means for transmitting the proposal list and three-dimensional model data to a terminal; means for capturing the user's body using a camera on the terminal and performing a virtual try-on using the captured data and the three-dimensional model data; and means for collecting user feedback and transmitting it to the server. This allows the user to virtually try on products at home and check the fit and design before purchasing. Furthermore, future proposals are improved based on the collected feedback, resulting in personalized fashion proposals.

[1593] A "user" is an entity that utilizes the system to browse, select, try on, and purchase products.

[1594] "Past purchase history" is data including information on products purchased by the user in the past.

[1595] "Feedback data" refers to data including ratings, comments, and opinions provided by users regarding products.

[1596] "Data collection means" is a mechanism for collecting users' past purchasing history and feedback data.

[1597] "Preprocessing means" is a mechanism for preparing collected data, complementing missing values, and extracting features.

[1598] "Features" are important elements extracted from data for analysis and prediction.

[1599] A "predictive model" is a computational model constructed to predict future behavior and preferences based on a user's preferences and behavior.

[1600] A "personalized suggestion list" is a list of product suggestions optimized for each user based on a predictive model.

[1601] "Three-dimensional model data" refers to three-dimensional digital representation data of a product used for virtual try-on.

[1602] A "terminal" is a device operated by a user, and is an electronic device such as a smartphone or tablet that includes a camera function and a display.

[1603] The "camera capture means" is a mechanism that uses the device's camera to capture the user's body measurements and pose.

[1604] "Virtual try-on" refers to the act of virtually trying on a product using user capture data and three-dimensional model data.

[1605] "Augmented reality technology" is a technology that displays digital information overlaid on images of the real world.

[1606] The "feedback collection means" is a mechanism by which users input their ratings and opinions about products and transmit that information to a server.

[1607] MODE FOR CARRYING OUT THE INVENTION

[1608] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI to learn their preferences, thereby providing personalized fashion suggestions and enabling virtual try-on sessions at home. This system is realized through the cooperation of a server, a device, and a user.

[1609] Server-side behavior

[1610] The server first collects the user's past purchase history and feedback data from a database. This collection is done using a database query method such as SQL. The collected data is then preprocessed, missing values ​​are filled in, and features are extracted. This is often done using data analysis libraries such as Python's Pandas or Numpy.

[1611] Using the preprocessed data, the server trains a generative AI model using deep learning frameworks such as TensorFlow and PyTorch. For example, the model is applied to learn the user's preference patterns, and a personalized recommendation list is generated based on the trained model. This recommendation list includes 3D model data for each product item.

[1612] The proposal list and 3D model data are sent from the server to the user's device using a REST API, with data exchanged in JSON format.

[1613] Operation on the terminal side

[1614] The device receives and stores the proposal list and 3D model data from the server. The device then uses the user's camera to capture the user's body dimensions and pose. The camera capture uses a computer vision library such as OpenCV. The captured data is used for virtual try-on and is then stored.

[1615] The device integrates the captured data with the 3D model data received from the server and performs a virtual try-on using augmented reality technology. Using frameworks such as ARKit (iOS) and ARCore (Android), the device overlays the virtual clothing on a real-time video image.

[1616] After the user has completed the virtual try-on, the device provides an interface for collecting user feedback, which is then sent to the server and used to improve future proposals.

[1617] User behavior

[1618] Users log in to the app using a smartphone or tablet and select the items they want to try on from a list of suggestions. Standard authentication methods are used for login. Next, users capture their entire body using the device's camera and begin the virtual try-on. They can view the virtual garments in real-time AR footage and evaluate the product's appearance and fit.

[1619] After the virtual try-on, the user enters feedback on the item they tried on in the app. For example, they rate it on whether the size fits or whether they like the color, and this information is sent to the server. The server can use this feedback to improve its suggestions for future purchases.

[1620] Specific examples

[1621] For example, suppose a user logs in to their smartphone and selects a red dress from a list of items suggested by the server. The user uses a camera to capture their body and uses augmented reality technology to virtually try on the red dress. The user checks how the dress looks on them and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server and reflected in future suggestions.

[1622] Prompt Sentence Examples

[1623] "Build a generative AI model that provides personalized fashion recommendations based on users' past purchase history and feedback data."

[1624] This invention allows users to efficiently try on and select products from the comfort of their own homes, significantly improving the convenience and satisfaction of online shopping.

[1625] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1626] Program processing flow

[1627] Server-side processing

[1628] Step 1: Data collection

[1629] The server collects the user's past purchase history and feedback data from a database.

[1630] Input: SQL query (e.g. SELECT FROM purchase_history WHERE user_id = ?)

[1631] Output: User purchase history and feedback data

[1632] What happens: The server connects to the database and executes an SQL query to retrieve the data.

[1633] Step 2: Data Preprocessing

[1634] The server cleans the collected data, imputes missing values, and extracts features.

[1635] Input: Purchase history and feedback data

[1636] Output: Preprocessed data

[1637] Specific operation: Create a data frame using the Pandas library, fill in missing values ​​with df.fillna(method='ffill'), and extract the necessary features.

[1638] Step 3: Training the generative AI model

[1639] The server uses the preprocessed data to train a generative AI model.

[1640] Input: Preprocessed data

[1641] Output: Trained model

[1642] Specific operation: Build a deep learning model using TensorFlow or PyTorch and train it with model.fit(data, labels).

[1643] Step 4: Create a list of suggestions

[1644] The server generates a list of suggestions optimized for each user based on the trained model.

[1645] Input: trained model, user data

[1646] Output: Proposal list and 3D model data

[1647] Specific operation: Create a suggestion list with predictions = model.predict(user_data) and load the related 3D model data from a file.

[1648] Step 5: Sending data

[1649] The server transmits the proposal list and the three-dimensional model data to the user's terminal.

[1650] Input: Proposal list and 3D model data

[1651] Output: None (transmission completion status)

[1652] Specific operation: Using the REST API, send data with requests.post('http: / / terminal URL / api', json=data).

[1653] Terminal side processing

[1654] Step 1: Receiving data

[1655] The terminal receives the proposal list and the three-dimensional model data received from the server.

[1656] Input: Data from the server

[1657] Output: Save the proposal list and 3D model data

[1658] Specific operation: Using the Flask framework, receive and save data using request.get_json().

[1659] Step 2: Performing a camera capture

[1660] The device uses the user's camera to capture the user's body measurements and pose.

[1661] Input: Device camera image

[1662] Output: Captured image or video data

[1663] Specific operation: Using OpenCV, capture camera footage with cv2.VideoCapture() and save the image with cv2.imwrite('user_image.jpg', frame).

[1664] Step 3: Virtual try-on with augmented reality

[1665] The device integrates the captured data with the three-dimensional model data received from the server and performs a virtual try-on using AR.

[1666] Input: Captured images and 3D model data

[1667] Output: Virtual try-on video

[1668] Specific operation: Using ARKit or ARCore, virtual clothing is overlaid on real-time video (e.g., ar_session.add(anchor)).

[1669] Step 4: Collect and submit feedback

[1670] The terminal provides an interface that allows the user to input feedback on the items they try on.

[1671] Input: User ratings and comments

[1672] Output: Sending feedback data

[1673] What it does: Collect feedback via an HTML form or in-app interface and send it to the server with requests.post('http: / / server_url / api / feedback', json=feedback_data).

[1674] User processing

[1675] Step 1: Log in and select an item

[1676] Users log in to the app using a smartphone or tablet and choose the items they want to try on from a list of suggestions.

[1677] Input: User ID and password

[1678] Output: Selected items

[1679] Specific behavior: Authenticate on the login screen and select an item from the suggestion list.

[1680] Step 2: Start the virtual try-on

[1681] The user captures their own body using the device's camera and begins the virtual try-on.

[1682] Input: Camera image

[1683] Output: Virtual try-on footage and feedback

[1684] Specific operations: While checking the video, select virtual clothing and try it on.

[1685] Step 3: Provide feedback

[1686] Users can enter and submit feedback on the items they try on within the app.

[1687] Input: Feedback data (e.g., size and color ratings)

[1688] Output: Feedback data sent to the server

[1689] Specific behavior: Enter feedback in the evaluation form within the app and press the submit button to send it to the server.

[1690] (Application example 1)

[1691] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1692] Traditional online shopping has the problem that users cannot actually try on products, which leads to mismatches in size and appearance, resulting in a high return rate. Furthermore, the lack of personalized suggestions makes it difficult to improve the user experience. To solve these issues, there is a need for a system that utilizes users' past purchase history and feedback data to make individually optimized fashion suggestions and enable virtual try-on sessions at home.

[1693] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1694] In this invention, the server includes means for collecting a user's past purchase history and feedback data, means for preprocessing the collected data and extracting features, means for learning user preferences using the preprocessed data and generating a predictive model, means for creating a personalized suggestion list for each user based on the predictive model, means for transmitting the suggestion list and 3D model data to the terminal, means for capturing the user's body using a camera in the terminal and performing a virtual try-on using the captured data and the 3D model data, means for collecting user feedback and transmitting it to the server, means for integrating the captured user's body dimension data and the 3D model data and performing a virtual try-on using augmented reality, and means for improving the suggestion list for future purchases using the user's feedback data. This allows users to efficiently try on and select products based on personalized suggestions, significantly improving the convenience and satisfaction of online shopping.

[1695] "User's past purchase history" is information about products purchased by the user in the past, and is data including purchase date and time, product name, category, price, quantity, and the like.

[1696] "Feedback data" is data collected by users providing ratings and comments about products they have purchased or tried on.

[1697] "Preprocessing" is the process of cleaning up the collected data, filling in missing values, and extracting important features.

[1698] "Features" are useful information extracted from data, and are variables or attributes used to learn user preferences and purchasing patterns.

[1699] A "predictive model" is a mathematical or algorithmic model built to learn user preferences and predict future purchasing behavior and tastes based on collected and preprocessed data.

[1700] A "suggestion list" is a list of products personalized for each user, generated based on a predictive model.

[1701] "3D Model Data" means data containing a three-dimensional representation of a product that is used to virtually try on the product.

[1702] A "terminal" is a computing device used by a user, such as a smartphone, tablet, or PC.

[1703] "Capturing the user's body using a camera" means capturing the user's body dimensions and pose using a camera mounted on the device and capturing them as data.

[1704] "Virtual try-on" is the integration of captured data and 3D model data, and the use of augmented reality technology to provide users with the experience of virtually trying on products.

[1705] "Augmented reality" is a technology that overlays digital elements onto real-world images to provide an interactive experience.

[1706] The "means for collecting feedback and transmitting it to the server" is a function for collecting data on users' try-on experiences and product evaluations, and transmitting the data to the server via a network.

[1707] "Means for improving the suggestion list from next time onwards" is a function that improves the algorithm for generating future suggestion lists based on feedback data collected from users, thereby achieving more advanced personalization.

[1708] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI to learn their preferences, thereby providing personalized fashion suggestions and enabling virtual try-on sessions at home. This system is realized through the cooperation of a server, a device, and a user.

[1709] Server-side behavior

[1710] The server first collects the user's past purchase history and feedback data from a database. This data is then preprocessed to fill in missing values ​​and extract important features. The preprocessed data is then used to train a generative AI model. Specifically, a deep learning algorithm is used to learn the user's preference patterns and generate a predictive model. Based on the trained model, a list of fashion suggestions optimized for each user is then generated and sent to the device, along with 3D model data for each product item.

[1711] Operation on the terminal side

[1712] The device receives the suggestion list and 3D model data from the server. When the user selects an item they want to try on from the suggestion list, the device uses its camera to capture the user's body measurements and pose. The captured data is processed in real time and used for the virtual try-on. Augmented reality (AR) technology is then used to integrate the captured data and the 3D model data, and the virtual garment is displayed superimposed on the user's real-time video. The device also provides an interface that collects feedback on the user's virtual try-on and sends it to the server.

[1713] User behavior

[1714] Users log in to the application using a smartphone or tablet and select the item they want to try on from a personalized list of suggestions provided by the server. Next, they use the device's camera to capture a photo of their body and begin the virtual try-on. During the virtual try-on, users can check the look and size of the product on the screen. They can then enter feedback on the item they tried on (e.g., whether the size fits, whether they like the color), and submit it within the app.

[1715] Specific examples

[1716] For example, suppose a user logs in and selects a blue shirt from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the blue shirt. The user checks how the shirt looks and evaluates whether the size and color suit them. After entering their evaluation results in the app and pressing the "Submit" button, this feedback is sent to the server. The server collects this feedback and reflects it in future suggestions.

[1717] Example prompts for generative AI models

[1718] Analyze users' past purchase history and feedback data to generate a personalized list of fashion suggestions. Provide 3D model data for each product.

[1719] This system allows users to efficiently try on and select products from the comfort of their own home, significantly improving the convenience and satisfaction of online shopping. Hardware used includes smartphones, tablets, and servers. Software used includes Python, OpenCV, Keras, REST API, and generative AI models.

[1720] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1721] Step 1:

[1722] The server collects users' past purchase history and feedback data from a database. The input is the purchase history data and feedback data in the database, and the output is the collected data. This data collection provides basic data for understanding users' preferences and behavioral patterns. Specifically, it executes queries to the database to retrieve related data.

[1723] Step 2:

[1724] The server preprocesses the collected data and extracts features. The input is the collected purchase history and feedback data, and the output is the preprocessed data and extracted features. Specific operations include data cleansing (filling in missing values ​​and removing outliers), normalization, and feature engineering. This converts the data into a format suitable for learning generative AI models.

[1725] Step 3:

[1726] The server uses the preprocessed data to train a generative AI model. The input is the preprocessed data and the generative AI model, and the output is a trained predictive model. Specifically, it uses a deep learning algorithm to learn the user's preference patterns and generate a predictive model. This builds a model that predicts the user's future purchasing behavior.

[1727] Step 4:

[1728] The server generates a personalized fashion suggestion list for each user based on the trained model and sends it to the device, including 3D model data for each product item. The input is the trained model and product database, and the output is a personalized suggestion list and 3D model data. Specifically, the server uses the trained model to evaluate the suitability of products for each user and generates an optimal product list.

[1729] Step 5:

[1730] The terminal receives the proposal list and 3D model data from the server. The input is the proposal list and 3D model data sent from the server, and the output is the received proposal list and 3D model data. Specifically, the terminal receives the data via the network and stores it in the terminal.

[1731] Step 6:

[1732] The user selects an item they want to try on from the suggestion list. The input is the suggestion list, and the output is the selected item. Specifically, the user selects an item using the interface within the application.

[1733] Step 7:

[1734] The device uses a camera to capture the user's body dimensions and pose. The input is the camera image, and the output is the captured body dimension data. Specifically, the device starts the camera and captures the user's entire body.

[1735] Step 8:

[1736] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology. The input is the captured data and 3D model data, and the output is a real-time video of the virtual try-on. Specifically, the device uses an AR engine to overlay the 3D model on the user's video and display it on the screen.

[1737] Step 9:

[1738] During the virtual try-on, the user checks the appearance and size of the product and provides feedback. The input is the user's confirmation result, and the output is feedback data. Specifically, the user checks the virtual try-on results on the screen and enters their rating and comments in the app's interface.

[1739] Step 10:

[1740] The terminal collects user feedback and transmits it to the server. The input is the user feedback data and the output is the transmitted feedback data. The specific operation is to collect the feedback data and transmit it to the server through the network.

[1741] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1742] This invention is a system that collects users' past purchase history and feedback data, and uses a generative AI and emotion engine to learn the user's preferences and emotions based on that data, thereby providing personalized fashion suggestions and enabling virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and the user.

[1743] Server-side behavior

[1744] 1. Data Collection and Preprocessing

[1745] The server collects the user's past purchase history and feedback data from a database.

[1746] The collected data is preprocessed, and data cleaning (removal of inappropriate data, completion of missing values) and feature extraction are performed.

[1747] 2. Model learning using generative AI

[1748] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[1749] 3. Emotion analysis using an emotion engine

[1750] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotion patterns into a model.

[1751] 4. Create a personalized suggestion list

[1752] Based on the learned preference model and emotion model, the server generates an optimized fashion suggestion list for each user.

[1753] The suggestion list includes 3D model data for each product item.

[1754] 5. Data transmission

[1755] The server sends the proposal list and 3D model data to the user's device.

[1756] Operation on the terminal side

[1757] 1. Data Reception

[1758] The terminal receives the proposal list and the 3D model data from the server.

[1759] 2. Performing camera capture

[1760] The device activates the user's camera and prepares to capture body measurements and poses.

[1761] 3. Emotion analysis

[1762] Using the device's camera and microphone, the emotion engine analyzes the user's facial expressions and voice to recognize the user's emotions in real time.

[1763] 4. Virtual try-on using augmented reality

[1764] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[1765] 5. Feedback Collection and Submission

[1766] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time.

[1767] Sends the user-entered feedback to the server.

[1768] User behavior

[1769] 1. Login and item selection

[1770] Users log in to the app using their smartphone or tablet and choose the items they want to try on from a list of suggestions.

[1771] 2. Start the virtual try-on

[1772] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[1773] 3. Emotion presentation

[1774] Users are also shown emotions (e.g., happy, excited) that the device recognizes in real time during the virtual try-on.

[1775] 4. Check the fitting results

[1776] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[1777] 5. Providing Feedback

[1778] Users enter feedback into an in-app form based on the items they try on and how they feel about them.

[1779] 6. Send Feedback

[1780] The user presses the "Submit" button to send the feedback to the server.

[1781] Specific examples

[1782] For example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. While trying it on, the device's camera and microphone analyze the user's facial expressions and voice using an emotion engine, which recognizes that the user has a positive emotional reaction to the dress. The user then enters their try-on experience (whether the size and color suit them) and their emotions (happy, excited, etc.) into the app and presses the "Send" button. This feedback is then sent to the server, which receives it as new data. The server learns from this feedback and reflects it in future suggestions.

[1783] The present invention not only allows users to efficiently try on and select products from the comfort of their own homes, but also allows them to receive personalized suggestions based on their emotional responses, further improving the convenience and satisfaction of online shopping.

[1784] The processing flow will be explained below.

[1785] Server-side processing steps

[1786] Step 1:

[1787] The server collects the user's past purchase history and feedback data from a database.

[1788] Step 2:

[1789] The server preprocesses the collected data, specifically cleaning the data (removing inappropriate data and filling in missing values) and extracting features.

[1790] Step 3:

[1791] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms to learn user preference patterns.

[1792] Step 4:

[1793] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotion patterns into a model.

[1794] Step 5:

[1795] The server generates a list of fashion suggestions optimized for each user based on the learned preference model and emotion model.

[1796] Step 6:

[1797] The server transmits the proposal list and the 3D model data to the user's terminal.

[1798] Terminal processing steps

[1799] Step 1:

[1800] The terminal receives the proposal list and the 3D model data from the server.

[1801] Step 2:

[1802] The device activates the user's camera and prepares to capture body measurements and poses.

[1803] Step 3:

[1804] The user stands in front of the camera, and the device captures the user's entire body and saves the data.

[1805] Step 4:

[1806] The device's emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotions.

[1807] Step 5:

[1808] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology, overlaying suggested clothing on the user's real-time video.

[1809] Step 6:

[1810] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time.

[1811] Step 7:

[1812] The feedback entered by the user is stored on the device and transmitted to the server.

[1813] User processing steps

[1814] Step 1:

[1815] Users log in to the app using their smartphone or tablet.

[1816] Step 2:

[1817] The user selects the product they wish to try on from the list of items suggested by the server.

[1818] Step 3:

[1819] The user begins the virtual try-on by capturing a picture of their body using the device's camera. They stand in front of the camera and their entire body is captured and saved.

[1820] Step 4:

[1821] The user's facial expressions and voice are analyzed by the emotion engine, and emotions are recognized in real time.

[1822] Step 5:

[1823] The user checks the appearance and size of the clothes being virtually tried on while looking at the screen of the terminal.

[1824] Step 6:

[1825] Users enter feedback into an in-app form based on the items they try on and how they feel about them.

[1826] Step 7:

[1827] The user presses the "Submit" button to send the feedback to the server.

[1828] Through these steps, the StyleSync+ AR Edition system provides personalized fashion suggestions based on the user's past data and real-time recognized emotions, and offers a virtual try-on experience at home.

[1829] Example 2

[1830] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1831] While online shopping has become commonplace in modern society, there are limited ways to check the actual fit and appearance of products before purchasing. This has led to an increase in returns and exchanges after purchase, resulting in a decline in user satisfaction. Furthermore, traditional methods are not sufficient to provide personalized product recommendations that take into account individual user preferences and emotions. This makes it difficult to improve the user experience and reduces the convenience of online shopping.

[1832] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1833] In this invention, the server includes means for collecting a user's past purchase history and feedback data, means for preprocessing the collected data and extracting features, means for learning the user's preferences using a deep learning algorithm with the preprocessed data and generating a predictive model, means for analyzing the user's past feedback data and identifying emotions, means for creating a personalized suggestion list for each user based on the generated predictive model and the results of the emotion analysis, and means for transmitting the suggestion list and 3D model data to a terminal. This allows users to efficiently try on and select products at home, as well as receive personalized suggestions based on their emotional reactions.

[1834] "User" refers to an individual who utilizes the system to virtually try on fashion items and provide feedback.

[1835] "Past purchase history" refers to a record of purchases made by the user up to now, and includes information such as product names, purchase dates, and quantities.

[1836] "Feedback data" refers to data including evaluations, impressions, and emotional expressions provided by users regarding products.

[1837] "Preprocessing" refers to a series of processes that convert collected data into a format suitable for analysis and learning, and includes data cleaning and feature extraction.

[1838] "Features" refer to important attributes or information extracted from data in data analysis and machine learning.

[1839] "Deep learning algorithm" refers to a type of machine learning that uses artificial neural networks to learn patterns from large amounts of data.

[1840] "Predictive Model" refers to a mathematical model trained to predict future preferences based on user tastes and behavior.

[1841] "Emotion engine" refers to technology that analyzes emotions from user feedback data and real-time facial expressions and voice.

[1842] A "personalized suggestion list" refers to a recommended list of specific fashion items created based on the preferences and emotional patterns of an individual user.

[1843] "3D model data" refers to digital data used to display fashion items in three dimensions, and includes formats such as glTF and OBJ.

[1844] "Terminal" refers to the device that a user uses to access the system and perform virtual try-on and emotion analysis, and primarily includes smartphones and tablets.

[1845] "Virtual try-on" refers to the process of virtually trying on clothes by combining captured data of the user's body with 3D model data.

[1846] "Augmented reality technology" refers to a technique that overlays digital information onto real-world images, providing users with an experience that makes them feel as if they are trying on clothes in real time.

[1847] "Feedback" refers to the ratings and impressions provided by users based on their fitting results and emotions, and includes data used to train the system.

[1848] This invention is a system that collects users' past purchase history and feedback data, and uses generative AI and an emotion engine to learn the user's preferences and emotions based on that data, and then makes personalized fashion suggestions. This system is realized through the cooperation of a server, a terminal, and the user.

[1849] First, the server collects the user's past purchase history and feedback data from a database (e.g., MySQL, PostgreSQL). The collected data is preprocessed to remove inappropriate data, fill in missing values, and extract features. During this preprocessing stage, the user's purchasing trends and product category information are analyzed.

[1850] The server then uses deep learning frameworks such as TensorFlow and PyTorch to train a generative AI model from the preprocessed data, which learns user preference patterns based on algorithms such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs).

[1851] Furthermore, the server utilizes an emotion engine (e.g., IBM Watson, Microsoft Azure Emotion API) to analyze emotions from user feedback data. This emotion analysis result is then combined with a generative AI model to create a personalized recommendation list based on emotions.

[1852] The generated recommendation list includes 3D model data (e.g., glTF, OBJ format) for each product item, and this data is sent to the user's device using a secure protocol (e.g., HTTPS).

[1853] The device receives the proposal list and 3D model data from the server. Then, it activates its camera to capture the user's body dimensions and pose, and acquires this data using a depth camera (e.g., Microsoft Kinect) or a standard camera. At the same time, it uses an emotion engine (e.g., OpenCV, Azure Cognitive Services) to analyze the user's facial expressions and voice and recognize emotions in real time.

[1854] The collected capture data and 3D model data are integrated, and the device uses augmented reality technology (e.g., ARKit, ARCore) to perform a virtual try-on experience, allowing users to see virtual clothing overlaid on real-world footage.

[1855] Users can use an interface on their device to provide feedback based on their fitting experience and their feelings. The feedback they provide is then sent back to the server and recorded as new data. This feedback is used in the system's overall learning process and is reflected in future recommendations.

[1856] As a concrete example, suppose a user logs in and selects a red dress from a list of items suggested by the server. The user captures their entire body using their smartphone camera and uses augmented reality technology to virtually try on the red dress. While trying it on, the device's camera and microphone analyze the user's facial expressions and voice using an emotion engine, which recognizes that the user has a positive emotional reaction to the dress. The user then enters their try-on experience (whether the size and color suit them) and their emotions (happy, excited, etc.) into the app and presses the "Submit" button. This sends the feedback to the server and receives it as new data. The server learns from this data and reflects it in future suggestions.

[1857] An example prompt is, "The user chooses a red dress, performs a virtual try-on, and records the positive emotion analyzed by the emotion engine."

[1858] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1859] Processing flow

[1860] Server-side behavior

[1861] Step 1:

[1862] Data collection and preprocessing

[1863] The server collects users' past purchase history and feedback data from a database as input. Since the collected data may contain inaccurate data, it performs data cleaning. It removes inappropriate data and fills in missing values ​​to organize the information. It also extracts features and tags each product item with the category and user purchasing trends. The preprocessed dataset is obtained as output.

[1864] Step 2:

[1865] Model learning using generative AI models

[1866] The server uses the preprocessed dataset as input to train a generative AI model using a deep learning framework (e.g., TensorFlow or PyTorch). It uses a recurrent neural network (RNN) or convolutional neural network (CNN) to learn user preference patterns. The output is a predictive model that reflects user preferences.

[1867] Step 3:

[1868] Emotion analysis using an emotion engine

[1869] The server takes the user's past feedback data as input and analyzes the emotions using an emotion engine (e.g., IBM Watson or Microsoft Azure Emotion API). It uses natural language processing (NLP) technology to analyze the feedback text and assign emotion labels such as positive or negative. The output is emotion-labeled feedback data.

[1870] Step 4:

[1871] Creating a personalized suggestion list

[1872] The server uses the generative AI model and the sentiment analysis results as inputs to generate a personalized fashion suggestion list. This suggestion list also includes 3D model data (e.g., glTF, OBJ format) for each product item. The output is the personalized suggestion list and 3D model data.

[1873] Step 5:

[1874] Sending data

[1875] The server takes the generated proposal list and 3D model data as input and sends them to the user's device. The data is securely transmitted using a secure protocol (e.g., HTTPS). As output, the proposal list and 3D model data sent to the device are obtained.

[1876] Operation on the terminal side

[1877] Step 1:

[1878] Data reception

[1879] The terminal receives as input the suggestion list and 3D model data sent from the server, prepares the received data for display, and obtains as output the suggestion list and 3D model data prepared in a displayable format.

[1880] Step 2:

[1881] Performing a camera capture

[1882] The device accepts user operations as input, activates a camera to capture the user's body dimensions and pose, and obtains body capture data using a depth camera (e.g., Microsoft Kinect) or a standard camera. The device obtains the capture data as output.

[1883] Step 3:

[1884] Emotion analysis

[1885] The device uses a camera and microphone as input, and analyzes the user's facial expressions and voice using an emotion engine (e.g., OpenCV, Azure Cognitive Services). The device recognizes the user's emotions in real time, and obtains real-time analyzed emotion data as output.

[1886] Step 4:

[1887] Augmented reality virtual try-on

[1888] The device integrates the captured data and 3D model data as input, and performs a virtual try-on using augmented reality technology (e.g., ARKit, ARCore). The device overlays the suggested clothing on a real-time video of the user. The output is a video of the virtual try-on using augmented reality technology.

[1889] Step 5:

[1890] Feedback collection and submission

[1891] The terminal accepts feedback based on the fitting results and emotions entered by the user and sends it to the server. It displays a feedback collection interface, prompts the user to enter information, and sends the entered data to the server using a secure protocol (e.g., HTTPS). The feedback data sent to the server is obtained as output.

[1892] User behavior

[1893] Step 1:

[1894] Login and item selection

[1895] As input, users log in to the app using their smartphone or tablet and select the items they want to try on from a list of suggestions. As output, they receive the selected items.

[1896] Step 2:

[1897] Start a virtual try-on

[1898] The user uses the device's camera as input to capture their body and begin the virtual try-on. The user stands in front of the camera, which captures and saves their entire body. The captured data is obtained as output.

[1899] Step 3:

[1900] emotion presentation

[1901] As input, the user confirms the emotions recognized by the device in real time during the virtual try-on. As output, real-time emotion data is obtained.

[1902] Step 4:

[1903] Check the fitting results

[1904] The user checks the appearance and size of the virtually tried-on clothes while looking at the screen of the device as input, and receives the confirmed fitting results as output.

[1905] Step 5:

[1906] Providing Feedback

[1907] The user inputs the items they tried on and their feedback based on the emotions they felt at the time into a form within the app. The input feedback data is obtained as output.

[1908] Step 6:

[1909] Send Feedback

[1910] As input, the user presses the "Send" button to send the feedback data to the server. The data is sent using a secure protocol (e.g., HTTPS). As output, the user gets the feedback data sent to the server.

[1911] (Application example 2)

[1912] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1913] In conventional online shopping environments, it is difficult for users to check the actual feel and fit of products before purchasing, which often leads to mistaken purchases and returns. Furthermore, product suggestions based on user preferences and emotions are rarely made, limiting the provision of a personalized shopping experience. This leads to issues such as lower user satisfaction and the inability to fully utilize the convenience of online shopping.

[1914] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1915] In this invention, the server includes: means for collecting a user's past purchase history and feedback data; means for preprocessing the collected data and extracting features; means for learning the user's preferences using the preprocessed data and generating a predictive model using a generative AI model; means for creating a personalized recommendation list for each user based on the predictive model; and means for transmitting the recommendation list and 3D model data to the terminal. This allows the user to not only receive personalized product recommendations but also virtually try on products at home using augmented reality technology. The system also includes means for analyzing the user's emotions in real time using a camera and microphone to improve the accuracy of the personalized recommendations, and means for collecting feedback based on the try-on results and emotions and transmitting the results to the server. This improves user satisfaction and convenience in online shopping.

[1916] "User's past purchase history" refers to information about products and services purchased by the user in the past.

[1917] "Feedback data" refers to data on evaluations and opinions regarding satisfaction and usability collected from users.

[1918] "Preprocessing" refers to the process of converting collected data into a form suitable for analysis.

[1919] "Features" refer to important attributes or characteristics of the data input into a machine learning model.

[1920] A "generative AI model" refers to an artificial intelligence model that uses technologies such as deep learning to learn and predict user preferences and behavior.

[1921] "Predictive model" refers to an algorithm created by a generative AI model to predict user behavior and preferences.

[1922] A "personalized suggestion list" refers to a list of products and services optimized based on a user's individual preferences and past purchasing history.

[1923] "3D model data" refers to the digital representation of the three-dimensional shape of a product.

[1924] "Augmented reality technology" refers to the technology of overlaying digital information onto images of the real world.

[1925] "Camera" refers to a device that captures images or videos.

[1926] "Capture data" refers to image and video data captured using a camera.

[1927] "Analyzing user emotions in real time" refers to the process of instantly analyzing data such as the user's facial expressions and voice to evaluate their current emotional state.

[1928] "Feedback" refers to opinions and ratings provided by users regarding their experience and satisfaction with the product.

[1929] This invention is a system that collects users' past purchase history and feedback data, and uses a generative AI model and emotion engine to learn the user's preferences and emotions based on that data, thereby providing personalized fashion suggestions and enabling virtual try-on at home. This system is realized through the cooperation of a server, a terminal, and the user.

[1930] Server-side behavior

[1931] Data collection and preprocessing

[1932] The server collects users' past purchase history and feedback data from a database. It preprocesses the collected data, cleaning it (removing inappropriate data and filling in missing values) and extracting features. This is done using Python, with libraries such as Pandas and Numpy used for data processing.

[1933] Model learning using generative AI models

[1934] The server uses the preprocessed data to train a generative AI model, which uses deep learning algorithms and frameworks such as TensorFlow and Keras to learn user preference patterns.

[1935] Emotion analysis

[1936] The server uses an emotion engine to analyze emotions from the user's past feedback data and incorporates the user's emotional patterns into a model using a deep learning model specialized for emotion analysis.

[1937] Creating a personalized suggestion list

[1938] Based on the trained preference and emotion models, the server generates a personalized fashion suggestion list, including 3D model data for each product item, which is updated in real time.

[1939] Sending data

[1940] The server communicates data using, for example, a REST API to send the proposal list and 3D model data to the user's device.

[1941] Operation on the terminal side

[1942] Data reception

[1943] The device receives the proposal list and 3D model data from the server using HTTP requests.

[1944] Camera and microphone activation and capture

[1945] The device activates the user's camera and microphone, capturing body measurements and poses, and preparing to analyze emotions in real time, using libraries such as OpenCV and MediaPipe.

[1946] Performing a virtual try-on

[1947] The device integrates the captured data and 3D model data, and performs a virtual try-on using augmented reality technology. Using Unity or ARKit / ARCore, the device overlays suggested clothing on the user's real-time video.

[1948] Feedback collection and submission

[1949] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their feelings at the time. An HTTP POST request is used to send the user-entered feedback to the server.

[1950] User behavior

[1951] Log in and receive personalized offers

[1952] The user logs in to the app using their smartphone or tablet and receives a list of suggested items from the server. This operation requires the user's account information.

[1953] Start a virtual try-on

[1954] Users can capture their body using the device's camera and begin the virtual try-on experience, giving them the experience of trying on products in real life, without leaving the comfort of their own home.

[1955] Real-time emotion display

[1956] The device uses a camera and microphone to analyze the user's emotions in real time while trying on clothes, and displays that information on the screen, allowing the user to check their own emotions.

[1957] Checking fitting results and providing feedback

[1958] The user checks the appearance and size of the clothes they try on on the device screen and inputs the results as feedback into the device. This feedback is sent to the server and reflected in future suggestions.

[1959] Examples of prompt statements

[1960] When a user logs in to receive personalized suggestions, the prompt text looks like this:

[1961] Text format

[1962] Get personalized fashion suggestions based on user ID 12345's past purchase history and feedback data.

[1963] In this way, the present invention provides personalized fashion suggestions based on the user's tastes and emotions, providing an enhanced online shopping experience.

[1964] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1965] Step 1:

[1966] The server collects the user's past purchase history and feedback data from the database. The input is the user ID, and the output is the past purchase history and feedback data. Specifically, it uses SQL queries to extract relevant data from the database.

[1967] Step 2:

[1968] The server preprocesses the collected data and extracts features. The input is the collected purchase history and feedback data, and the output is a cleaned dataset and features. Specific operations include cleaning the data (filling in missing values ​​and removing inappropriate data) and extracting features using Pandas.

[1969] Step 3:

[1970] The server uses the preprocessed data to train a generative AI model. The input is the preprocessed data and features, and the output is a trained model. Specifically, it learns user preference patterns using deep learning algorithms using TensorFlow and Keras.

[1971] Step 4:

[1972] The server uses an emotion engine to analyze emotions from the user's past feedback data. The input is the feedback data, and the output is emotion features. Specifically, it extracts emotions from text data using natural language processing technology (e.g., an emotion analysis model).

[1973] Step 5:

[1974] The server generates a personalized fashion suggestion list for each user based on the trained preference and emotion models. The input is the trained model, and the output is the suggestion list and 3D model data. Specifically, it selects items that match the user's preferences and creates a list including the 3D model data.

[1975] Step 6:

[1976] The server sends the proposal list and 3D model data to the device. The input is the proposal list and 3D model data, and the output is the successful data transmission to the device. Specifically, the data is sent using the REST API.

[1977] Step 7:

[1978] The device receives the proposal list and 3D model data from the server. The input is the proposal list sent from the server, and the output is the proposal list displayed on the device. Specifically, the data is received using an HTTP GET request.

[1979] Step 8:

[1980] The device activates the camera and microphone, and prepares to capture the user's body and analyze emotions in real time. The input is real-time data from the camera and microphone, and the output is captured data and emotion data. Specifically, the device uses OpenCV to capture camera images and perform emotion analysis in real time.

[1981] Step 9:

[1982] The device integrates the captured data and 3D model data and performs a virtual try-on using augmented reality technology. The input is the captured data and 3D model data, and the output is an image of the virtual try-on. Specifically, using Unity or ARKit / ARCore, the suggested clothing is overlaid on the user's real-time video.

[1983] Step 10:

[1984] The terminal displays an interface that allows the user to provide feedback based on the fitting results and their emotions. The input is the user's feedback, and the output is the collected feedback data. Specifically, the terminal displays a GUI form to receive user input.

[1985] Step 11:

[1986] The terminal sends the feedback input by the user to the server. The input is the feedback data, and the output is the successful data transmission to the server. Specifically, the feedback data is sent using an HTTP POST request.

[1987] Step 12:

[1988] The server stores the received feedback data in a database and updates the data to reflect future proposals. The input is the feedback data, and the output is the updated database. Specifically, it inserts new data and updates existing data using SQL queries.

[1989] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1990] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1991] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1992] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1993] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1994] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1995] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1996] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1997] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1998] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1999] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2000] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2001] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2002] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2003] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2004] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2005] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2006] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2007] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2008] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2009] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2010] The following is further disclosed regarding the above embodiment.

[2011] (Claim 1)

[2012] A means for collecting user past purchase history and feedback data;

[2013] A means for preprocessing the collected data and extracting features;

[2014] means for learning user preferences using the preprocessed data and generating a predic...

Claims

1. A means for collecting user past purchase history and feedback data; A means for preprocessing the collected data and extracting features; means for learning user preferences using the preprocessed data and generating a predictive model; a means for generating a personalized list of suggestions for each user based on the predictive model; means for transmitting the proposal list and the 3D model data to the terminal; a means for capturing a user's body using a camera in the terminal and performing a virtual try-on using the captured data and 3D model data; means for collecting and transmitting user feedback to a server; A system including:

2. 10. The system of claim 1, wherein the means for learning user preferences uses a deep learning model.

3. 2. The system according to claim 1, wherein the means for performing the virtual try-on uses augmented reality technology.

Images