System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring a digital video of clothing and footwear owned by a user; means for analyzing the acquired digital video and extracting category information, color information, and material information of the clothing and the footwear; means for generating optimal coordination based on the extracted information; means for proposing a new item based on a budget of the user; and means for transmitting the generated coordination and the proposal of the new item to a terminal of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The challenges facing users are that they spend too much time on coordinating their daily fashion and are unable to effectively utilize the items they already own. Furthermore, when selecting new items, optimal suggestions based on budget and style are not available. It is necessary to provide a system that can solve these challenges and allow users to easily enjoy unique and sophisticated fashion. [Means for solving the problem]

[0005] The present invention provides a system that acquires digital images of clothing and footwear owned by a user and analyzes the digital images to extract category, color, and material information for the clothing and footwear. It also includes a system that generates optimal outfits based on the extracted information and a system that suggests new items based on the user's budget. The system also includes a system that transmits the generated outfits and suggestions for new items to the user's device. This system allows users to make the most of their existing items, instantly create optimal outfits, and receive suggestions for new items that fit their budget.

[0006] "User" refers to any individual who uses this system.

[0007] "Clothing" refers to all clothes and accessories that a user wears on a daily basis.

[0008] "Footwear" refers to items that cover the feet, such as shoes or boots worn by a user.

[0009] "Digital Footage" refers to video clips of clothing and footwear captured by a user's smartphone or other imaging device.

[0010] "Capture means" refers to the processes and techniques used to capture digital video from a user's device into the system.

[0011] "Means for analyzing" refers to algorithms and technologies for analyzing the content of digital video and extracting category information, color information, and material information of clothing and footwear.

[0012] "Category information" refers to information about the type of clothing and footwear analyzed (e.g., tops, bottoms, shoes).

[0013] "Color information" refers to information about the color of the analyzed clothing and footwear.

[0014] "Material information" refers to information about the materials used to make the analyzed clothing and footwear.

[0015] "Means for generating" refers to the process and technology for creating the optimal outfit for the user based on the analyzed information.

[0016] "Means for suggesting" refers to the process and techniques for selecting and suggesting new items based on the user's budget and preferences.

[0017] "Transmitting means" refers to the process and technology for transferring the generated coordinates and new item suggestions to the user's terminal.

[0018] "Device" refers to a smartphone, tablet, or other computing device used by a User. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention relates to a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The system of the present invention is designed to enable users to make the most of the items they already own and easily enjoy unique and effective fashion.

[0041] Program processing

[0042] Video recording and uploading

[0043] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. After filming is complete, the device uploads the video data to the server. When uploaded, the video data is encrypted and transmitted via secure communication.

[0044] Video analysis and item recognition

[0045] The server decompresses the received video data and inputs it into an AI model for video analysis. The AI ​​model then recognizes the clothing and footwear in each frame and extracts category information (e.g., tops, bottoms, shoes), color information, pattern information, and material information. Based on the results of this analysis, all data is recorded in the user's profile.

[0046] Coordinate generation

[0047] The server generates the optimal outfit based on the analyzed item information. This takes into account fashion rules and the latest trends. It also reflects the user's past selection history and style preferences. For example, if a user has a white shirt, blue jeans, and red sneakers, the server will suggest a casual outfit combining these items.

[0048] New item proposals

[0049] The server then suggests new items based on the user's budget and style preferences. It retrieves the latest price information from the shopping API and a database of partner retailers, and uses that data to select the new items that best suit the user. For example, it suggests a black jacket or accessories that go well with a casual style.

[0050] Displaying the results

[0051] Finally, the server sends the generated outfits and new item suggestions to the device, which receives them and displays them in the user's smartphone app. The user can then review the suggestions and save their favorite outfits, or click a link to purchase the new items on an online shopping site.

[0052] Specific examples

[0053] 1. A user launches a smartphone app and takes a photo of a white shirt, blue jeans, and red sneakers.

[0054] 2. The device uploads the video data to the server.

[0055] 3. The server analyzes the video and recognizes each item (white shirt, blue jeans, red sneakers).

[0056] 4. Based on the analysis results, the server generates the optimal casual outfit combining a white shirt, blue jeans, and red sneakers.

[0057] 5. The server suggests new items, such as a black jacket or accessories, based on the user's budget and preferences.

[0058] 6. The device receives the generated outfit and new item suggestions and displays them to the user. The user can save the outfit or click a link to purchase the new item.

[0059] In this way, users can easily incorporate new fashion items while utilizing items they already have.

[0060] The processing flow will be explained below.

[0061] Step 1:

[0062] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. After filming is complete, the video data is automatically saved in the app.

[0063] Step 2:

[0064] Prepare the video data captured by the device for uploading to the server. Encrypt the video data before uploading.

[0065] Step 3:

[0066] The device sends the encrypted video data over the network to the server, where the network connection is verified and the data integrity is checked during transmission.

[0067] Step 4:

[0068] The server decompresses the video data received from the device, and after decompression, the video data is input into the AI ​​analysis module.

[0069] Step 5:

[0070] The AI ​​analysis module on the server analyzes the video data, extracts images of clothing and footwear from each frame of the video, and uses object recognition algorithms to identify each item. Information such as category, color, pattern, and material of each identified item is then extracted.

[0071] Step 6:

[0072] The server generates the optimal outfit for the user based on the analyzed item information. The generation process uses pre-programmed fashion rules and trend data, and also takes into account the user's past style history and preferences.

[0073] Step 7:

[0074] The server references the user's budget information and preferred style to suggest new items, and uses an external shopping API to get the latest price information and select new items to include in the suggestions.

[0075] Step 8:

[0076] The server packages the generated outfits and new item suggestions together and sends them to the device. The package includes image data, text information, a purchase link, and more.

[0077] Step 9:

[0078] The device extracts the coordination results and new item suggestions received from the server and displays them within the app. A notification is sent to the user to inform them of new suggestions.

[0079] Step 10:

[0080] Users can view suggested outfits and new items within the app, save their favorite outfits, and click a purchase link to access the online shopping site for suggested new items.

[0081] The above are the specific processing steps of the system based on the claims.

[0082] Example 1

[0083] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0084] In modern life, users need to be able to effectively utilize the clothing and footwear they own and easily incorporate the latest fashion styles. However, many users find it difficult to devote time and effort to choosing outfits, which prevents them from making the most of the items they own. Another problem is that it is difficult to instantly obtain appropriate price information when selecting or purchasing new items.

[0085] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0086] In this invention, the server includes: means for acquiring digital images of the user's clothing and footwear; means for encrypting the acquired digital images and securely transmitting them from the user's device to the server; means for decompressing the digital images received by the server and inputting them into an AI model for video analysis; means for analyzing the images using the AI ​​model and extracting information about the clothing and footwear's category, color, pattern, and material; means for saving the extracted information in the user's profile; means for generating optimal outfits based on the saved information and taking into account fashion rules and trend information; means for suggesting new items from a shopping API or retailer database based on the user's budget and price information for new items; and means for transmitting the generated outfits and suggested new items to the user's device. This allows users to make the most of their existing clothing and footwear and easily incorporate the latest fashion styles. Furthermore, when selecting or purchasing new items, users can obtain appropriate price information in real time, providing a comfortable shopping experience.

[0087] "User" refers to an individual who uses the system to obtain digital images of clothing and footwear and receive suggestions for coordination and new items.

[0088] "Clothing and footwear" refers to fashion-related items such as clothes and shoes owned by a user.

[0089] "Digital footage" refers to visual image data captured using a smartphone or other device.

[0090] "Analysis" refers to the process of extracting specific information from digital video, and includes image recognition technology.

[0091] "Category information" is information for classifying clothing and footwear in digital video, and indicates types such as tops, bottoms, and shoes.

[0092] "Color information" refers to information about the color of items in a digital image.

[0093] "Pattern" refers to information about the design or pattern of an item within a digital image.

[0094] "Material information" refers to information about the material (e.g., cotton, denim, etc.) of an item in a digital image.

[0095] A "profile" refers to a database or data store where information about a user is recorded.

[0096] "Fashion rules" refer to guidelines and standards for creating outfits, including those based on seasons and trends.

[0097] "Trend information" refers to information about current fashion trends.

[0098] "Coordination" refers to combining multiple items of clothing and footwear to create a stylish outfit.

[0099] "Shopping API" refers to an interface to an external database used to obtain information about new items.

[0100] "Retailer database" refers to a database of product information managed by a particular retailer.

[0101] "Terminal" refers to a digital device operated by a user, including a smartphone or tablet.

[0102] "Encryption" refers to the process of transforming content to preserve data security.

[0103] "Secure communications" refers to a means of communication that ensures that data is transmitted safely and without tampering.

[0104] "AI model" refers to an artificial intelligence algorithm for analyzing digital video.

[0105] This invention relates to a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The system of the present invention is designed to enable users to make the most of the items they already own and easily enjoy unique and effective fashion.

[0106] Users launch a dedicated application on their smartphone or other device. The system begins by taking a video of the clothes and shoes in their closet and tapping the "Upload" button. The device then encrypts the video data using the SSL / TLS protocol and securely transmits it to the server.

[0107] The server decompresses the received video data. It then analyzes each frame using a generative AI model that uses an object detection model such as YOLO (You Only Look Once) to extract information about the category (tops, bottoms, shoes, etc.), color, pattern, and material of the clothing and footwear in the video. The analysis results are stored in the user's profile database.

[0108] The server then generates optimal outfits based on the saved item information, referring to the latest fashion rules and trend information. This process also takes into account the user's past outfit history and style preferences. A concrete example of this system is suggesting casual outfits for a user who owns a white shirt, blue jeans, and red sneakers.

[0109] The server also uses a shopping API and retailer database to suggest new items based on the user's budget and style preferences, such as a black jacket and accessories that complement a casual style, along with up-to-date pricing information.

[0110] Finally, the server sends the generated outfits and new item suggestions to the device. The device receives them and displays them in the user's smartphone app. The user can review the suggested outfits and, if they like them, tap the "Save" button to save them. They can also purchase the new items on an online shopping site by clicking the "Purchase Link" for the items.

[0111] As a concrete example, the following prompt sentences can be effectively input into a generative AI model:

[0112] "Please suggest a casual outfit using a white shirt, blue jeans, and red sneakers. Also, please suggest some new items that would go well with this casual style."

[0113] Based on this prompt, the generative AI model can provide users with optimal outfits and suggestions for new items, allowing them to easily incorporate new fashion items while utilizing the items they already own.

[0114] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0115] Step 1:

[0116] The user launches the smartphone app and follows the instructions in the app to take a video of the clothes and shoes in their closet. For example, the user moves the camera to show off a white shirt, blue jeans, and red sneakers individually. The input is video data of the items in the closet. The output is video data stored in the device's internal storage.

[0117] Step 2:

[0118] The user taps the "Upload" button to send the video data they have taken to the server. The device encrypts this video data using the SSL / TLS protocol and sends it securely to the server. The input is the video data stored on the device. The output is the encrypted video data sent to the server.

[0119] Step 3:

[0120] The server receives the received video data and first decrypts it. Then it divides the data into frames. The input is the encrypted video data. The output is the decomposed frame data.

[0121] Step 4:

[0122] The server inputs the decomposed frame data into a generative AI model for video analysis. An AI model (e.g., YOLO) is used to recognize clothing and footwear from each frame. The input is frame-by-frame video data. The output is category information, color information, pattern information, and material information for each item.

[0123] Step 5:

[0124] The server saves the extracted item information in the user profile. For example, a white shirt is recorded in the database as "Tops, White, Solid, Cotton." The input is category information, color information, pattern information, and material information. The output is updated user profile data.

[0125] Step 6:

[0126] The server generates optimal outfits based on the saved item information and the latest fashion rules and trend information. For example, it suggests a casual style that combines a white shirt, blue jeans, and red sneakers. The input is the user profile information, fashion rules, and trend information. The output is optimal outfit data.

[0127] Step 7:

[0128] The server queries the shopping API and retailer database to suggest new items based on the user's budget and style preferences. For example, select a black jacket and accessories that go well with a casual style and obtain their price information. The input is the user profile information and price information obtained from the shopping API. The output is the suggested new item data.

[0129] Step 8:

[0130] The server sends the generated coordinates and new item proposals to the terminal. The input is the optimal coordinate data and new item proposal data. The output is the data to be sent to the terminal.

[0131] Step 9:

[0132] The device displays the received coordinations and new item suggestions within the smartphone app. The user can check the displayed content and save their favorite coordinations by tapping the "Save" button, or click the "Purchase Link" for the new item to purchase it on the online shopping site. The input is the data sent from the server. The output is the coordination and new item information displayed on the user interface.

[0133] (Application example 1)

[0134] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0135] Conventional fashion coordination systems require users to manually manage their clothing and footwear, which is time-consuming and makes it difficult to suggest optimal outfits. Furthermore, they often fail to adequately suggest new items that take into account the user's budget, style preferences, and the latest fashion trends. Furthermore, their ability to obtain the latest price information from external shopping information databases is limited, making them inconvenient for users.

[0136] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0137] In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, means for generating optimal outfits based on the extracted information, means for suggesting new items based on the user's budget and style preferences, means for transmitting the generated outfits and suggestions for new items to the user's communication terminal, and means for acquiring the latest price information from an external shopping information database. This allows the user to easily manage their items and receive suggestions for optimal outfits and the latest fashion items.

[0138] "Digital video" refers to video data of clothing and footwear owned by a user photographed with a digital device such as a camera or smartphone.

[0139] "Category information" is information indicating the type and shape of clothing and footwear extracted from digital video, such as tops, bottoms, shoes, etc.

[0140] "Color information" is information that indicates the color of clothing and footwear extracted from digital video.

[0141] "Material information" is information that indicates the material and fabric of the clothing and footwear extracted from the digital video.

[0142] "Coordination" refers to creating optimal combinations based on extracted information, and proposing effective combinations of clothing and footwear for the user to wear.

[0143] "New Items" are new clothing or footwear that the user does not already own, but that are suggested based on budget and style preferences.

[0144] A "communication terminal" is a device used by a user, such as a smartphone, tablet, or PC, which is a device for sending and receiving information.

[0145] "Shopping information database" refers to a database of an external online shopping platform or retail store for obtaining product information and price information.

[0146] A "generative AI model" is a machine learning model that uses artificial intelligence to extract information from digital images and generate optimal outfits.

[0147] A "prompt" is an instruction sentence that is input into a generative AI model to obtain a specific output.

[0148] This invention is a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The entire system is composed of a user's communication terminal and a server.

[0149] First, the user launches the smartphone app and takes a video of the clothes and footwear in their closet. After taking the video, the user's device uploads the video data to a server using a secure protocol (e.g., HTTPS). When uploaded, the video data is encrypted and transmitted via secure communication.

[0150] The server decompresses the received video data and uses a generative AI model (e.g., YOLOv5 or TensorFlow) for video analysis. The generative AI model recognizes the clothing and footwear in each frame and extracts category information (tops, bottoms, shoes, etc.), color information, pattern information, and material information. The analysis results are recorded in a NoSQL database (e.g., MongoDB).

[0151] Next, the server generates the optimal outfit based on the extracted item information, using algorithms (e.g., Python's scikit-learn, pandas) to take into account fashion rules, the latest trends, and the user's past selection history and style preferences.

[0152] Furthermore, the server suggests new items based on the user's budget and style preferences. When making suggestions, the server retrieves the latest price information from a shopping information database (e.g., Rakuten API, Amazon Product Advertising API) and selects the most suitable items based on that data. At this time, the server selects items by inputting prompt statements into the generative AI model.

[0153] Finally, the server sends the generated outfits and new item suggestions to the user's communication device. The user can check them in the smartphone app, save the suggested outfits, or click a link to purchase the new items on an online shopping site.

[0154] For example, a user launches a smartphone app, takes a photo of a white shirt, blue jeans, and red sneakers, and then inputs the following prompt into a generative AI model:

[0155] "List the items the user owns

[0156] White shirt, blue jeans, red sneakers

[0157] Based on these, please suggest the best fashion coordination and new items available for purchase.

[0158] My budget is under 5,000 yen, and I prefer a casual style.

[0159] This allows users to easily manage the items they own and receive suggestions for optimal coordination and the latest fashion items.

[0160] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0161] Step 1:

[0162] The user starts the smartphone app and takes a video of the clothes and footwear in their closet.

[0163] Input: A user uses a smartphone to take a video of the clothes and footwear in their closet.

[0164] Output: Recorded video data.

[0165] Specific operation: The user launches the smartphone app, takes a video of the clothing and footwear from an appropriate angle, and saves it within the app.

[0166] Step 2:

[0167] The device uploads the video data to the server.

[0168] Input: Video data captured in step 1.

[0169] Output: The encrypted video data is sent to the server.

[0170] Specific operation: The smartphone app encrypts the video data using SSL / TLS and securely uploads it to the server using the HTTPS protocol.

[0171] Step 3:

[0172] The server decompresses the received video data and inputs it into a generative AI model for video analysis.

[0173] Input: Encrypted video data.

[0174] Output: The decompressed video data is input into the AI ​​model.

[0175] Specific operation: The server decompresses the video data and preprocesses it for AI analysis using YOLOv5 and TensorFlow.

[0176] Step 4:

[0177] The server uses an AI model to analyze the video data and extract category, color, and material information.

[0178] Input: Decompressed video data.

[0179] Output: Category information, color information, and material information are extracted.

[0180] How it works: The server applies the generative AI model to detect clothing and footwear in each frame and identify their category, color, and material.

[0181] Step 5:

[0182] The server generates the optimal coordination based on the analyzed item information.

[0183] Input: Category information, color information and material information.

[0184] Output: Optimal coordinate information.

[0185] Specific operation: The server generates outfits using an algorithm (e.g., scikit-learn, pandas) based on fashion rules and trend information stored in a database, as well as the user's past selection history.

[0186] Step 6:

[0187] The server suggests new items based on the user's budget and style preferences.

[0188] Input: User's budget, style preferences, and coordinated outfit information.

[0189] Output: Proposed new item information.

[0190] Specific operation: The server queries the shopping information database via an API (e.g., Rakuten API, Amazon Product Advertising API) to search for new items that fit the style and are within budget.

[0191] Step 7:

[0192] The server uses a generative AI model to input prompts to suggest new items.

[0193] Input: A user request in the form of a prompt statement.

[0194] Output: Information about the selected new item.

[0195] Specific operation: The server inputs the following prompt into the generated AI model and suggests a new item:

[0196] "List the items the user owns

[0197] White shirt, blue jeans, red sneakers

[0198] Based on these, please suggest the best fashion coordination and new items available for purchase.

[0199] My budget is under 5,000 yen, and I prefer a casual style.

[0200] Step 8:

[0201] The server transmits the generated coordinates and new item suggestions to the user's communication terminal.

[0202] Input: optimal coordination information, new item information.

[0203] Output: Coordination and new item suggestions are displayed on the user's device.

[0204] How it works: The server packages the generated outfits and new item suggestions in JSON format and sends them to the user's device using the HTTPS protocol. The user can then view them in the app, save the outfits, or click on a link to purchase the new items.

[0205] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0206] This invention combines a system that captures a user's clothing and footwear as digital images, analyzes the images, and suggests optimal outfits and new items, with an emotion engine that recognizes the user's emotions.The system of the present invention is designed to effectively utilize the items a user owns, allowing them to easily enjoy unique and effective fashion.

[0207] Program processing

[0208] Video recording and uploading

[0209] Users launch the smartphone app and take videos of the clothes and shoes in their closet. After filming is complete, the video data is automatically saved in the app.

[0210] Uploading video data

[0211] The device prepares the captured video data for uploading to the server, encrypts the video data, and then transmits the encrypted video data to the server via the network.

[0212] Video analysis and item recognition

[0213] The server decompresses the video data received from the device and inputs it into the AI ​​analysis module, where the AI ​​model recognizes clothing and footwear from each frame of the video and extracts category, color, and material information. The analysis results are recorded in the user's profile.

[0214] Coordinate generation

[0215] The server generates the optimal outfit based on the analyzed item information. This generation process takes into account fashion rules and the latest trends, as well as the user's past style history and preferences. The generated outfit is designed to be optimal for the user.

[0216] New item proposals

[0217] The server suggests new items based on the user's budget. It obtains the latest price information from an external shopping information database and selects new items based on that information. For example, it may suggest jackets and accessories that go well with a casual style.

[0218] Acquisition and analysis of emotional information

[0219] When a user inputs their facial expressions and voice into the smartphone app, the device analyzes them with an emotion engine to identify the user's emotional state. For example, if the user is smiling, it will extract emotional information such as "joy," and if the user has a serious expression, it will extract emotional information such as "concentration."

[0220] Coordination and new item suggestions that take emotions into account

[0221] The server then uses the emotional information recognized by the emotion engine to suggest outfits and new items that match the user's current mood. For example, if the server detects that the user is in a high mood, it will suggest outfits that combine bright colors and comfortable clothing.

[0222] Displaying the results

[0223] Finally, the server sends the generated outfits and new item suggestions to the device, which displays the results within the app and notifies the user, allowing the user to review the suggested outfits and new items and save or purchase them as needed.

[0224] Specific examples

[0225] 1. A user launches a smartphone app and records a video of themselves wearing a white shirt, blue jeans, and red sneakers.

[0226] 2. The device uploads the video data to the server.

[0227] 3. The server analyzes the video and recognizes the white shirt, blue jeans, and red sneakers.

[0228] 4. The server generates the optimal casual outfit consisting of a white shirt, blue jeans, and red sneakers.

[0229] 5. The server suggests new items, such as a black jacket or accessories, based on the user's budget.

[0230] 6. The user inputs their facial expressions and voice within the app, and the device analyzes them with its emotion engine. For example, if the user is smiling, it will be recognized as "joy."

[0231] 7. Based on the emotional information recognized as "joy," the server will further suggest coordination of bright colors and comfortable textures.

[0232] 8. The device receives the generated outfits and new item suggestions and displays them to the user. The user can like the suggestions and click the save or purchase link.

[0233] In this way, users can easily incorporate new fashion items based on their emotions while using items they already own.

[0234] The processing flow will be explained below.

[0235] Step 1:

[0236] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. Once the video is complete, it is automatically saved in the app.

[0237] Step 2:

[0238] The device prepares to upload the video data it has taken to the server, encrypts the video content, and once encryption is complete, sends the video data to the server via the network.

[0239] Step 3:

[0240] The server decompresses the video data received from the device, and the decompressed data is input into the AI ​​analysis module.

[0241] Step 4:

[0242] The AI ​​analysis module in the server performs image analysis to identify clothes and shoes from each frame of the video, and uses object recognition algorithms to extract information about the category, color, pattern, and material of the clothing and footwear.

[0243] Step 5:

[0244] The server then analyzes each item and generates the optimal outfit using the user's existing items, taking into account the latest fashion trends and style rules.

[0245] Step 6:

[0246] The server uses an emotion engine to understand the user's emotions by analyzing the user's facial expressions and voice data. Facial expression analysis recognizes the user's facial features and determines their emotional state, such as smiling or serious.

[0247] Step 7:

[0248] The server analyzes the user's emotional information and then suggests outfits that match the user's current mood. For example, if the user is in a happy state, the server suggests outfits that include bright colors and items with positive impressions.

[0249] Step 8:

[0250] At the same time, the server also suggests new items, referencing an external shopping information database to select the latest items that fit the user's budget and include them in the suggestions.

[0251] Step 9:

[0252] The server packages the generated outfits and new item suggestions and sends them to the device. The package includes image data, detailed information, and a purchase link.

[0253] Step 10:

[0254] The device unpacks the result package received from the server and displays it within the smartphone app, and a notification is sent to the user to inform them of new suggestions.

[0255] Step 11:

[0256] Users can view suggested outfits and new items within the app, and can save their favorite outfits or items within the app, or click a purchase link to purchase them on an online shopping site.

[0257] This allows users to intuitively and easily receive optimal fashion coordination based on their emotional state and the items they own, as well as new suggested items.

[0258] Example 2

[0259] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0260] Today's consumers want to enjoy creating unique and optimal fashion coordination while effectively utilizing the clothing and footwear they own. However, conventional systems have difficulty in suggesting real-time coordination and new items that take the user's emotional state into account, which has led to issues in not being able to increase user satisfaction. Furthermore, in order to make suggestions that match the user's budget and preferred style, it is necessary to consider many factors simultaneously, which increases the system's complexity.

[0261] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, means for generating an optimal outfit based on the extracted information, means for suggesting new items based on the user's budget, means for suggesting outfits and new items that match the user's current mood using an emotion engine that recognizes the user's emotions, and means for transmitting the generated outfits and new item suggestions to the user's terminal. This allows the user to easily incorporate new fashion items based on their emotions while utilizing items they already own.

[0262] "User" refers to a consumer who uses this system.

[0263] "Clothing and footwear" refers to fashion items such as clothes and shoes that are owned by a user and stored in a closet.

[0264] "Digital video" refers to image and video data captured by a camera such as a smartphone and stored on a digital device.

[0265] "Analysis" refers to the processing of data to extract specific information from captured digital images.

[0266] "Category information" is information that indicates the type of clothing or footwear, and includes, for example, "shirts," "jeans," and "sneakers."

[0267] "Color information" refers to information about the color of clothing or footwear, and refers to data that identifies colors such as "white," "blue," and "red."

[0268] "Material information" is information about the materials used to manufacture clothing or footwear, and includes, for example, "cotton," "denim," and "leather."

[0269] "Generating coordination" refers to proposing the optimal style by combining multiple pieces of clothing and footwear based on fashion rules, trend information, the user's preferences, and past style history.

[0270] "Suggesting new items" means selecting the latest fashion items from an external shopping information database based on the user's budget and style, and encouraging them to purchase them.

[0271] An "emotion engine" refers to a software module that analyzes a user's facial expressions and voice data to identify their emotional state.

[0272] A "terminal" is a digital device used by a user, including a smartphone, tablet, computer, etc.

[0273] "Server" refers to a computer system that provides functions such as data analysis, coordinate generation, new item suggestions, and emotional information processing.

[0274] "Trend information" is data about current trends and popular styles in the fashion industry.

[0275] This invention is a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. This system also incorporates an emotion engine that recognizes the user's emotions, allowing it to suggest outfits that match the user's mood.

[0276] The specific operation of the system will now be described.

[0277] 1. Video recording and data storage

[0278] Users launch the smartphone app and take videos of the clothes and shoes in their closet. The smartphone can be an iPhone or an Android device, and the video data is automatically saved in the app.

[0279] 2. Uploading and Encrypting Video Data

[0280] To upload the captured video data to the server, the device first encrypts the video data using the AES-256 algorithm, and then transmits the encrypted video data to the server via the Internet.

[0281] 3. Video analysis and item recognition

[0282] The server decompresses the video data received from the device and inputs it into an AI analysis module, such as TensorFlow or PyTorch. The server analyzes the video frame by frame and extracts information about the category, color, and material of the clothing and footwear. For example, it uses an object detection algorithm (such as YOLO or SSD) to identify the items.

[0283] 4. Optimal Coordination Generation

[0284] The server then generates the optimal outfit based on the extracted item information, taking into account fashion rules and the latest trends. The generation process also takes into account the user's past style history and preferences. This process utilizes fashion-related APIs from Google Cloud AI and Amazon Web Services.

[0285] 5. Proposal for new items

[0286] The server retrieves up-to-date price information from external shopping information databases (e.g., Shopify or Amazon APIs) and suggests new items based on the user's budget, such as a black jacket or accessories.

[0287] 6. Acquisition and analysis of emotional information

[0288] When a user inputs their facial expressions or voice into the smartphone app, the device analyzes them with an emotion engine. For example, it uses Microsoft Azure's Emotion API or Google Cloud's Vision AI to extract emotional information such as "joy" or "concentration."

[0289] 7. Emotion-based recommendations

[0290] The server then suggests outfits and new items that match the user's mood based on the emotional information recognized by the emotion engine. For example, if the user is recognized as feeling "joy," the server will suggest outfits that combine bright colors and comfortable clothing.

[0291] 8. Displaying the results

[0292] The server then sends the resulting coordinates and new item suggestions to the device, which displays the results within the app. The user can then review the suggested coordinates and new items and click on the links to save or purchase them.

[0293] Specific examples

[0294] Suppose a user launches a smartphone app and takes a video of themselves wearing a white shirt, blue jeans, and red sneakers. The device encrypts the video data and uploads it to the server. The server analyzes the video and extracts information about each item. Based on the extracted information, the server generates a casual outfit that combines a white shirt, blue jeans, and red sneakers. It also suggests other items, such as a black jacket and accessories, based on the user's budget. If the user smiles within the app, the emotion engine recognizes "happiness" and further suggests bright color coordination based on this. The final results are sent to the device, where the user can review the suggestions and save or purchase them within the app.

[0295] Prompt Sentence Examples

[0296] "Please suggest the perfect casual outfit based on the white shirt, blue jeans, and red sneakers I have on hand. Also, please suggest new items within a budget of 10,000 yen. And please suggest an outfit that you think will please me."

[0297] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0298] Step 1: Recording video and saving data

[0299] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. The smartphone used can be an iPhone or an Android device. After filming, the video data is automatically saved within the app.

[0300] Input: User-recorded video

[0301] Output: Video data stored on the smartphone

[0302] Step 2: Prepare and encrypt video data for upload

[0303] The device prepares the captured video data for upload to the server. First, it encrypts the video data using the AES-256 algorithm. This encryption process runs in the background and a notification is displayed to the user.

[0304] Input: Video data stored on your smartphone

[0305] Output: Encrypted video data

[0306] Step 3: Sending video data to the server

[0307] The device sends encrypted video data to a server via the Internet, and HTTPS is generally used as the protocol for network connection.

[0308] Input: Encrypted video data

[0309] Output: Encrypted video data sent to the server

[0310] Step 4: Extract and analyze the video data

[0311] The server decompresses the received video data and inputs it into the AI ​​analysis module. The data is sequentially decomposed frame by frame. The analysis modules used include TensorFlow and PyTorch. The analysis uses object detection algorithms (YOLO and SSD) to extract category, color, and material information for clothing and footwear.

[0312] Input: Encrypted video data sent to the server

[0313] Output: Clothing and footwear category information, color information, material information

[0314] Step 5: Generate optimal coordinates

[0315] The server then generates the optimal outfit based on the extracted item information, taking into account fashion rules and the latest trends. This process also takes into account the user's past style history and preferences. The APIs used include fashion-related APIs from Google Cloud AI and Amazon Web Services.

[0316] Input: Clothing and footwear category information, color information, material information

[0317] Output: Generated coordinate information

[0318] Step 6: Propose new items

[0319] The server retrieves up-to-date price information from external shopping databases (e.g., Shopify or Amazon's API) and suggests new items based on the user's budget, such as a black jacket or accessories.

[0320] Input: User's budget information, price information from external shopping database

[0321] Output: Information about the proposed new item

[0322] Step 7: Acquiring and analyzing emotional information

[0323] Users input their facial expressions and voices into the smartphone app, which the device then analyzes using an emotion engine. The analysis engines used include Microsoft Azure's Emotion API and Google Cloud's Vision AI. The analyzed emotional information is extracted as "joy," "concentration," etc.

[0324] Input: User's facial expression data and voice data

[0325] Output: Recognized emotion information

[0326] Step 8: Emotion-Based Proposals

[0327] The server then suggests outfits and new items that match the user's mood based on the emotional information recognized by the emotion engine. For example, if the user is recognized as feeling "joy," the server will suggest outfits that combine bright colors and comfortable clothing.

[0328] Input: Recognized emotion information, generated coordinate information, new item information

[0329] Output: Emotion-based coordinate suggestions and new item information

[0330] Step 9: View the results

[0331] The server sends the final coordinated outfits and new item suggestions to the device, which displays the results in an easy-to-read format for the user, allowing the user to check the suggested coordinates and new items and click links to save or purchase them.

[0332] Input: Proposed coordination and new item information

[0333] Output: The final result that is displayed to the user

[0334] (Application example 2)

[0335] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0336] Existing clothing and footwear coordination suggestion systems have difficulty effectively utilizing the items a user owns and making optimal suggestions for each individual user. Furthermore, there is a lack of systems that can respond to emotional changes and real-time customer service in physical stores, making it difficult to improve user satisfaction.

[0337] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, and means for generating optimal coordination based on the extracted information. This makes it possible to propose coordination ideas and new items that take into account the emotional state of the user based on information about the items owned by the user.

[0338] A "user" is a person who uses the system to coordinate their own clothing and footwear.

[0339] "Digital images" refers to image and video data of clothing and footwear owned by the user.

[0340] "Analysis" refers to the process of extracting category information, color information, and material information of clothing and footwear from digital video.

[0341] "Category information" is information that indicates to which type the clothing and footwear belong.

[0342] "Color information" is information about the color of clothing and footwear.

[0343] "Material information" is information about the fabric and material of clothing and footwear.

[0344] "Coordination" refers to combining clothing and footwear that the user owns to create a single style.

[0345] "New items" are new products such as clothing, footwear, and accessories that the user does not yet own but are suggested to further improve coordination and style.

[0346] An "emotion engine" is a technology that analyzes a user's facial expressions and voice to recognize their current emotional state.

[0347] "Terminal" refers to the device a user uses to access the system, such as a smartphone or smart glasses.

[0348] A "server" is a computer system that has the function of analyzing digital video, generating coordinates, and sending the results to the user's terminal.

[0349] "Budget" is information indicating the range of the amount of money that the user can allocate to purchasing new items.

[0350] "Emotion information" is data about the user's emotional state obtained by the emotion engine.

[0351] This invention combines a system that captures a user's clothing and footwear as digital images, analyzes the images, and suggests optimal outfits and new items, with an emotion engine that recognizes the user's emotions.The system of the present invention is designed to effectively utilize the items a user owns, allowing them to easily enjoy unique and effective fashion.

[0352] This system mainly performs the following processes:

[0353] First, a user uses a device such as a smartphone or smart glasses to capture digital images of their clothing and footwear. The images are then stored locally on the device, encrypted, and uploaded to a server. The hardware used can be smart glasses (e.g., a typical smart glasses device) or a smartphone (e.g., a typical smartphone device).

[0354] The server decrypts the received encrypted video data and uses an AI analysis module to recognize the clothing and footwear in the video. Common data decoding techniques are used for the decryption, and deep learning frameworks such as TensorFlow are used for the analysis. This analysis extracts information about the category, color, and material.

[0355] The server then generates the optimal outfit based on the analysis results. Factors taken into consideration include fashion rules, the latest trends, and the user's past style history and preferences. The generated outfit also includes suggestions for new items based on the user's budget. A common API is used to obtain the latest price information from an external shopping information database.

[0356] Furthermore, when the user inputs facial expressions or voice via the device, this is analyzed by the emotion engine to identify the user's emotional state. For example, if the user is smiling, emotional information such as "joy" is extracted, and if the user has a serious expression, emotional information such as "concentration" is extracted. The emotion engine uses emotion recognition software such as EmoPy and Affectiva.

[0357] Based on this emotional information, the server can suggest outfits and new items that match the user's current mood. For example, if the server detects that the user is in a high mood, it can suggest outfits that combine bright colors and comfortable clothing.

[0358] The final coordinated outfits and new item suggestions are sent from the server to the user's device, where they are displayed in the app and notified to the user. The user can then review the suggested coordinates and new items and save or purchase them as needed.

[0359] For example, a user puts on smart glasses, takes a photo of a white shirt, blue jeans, and red sneakers they have selected in a store, and sends the image data to a server. The server analyzes the photo and suggests the optimal casual outfit combination of a white shirt, blue jeans, and red sneakers. At the same time, it also suggests new items such as a black jacket or accessories based on the latest price information obtained from an external database.

[0360] An example of a prompt for a generative AI model is as follows:

[0361] "Please suggest the best outfit for your customer using the following items:

[0362] Items: White shirt, blue jeans, red sneakers

[0363] Emotion: Joy

[0364] Budget: Under 10,000 yen

[0365] Suggested style: Casual

[0366] In this way, a system is realized that takes into account the user's emotions and makes optimal fashion suggestions in real time.

[0367] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0368] Step 1:

[0369] The user takes a picture of their clothing and footwear using a smartphone or smart glasses. Specifically, the camera on the smart glasses is used to capture video of the clothing and footwear, obtaining digital video data. This video data is stored in the device. The input is the digital video captured by the user, and the output is the digital video data stored in the device.

[0370] Step 2:

[0371] The device uploads the captured digital video data to a server. Specifically, the digital video data is encrypted and sent to the server via a network. The input is the digital video data stored in the device, and the output is the encrypted digital video data sent to the server.

[0372] Step 3:

[0373] The server decrypts the received encrypted video data and uses an AI analysis module to recognize the clothing and footwear in the video. The input is the encrypted digital video data received by the server, and the output is the analysis results including category information, color information, and material information. Specifically, the AI ​​analysis module (using TensorFlow, for example) recognizes the clothing and footwear from each frame of the video and extracts their features.

[0374] Step 4:

[0375] The server generates the optimal outfit based on the analysis results. The input is the analyzed clothing and footwear information (category information, color information, material information), and the output is the optimal outfit information. Specifically, the outfit is generated using an AI model that takes into account the user's past style history and preferences, fashion rules, and trend information.

[0376] Step 5:

[0377] The server proposes new items based on the user's budget. The input is the user's budget information and existing coordinate information, and the output is information on the proposed new items. Specifically, the server obtains the latest price information from an external shopping information database and selects the optimal items within the user's budget.

[0378] Step 6:

[0379] The user inputs facial expressions and voice data via the device, which then analyzes them using an emotion engine. The input is the user's facial and voice data, and the output is emotional information. Specifically, facial expressions and voice are captured using the camera and microphone of a smartphone or smart glasses, and emotions are analyzed using emotion recognition software (e.g., EmoPy or Affectiva).

[0380] Step 7:

[0381] The server proposes outfits and new items that match the user's current mood based on the emotional information recognized by the emotion engine. The input is emotional information and existing outfit information, and the output is optimal outfit information based on the emotion and suggested new items. Specifically, a generative AI model is used to propose items that match the user's emotional state.

[0382] Step 8:

[0383] The server sends the generated coordinates and new item suggestions to the device. The input is the generated coordinate information and new item suggestion information, and the output is the coordinates and new item suggestions sent to the device. Specifically, the information is sent to the device via the network, and the device displays it within the app.

[0384] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0385] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0386] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0387] [Second embodiment]

[0388] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0389] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0390] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0391] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0392] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0393] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0394] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0395] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0396] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0397] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0398] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0399] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0400] This invention relates to a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The system of the present invention is designed to enable users to make the most of the items they already own and easily enjoy unique and effective fashion.

[0401] Program processing

[0402] Video recording and uploading

[0403] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. After filming is complete, the device uploads the video data to the server. When uploaded, the video data is encrypted and transmitted via secure communication.

[0404] Video analysis and item recognition

[0405] The server decompresses the received video data and inputs it into an AI model for video analysis. The AI ​​model then recognizes the clothing and footwear in each frame and extracts category information (e.g., tops, bottoms, shoes), color information, pattern information, and material information. Based on the results of this analysis, all data is recorded in the user's profile.

[0406] Coordinate generation

[0407] The server generates the optimal outfit based on the analyzed item information. This takes into account fashion rules and the latest trends. It also reflects the user's past selection history and style preferences. For example, if a user has a white shirt, blue jeans, and red sneakers, the server will suggest a casual outfit combining these items.

[0408] New item proposals

[0409] The server then suggests new items based on the user's budget and style preferences. It retrieves the latest price information from the shopping API and a database of partner retailers, and uses that data to select the new items that best suit the user. For example, it suggests a black jacket or accessories that go well with a casual style.

[0410] Displaying the results

[0411] Finally, the server sends the generated outfits and new item suggestions to the device, which receives them and displays them in the user's smartphone app. The user can then review the suggestions and save their favorite outfits, or click a link to purchase the new items on an online shopping site.

[0412] Specific examples

[0413] 1. A user launches a smartphone app and takes a photo of a white shirt, blue jeans, and red sneakers.

[0414] 2. The device uploads the video data to the server.

[0415] 3. The server analyzes the video and recognizes each item (white shirt, blue jeans, red sneakers).

[0416] 4. Based on the analysis results, the server generates the optimal casual outfit combining a white shirt, blue jeans, and red sneakers.

[0417] 5. The server suggests new items, such as a black jacket or accessories, based on the user's budget and preferences.

[0418] 6. The device receives the generated outfit and new item suggestions and displays them to the user. The user can save the outfit or click a link to purchase the new item.

[0419] In this way, users can easily incorporate new fashion items while utilizing items they already have.

[0420] The processing flow will be explained below.

[0421] Step 1:

[0422] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. After filming is complete, the video data is automatically saved in the app.

[0423] Step 2:

[0424] Prepare the video data captured by the device for uploading to the server. Encrypt the video data before uploading.

[0425] Step 3:

[0426] The device sends the encrypted video data over the network to the server, where the network connection is verified and the data integrity is checked during transmission.

[0427] Step 4:

[0428] The server decompresses the video data received from the device, and after decompression, the video data is input into the AI ​​analysis module.

[0429] Step 5:

[0430] The AI ​​analysis module on the server analyzes the video data, extracts images of clothing and footwear from each frame of the video, and uses object recognition algorithms to identify each item. Information such as category, color, pattern, and material of each identified item is then extracted.

[0431] Step 6:

[0432] The server generates the optimal outfit for the user based on the analyzed item information. The generation process uses pre-programmed fashion rules and trend data, and also takes into account the user's past style history and preferences.

[0433] Step 7:

[0434] The server references the user's budget information and preferred style to suggest new items, and uses an external shopping API to get the latest price information and select new items to include in the suggestions.

[0435] Step 8:

[0436] The server packages the generated outfits and new item suggestions together and sends them to the device. The package includes image data, text information, a purchase link, and more.

[0437] Step 9:

[0438] The device extracts the coordination results and new item suggestions received from the server and displays them within the app. A notification is sent to the user to inform them of new suggestions.

[0439] Step 10:

[0440] Users can view suggested outfits and new items within the app, save their favorite outfits, and click a purchase link to access the online shopping site for suggested new items.

[0441] The above are the specific processing steps of the system based on the claims.

[0442] Example 1

[0443] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0444] In modern life, users need to be able to effectively utilize the clothing and footwear they own and easily incorporate the latest fashion styles. However, many users find it difficult to devote time and effort to choosing outfits, which prevents them from making the most of the items they own. Another problem is that it is difficult to instantly obtain appropriate price information when selecting or purchasing new items.

[0445] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0446] In this invention, the server includes: means for acquiring digital images of the user's clothing and footwear; means for encrypting the acquired digital images and securely transmitting them from the user's device to the server; means for decompressing the digital images received by the server and inputting them into an AI model for video analysis; means for analyzing the images using the AI ​​model and extracting information about the clothing and footwear's category, color, pattern, and material; means for saving the extracted information in the user's profile; means for generating optimal outfits based on the saved information and taking into account fashion rules and trend information; means for suggesting new items from a shopping API or retailer database based on the user's budget and price information for new items; and means for transmitting the generated outfits and suggested new items to the user's device. This allows users to make the most of their existing clothing and footwear and easily incorporate the latest fashion styles. Furthermore, when selecting or purchasing new items, users can obtain appropriate price information in real time, providing a comfortable shopping experience.

[0447] "User" refers to an individual who uses the system to obtain digital images of clothing and footwear and receive suggestions for coordination and new items.

[0448] "Clothing and footwear" refers to fashion-related items such as clothes and shoes owned by a user.

[0449] "Digital footage" refers to visual image data captured using a smartphone or other device.

[0450] "Analysis" refers to the process of extracting specific information from digital video, and includes image recognition technology.

[0451] "Category information" is information for classifying clothing and footwear in digital video, and indicates types such as tops, bottoms, and shoes.

[0452] "Color information" refers to information about the color of items in a digital image.

[0453] "Pattern" refers to information about the design or pattern of an item within a digital image.

[0454] "Material information" refers to information about the material (e.g., cotton, denim, etc.) of an item in a digital image.

[0455] A "profile" refers to a database or data store where information about a user is recorded.

[0456] "Fashion rules" refer to guidelines and standards for creating outfits, including those based on seasons and trends.

[0457] "Trend information" refers to information about current fashion trends.

[0458] "Coordination" refers to combining multiple items of clothing and footwear to create a stylish outfit.

[0459] "Shopping API" refers to an interface to an external database used to obtain information about new items.

[0460] "Retailer database" refers to a database of product information managed by a particular retailer.

[0461] "Terminal" refers to a digital device operated by a user, including a smartphone or tablet.

[0462] "Encryption" refers to the process of transforming content to preserve data security.

[0463] "Secure communications" refers to a means of communication that ensures that data is transmitted safely and without tampering.

[0464] "AI model" refers to an artificial intelligence algorithm for analyzing digital video.

[0465] This invention relates to a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The system of the present invention is designed to enable users to make the most of the items they already own and easily enjoy unique and effective fashion.

[0466] Users launch a dedicated application on their smartphone or other device. The system begins by taking a video of the clothes and shoes in their closet and tapping the "Upload" button. The device then encrypts the video data using the SSL / TLS protocol and securely transmits it to the server.

[0467] The server decompresses the received video data. It then analyzes each frame using a generative AI model that uses an object detection model such as YOLO (You Only Look Once) to extract information about the category (tops, bottoms, shoes, etc.), color, pattern, and material of the clothing and footwear in the video. The analysis results are stored in the user's profile database.

[0468] The server then generates optimal outfits based on the saved item information, referring to the latest fashion rules and trend information. This process also takes into account the user's past outfit history and style preferences. A concrete example of this system is suggesting casual outfits for a user who owns a white shirt, blue jeans, and red sneakers.

[0469] The server also uses a shopping API and retailer database to suggest new items based on the user's budget and style preferences, such as a black jacket and accessories that complement a casual style, along with up-to-date pricing information.

[0470] Finally, the server sends the generated outfits and new item suggestions to the device. The device receives them and displays them in the user's smartphone app. The user can review the suggested outfits and, if they like them, tap the "Save" button to save them. They can also purchase the new items on an online shopping site by clicking the "Purchase Link" for the items.

[0471] As a concrete example, the following prompt sentences can be effectively input into a generative AI model:

[0472] "Please suggest a casual outfit using a white shirt, blue jeans, and red sneakers. Also, please suggest some new items that would go well with this casual style."

[0473] Based on this prompt, the generative AI model can provide users with optimal outfits and suggestions for new items, allowing them to easily incorporate new fashion items while utilizing the items they already own.

[0474] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0475] Step 1:

[0476] The user launches the smartphone app and follows the instructions in the app to take a video of the clothes and shoes in their closet. For example, the user moves the camera to show off a white shirt, blue jeans, and red sneakers individually. The input is video data of the items in the closet. The output is video data stored in the device's internal storage.

[0477] Step 2:

[0478] The user taps the "Upload" button to send the video data they have taken to the server. The device encrypts this video data using the SSL / TLS protocol and sends it securely to the server. The input is the video data stored on the device. The output is the encrypted video data sent to the server.

[0479] Step 3:

[0480] The server receives the received video data and first decrypts it. Then it divides the data into frames. The input is the encrypted video data. The output is the decomposed frame data.

[0481] Step 4:

[0482] The server inputs the decomposed frame data into a generative AI model for video analysis. An AI model (e.g., YOLO) is used to recognize clothing and footwear from each frame. The input is frame-by-frame video data. The output is category information, color information, pattern information, and material information for each item.

[0483] Step 5:

[0484] The server saves the extracted item information in the user profile. For example, a white shirt is recorded in the database as "Tops, White, Solid, Cotton." The input is category information, color information, pattern information, and material information. The output is updated user profile data.

[0485] Step 6:

[0486] The server generates optimal outfits based on the saved item information and the latest fashion rules and trend information. For example, it suggests a casual style that combines a white shirt, blue jeans, and red sneakers. The input is the user profile information, fashion rules, and trend information. The output is optimal outfit data.

[0487] Step 7:

[0488] The server queries the shopping API and retailer database to suggest new items based on the user's budget and style preferences. For example, select a black jacket and accessories that go well with a casual style and obtain their price information. The input is the user profile information and price information obtained from the shopping API. The output is the suggested new item data.

[0489] Step 8:

[0490] The server sends the generated coordinates and new item proposals to the terminal. The input is the optimal coordinate data and new item proposal data. The output is the data to be sent to the terminal.

[0491] Step 9:

[0492] The device displays the received coordinations and new item suggestions within the smartphone app. The user can check the displayed content and save their favorite coordinations by tapping the "Save" button, or click the "Purchase Link" for the new item to purchase it on the online shopping site. The input is the data sent from the server. The output is the coordination and new item information displayed on the user interface.

[0493] (Application example 1)

[0494] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0495] Conventional fashion coordination systems require users to manually manage their clothing and footwear, which is time-consuming and makes it difficult to suggest optimal outfits. Furthermore, they often fail to adequately suggest new items that take into account the user's budget, style preferences, and the latest fashion trends. Furthermore, their ability to obtain the latest price information from external shopping information databases is limited, making them inconvenient for users.

[0496] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0497] In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, means for generating optimal outfits based on the extracted information, means for suggesting new items based on the user's budget and style preferences, means for transmitting the generated outfits and suggestions for new items to the user's communication terminal, and means for acquiring the latest price information from an external shopping information database. This allows the user to easily manage their items and receive suggestions for optimal outfits and the latest fashion items.

[0498] "Digital video" refers to video data of clothing and footwear owned by a user photographed with a digital device such as a camera or smartphone.

[0499] "Category information" is information indicating the type and shape of clothing and footwear extracted from digital video, such as tops, bottoms, shoes, etc.

[0500] "Color information" is information that indicates the color of clothing and footwear extracted from digital video.

[0501] "Material information" is information that indicates the material and fabric of the clothing and footwear extracted from the digital video.

[0502] "Coordination" refers to creating optimal combinations based on extracted information, and proposing effective combinations of clothing and footwear for the user to wear.

[0503] "New Items" are new clothing or footwear that the user does not already own, but that are suggested based on budget and style preferences.

[0504] A "communication terminal" is a device used by a user, such as a smartphone, tablet, or PC, which is a device for sending and receiving information.

[0505] "Shopping information database" refers to a database of an external online shopping platform or retail store for obtaining product information and price information.

[0506] A "generative AI model" is a machine learning model that uses artificial intelligence to extract information from digital images and generate optimal outfits.

[0507] A "prompt" is an instruction sentence that is input into a generative AI model to obtain a specific output.

[0508] This invention is a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The entire system is composed of a user's communication terminal and a server.

[0509] First, the user launches the smartphone app and takes a video of the clothes and footwear in their closet. After taking the video, the user's device uploads the video data to a server using a secure protocol (e.g., HTTPS). When uploaded, the video data is encrypted and transmitted via secure communication.

[0510] The server decompresses the received video data and uses a generative AI model (e.g., YOLOv5 or TensorFlow) for video analysis. The generative AI model recognizes the clothing and footwear in each frame and extracts category information (tops, bottoms, shoes, etc.), color information, pattern information, and material information. The analysis results are recorded in a NoSQL database (e.g., MongoDB).

[0511] Next, the server generates the optimal outfit based on the extracted item information, using algorithms (e.g., Python's scikit-learn, pandas) to take into account fashion rules, the latest trends, and the user's past selection history and style preferences.

[0512] Furthermore, the server suggests new items based on the user's budget and style preferences. When making suggestions, the server retrieves the latest price information from a shopping information database (e.g., Rakuten API, Amazon Product Advertising API) and selects the most suitable items based on that data. At this time, the server selects items by inputting prompt statements into the generative AI model.

[0513] Finally, the server sends the generated outfits and new item suggestions to the user's communication device. The user can check them in the smartphone app, save the suggested outfits, or click a link to purchase the new items on an online shopping site.

[0514] For example, a user launches a smartphone app, takes a photo of a white shirt, blue jeans, and red sneakers, and then inputs the following prompt into a generative AI model:

[0515] "List the items the user owns

[0516] White shirt, blue jeans, red sneakers

[0517] Based on these, please suggest the best fashion coordination and new items available for purchase.

[0518] My budget is under 5,000 yen, and I prefer a casual style.

[0519] This allows users to easily manage the items they own and receive suggestions for optimal coordination and the latest fashion items.

[0520] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0521] Step 1:

[0522] The user starts the smartphone app and takes a video of the clothes and footwear in their closet.

[0523] Input: A user uses a smartphone to take a video of the clothes and footwear in their closet.

[0524] Output: Recorded video data.

[0525] Specific operation: The user launches the smartphone app, takes a video of the clothing and footwear from an appropriate angle, and saves it within the app.

[0526] Step 2:

[0527] The device uploads the video data to the server.

[0528] Input: Video data captured in step 1.

[0529] Output: The encrypted video data is sent to the server.

[0530] Specific operation: The smartphone app encrypts the video data using SSL / TLS and securely uploads it to the server using the HTTPS protocol.

[0531] Step 3:

[0532] The server decompresses the received video data and inputs it into a generative AI model for video analysis.

[0533] Input: Encrypted video data.

[0534] Output: The decompressed video data is input into the AI ​​model.

[0535] Specific operation: The server decompresses the video data and preprocesses it for AI analysis using YOLOv5 and TensorFlow.

[0536] Step 4:

[0537] The server uses an AI model to analyze the video data and extract category, color, and material information.

[0538] Input: Decompressed video data.

[0539] Output: Category information, color information, and material information are extracted.

[0540] How it works: The server applies the generative AI model to detect clothing and footwear in each frame and identify their category, color, and material.

[0541] Step 5:

[0542] The server generates the optimal coordination based on the analyzed item information.

[0543] Input: Category information, color information and material information.

[0544] Output: Optimal coordinate information.

[0545] Specific operation: The server generates outfits using an algorithm (e.g., scikit-learn, pandas) based on fashion rules and trend information stored in a database, as well as the user's past selection history.

[0546] Step 6:

[0547] The server suggests new items based on the user's budget and style preferences.

[0548] Input: User's budget, style preferences, and coordinated outfit information.

[0549] Output: Proposed new item information.

[0550] Specific operation: The server queries the shopping information database via an API (e.g., Rakuten API, Amazon Product Advertising API) to search for new items that fit the style and are within budget.

[0551] Step 7:

[0552] The server uses a generative AI model to input prompts to suggest new items.

[0553] Input: A user request in the form of a prompt statement.

[0554] Output: Information about the selected new item.

[0555] Specific operation: The server inputs the following prompt into the generated AI model and suggests a new item:

[0556] "List the items the user owns

[0557] White shirt, blue jeans, red sneakers

[0558] Based on these, please suggest the best fashion coordination and new items available for purchase.

[0559] My budget is under 5,000 yen, and I prefer a casual style.

[0560] Step 8:

[0561] The server transmits the generated coordinates and new item suggestions to the user's communication terminal.

[0562] Input: optimal coordination information, new item information.

[0563] Output: Coordination and new item suggestions are displayed on the user's device.

[0564] How it works: The server packages the generated outfits and new item suggestions in JSON format and sends them to the user's device using the HTTPS protocol. The user can then view them in the app, save the outfits, or click on a link to purchase the new items.

[0565] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0566] This invention combines a system that captures a user's clothing and footwear as digital images, analyzes the images, and suggests optimal outfits and new items, with an emotion engine that recognizes the user's emotions.The system of the present invention is designed to effectively utilize the items a user owns, allowing them to easily enjoy unique and effective fashion.

[0567] Program processing

[0568] Video recording and uploading

[0569] Users launch the smartphone app and take videos of the clothes and shoes in their closet. After filming is complete, the video data is automatically saved in the app.

[0570] Uploading video data

[0571] The device prepares the captured video data for uploading to the server, encrypts the video data, and then transmits the encrypted video data to the server via the network.

[0572] Video analysis and item recognition

[0573] The server decompresses the video data received from the device and inputs it into the AI ​​analysis module, where the AI ​​model recognizes clothing and footwear from each frame of the video and extracts category, color, and material information. The analysis results are recorded in the user's profile.

[0574] Coordinate generation

[0575] The server generates the optimal outfit based on the analyzed item information. This generation process takes into account fashion rules and the latest trends, as well as the user's past style history and preferences. The generated outfit is designed to be optimal for the user.

[0576] New item proposals

[0577] The server suggests new items based on the user's budget. It obtains the latest price information from an external shopping information database and selects new items based on that information. For example, it may suggest jackets and accessories that go well with a casual style.

[0578] Acquisition and analysis of emotional information

[0579] When a user inputs their facial expressions and voice into the smartphone app, the device analyzes them with an emotion engine to identify the user's emotional state. For example, if the user is smiling, it will extract emotional information such as "joy," and if the user has a serious expression, it will extract emotional information such as "concentration."

[0580] Coordination and new item suggestions that take emotions into account

[0581] The server then uses the emotional information recognized by the emotion engine to suggest outfits and new items that match the user's current mood. For example, if the server detects that the user is in a high mood, it will suggest outfits that combine bright colors and comfortable clothing.

[0582] Displaying the results

[0583] Finally, the server sends the generated outfits and new item suggestions to the device, which displays the results within the app and notifies the user, allowing the user to review the suggested outfits and new items and save or purchase them as needed.

[0584] Specific examples

[0585] 1. A user launches a smartphone app and records a video of themselves wearing a white shirt, blue jeans, and red sneakers.

[0586] 2. The device uploads the video data to the server.

[0587] 3. The server analyzes the video and recognizes the white shirt, blue jeans, and red sneakers.

[0588] 4. The server generates the optimal casual outfit consisting of a white shirt, blue jeans, and red sneakers.

[0589] 5. The server suggests new items, such as a black jacket or accessories, based on the user's budget.

[0590] 6. The user inputs their facial expressions and voice within the app, and the device analyzes them with its emotion engine. For example, if the user is smiling, it will be recognized as "joy."

[0591] 7. Based on the emotional information recognized as "joy," the server will further suggest coordination of bright colors and comfortable textures.

[0592] 8. The device receives the generated outfits and new item suggestions and displays them to the user. The user can like the suggestions and click the save or purchase link.

[0593] In this way, users can easily incorporate new fashion items based on their emotions while using items they already own.

[0594] The processing flow will be explained below.

[0595] Step 1:

[0596] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. Once the video is complete, it is automatically saved in the app.

[0597] Step 2:

[0598] The device prepares to upload the video data it has taken to the server, encrypts the video content, and once encryption is complete, sends the video data to the server via the network.

[0599] Step 3:

[0600] The server decompresses the video data received from the device, and the decompressed data is input into the AI ​​analysis module.

[0601] Step 4:

[0602] The AI ​​analysis module in the server performs image analysis to identify clothes and shoes from each frame of the video, and uses object recognition algorithms to extract information about the category, color, pattern, and material of the clothing and footwear.

[0603] Step 5:

[0604] The server then analyzes each item and generates the optimal outfit using the user's existing items, taking into account the latest fashion trends and style rules.

[0605] Step 6:

[0606] The server uses an emotion engine to understand the user's emotions by analyzing the user's facial expressions and voice data. Facial expression analysis recognizes the user's facial features and determines their emotional state, such as smiling or serious.

[0607] Step 7:

[0608] The server analyzes the user's emotional information and then suggests outfits that match the user's current mood. For example, if the user is in a happy state, the server suggests outfits that include bright colors and items with positive impressions.

[0609] Step 8:

[0610] At the same time, the server also suggests new items, referencing an external shopping information database to select the latest items that fit the user's budget and include them in the suggestions.

[0611] Step 9:

[0612] The server packages the generated outfits and new item suggestions and sends them to the device. The package includes image data, detailed information, and a purchase link.

[0613] Step 10:

[0614] The device unpacks the result package received from the server and displays it within the smartphone app, and a notification is sent to the user to inform them of new suggestions.

[0615] Step 11:

[0616] Users can view suggested outfits and new items within the app, and can save their favorite outfits or items within the app, or click a purchase link to purchase them on an online shopping site.

[0617] This allows users to intuitively and easily receive optimal fashion coordination based on their emotional state and the items they own, as well as new suggested items.

[0618] Example 2

[0619] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0620] Today's consumers want to enjoy creating unique and optimal fashion coordination while effectively utilizing the clothing and footwear they own. However, conventional systems have difficulty in suggesting real-time coordination and new items that take the user's emotional state into account, which has led to issues in not being able to increase user satisfaction. Furthermore, in order to make suggestions that match the user's budget and preferred style, it is necessary to consider many factors simultaneously, which increases the system's complexity.

[0621] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, means for generating an optimal outfit based on the extracted information, means for suggesting new items based on the user's budget, means for suggesting outfits and new items that match the user's current mood using an emotion engine that recognizes the user's emotions, and means for transmitting the generated outfits and new item suggestions to the user's terminal. This allows the user to easily incorporate new fashion items based on their emotions while utilizing items they already own.

[0622] "User" refers to a consumer who uses this system.

[0623] "Clothing and footwear" refers to fashion items such as clothes and shoes that are owned by a user and stored in a closet.

[0624] "Digital video" refers to image and video data captured by a camera such as a smartphone and stored on a digital device.

[0625] "Analysis" refers to the processing of data to extract specific information from captured digital images.

[0626] "Category information" is information that indicates the type of clothing or footwear, and includes, for example, "shirts," "jeans," and "sneakers."

[0627] "Color information" refers to information about the color of clothing or footwear, and refers to data that identifies colors such as "white," "blue," and "red."

[0628] "Material information" is information about the materials used to manufacture clothing or footwear, and includes, for example, "cotton," "denim," and "leather."

[0629] "Generating coordination" refers to proposing the optimal style by combining multiple pieces of clothing and footwear based on fashion rules, trend information, the user's preferences, and past style history.

[0630] "Suggesting new items" means selecting the latest fashion items from an external shopping information database based on the user's budget and style, and encouraging them to purchase them.

[0631] An "emotion engine" refers to a software module that analyzes a user's facial expressions and voice data to identify their emotional state.

[0632] A "terminal" is a digital device used by a user, including a smartphone, tablet, computer, etc.

[0633] "Server" refers to a computer system that provides functions such as data analysis, coordinate generation, new item suggestions, and emotional information processing.

[0634] "Trend information" is data about current trends and popular styles in the fashion industry.

[0635] This invention is a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. This system also incorporates an emotion engine that recognizes the user's emotions, allowing it to suggest outfits that match the user's mood.

[0636] The specific operation of the system will now be described.

[0637] 1. Video recording and data storage

[0638] Users launch the smartphone app and take videos of the clothes and shoes in their closet. The smartphone can be an iPhone or an Android device, and the video data is automatically saved in the app.

[0639] 2. Uploading and Encrypting Video Data

[0640] To upload the captured video data to the server, the device first encrypts the video data using the AES-256 algorithm, and then transmits the encrypted video data to the server via the Internet.

[0641] 3. Video analysis and item recognition

[0642] The server decompresses the video data received from the device and inputs it into an AI analysis module, such as TensorFlow or PyTorch. The server analyzes the video frame by frame and extracts information about the category, color, and material of the clothing and footwear. For example, it uses an object detection algorithm (such as YOLO or SSD) to identify the items.

[0643] 4. Optimal Coordination Generation

[0644] The server then generates the optimal outfit based on the extracted item information, taking into account fashion rules and the latest trends. The generation process also takes into account the user's past style history and preferences. This process utilizes fashion-related APIs from Google Cloud AI and Amazon Web Services.

[0645] 5. Proposal for new items

[0646] The server retrieves up-to-date price information from external shopping information databases (e.g., Shopify or Amazon APIs) and suggests new items based on the user's budget, such as a black jacket or accessories.

[0647] 6. Acquisition and analysis of emotional information

[0648] When a user inputs their facial expressions or voice into the smartphone app, the device analyzes them with an emotion engine. For example, it uses Microsoft Azure's Emotion API or Google Cloud's Vision AI to extract emotional information such as "joy" or "concentration."

[0649] 7. Emotion-based recommendations

[0650] The server then suggests outfits and new items that match the user's mood based on the emotional information recognized by the emotion engine. For example, if the user is recognized as feeling "joy," the server will suggest outfits that combine bright colors and comfortable clothing.

[0651] 8. Displaying the results

[0652] The server then sends the resulting coordinates and new item suggestions to the device, which displays the results within the app. The user can then review the suggested coordinates and new items and click on the links to save or purchase them.

[0653] Specific examples

[0654] Suppose a user launches a smartphone app and takes a video of themselves wearing a white shirt, blue jeans, and red sneakers. The device encrypts the video data and uploads it to the server. The server analyzes the video and extracts information about each item. Based on the extracted information, the server generates a casual outfit that combines a white shirt, blue jeans, and red sneakers. It also suggests other items, such as a black jacket and accessories, based on the user's budget. If the user smiles within the app, the emotion engine recognizes "happiness" and further suggests bright color coordination based on this. The final results are sent to the device, where the user can review the suggestions and save or purchase them within the app.

[0655] Prompt Sentence Examples

[0656] "Please suggest the perfect casual outfit based on the white shirt, blue jeans, and red sneakers I have on hand. Also, please suggest new items within a budget of 10,000 yen. And please suggest an outfit that you think will please me."

[0657] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0658] Step 1: Recording video and saving data

[0659] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. The smartphone used can be an iPhone or an Android device. After filming, the video data is automatically saved within the app.

[0660] Input: User-recorded video

[0661] Output: Video data stored on the smartphone

[0662] Step 2: Prepare and encrypt video data for upload

[0663] The device prepares the captured video data for upload to the server. First, it encrypts the video data using the AES-256 algorithm. This encryption process runs in the background and a notification is displayed to the user.

[0664] Input: Video data stored on your smartphone

[0665] Output: Encrypted video data

[0666] Step 3: Sending video data to the server

[0667] The device sends encrypted video data to a server via the Internet, and HTTPS is generally used as the protocol for network connection.

[0668] Input: Encrypted video data

[0669] Output: Encrypted video data sent to the server

[0670] Step 4: Extract and analyze the video data

[0671] The server decompresses the received video data and inputs it into the AI ​​analysis module. The data is sequentially decomposed frame by frame. The analysis modules used include TensorFlow and PyTorch. The analysis uses object detection algorithms (YOLO and SSD) to extract category, color, and material information for clothing and footwear.

[0672] Input: Encrypted video data sent to the server

[0673] Output: Clothing and footwear category information, color information, material information

[0674] Step 5: Generate optimal coordinates

[0675] The server then generates the optimal outfit based on the extracted item information, taking into account fashion rules and the latest trends. This process also takes into account the user's past style history and preferences. The APIs used include fashion-related APIs from Google Cloud AI and Amazon Web Services.

[0676] Input: Clothing and footwear category information, color information, material information

[0677] Output: Generated coordinate information

[0678] Step 6: Propose new items

[0679] The server retrieves up-to-date price information from external shopping databases (e.g., Shopify or Amazon's API) and suggests new items based on the user's budget, such as a black jacket or accessories.

[0680] Input: User's budget information, price information from external shopping database

[0681] Output: Information about the proposed new item

[0682] Step 7: Acquiring and analyzing emotional information

[0683] Users input their facial expressions and voices into the smartphone app, which the device then analyzes using an emotion engine. The analysis engines used include Microsoft Azure's Emotion API and Google Cloud's Vision AI. The analyzed emotional information is extracted as "joy," "concentration," etc.

[0684] Input: User's facial expression data and voice data

[0685] Output: Recognized emotion information

[0686] Step 8: Emotion-Based Proposals

[0687] The server then suggests outfits and new items that match the user's mood based on the emotional information recognized by the emotion engine. For example, if the user is recognized as feeling "joy," the server will suggest outfits that combine bright colors and comfortable clothing.

[0688] Input: Recognized emotion information, generated coordinate information, new item information

[0689] Output: Emotion-based coordinate suggestions and new item information

[0690] Step 9: View the results

[0691] The server sends the final coordinated outfits and new item suggestions to the device, which displays the results in an easy-to-read format for the user, allowing the user to check the suggested coordinates and new items and click links to save or purchase them.

[0692] Input: Proposed coordination and new item information

[0693] Output: The final result that is displayed to the user

[0694] (Application example 2)

[0695] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0696] Existing clothing and footwear coordination suggestion systems have difficulty effectively utilizing the items a user owns and making optimal suggestions for each individual user. Furthermore, there is a lack of systems that can respond to emotional changes and real-time customer service in physical stores, making it difficult to improve user satisfaction.

[0697] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, and means for generating optimal coordination based on the extracted information. This makes it possible to propose coordination ideas and new items that take into account the emotional state of the user based on information about the items owned by the user.

[0698] A "user" is a person who uses the system to coordinate their own clothing and footwear.

[0699] "Digital images" refers to image and video data of clothing and footwear owned by the user.

[0700] "Analysis" refers to the process of extracting category information, color information, and material information of clothing and footwear from digital video.

[0701] "Category information" is information that indicates to which type the clothing and footwear belong.

[0702] "Color information" is information about the color of clothing and footwear.

[0703] "Material information" is information about the fabric and material of clothing and footwear.

[0704] "Coordination" refers to combining clothing and footwear that the user owns to create a single style.

[0705] "New items" are new products such as clothing, footwear, and accessories that the user does not yet own but are suggested to further improve coordination and style.

[0706] An "emotion engine" is a technology that analyzes a user's facial expressions and voice to recognize their current emotional state.

[0707] "Terminal" refers to the device a user uses to access the system, such as a smartphone or smart glasses.

[0708] A "server" is a computer system that has the function of analyzing digital video, generating coordinates, and sending the results to the user's terminal.

[0709] "Budget" is information indicating the range of the amount of money that the user can allocate to purchasing new items.

[0710] "Emotion information" is data about the user's emotional state obtained by the emotion engine.

[0711] This invention combines a system that captures a user's clothing and footwear as digital images, analyzes the images, and suggests optimal outfits and new items, with an emotion engine that recognizes the user's emotions.The system of the present invention is designed to effectively utilize the items a user owns, allowing them to easily enjoy unique and effective fashion.

[0712] This system mainly performs the following processes:

[0713] First, a user uses a device such as a smartphone or smart glasses to capture digital images of their clothing and footwear. The images are then stored locally on the device, encrypted, and uploaded to a server. The hardware used can be smart glasses (e.g., a typical smart glasses device) or a smartphone (e.g., a typical smartphone device).

[0714] The server decrypts the received encrypted video data and uses an AI analysis module to recognize the clothing and footwear in the video. Common data decoding techniques are used for the decryption, and deep learning frameworks such as TensorFlow are used for the analysis. This analysis extracts information about the category, color, and material.

[0715] The server then generates the optimal outfit based on the analysis results. Factors taken into consideration include fashion rules, the latest trends, and the user's past style history and preferences. The generated outfit also includes suggestions for new items based on the user's budget. A common API is used to obtain the latest price information from an external shopping information database.

[0716] Furthermore, when the user inputs facial expressions or voice via the device, this is analyzed by the emotion engine to identify the user's emotional state. For example, if the user is smiling, emotional information such as "joy" is extracted, and if the user has a serious expression, emotional information such as "concentration" is extracted. The emotion engine uses emotion recognition software such as EmoPy and Affectiva.

[0717] Based on this emotional information, the server can suggest outfits and new items that match the user's current mood. For example, if the server detects that the user is in a high mood, it can suggest outfits that combine bright colors and comfortable clothing.

[0718] The final coordinated outfits and new item suggestions are sent from the server to the user's device, where they are displayed in the app and notified to the user. The user can then review the suggested coordinates and new items and save or purchase them as needed.

[0719] For example, a user puts on smart glasses, takes a photo of a white shirt, blue jeans, and red sneakers they have selected in a store, and sends the image data to a server. The server analyzes the photo and suggests the optimal casual outfit combination of a white shirt, blue jeans, and red sneakers. At the same time, it also suggests new items such as a black jacket or accessories based on the latest price information obtained from an external database.

[0720] An example of a prompt for a generative AI model is as follows:

[0721] "Please suggest the best outfit for your customer using the following items:

[0722] Items: White shirt, blue jeans, red sneakers

[0723] Emotion: Joy

[0724] Budget: Under 10,000 yen

[0725] Suggested style: Casual

[0726] In this way, a system is realized that takes into account the user's emotions and makes optimal fashion suggestions in real time.

[0727] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0728] Step 1:

[0729] The user takes a picture of their clothing and footwear using a smartphone or smart glasses. Specifically, the camera on the smart glasses is used to capture video of the clothing and footwear, obtaining digital video data. This video data is stored in the device. The input is the digital video captured by the user, and the output is the digital video data stored in the device.

[0730] Step 2:

[0731] The device uploads the captured digital video data to a server. Specifically, the digital video data is encrypted and sent to the server via a network. The input is the digital video data stored in the device, and the output is the encrypted digital video data sent to the server.

[0732] Step 3:

[0733] The server decrypts the received encrypted video data and uses an AI analysis module to recognize the clothing and footwear in the video. The input is the encrypted digital video data received by the server, and the output is the analysis results including category information, color information, and material information. Specifically, the AI ​​analysis module (using TensorFlow, for example) recognizes the clothing and footwear from each frame of the video and extracts their features.

[0734] Step 4:

[0735] The server generates the optimal outfit based on the analysis results. The input is the analyzed clothing and footwear information (category information, color information, material information), and the output is the optimal outfit information. Specifically, the outfit is generated using an AI model that takes into account the user's past style history and preferences, fashion rules, and trend information.

[0736] Step 5:

[0737] The server proposes new items based on the user's budget. The input is the user's budget information and existing coordinate information, and the output is information on the proposed new items. Specifically, the server obtains the latest price information from an external shopping information database and selects the optimal items within the user's budget.

[0738] Step 6:

[0739] The user inputs facial expressions and voice data via the device, which then analyzes them using an emotion engine. The input is the user's facial and voice data, and the output is emotional information. Specifically, facial expressions and voice are captured using the camera and microphone of a smartphone or smart glasses, and emotions are analyzed using emotion recognition software (e.g., EmoPy or Affectiva).

[0740] Step 7:

[0741] The server proposes outfits and new items that match the user's current mood based on the emotional information recognized by the emotion engine. The input is emotional information and existing outfit information, and the output is optimal outfit information based on the emotion and suggested new items. Specifically, a generative AI model is used to propose items that match the user's emotional state.

[0742] Step 8:

[0743] The server sends the generated coordinates and new item suggestions to the device. The input is the generated coordinate information and new item suggestion information, and the output is the coordinates and new item suggestions sent to the device. Specifically, the information is sent to the device via the network, and the device displays it within the app.

[0744] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0745] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0746] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0747] [Third embodiment]

[0748] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0749] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0750] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0751] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0752] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0753] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0754] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0755] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0756] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0757] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0758] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0759] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0760] This invention relates to a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The system of the present invention is designed to enable users to make the most of the items they already own and easily enjoy unique and effective fashion.

[0761] Program processing

[0762] Video recording and uploading

[0763] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. After filming is complete, the device uploads the video data to the server. When uploaded, the video data is encrypted and transmitted via secure communication.

[0764] Video analysis and item recognition

[0765] The server decompresses the received video data and inputs it into an AI model for video analysis. The AI ​​model then recognizes the clothing and footwear in each frame and extracts category information (e.g., tops, bottoms, shoes), color information, pattern information, and material information. Based on the results of this analysis, all data is recorded in the user's profile.

[0766] Coordinate generation

[0767] The server generates the optimal outfit based on the analyzed item information. This takes into account fashion rules and the latest trends. It also reflects the user's past selection history and style preferences. For example, if a user has a white shirt, blue jeans, and red sneakers, the server will suggest a casual outfit combining these items.

[0768] New item proposals

[0769] The server then suggests new items based on the user's budget and style preferences. It retrieves the latest price information from the shopping API and a database of partner retailers, and uses that data to select the new items that best suit the user. For example, it suggests a black jacket or accessories that go well with a casual style.

[0770] Displaying the results

[0771] Finally, the server sends the generated outfits and new item suggestions to the device, which receives them and displays them in the user's smartphone app. The user can then review the suggestions and save their favorite outfits, or click a link to purchase the new items on an online shopping site.

[0772] Specific examples

[0773] 1. A user launches a smartphone app and takes a photo of a white shirt, blue jeans, and red sneakers.

[0774] 2. The device uploads the video data to the server.

[0775] 3. The server analyzes the video and recognizes each item (white shirt, blue jeans, red sneakers).

[0776] 4. Based on the analysis results, the server generates the optimal casual outfit combining a white shirt, blue jeans, and red sneakers.

[0777] 5. The server suggests new items, such as a black jacket or accessories, based on the user's budget and preferences.

[0778] 6. The device receives the generated outfit and new item suggestions and displays them to the user. The user can save the outfit or click a link to purchase the new item.

[0779] In this way, users can easily incorporate new fashion items while utilizing items they already have.

[0780] The processing flow will be explained below.

[0781] Step 1:

[0782] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. After filming is complete, the video data is automatically saved in the app.

[0783] Step 2:

[0784] Prepare the video data captured by the device for uploading to the server. Encrypt the video data before uploading.

[0785] Step 3:

[0786] The device sends the encrypted video data over the network to the server, where the network connection is verified and the data integrity is checked during transmission.

[0787] Step 4:

[0788] The server decompresses the video data received from the device, and after decompression, the video data is input into the AI ​​analysis module.

[0789] Step 5:

[0790] The AI ​​analysis module on the server analyzes the video data, extracts images of clothing and footwear from each frame of the video, and uses object recognition algorithms to identify each item. Information such as category, color, pattern, and material of each identified item is then extracted.

[0791] Step 6:

[0792] The server generates the optimal outfit for the user based on the analyzed item information. The generation process uses pre-programmed fashion rules and trend data, and also takes into account the user's past style history and preferences.

[0793] Step 7:

[0794] The server references the user's budget information and preferred style to suggest new items, and uses an external shopping API to get the latest price information and select new items to include in the suggestions.

[0795] Step 8:

[0796] The server packages the generated outfits and new item suggestions together and sends them to the device. The package includes image data, text information, a purchase link, and more.

[0797] Step 9:

[0798] The device extracts the coordination results and new item suggestions received from the server and displays them within the app. A notification is sent to the user to inform them of new suggestions.

[0799] Step 10:

[0800] Users can view suggested outfits and new items within the app, save their favorite outfits, and click a purchase link to access the online shopping site for suggested new items.

[0801] The above are the specific processing steps of the system based on the claims.

[0802] Example 1

[0803] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0804] In modern life, users need to be able to effectively utilize the clothing and footwear they own and easily incorporate the latest fashion styles. However, many users find it difficult to devote time and effort to choosing outfits, which prevents them from making the most of the items they own. Another problem is that it is difficult to instantly obtain appropriate price information when selecting or purchasing new items.

[0805] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0806] In this invention, the server includes: means for acquiring digital images of the user's clothing and footwear; means for encrypting the acquired digital images and securely transmitting them from the user's device to the server; means for decompressing the digital images received by the server and inputting them into an AI model for video analysis; means for analyzing the images using the AI ​​model and extracting information about the clothing and footwear's category, color, pattern, and material; means for saving the extracted information in the user's profile; means for generating optimal outfits based on the saved information and taking into account fashion rules and trend information; means for suggesting new items from a shopping API or retailer database based on the user's budget and price information for new items; and means for transmitting the generated outfits and suggested new items to the user's device. This allows users to make the most of their existing clothing and footwear and easily incorporate the latest fashion styles. Furthermore, when selecting or purchasing new items, users can obtain appropriate price information in real time, providing a comfortable shopping experience.

[0807] "User" refers to an individual who uses the system to obtain digital images of clothing and footwear and receive suggestions for coordination and new items.

[0808] "Clothing and footwear" refers to fashion-related items such as clothes and shoes owned by a user.

[0809] "Digital footage" refers to visual image data captured using a smartphone or other device.

[0810] "Analysis" refers to the process of extracting specific information from digital video, and includes image recognition technology.

[0811] "Category information" is information for classifying clothing and footwear in digital video, and indicates types such as tops, bottoms, and shoes.

[0812] "Color information" refers to information about the color of items in a digital image.

[0813] "Pattern" refers to information about the design or pattern of an item within a digital image.

[0814] "Material information" refers to information about the material (e.g., cotton, denim, etc.) of an item in a digital image.

[0815] A "profile" refers to a database or data store where information about a user is recorded.

[0816] "Fashion rules" refer to guidelines and standards for creating outfits, including those based on seasons and trends.

[0817] "Trend information" refers to information about current fashion trends.

[0818] "Coordination" refers to combining multiple items of clothing and footwear to create a stylish outfit.

[0819] "Shopping API" refers to an interface to an external database used to obtain information about new items.

[0820] "Retailer database" refers to a database of product information managed by a particular retailer.

[0821] "Terminal" refers to a digital device operated by a user, including a smartphone or tablet.

[0822] "Encryption" refers to the process of transforming content to preserve data security.

[0823] "Secure communications" refers to a means of communication that ensures that data is transmitted safely and without tampering.

[0824] "AI model" refers to an artificial intelligence algorithm for analyzing digital video.

[0825] This invention relates to a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The system of the present invention is designed to enable users to make the most of the items they already own and easily enjoy unique and effective fashion.

[0826] Users launch a dedicated application on their smartphone or other device. The system begins by taking a video of the clothes and shoes in their closet and tapping the "Upload" button. The device then encrypts the video data using the SSL / TLS protocol and securely transmits it to the server.

[0827] The server decompresses the received video data. It then analyzes each frame using a generative AI model that uses an object detection model such as YOLO (You Only Look Once) to extract information about the category (tops, bottoms, shoes, etc.), color, pattern, and material of the clothing and footwear in the video. The analysis results are stored in the user's profile database.

[0828] The server then generates optimal outfits based on the saved item information, referring to the latest fashion rules and trend information. This process also takes into account the user's past outfit history and style preferences. A concrete example of this system is suggesting casual outfits for a user who owns a white shirt, blue jeans, and red sneakers.

[0829] The server also uses a shopping API and retailer database to suggest new items based on the user's budget and style preferences, such as a black jacket and accessories that complement a casual style, along with up-to-date pricing information.

[0830] Finally, the server sends the generated outfits and new item suggestions to the device. The device receives them and displays them in the user's smartphone app. The user can review the suggested outfits and, if they like them, tap the "Save" button to save them. They can also purchase the new items on an online shopping site by clicking the "Purchase Link" for the items.

[0831] As a concrete example, the following prompt sentences can be effectively input into a generative AI model:

[0832] "Please suggest a casual outfit using a white shirt, blue jeans, and red sneakers. Also, please suggest some new items that would go well with this casual style."

[0833] Based on this prompt, the generative AI model can provide users with optimal outfits and suggestions for new items, allowing them to easily incorporate new fashion items while utilizing the items they already own.

[0834] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0835] Step 1:

[0836] The user launches the smartphone app and follows the instructions in the app to take a video of the clothes and shoes in their closet. For example, the user moves the camera to show off a white shirt, blue jeans, and red sneakers individually. The input is video data of the items in the closet. The output is video data stored in the device's internal storage.

[0837] Step 2:

[0838] The user taps the "Upload" button to send the video data they have taken to the server. The device encrypts this video data using the SSL / TLS protocol and sends it securely to the server. The input is the video data stored on the device. The output is the encrypted video data sent to the server.

[0839] Step 3:

[0840] The server receives the received video data and first decrypts it. Then it divides the data into frames. The input is the encrypted video data. The output is the decomposed frame data.

[0841] Step 4:

[0842] The server inputs the decomposed frame data into a generative AI model for video analysis. An AI model (e.g., YOLO) is used to recognize clothing and footwear from each frame. The input is frame-by-frame video data. The output is category information, color information, pattern information, and material information for each item.

[0843] Step 5:

[0844] The server saves the extracted item information in the user profile. For example, a white shirt is recorded in the database as "Tops, White, Solid, Cotton." The input is category information, color information, pattern information, and material information. The output is updated user profile data.

[0845] Step 6:

[0846] The server generates optimal outfits based on the saved item information and the latest fashion rules and trend information. For example, it suggests a casual style that combines a white shirt, blue jeans, and red sneakers. The input is the user profile information, fashion rules, and trend information. The output is optimal outfit data.

[0847] Step 7:

[0848] The server queries the shopping API and retailer database to suggest new items based on the user's budget and style preferences. For example, select a black jacket and accessories that go well with a casual style and obtain their price information. The input is the user profile information and price information obtained from the shopping API. The output is the suggested new item data.

[0849] Step 8:

[0850] The server sends the generated coordinates and new item proposals to the terminal. The input is the optimal coordinate data and new item proposal data. The output is the data to be sent to the terminal.

[0851] Step 9:

[0852] The device displays the received coordinations and new item suggestions within the smartphone app. The user can check the displayed content and save their favorite coordinations by tapping the "Save" button, or click the "Purchase Link" for the new item to purchase it on the online shopping site. The input is the data sent from the server. The output is the coordination and new item information displayed on the user interface.

[0853] (Application example 1)

[0854] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0855] Conventional fashion coordination systems require users to manually manage their clothing and footwear, which is time-consuming and makes it difficult to suggest optimal outfits. Furthermore, they often fail to adequately suggest new items that take into account the user's budget, style preferences, and the latest fashion trends. Furthermore, their ability to obtain the latest price information from external shopping information databases is limited, making them inconvenient for users.

[0856] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0857] In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, means for generating optimal outfits based on the extracted information, means for suggesting new items based on the user's budget and style preferences, means for transmitting the generated outfits and suggestions for new items to the user's communication terminal, and means for acquiring the latest price information from an external shopping information database. This allows the user to easily manage their items and receive suggestions for optimal outfits and the latest fashion items.

[0858] "Digital video" refers to video data of clothing and footwear owned by a user photographed with a digital device such as a camera or smartphone.

[0859] "Category information" is information indicating the type and shape of clothing and footwear extracted from digital video, such as tops, bottoms, shoes, etc.

[0860] "Color information" is information that indicates the color of clothing and footwear extracted from digital video.

[0861] "Material information" is information that indicates the material and fabric of the clothing and footwear extracted from the digital video.

[0862] "Coordination" refers to creating optimal combinations based on extracted information, and proposing effective combinations of clothing and footwear for the user to wear.

[0863] "New Items" are new clothing or footwear that the user does not already own, but that are suggested based on budget and style preferences.

[0864] A "communication terminal" is a device used by a user, such as a smartphone, tablet, or PC, which is a device for sending and receiving information.

[0865] "Shopping information database" refers to a database of an external online shopping platform or retail store for obtaining product information and price information.

[0866] A "generative AI model" is a machine learning model that uses artificial intelligence to extract information from digital images and generate optimal outfits.

[0867] A "prompt" is an instruction sentence that is input into a generative AI model to obtain a specific output.

[0868] This invention is a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The entire system is composed of a user's communication terminal and a server.

[0869] First, the user launches the smartphone app and takes a video of the clothes and footwear in their closet. After taking the video, the user's device uploads the video data to a server using a secure protocol (e.g., HTTPS). When uploaded, the video data is encrypted and transmitted via secure communication.

[0870] The server decompresses the received video data and uses a generative AI model (e.g., YOLOv5 or TensorFlow) for video analysis. The generative AI model recognizes the clothing and footwear in each frame and extracts category information (tops, bottoms, shoes, etc.), color information, pattern information, and material information. The analysis results are recorded in a NoSQL database (e.g., MongoDB).

[0871] Next, the server generates the optimal outfit based on the extracted item information, using algorithms (e.g., Python's scikit-learn, pandas) to take into account fashion rules, the latest trends, and the user's past selection history and style preferences.

[0872] Furthermore, the server suggests new items based on the user's budget and style preferences. When making suggestions, the server retrieves the latest price information from a shopping information database (e.g., Rakuten API, Amazon Product Advertising API) and selects the most suitable items based on that data. At this time, the server selects items by inputting prompt statements into the generative AI model.

[0873] Finally, the server sends the generated outfits and new item suggestions to the user's communication device. The user can check them in the smartphone app, save the suggested outfits, or click a link to purchase the new items on an online shopping site.

[0874] For example, a user launches a smartphone app, takes a photo of a white shirt, blue jeans, and red sneakers, and then inputs the following prompt into a generative AI model:

[0875] "List the items the user owns

[0876] White shirt, blue jeans, red sneakers

[0877] Based on these, please suggest the best fashion coordination and new items available for purchase.

[0878] My budget is under 5,000 yen, and I prefer a casual style.

[0879] This allows users to easily manage the items they own and receive suggestions for optimal coordination and the latest fashion items.

[0880] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0881] Step 1:

[0882] The user starts the smartphone app and takes a video of the clothes and footwear in their closet.

[0883] Input: A user uses a smartphone to take a video of the clothes and footwear in their closet.

[0884] Output: Recorded video data.

[0885] Specific operation: The user launches the smartphone app, takes a video of the clothing and footwear from an appropriate angle, and saves it within the app.

[0886] Step 2:

[0887] The device uploads the video data to the server.

[0888] Input: Video data captured in step 1.

[0889] Output: The encrypted video data is sent to the server.

[0890] Specific operation: The smartphone app encrypts the video data using SSL / TLS and securely uploads it to the server using the HTTPS protocol.

[0891] Step 3:

[0892] The server decompresses the received video data and inputs it into a generative AI model for video analysis.

[0893] Input: Encrypted video data.

[0894] Output: The decompressed video data is input into the AI ​​model.

[0895] Specific operation: The server decompresses the video data and preprocesses it for AI analysis using YOLOv5 and TensorFlow.

[0896] Step 4:

[0897] The server uses an AI model to analyze the video data and extract category, color, and material information.

[0898] Input: Decompressed video data.

[0899] Output: Category information, color information, and material information are extracted.

[0900] How it works: The server applies the generative AI model to detect clothing and footwear in each frame and identify their category, color, and material.

[0901] Step 5:

[0902] The server generates the optimal coordination based on the analyzed item information.

[0903] Input: Category information, color information and material information.

[0904] Output: Optimal coordinate information.

[0905] Specific operation: The server generates outfits using an algorithm (e.g., scikit-learn, pandas) based on fashion rules and trend information stored in a database, as well as the user's past selection history.

[0906] Step 6:

[0907] The server suggests new items based on the user's budget and style preferences.

[0908] Input: User's budget, style preferences, and coordinated outfit information.

[0909] Output: Proposed new item information.

[0910] Specific operation: The server queries the shopping information database via an API (e.g., Rakuten API, Amazon Product Advertising API) to search for new items that fit the style and are within budget.

[0911] Step 7:

[0912] The server uses a generative AI model to input prompts to suggest new items.

[0913] Input: A user request in the form of a prompt statement.

[0914] Output: Information about the selected new item.

[0915] Specific operation: The server inputs the following prompt into the generated AI model and suggests a new item:

[0916] "List the items the user owns

[0917] White shirt, blue jeans, red sneakers

[0918] Based on these, please suggest the best fashion coordination and new items available for purchase.

[0919] My budget is under 5,000 yen, and I prefer a casual style.

[0920] Step 8:

[0921] The server transmits the generated coordinates and new item suggestions to the user's communication terminal.

[0922] Input: optimal coordination information, new item information.

[0923] Output: Coordination and new item suggestions are displayed on the user's device.

[0924] How it works: The server packages the generated outfits and new item suggestions in JSON format and sends them to the user's device using the HTTPS protocol. The user can then view them in the app, save the outfits, or click on a link to purchase the new items.

[0925] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0926] This invention combines a system that captures a user's clothing and footwear as digital images, analyzes the images, and suggests optimal outfits and new items, with an emotion engine that recognizes the user's emotions.The system of the present invention is designed to effectively utilize the items a user owns, allowing them to easily enjoy unique and effective fashion.

[0927] Program processing

[0928] Video recording and uploading

[0929] Users launch the smartphone app and take videos of the clothes and shoes in their closet. After filming is complete, the video data is automatically saved in the app.

[0930] Uploading video data

[0931] The device prepares the captured video data for uploading to the server, encrypts the video data, and then transmits the encrypted video data to the server via the network.

[0932] Video analysis and item recognition

[0933] The server decompresses the video data received from the device and inputs it into the AI ​​analysis module, where the AI ​​model recognizes clothing and footwear from each frame of the video and extracts category, color, and material information. The analysis results are recorded in the user's profile.

[0934] Coordinate generation

[0935] The server generates the optimal outfit based on the analyzed item information. This generation process takes into account fashion rules and the latest trends, as well as the user's past style history and preferences. The generated outfit is designed to be optimal for the user.

[0936] New item proposals

[0937] The server suggests new items based on the user's budget. It obtains the latest price information from an external shopping information database and selects new items based on that information. For example, it may suggest jackets and accessories that go well with a casual style.

[0938] Acquisition and analysis of emotional information

[0939] When a user inputs their facial expressions and voice into the smartphone app, the device analyzes them with an emotion engine to identify the user's emotional state. For example, if the user is smiling, it will extract emotional information such as "joy," and if the user has a serious expression, it will extract emotional information such as "concentration."

[0940] Coordination and new item suggestions that take emotions into account

[0941] The server then uses the emotional information recognized by the emotion engine to suggest outfits and new items that match the user's current mood. For example, if the server detects that the user is in a high mood, it will suggest outfits that combine bright colors and comfortable clothing.

[0942] Displaying the results

[0943] Finally, the server sends the generated outfits and new item suggestions to the device, which displays the results within the app and notifies the user, allowing the user to review the suggested outfits and new items and save or purchase them as needed.

[0944] Specific examples

[0945] 1. A user launches a smartphone app and records a video of themselves wearing a white shirt, blue jeans, and red sneakers.

[0946] 2. The device uploads the video data to the server.

[0947] 3. The server analyzes the video and recognizes the white shirt, blue jeans, and red sneakers.

[0948] 4. The server generates the optimal casual outfit consisting of a white shirt, blue jeans, and red sneakers.

[0949] 5. The server suggests new items, such as a black jacket or accessories, based on the user's budget.

[0950] 6. The user inputs their facial expressions and voice within the app, and the device analyzes them with its emotion engine. For example, if the user is smiling, it will be recognized as "joy."

[0951] 7. Based on the emotional information recognized as "joy," the server will further suggest coordination of bright colors and comfortable textures.

[0952] 8. The device receives the generated outfits and new item suggestions and displays them to the user. The user can like the suggestions and click the save or purchase link.

[0953] In this way, users can easily incorporate new fashion items based on their emotions while using items they already own.

[0954] The processing flow will be explained below.

[0955] Step 1:

[0956] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. Once the video is complete, it is automatically saved in the app.

[0957] Step 2:

[0958] The device prepares to upload the video data it has taken to the server, encrypts the video content, and once encryption is complete, sends the video data to the server via the network.

[0959] Step 3:

[0960] The server decompresses the video data received from the device, and the decompressed data is input into the AI ​​analysis module.

[0961] Step 4:

[0962] The AI ​​analysis module in the server performs image analysis to identify clothes and shoes from each frame of the video, and uses object recognition algorithms to extract information about the category, color, pattern, and material of the clothing and footwear.

[0963] Step 5:

[0964] The server then analyzes each item and generates the optimal outfit using the user's existing items, taking into account the latest fashion trends and style rules.

[0965] Step 6:

[0966] The server uses an emotion engine to understand the user's emotions by analyzing the user's facial expressions and voice data. Facial expression analysis recognizes the user's facial features and determines their emotional state, such as smiling or serious.

[0967] Step 7:

[0968] The server analyzes the user's emotional information and then suggests outfits that match the user's current mood. For example, if the user is in a happy state, the server suggests outfits that include bright colors and items with positive impressions.

[0969] Step 8:

[0970] At the same time, the server also suggests new items, referencing an external shopping information database to select the latest items that fit the user's budget and include them in the suggestions.

[0971] Step 9:

[0972] The server packages the generated outfits and new item suggestions and sends them to the device. The package includes image data, detailed information, and a purchase link.

[0973] Step 10:

[0974] The device unpacks the result package received from the server and displays it within the smartphone app, and a notification is sent to the user to inform them of new suggestions.

[0975] Step 11:

[0976] Users can view suggested outfits and new items within the app, and can save their favorite outfits or items within the app, or click a purchase link to purchase them on an online shopping site.

[0977] This allows users to intuitively and easily receive optimal fashion coordination based on their emotional state and the items they own, as well as new suggested items.

[0978] Example 2

[0979] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0980] Today's consumers want to enjoy creating unique and optimal fashion coordination while effectively utilizing the clothing and footwear they own. However, conventional systems have difficulty in suggesting real-time coordination and new items that take the user's emotional state into account, which has led to issues in not being able to increase user satisfaction. Furthermore, in order to make suggestions that match the user's budget and preferred style, it is necessary to consider many factors simultaneously, which increases the system's complexity.

[0981] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, means for generating an optimal outfit based on the extracted information, means for suggesting new items based on the user's budget, means for suggesting outfits and new items that match the user's current mood using an emotion engine that recognizes the user's emotions, and means for transmitting the generated outfits and new item suggestions to the user's terminal. This allows the user to easily incorporate new fashion items based on their emotions while utilizing items they already own.

[0982] "User" refers to a consumer who uses this system.

[0983] "Clothing and footwear" refers to fashion items such as clothes and shoes that are owned by a user and stored in a closet.

[0984] "Digital video" refers to image and video data captured by a camera such as a smartphone and stored on a digital device.

[0985] "Analysis" refers to the processing of data to extract specific information from captured digital images.

[0986] "Category information" is information that indicates the type of clothing or footwear, and includes, for example, "shirts," "jeans," and "sneakers."

[0987] "Color information" refers to information about the color of clothing or footwear, and refers to data that identifies colors such as "white," "blue," and "red."

[0988] "Material information" is information about the materials used to manufacture clothing or footwear, and includes, for example, "cotton," "denim," and "leather."

[0989] "Generating coordination" refers to proposing the optimal style by combining multiple pieces of clothing and footwear based on fashion rules, trend information, the user's preferences, and past style history.

[0990] "Suggesting new items" means selecting the latest fashion items from an external shopping information database based on the user's budget and style, and encouraging them to purchase them.

[0991] An "emotion engine" refers to a software module that analyzes a user's facial expressions and voice data to identify their emotional state.

[0992] A "terminal" is a digital device used by a user, including a smartphone, tablet, computer, etc.

[0993] "Server" refers to a computer system that provides functions such as data analysis, coordinate generation, new item suggestions, and emotional information processing.

[0994] "Trend information" is data about current trends and popular styles in the fashion industry.

[0995] This invention is a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. This system also incorporates an emotion engine that recognizes the user's emotions, allowing it to suggest outfits that match the user's mood.

[0996] The specific operation of the system will now be described.

[0997] 1. Video recording and data storage

[0998] Users launch the smartphone app and take videos of the clothes and shoes in their closet. The smartphone can be an iPhone or an Android device, and the video data is automatically saved in the app.

[0999] 2. Uploading and Encrypting Video Data

[1000] To upload the captured video data to the server, the device first encrypts the video data using the AES-256 algorithm, and then transmits the encrypted video data to the server via the Internet.

[1001] 3. Video analysis and item recognition

[1002] The server decompresses the video data received from the device and inputs it into an AI analysis module, such as TensorFlow or PyTorch. The server analyzes the video frame by frame and extracts information about the category, color, and material of the clothing and footwear. For example, it uses an object detection algorithm (such as YOLO or SSD) to identify the items.

[1003] 4. Optimal Coordination Generation

[1004] The server then generates the optimal outfit based on the extracted item information, taking into account fashion rules and the latest trends. The generation process also takes into account the user's past style history and preferences. This process utilizes fashion-related APIs from Google Cloud AI and Amazon Web Services.

[1005] 5. Proposal for new items

[1006] The server retrieves up-to-date price information from external shopping information databases (e.g., Shopify or Amazon APIs) and suggests new items based on the user's budget, such as a black jacket or accessories.

[1007] 6. Acquisition and analysis of emotional information

[1008] When a user inputs their facial expressions or voice into the smartphone app, the device analyzes them with an emotion engine. For example, it uses Microsoft Azure's Emotion API or Google Cloud's Vision AI to extract emotional information such as "joy" or "concentration."

[1009] 7. Emotion-based recommendations

[1010] The server then suggests outfits and new items that match the user's mood based on the emotional information recognized by the emotion engine. For example, if the user is recognized as feeling "joy," the server will suggest outfits that combine bright colors and comfortable clothing.

[1011] 8. Displaying the results

[1012] The server then sends the resulting coordinates and new item suggestions to the device, which displays the results within the app. The user can then review the suggested coordinates and new items and click on the links to save or purchase them.

[1013] Specific examples

[1014] Suppose a user launches a smartphone app and takes a video of themselves wearing a white shirt, blue jeans, and red sneakers. The device encrypts the video data and uploads it to the server. The server analyzes the video and extracts information about each item. Based on the extracted information, the server generates a casual outfit that combines a white shirt, blue jeans, and red sneakers. It also suggests other items, such as a black jacket and accessories, based on the user's budget. If the user smiles within the app, the emotion engine recognizes "happiness" and further suggests bright color coordination based on this. The final results are sent to the device, where the user can review the suggestions and save or purchase them within the app.

[1015] Prompt Sentence Examples

[1016] "Please suggest the perfect casual outfit based on the white shirt, blue jeans, and red sneakers I have on hand. Also, please suggest new items within a budget of 10,000 yen. And please suggest an outfit that you think will please me."

[1017] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1018] Step 1: Recording video and saving data

[1019] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. The smartphone used can be an iPhone or an Android device. After filming, the video data is automatically saved within the app.

[1020] Input: User-recorded video

[1021] Output: Video data stored on the smartphone

[1022] Step 2: Prepare and encrypt video data for upload

[1023] The device prepares the captured video data for upload to the server. First, it encrypts the video data using the AES-256 algorithm. This encryption process runs in the background and a notification is displayed to the user.

[1024] Input: Video data stored on your smartphone

[1025] Output: Encrypted video data

[1026] Step 3: Sending video data to the server

[1027] The device sends encrypted video data to a server via the Internet, and HTTPS is generally used as the protocol for network connection.

[1028] Input: Encrypted video data

[1029] Output: Encrypted video data sent to the server

[1030] Step 4: Extract and analyze the video data

[1031] The server decompresses the received video data and inputs it into the AI ​​analysis module. The data is sequentially decomposed frame by frame. The analysis modules used include TensorFlow and PyTorch. The analysis uses object detection algorithms (YOLO and SSD) to extract category, color, and material information for clothing and footwear.

[1032] Input: Encrypted video data sent to the server

[1033] Output: Clothing and footwear category information, color information, material information

[1034] Step 5: Generate optimal coordinates

[1035] The server then generates the optimal outfit based on the extracted item information, taking into account fashion rules and the latest trends. This process also takes into account the user's past style history and preferences. The APIs used include fashion-related APIs from Google Cloud AI and Amazon Web Services.

[1036] Input: Clothing and footwear category information, color information, material information

[1037] Output: Generated coordinate information

[1038] Step 6: Propose new items

[1039] The server retrieves up-to-date price information from external shopping databases (e.g., Shopify or Amazon's API) and suggests new items based on the user's budget, such as a black jacket or accessories.

[1040] Input: User's budget information, price information from external shopping database

[1041] Output: Information about the proposed new item

[1042] Step 7: Acquiring and analyzing emotional information

[1043] Users input their facial expressions and voices into the smartphone app, which the device then analyzes using an emotion engine. The analysis engines used include Microsoft Azure's Emotion API and Google Cloud's Vision AI. The analyzed emotional information is extracted as "joy," "concentration," etc.

[1044] Input: User's facial expression data and voice data

[1045] Output: Recognized emotion information

[1046] Step 8: Emotion-Based Proposals

[1047] The server then suggests outfits and new items that match the user's mood based on the emotional information recognized by the emotion engine. For example, if the user is recognized as feeling "joy," the server will suggest outfits that combine bright colors and comfortable clothing.

[1048] Input: Recognized emotion information, generated coordinate information, new item information

[1049] Output: Emotion-based coordinate suggestions and new item information

[1050] Step 9: View the results

[1051] The server sends the final coordinated outfits and new item suggestions to the device, which displays the results in an easy-to-read format for the user, allowing the user to check the suggested coordinates and new items and click links to save or purchase them.

[1052] Input: Proposed coordination and new item information

[1053] Output: The final result that is displayed to the user

[1054] (Application example 2)

[1055] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1056] Existing clothing and footwear coordination suggestion systems have difficulty effectively utilizing the items a user owns and making optimal suggestions for each individual user. Furthermore, there is a lack of systems that can respond to emotional changes and real-time customer service in physical stores, making it difficult to improve user satisfaction.

[1057] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, and means for generating optimal coordination based on the extracted information. This makes it possible to propose coordination ideas and new items that take into account the emotional state of the user based on information about the items owned by the user.

[1058] A "user" is a person who uses the system to coordinate their own clothing and footwear.

[1059] "Digital images" refers to image and video data of clothing and footwear owned by the user.

[1060] "Analysis" refers to the process of extracting category information, color information, and material information of clothing and footwear from digital video.

[1061] "Category information" is information that indicates to which type the clothing and footwear belong.

[1062] "Color information" is information about the color of clothing and footwear.

[1063] "Material information" is information about the fabric and material of clothing and footwear.

[1064] "Coordination" refers to combining clothing and footwear that the user owns to create a single style.

[1065] "New items" are new products such as clothing, footwear, and accessories that the user does not yet own but are suggested to further improve coordination and style.

[1066] An "emotion engine" is a technology that analyzes a user's facial expressions and voice to recognize their current emotional state.

[1067] "Terminal" refers to the device a user uses to access the system, such as a smartphone or smart glasses.

[1068] A "server" is a computer system that has the function of analyzing digital video, generating coordinates, and sending the results to the user's terminal.

[1069] "Budget" is information indicating the range of the amount of money that the user can allocate to purchasing new items.

[1070] "Emotion information" is data about the user's emotional state obtained by the emotion engine.

[1071] This invention combines a system that captures a user's clothing and footwear as digital images, analyzes the images, and suggests optimal outfits and new items, with an emotion engine that recognizes the user's emotions.The system of the present invention is designed to effectively utilize the items a user owns, allowing them to easily enjoy unique and effective fashion.

[1072] This system mainly performs the following processes:

[1073] First, a user uses a device such as a smartphone or smart glasses to capture digital images of their clothing and footwear. The images are then stored locally on the device, encrypted, and uploaded to a server. The hardware used can be smart glasses (e.g., a typical smart glasses device) or a smartphone (e.g., a typical smartphone device).

[1074] The server decrypts the received encrypted video data and uses an AI analysis module to recognize the clothing and footwear in the video. Common data decoding techniques are used for the decryption, and deep learning frameworks such as TensorFlow are used for the analysis. This analysis extracts information about the category, color, and material.

[1075] The server then generates the optimal outfit based on the analysis results. Factors taken into consideration include fashion rules, the latest trends, and the user's past style history and preferences. The generated outfit also includes suggestions for new items based on the user's budget. A common API is used to obtain the latest price information from an external shopping information database.

[1076] Furthermore, when the user inputs facial expressions or voice via the device, this is analyzed by the emotion engine to identify the user's emotional state. For example, if the user is smiling, emotional information such as "joy" is extracted, and if the user has a serious expression, emotional information such as "concentration" is extracted. The emotion engine uses emotion recognition software such as EmoPy and Affectiva.

[1077] Based on this emotional information, the server can suggest outfits and new items that match the user's current mood. For example, if the server detects that the user is in a high mood, it can suggest outfits that combine bright colors and comfortable clothing.

[1078] The final coordinated outfits and new item suggestions are sent from the server to the user's device, where they are displayed in the app and notified to the user. The user can then review the suggested coordinates and new items and save or purchase them as needed.

[1079] For example, a user puts on smart glasses, takes a photo of a white shirt, blue jeans, and red sneakers they have selected in a store, and sends the image data to a server. The server analyzes the photo and suggests the optimal casual outfit combination of a white shirt, blue jeans, and red sneakers. At the same time, it also suggests new items such as a black jacket or accessories based on the latest price information obtained from an external database.

[1080] An example of a prompt for a generative AI model is as follows:

[1081] "Please suggest the best outfit for your customer using the following items:

[1082] Items: White shirt, blue jeans, red sneakers

[1083] Emotion: Joy

[1084] Budget: Under 10,000 yen

[1085] Suggested style: Casual

[1086] In this way, a system is realized that takes into account the user's emotions and makes optimal fashion suggestions in real time.

[1087] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1088] Step 1:

[1089] The user takes a picture of their clothing and footwear using a smartphone or smart glasses. Specifically, the camera on the smart glasses is used to capture video of the clothing and footwear, obtaining digital video data. This video data is stored in the device. The input is the digital video captured by the user, and the output is the digital video data stored in the device.

[1090] Step 2:

[1091] The device uploads the captured digital video data to a server. Specifically, the digital video data is encrypted and sent to the server via a network. The input is the digital video data stored in the device, and the output is the encrypted digital video data sent to the server.

[1092] Step 3:

[1093] The server decrypts the received encrypted video data and uses an AI analysis module to recognize the clothing and footwear in the video. The input is the encrypted digital video data received by the server, and the output is the analysis results including category information, color information, and material information. Specifically, the AI ​​analysis module (using TensorFlow, for example) recognizes the clothing and footwear from each frame of the video and extracts their features.

[1094] Step 4:

[1095] The server generates the optimal outfit based on the analysis results. The input is the analyzed clothing and footwear information (category information, color information, material information), and the output is the optimal outfit information. Specifically, the outfit is generated using an AI model that takes into account the user's past style history and preferences, fashion rules, and trend information.

[1096] Step 5:

[1097] The server proposes new items based on the user's budget. The input is the user's budget information and existing coordinate information, and the output is information on the proposed new items. Specifically, the server obtains the latest price information from an external shopping information database and selects the optimal items within the user's budget.

[1098] Step 6:

[1099] The user inputs facial expressions and voice data via the device, which then analyzes them using an emotion engine. The input is the user's facial and voice data, and the output is emotional information. Specifically, facial expressions and voice are captured using the camera and microphone of a smartphone or smart glasses, and emotions are analyzed using emotion recognition software (e.g., EmoPy or Affectiva).

[1100] Step 7:

[1101] The server proposes outfits and new items that match the user's current mood based on the emotional information recognized by the emotion engine. The input is emotional information and existing outfit information, and the output is optimal outfit information based on the emotion and suggested new items. Specifically, a generative AI model is used to propose items that match the user's emotional state.

[1102] Step 8:

[1103] The server sends the generated coordinates and new item suggestions to the device. The input is the generated coordinate information and new item suggestion information, and the output is the coordinates and new item suggestions sent to the device. Specifically, the information is sent to the device via the network, and the device displays it within the app.

[1104] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1105] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1106] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1107] [Fourth embodiment]

[1108] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1109] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1110] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1111] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1112] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1113] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1114] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1115] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1116] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1117] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1118] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1119] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1120] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1121] This invention relates to a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The system of the present invention is designed to enable users to make the most of the items they already own and easily enjoy unique and effective fashion.

[1122] Program processing

[1123] Video recording and uploading

[1124] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. After filming is complete, the device uploads the video data to the server. When uploaded, the video data is encrypted and transmitted via secure communication.

[1125] Video analysis and item recognition

[1126] The server decompresses the received video data and inputs it into an AI model for video analysis. The AI ​​model then recognizes the clothing and footwear in each frame and extracts category information (e.g., tops, bottoms, shoes), color information, pattern information, and material information. Based on the results of this analysis, all data is recorded in the user's profile.

[1127] Coordinate generation

[1128] The server generates the optimal outfit based on the analyzed item information. This takes into account fashion rules and the latest trends. It also reflects the user's past selection history and style preferences. For example, if a user has a white shirt, blue jeans, and red sneakers, the server will suggest a casual outfit combining these items.

[1129] New item proposals

[1130] The server then suggests new items based on the user's budget and style preferences. It retrieves the latest price information from the shopping API and a database of partner retailers, and uses that data to select the new items that best suit the user. For example, it suggests a black jacket or accessories that go well with a casual style.

[1131] Displaying the results

[1132] Finally, the server sends the generated outfits and new item suggestions to the device, which receives them and displays them in the user's smartphone app. The user can then review the suggestions and save their favorite outfits, or click a link to purchase the new items on an online shopping site.

[1133] Specific examples

[1134] 1. A user launches a smartphone app and takes a photo of a white shirt, blue jeans, and red sneakers.

[1135] 2. The device uploads the video data to the server.

[1136] 3. The server analyzes the video and recognizes each item (white shirt, blue jeans, red sneakers).

[1137] 4. Based on the analysis results, the server generates the optimal casual outfit combining a white shirt, blue jeans, and red sneakers.

[1138] 5. The server suggests new items, such as a black jacket or accessories, based on the user's budget and preferences.

[1139] 6. The device receives the generated outfit and new item suggestions and displays them to the user. The user can save the outfit or click a link to purchase the new item.

[1140] In this way, users can easily incorporate new fashion items while utilizing items they already have.

[1141] The processing flow will be explained below.

[1142] Step 1:

[1143] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. After filming is complete, the video data is automatically saved in the app.

[1144] Step 2:

[1145] Prepare the video data captured by the device for uploading to the server. Encrypt the video data before uploading.

[1146] Step 3:

[1147] The device sends the encrypted video data over the network to the server, where the network connection is verified and the data integrity is checked during transmission.

[1148] Step 4:

[1149] The server decompresses the video data received from the device, and after decompression, the video data is input into the AI ​​analysis module.

[1150] Step 5:

[1151] The AI ​​analysis module on the server analyzes the video data, extracts images of clothing and footwear from each frame of the video, and uses object recognition algorithms to identify each item. Information such as category, color, pattern, and material of each identified item is then extracted.

[1152] Step 6:

[1153] The server generates the optimal outfit for the user based on the analyzed item information. The generation process uses pre-programmed fashion rules and trend data, and also takes into account the user's past style history and preferences.

[1154] Step 7:

[1155] The server references the user's budget information and preferred style to suggest new items, and uses an external shopping API to get the latest price information and select new items to include in the suggestions.

[1156] Step 8:

[1157] The server packages the generated outfits and new item suggestions together and sends them to the device. The package includes image data, text information, a purchase link, and more.

[1158] Step 9:

[1159] The device extracts the coordination results and new item suggestions received from the server and displays them within the app. A notification is sent to the user to inform them of new suggestions.

[1160] Step 10:

[1161] Users can view suggested outfits and new items within the app, save their favorite outfits, and click a purchase link to access the online shopping site for suggested new items.

[1162] The above are the specific processing steps of the system based on the claims.

[1163] Example 1

[1164] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1165] In modern life, users need to be able to effectively utilize the clothing and footwear they own and easily incorporate the latest fashion styles. However, many users find it difficult to devote time and effort to choosing outfits, which prevents them from making the most of the items they own. Another problem is that it is difficult to instantly obtain appropriate price information when selecting or purchasing new items.

[1166] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1167] In this invention, the server includes: means for acquiring digital images of the user's clothing and footwear; means for encrypting the acquired digital images and securely transmitting them from the user's device to the server; means for decompressing the digital images received by the server and inputting them into an AI model for video analysis; means for analyzing the images using the AI ​​model and extracting information about the clothing and footwear's category, color, pattern, and material; means for saving the extracted information in the user's profile; means for generating optimal outfits based on the saved information and taking into account fashion rules and trend information; means for suggesting new items from a shopping API or retailer database based on the user's budget and price information for new items; and means for transmitting the generated outfits and suggested new items to the user's device. This allows users to make the most of their existing clothing and footwear and easily incorporate the latest fashion styles. Furthermore, when selecting or purchasing new items, users can obtain appropriate price information in real time, providing a comfortable shopping experience.

[1168] "User" refers to an individual who uses the system to obtain digital images of clothing and footwear and receive suggestions for coordination and new items.

[1169] "Clothing and footwear" refers to fashion-related items such as clothes and shoes owned by a user.

[1170] "Digital footage" refers to visual image data captured using a smartphone or other device.

[1171] "Analysis" refers to the process of extracting specific information from digital video, and includes image recognition technology.

[1172] "Category information" is information for classifying clothing and footwear in digital video, and indicates types such as tops, bottoms, and shoes.

[1173] "Color information" refers to information about the color of items in a digital image.

[1174] "Pattern" refers to information about the design or pattern of an item within a digital image.

[1175] "Material information" refers to information about the material (e.g., cotton, denim, etc.) of an item in a digital image.

[1176] A "profile" refers to a database or data store where information about a user is recorded.

[1177] "Fashion rules" refer to guidelines and standards for creating outfits, including those based on seasons and trends.

[1178] "Trend information" refers to information about current fashion trends.

[1179] "Coordination" refers to combining multiple items of clothing and footwear to create a stylish outfit.

[1180] "Shopping API" refers to an interface to an external database used to obtain information about new items.

[1181] "Retailer database" refers to a database of product information managed by a particular retailer.

[1182] "Terminal" refers to a digital device operated by a user, including a smartphone or tablet.

[1183] "Encryption" refers to the process of transforming content to preserve data security.

[1184] "Secure communications" refers to a means of communication that ensures that data is transmitted safely and without tampering.

[1185] "AI model" refers to an artificial intelligence algorithm for analyzing digital video.

[1186] This invention relates to a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The system of the present invention is designed to enable users to make the most of the items they already own and easily enjoy unique and effective fashion.

[1187] Users launch a dedicated application on their smartphone or other device. The system begins by taking a video of the clothes and shoes in their closet and tapping the "Upload" button. The device then encrypts the video data using the SSL / TLS protocol and securely transmits it to the server.

[1188] The server decompresses the received video data. It then analyzes each frame using a generative AI model that uses an object detection model such as YOLO (You Only Look Once) to extract information about the category (tops, bottoms, shoes, etc.), color, pattern, and material of the clothing and footwear in the video. The analysis results are stored in the user's profile database.

[1189] The server then generates optimal outfits based on the saved item information, referring to the latest fashion rules and trend information. This process also takes into account the user's past outfit history and style preferences. A concrete example of this system is suggesting casual outfits for a user who owns a white shirt, blue jeans, and red sneakers.

[1190] The server also uses a shopping API and retailer database to suggest new items based on the user's budget and style preferences, such as a black jacket and accessories that complement a casual style, along with up-to-date pricing information.

[1191] Finally, the server sends the generated outfits and new item suggestions to the device. The device receives them and displays them in the user's smartphone app. The user can review the suggested outfits and, if they like them, tap the "Save" button to save them. They can also purchase the new items on an online shopping site by clicking the "Purchase Link" for the items.

[1192] As a concrete example, the following prompt sentences can be effectively input into a generative AI model:

[1193] "Please suggest a casual outfit using a white shirt, blue jeans, and red sneakers. Also, please suggest some new items that would go well with this casual style."

[1194] Based on this prompt, the generative AI model can provide users with optimal outfits and suggestions for new items, allowing them to easily incorporate new fashion items while utilizing the items they already own.

[1195] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1196] Step 1:

[1197] The user launches the smartphone app and follows the instructions in the app to take a video of the clothes and shoes in their closet. For example, the user moves the camera to show off a white shirt, blue jeans, and red sneakers individually. The input is video data of the items in the closet. The output is video data stored in the device's internal storage.

[1198] Step 2:

[1199] The user taps the "Upload" button to send the video data they have taken to the server. The device encrypts this video data using the SSL / TLS protocol and sends it securely to the server. The input is the video data stored on the device. The output is the encrypted video data sent to the server.

[1200] Step 3:

[1201] The server receives the received video data and first decrypts it. Then it divides the data into frames. The input is the encrypted video data. The output is the decomposed frame data.

[1202] Step 4:

[1203] The server inputs the decomposed frame data into a generative AI model for video analysis. An AI model (e.g., YOLO) is used to recognize clothing and footwear from each frame. The input is frame-by-frame video data. The output is category information, color information, pattern information, and material information for each item.

[1204] Step 5:

[1205] The server saves the extracted item information in the user profile. For example, a white shirt is recorded in the database as "Tops, White, Solid, Cotton." The input is category information, color information, pattern information, and material information. The output is updated user profile data.

[1206] Step 6:

[1207] The server generates optimal outfits based on the saved item information and the latest fashion rules and trend information. For example, it suggests a casual style that combines a white shirt, blue jeans, and red sneakers. The input is the user profile information, fashion rules, and trend information. The output is optimal outfit data.

[1208] Step 7:

[1209] The server queries the shopping API and retailer database to suggest new items based on the user's budget and style preferences. For example, select a black jacket and accessories that go well with a casual style and obtain their price information. The input is the user profile information and price information obtained from the shopping API. The output is the suggested new item data.

[1210] Step 8:

[1211] The server sends the generated coordinates and new item proposals to the terminal. The input is the optimal coordinate data and new item proposal data. The output is the data to be sent to the terminal.

[1212] Step 9:

[1213] The device displays the received coordinations and new item suggestions within the smartphone app. The user can check the displayed content and save their favorite coordinations by tapping the "Save" button, or click the "Purchase Link" for the new item to purchase it on the online shopping site. The input is the data sent from the server. The output is the coordination and new item information displayed on the user interface.

[1214] (Application example 1)

[1215] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1216] Conventional fashion coordination systems require users to manually manage their clothing and footwear, which is time-consuming and makes it difficult to suggest optimal outfits. Furthermore, they often fail to adequately suggest new items that take into account the user's budget, style preferences, and the latest fashion trends. Furthermore, their ability to obtain the latest price information from external shopping information databases is limited, making them inconvenient for users.

[1217] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1218] In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, means for generating optimal outfits based on the extracted information, means for suggesting new items based on the user's budget and style preferences, means for transmitting the generated outfits and suggestions for new items to the user's communication terminal, and means for acquiring the latest price information from an external shopping information database. This allows the user to easily manage their items and receive suggestions for optimal outfits and the latest fashion items.

[1219] "Digital video" refers to video data of clothing and footwear owned by a user photographed with a digital device such as a camera or smartphone.

[1220] "Category information" is information indicating the type and shape of clothing and footwear extracted from digital video, such as tops, bottoms, shoes, etc.

[1221] "Color information" is information that indicates the color of clothing and footwear extracted from digital video.

[1222] "Material information" is information that indicates the material and fabric of the clothing and footwear extracted from the digital video.

[1223] "Coordination" refers to creating optimal combinations based on extracted information, and proposing effective combinations of clothing and footwear for the user to wear.

[1224] "New Items" are new clothing or footwear that the user does not already own, but that are suggested based on budget and style preferences.

[1225] A "communication terminal" is a device used by a user, such as a smartphone, tablet, or PC, which is a device for sending and receiving information.

[1226] "Shopping information database" refers to a database of an external online shopping platform or retail store for obtaining product information and price information.

[1227] A "generative AI model" is a machine learning model that uses artificial intelligence to extract information from digital images and generate optimal outfits.

[1228] A "prompt" is an instruction sentence that is input into a generative AI model to obtain a specific output.

[1229] This invention is a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. The entire system is composed of a user's communication terminal and a server.

[1230] First, the user launches the smartphone app and takes a video of the clothes and footwear in their closet. After taking the video, the user's device uploads the video data to a server using a secure protocol (e.g., HTTPS). When uploaded, the video data is encrypted and transmitted via secure communication.

[1231] The server decompresses the received video data and uses a generative AI model (e.g., YOLOv5 or TensorFlow) for video analysis. The generative AI model recognizes the clothing and footwear in each frame and extracts category information (tops, bottoms, shoes, etc.), color information, pattern information, and material information. The analysis results are recorded in a NoSQL database (e.g., MongoDB).

[1232] Next, the server generates the optimal outfit based on the extracted item information, using algorithms (e.g., Python's scikit-learn, pandas) to take into account fashion rules, the latest trends, and the user's past selection history and style preferences.

[1233] Furthermore, the server suggests new items based on the user's budget and style preferences. When making suggestions, the server retrieves the latest price information from a shopping information database (e.g., Rakuten API, Amazon Product Advertising API) and selects the most suitable items based on that data. At this time, the server selects items by inputting prompt statements into the generative AI model.

[1234] Finally, the server sends the generated outfits and new item suggestions to the user's communication device. The user can check them in the smartphone app, save the suggested outfits, or click a link to purchase the new items on an online shopping site.

[1235] For example, a user launches a smartphone app, takes a photo of a white shirt, blue jeans, and red sneakers, and then inputs the following prompt into a generative AI model:

[1236] "List the items the user owns

[1237] White shirt, blue jeans, red sneakers

[1238] Based on these, please suggest the best fashion coordination and new items available for purchase.

[1239] My budget is under 5,000 yen, and I prefer a casual style.

[1240] This allows users to easily manage the items they own and receive suggestions for optimal coordination and the latest fashion items.

[1241] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1242] Step 1:

[1243] The user starts the smartphone app and takes a video of the clothes and footwear in their closet.

[1244] Input: A user uses a smartphone to take a video of the clothes and footwear in their closet.

[1245] Output: Recorded video data.

[1246] Specific operation: The user launches the smartphone app, takes a video of the clothing and footwear from an appropriate angle, and saves it within the app.

[1247] Step 2:

[1248] The device uploads the video data to the server.

[1249] Input: Video data captured in step 1.

[1250] Output: The encrypted video data is sent to the server.

[1251] Specific operation: The smartphone app encrypts the video data using SSL / TLS and securely uploads it to the server using the HTTPS protocol.

[1252] Step 3:

[1253] The server decompresses the received video data and inputs it into a generative AI model for video analysis.

[1254] Input: Encrypted video data.

[1255] Output: The decompressed video data is input into the AI ​​model.

[1256] Specific operation: The server decompresses the video data and preprocesses it for AI analysis using YOLOv5 and TensorFlow.

[1257] Step 4:

[1258] The server uses an AI model to analyze the video data and extract category, color, and material information.

[1259] Input: Decompressed video data.

[1260] Output: Category information, color information, and material information are extracted.

[1261] How it works: The server applies the generative AI model to detect clothing and footwear in each frame and identify their category, color, and material.

[1262] Step 5:

[1263] The server generates the optimal coordination based on the analyzed item information.

[1264] Input: Category information, color information and material information.

[1265] Output: Optimal coordinate information.

[1266] Specific operation: The server generates outfits using an algorithm (e.g., scikit-learn, pandas) based on fashion rules and trend information stored in a database, as well as the user's past selection history.

[1267] Step 6:

[1268] The server suggests new items based on the user's budget and style preferences.

[1269] Input: User's budget, style preferences, and coordinated outfit information.

[1270] Output: Proposed new item information.

[1271] Specific operation: The server queries the shopping information database via an API (e.g., Rakuten API, Amazon Product Advertising API) to search for new items that fit the style and are within budget.

[1272] Step 7:

[1273] The server uses a generative AI model to input prompts to suggest new items.

[1274] Input: A user request in the form of a prompt statement.

[1275] Output: Information about the selected new item.

[1276] Specific operation: The server inputs the following prompt into the generated AI model and suggests a new item:

[1277] "List the items the user owns

[1278] White shirt, blue jeans, red sneakers

[1279] Based on these, please suggest the best fashion coordination and new items available for purchase.

[1280] My budget is under 5,000 yen, and I prefer a casual style.

[1281] Step 8:

[1282] The server transmits the generated coordinates and new item suggestions to the user's communication terminal.

[1283] Input: optimal coordination information, new item information.

[1284] Output: Coordination and new item suggestions are displayed on the user's device.

[1285] How it works: The server packages the generated outfits and new item suggestions in JSON format and sends them to the user's device using the HTTPS protocol. The user can then view them in the app, save the outfits, or click on a link to purchase the new items.

[1286] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1287] This invention combines a system that captures a user's clothing and footwear as digital images, analyzes the images, and suggests optimal outfits and new items, with an emotion engine that recognizes the user's emotions.The system of the present invention is designed to effectively utilize the items a user owns, allowing them to easily enjoy unique and effective fashion.

[1288] Program processing

[1289] Video recording and uploading

[1290] Users launch the smartphone app and take videos of the clothes and shoes in their closet. After filming is complete, the video data is automatically saved in the app.

[1291] Uploading video data

[1292] The device prepares the captured video data for uploading to the server, encrypts the video data, and then transmits the encrypted video data to the server via the network.

[1293] Video analysis and item recognition

[1294] The server decompresses the video data received from the device and inputs it into the AI ​​analysis module, where the AI ​​model recognizes clothing and footwear from each frame of the video and extracts category, color, and material information. The analysis results are recorded in the user's profile.

[1295] Coordinate generation

[1296] The server generates the optimal outfit based on the analyzed item information. This generation process takes into account fashion rules and the latest trends, as well as the user's past style history and preferences. The generated outfit is designed to be optimal for the user.

[1297] New item proposals

[1298] The server suggests new items based on the user's budget. It obtains the latest price information from an external shopping information database and selects new items based on that information. For example, it may suggest jackets and accessories that go well with a casual style.

[1299] Acquisition and analysis of emotional information

[1300] When a user inputs their facial expressions and voice into the smartphone app, the device analyzes them with an emotion engine to identify the user's emotional state. For example, if the user is smiling, it will extract emotional information such as "joy," and if the user has a serious expression, it will extract emotional information such as "concentration."

[1301] Coordination and new item suggestions that take emotions into account

[1302] The server then uses the emotional information recognized by the emotion engine to suggest outfits and new items that match the user's current mood. For example, if the server detects that the user is in a high mood, it will suggest outfits that combine bright colors and comfortable clothing.

[1303] Displaying the results

[1304] Finally, the server sends the generated outfits and new item suggestions to the device, which displays the results within the app and notifies the user, allowing the user to review the suggested outfits and new items and save or purchase them as needed.

[1305] Specific examples

[1306] 1. A user launches a smartphone app and records a video of themselves wearing a white shirt, blue jeans, and red sneakers.

[1307] 2. The device uploads the video data to the server.

[1308] 3. The server analyzes the video and recognizes the white shirt, blue jeans, and red sneakers.

[1309] 4. The server generates the optimal casual outfit consisting of a white shirt, blue jeans, and red sneakers.

[1310] 5. The server suggests new items, such as a black jacket or accessories, based on the user's budget.

[1311] 6. The user inputs their facial expressions and voice within the app, and the device analyzes them with its emotion engine. For example, if the user is smiling, it will be recognized as "joy."

[1312] 7. Based on the emotional information recognized as "joy," the server will further suggest coordination of bright colors and comfortable textures.

[1313] 8. The device receives the generated outfits and new item suggestions and displays them to the user. The user can like the suggestions and click the save or purchase link.

[1314] In this way, users can easily incorporate new fashion items based on their emotions while using items they already own.

[1315] The processing flow will be explained below.

[1316] Step 1:

[1317] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. Once the video is complete, it is automatically saved in the app.

[1318] Step 2:

[1319] The device prepares to upload the video data it has taken to the server, encrypts the video content, and once encryption is complete, sends the video data to the server via the network.

[1320] Step 3:

[1321] The server decompresses the video data received from the device, and the decompressed data is input into the AI ​​analysis module.

[1322] Step 4:

[1323] The AI ​​analysis module in the server performs image analysis to identify clothes and shoes from each frame of the video, and uses object recognition algorithms to extract information about the category, color, pattern, and material of the clothing and footwear.

[1324] Step 5:

[1325] The server then analyzes each item and generates the optimal outfit using the user's existing items, taking into account the latest fashion trends and style rules.

[1326] Step 6:

[1327] The server uses an emotion engine to understand the user's emotions by analyzing the user's facial expressions and voice data. Facial expression analysis recognizes the user's facial features and determines their emotional state, such as smiling or serious.

[1328] Step 7:

[1329] The server analyzes the user's emotional information and then suggests outfits that match the user's current mood. For example, if the user is in a happy state, the server suggests outfits that include bright colors and items with positive impressions.

[1330] Step 8:

[1331] At the same time, the server also suggests new items, referencing an external shopping information database to select the latest items that fit the user's budget and include them in the suggestions.

[1332] Step 9:

[1333] The server packages the generated outfits and new item suggestions and sends them to the device. The package includes image data, detailed information, and a purchase link.

[1334] Step 10:

[1335] The device unpacks the result package received from the server and displays it within the smartphone app, and a notification is sent to the user to inform them of new suggestions.

[1336] Step 11:

[1337] Users can view suggested outfits and new items within the app, and can save their favorite outfits or items within the app, or click a purchase link to purchase them on an online shopping site.

[1338] This allows users to intuitively and easily receive optimal fashion coordination based on their emotional state and the items they own, as well as new suggested items.

[1339] Example 2

[1340] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1341] Today's consumers want to enjoy creating unique and optimal fashion coordination while effectively utilizing the clothing and footwear they own. However, conventional systems have difficulty in suggesting real-time coordination and new items that take the user's emotional state into account, which has led to issues in not being able to increase user satisfaction. Furthermore, in order to make suggestions that match the user's budget and preferred style, it is necessary to consider many factors simultaneously, which increases the system's complexity.

[1342] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, means for generating an optimal outfit based on the extracted information, means for suggesting new items based on the user's budget, means for suggesting outfits and new items that match the user's current mood using an emotion engine that recognizes the user's emotions, and means for transmitting the generated outfits and new item suggestions to the user's terminal. This allows the user to easily incorporate new fashion items based on their emotions while utilizing items they already own.

[1343] "User" refers to a consumer who uses this system.

[1344] "Clothing and footwear" refers to fashion items such as clothes and shoes that are owned by a user and stored in a closet.

[1345] "Digital video" refers to image and video data captured by a camera such as a smartphone and stored on a digital device.

[1346] "Analysis" refers to the processing of data to extract specific information from captured digital images.

[1347] "Category information" is information that indicates the type of clothing or footwear, and includes, for example, "shirts," "jeans," and "sneakers."

[1348] "Color information" refers to information about the color of clothing or footwear, and refers to data that identifies colors such as "white," "blue," and "red."

[1349] "Material information" is information about the materials used to manufacture clothing or footwear, and includes, for example, "cotton," "denim," and "leather."

[1350] "Generating coordination" refers to proposing the optimal style by combining multiple pieces of clothing and footwear based on fashion rules, trend information, the user's preferences, and past style history.

[1351] "Suggesting new items" means selecting the latest fashion items from an external shopping information database based on the user's budget and style, and encouraging them to purchase them.

[1352] An "emotion engine" refers to a software module that analyzes a user's facial expressions and voice data to identify their emotional state.

[1353] A "terminal" is a digital device used by a user, including a smartphone, tablet, computer, etc.

[1354] "Server" refers to a computer system that provides functions such as data analysis, coordinate generation, new item suggestions, and emotional information processing.

[1355] "Trend information" is data about current trends and popular styles in the fashion industry.

[1356] This invention is a system that acquires digital images of a user's clothing and footwear, analyzes the images, and suggests optimal outfits and new items. This system also incorporates an emotion engine that recognizes the user's emotions, allowing it to suggest outfits that match the user's mood.

[1357] The specific operation of the system will now be described.

[1358] 1. Video recording and data storage

[1359] Users launch the smartphone app and take videos of the clothes and shoes in their closet. The smartphone can be an iPhone or an Android device, and the video data is automatically saved in the app.

[1360] 2. Uploading and Encrypting Video Data

[1361] To upload the captured video data to the server, the device first encrypts the video data using the AES-256 algorithm, and then transmits the encrypted video data to the server via the Internet.

[1362] 3. Video analysis and item recognition

[1363] The server decompresses the video data received from the device and inputs it into an AI analysis module, such as TensorFlow or PyTorch. The server analyzes the video frame by frame and extracts information about the category, color, and material of the clothing and footwear. For example, it uses an object detection algorithm (such as YOLO or SSD) to identify the items.

[1364] 4. Optimal Coordination Generation

[1365] The server then generates the optimal outfit based on the extracted item information, taking into account fashion rules and the latest trends. The generation process also takes into account the user's past style history and preferences. This process utilizes fashion-related APIs from Google Cloud AI and Amazon Web Services.

[1366] 5. Proposal for new items

[1367] The server retrieves up-to-date price information from external shopping information databases (e.g., Shopify or Amazon APIs) and suggests new items based on the user's budget, such as a black jacket or accessories.

[1368] 6. Acquisition and analysis of emotional information

[1369] When a user inputs their facial expressions or voice into the smartphone app, the device analyzes them with an emotion engine. For example, it uses Microsoft Azure's Emotion API or Google Cloud's Vision AI to extract emotional information such as "joy" or "concentration."

[1370] 7. Emotion-based recommendations

[1371] The server then suggests outfits and new items that match the user's mood based on the emotional information recognized by the emotion engine. For example, if the user is recognized as feeling "joy," the server will suggest outfits that combine bright colors and comfortable clothing.

[1372] 8. Displaying the results

[1373] The server then sends the resulting coordinates and new item suggestions to the device, which displays the results within the app. The user can then review the suggested coordinates and new items and click on the links to save or purchase them.

[1374] Specific examples

[1375] Suppose a user launches a smartphone app and takes a video of themselves wearing a white shirt, blue jeans, and red sneakers. The device encrypts the video data and uploads it to the server. The server analyzes the video and extracts information about each item. Based on the extracted information, the server generates a casual outfit that combines a white shirt, blue jeans, and red sneakers. It also suggests other items, such as a black jacket and accessories, based on the user's budget. If the user smiles within the app, the emotion engine recognizes "happiness" and further suggests bright color coordination based on this. The final results are sent to the device, where the user can review the suggestions and save or purchase them within the app.

[1376] Prompt Sentence Examples

[1377] "Please suggest the perfect casual outfit based on the white shirt, blue jeans, and red sneakers I have on hand. Also, please suggest new items within a budget of 10,000 yen. And please suggest an outfit that you think will please me."

[1378] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1379] Step 1: Recording video and saving data

[1380] The user launches the smartphone app and takes a video of the clothes and shoes in their closet. The smartphone used can be an iPhone or an Android device. After filming, the video data is automatically saved within the app.

[1381] Input: User-recorded video

[1382] Output: Video data stored on the smartphone

[1383] Step 2: Prepare and encrypt video data for upload

[1384] The device prepares the captured video data for upload to the server. First, it encrypts the video data using the AES-256 algorithm. This encryption process runs in the background and a notification is displayed to the user.

[1385] Input: Video data stored on your smartphone

[1386] Output: Encrypted video data

[1387] Step 3: Sending video data to the server

[1388] The device sends encrypted video data to a server via the Internet, and HTTPS is generally used as the protocol for network connection.

[1389] Input: Encrypted video data

[1390] Output: Encrypted video data sent to the server

[1391] Step 4: Extract and analyze the video data

[1392] The server decompresses the received video data and inputs it into the AI ​​analysis module. The data is sequentially decomposed frame by frame. The analysis modules used include TensorFlow and PyTorch. The analysis uses object detection algorithms (YOLO and SSD) to extract category, color, and material information for clothing and footwear.

[1393] Input: Encrypted video data sent to the server

[1394] Output: Clothing and footwear category information, color information, material information

[1395] Step 5: Generate optimal coordinates

[1396] The server then generates the optimal outfit based on the extracted item information, taking into account fashion rules and the latest trends. This process also takes into account the user's past style history and preferences. The APIs used include fashion-related APIs from Google Cloud AI and Amazon Web Services.

[1397] Input: Clothing and footwear category information, color information, material information

[1398] Output: Generated coordinate information

[1399] Step 6: Propose new items

[1400] The server retrieves up-to-date price information from external shopping databases (e.g., Shopify or Amazon's API) and suggests new items based on the user's budget, such as a black jacket or accessories.

[1401] Input: User's budget information, price information from external shopping database

[1402] Output: Information about the proposed new item

[1403] Step 7: Acquiring and analyzing emotional information

[1404] Users input their facial expressions and voices into the smartphone app, which the device then analyzes using an emotion engine. The analysis engines used include Microsoft Azure's Emotion API and Google Cloud's Vision AI. The analyzed emotional information is extracted as "joy," "concentration," etc.

[1405] Input: User's facial expression data and voice data

[1406] Output: Recognized emotion information

[1407] Step 8: Emotion-Based Proposals

[1408] The server then suggests outfits and new items that match the user's mood based on the emotional information recognized by the emotion engine. For example, if the user is recognized as feeling "joy," the server will suggest outfits that combine bright colors and comfortable clothing.

[1409] Input: Recognized emotion information, generated coordinate information, new item information

[1410] Output: Emotion-based coordinate suggestions and new item information

[1411] Step 9: View the results

[1412] The server sends the final coordinated outfits and new item suggestions to the device, which displays the results in an easy-to-read format for the user, allowing the user to check the suggested coordinates and new items and click links to save or purchase them.

[1413] Input: Proposed coordination and new item information

[1414] Output: The final result that is displayed to the user

[1415] (Application example 2)

[1416] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1417] Existing clothing and footwear coordination suggestion systems have difficulty effectively utilizing the items a user owns and making optimal suggestions for each individual user. Furthermore, there is a lack of systems that can respond to emotional changes and real-time customer service in physical stores, making it difficult to improve user satisfaction.

[1418] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring digital images of clothing and footwear owned by the user, means for analyzing the acquired digital images and extracting category information, color information, and material information of the clothing and footwear, and means for generating optimal coordination based on the extracted information. This makes it possible to propose coordination ideas and new items that take into account the emotional state of the user based on information about the items owned by the user.

[1419] A "user" is a person who uses the system to coordinate their own clothing and footwear.

[1420] "Digital images" refers to image and video data of clothing and footwear owned by the user.

[1421] "Analysis" refers to the process of extracting category information, color information, and material information of clothing and footwear from digital video.

[1422] "Category information" is information that indicates to which type the clothing and footwear belong.

[1423] "Color information" is information about the color of clothing and footwear.

[1424] "Material information" is information about the fabric and material of clothing and footwear.

[1425] "Coordination" refers to combining clothing and footwear that the user owns to create a single style.

[1426] "New items" are new products such as clothing, footwear, and accessories that the user does not yet own but are suggested to further improve coordination and style.

[1427] An "emotion engine" is a technology that analyzes a user's facial expressions and voice to recognize their current emotional state.

[1428] "Terminal" refers to the device a user uses to access the system, such as a smartphone or smart glasses.

[1429] A "server" is a computer system that has the function of analyzing digital video, generating coordinates, and sending the results to the user's terminal.

[1430] "Budget" is information indicating the range of the amount of money that the user can allocate to purchasing new items.

[1431] "Emotion information" is data about the user's emotional state obtained by the emotion engine.

[1432] This invention combines a system that captures a user's clothing and footwear as digital images, analyzes the images, and suggests optimal outfits and new items, with an emotion engine that recognizes the user's emotions.The system of the present invention is designed to effectively utilize the items a user owns, allowing them to easily enjoy unique and effective fashion.

[1433] This system mainly performs the following processes:

[1434] First, a user uses a device such as a smartphone or smart glasses to capture digital images of their clothing and footwear. The images are then stored locally on the device, encrypted, and uploaded to a server. The hardware used can be smart glasses (e.g., a typical smart glasses device) or a smartphone (e.g., a typical smartphone device).

[1435] The server decrypts the received encrypted video data and uses an AI analysis module to recognize the clothing and footwear in the video. Common data decoding techniques are used for the decryption, and deep learning frameworks such as TensorFlow are used for the analysis. This analysis extracts information about the category, color, and material.

[1436] The server then generates the optimal outfit based on the analysis results. Factors taken into consideration include fashion rules, the latest trends, and the user's past style history and preferences. The generated outfit also includes suggestions for new items based on the user's budget. A common API is used to obtain the latest price information from an external shopping information database.

[1437] Furthermore, when the user inputs facial expressions or voice via the device, this is analyzed by the emotion engine to identify the user's emotional state. For example, if the user is smiling, emotional information such as "joy" is extracted, and if the user has a serious expression, emotional information such as "concentration" is extracted. The emotion engine uses emotion recognition software such as EmoPy and Affectiva.

[1438] Based on this emotional information, the server can suggest outfits and new items that match the user's current mood. For example, if the server detects that the user is in a high mood, it can suggest outfits that combine bright colors and comfortable clothing.

[1439] The final coordinated outfits and new item suggestions are sent from the server to the user's device, where they are displayed in the app and notified to the user. The user can then review the suggested coordinates and new items and save or purchase them as needed.

[1440] For example, a user puts on smart glasses, takes a photo of a white shirt, blue jeans, and red sneakers they have selected in a store, and sends the image data to a server. The server analyzes the photo and suggests the optimal casual outfit combination of a white shirt, blue jeans, and red sneakers. At the same time, it also suggests new items such as a black jacket or accessories based on the latest price information obtained from an external database.

[1441] An example of a prompt for a generative AI model is as follows:

[1442] "Please suggest the best outfit for your customer using the following items:

[1443] Items: White shirt, blue jeans, red sneakers

[1444] Emotion: Joy

[1445] Budget: Under 10,000 yen

[1446] Suggested style: Casual

[1447] In this way, a system is realized that takes into account the user's emotions and makes optimal fashion suggestions in real time.

[1448] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1449] Step 1:

[1450] The user takes a picture of their clothing and footwear using a smartphone or smart glasses. Specifically, the camera on the smart glasses is used to capture video of the clothing and footwear, obtaining digital video data. This video data is stored in the device. The input is the digital video captured by the user, and the output is the digital video data stored in the device.

[1451] Step 2:

[1452] The device uploads the captured digital video data to a server. Specifically, the digital video data is encrypted and sent to the server via a network. The input is the digital video data stored in the device, and the output is the encrypted digital video data sent to the server.

[1453] Step 3:

[1454] The server decrypts the received encrypted video data and uses an AI analysis module to recognize the clothing and footwear in the video. The input is the encrypted digital video data received by the server, and the output is the analysis results including category information, color information, and material information. Specifically, the AI ​​analysis module (using TensorFlow, for example) recognizes the clothing and footwear from each frame of the video and extracts their features.

[1455] Step 4:

[1456] The server generates the optimal outfit based on the analysis results. The input is the analyzed clothing and footwear information (category information, color information, material information), and the output is the optimal outfit information. Specifically, the outfit is generated using an AI model that takes into account the user's past style history and preferences, fashion rules, and trend information.

[1457] Step 5:

[1458] The server proposes new items based on the user's budget. The input is the user's budget information and existing coordinate information, and the output is information on the proposed new items. Specifically, the server obtains the latest price information from an external shopping information database and selects the optimal items within the user's budget.

[1459] Step 6:

[1460] The user inputs facial expressions and voice data via the device, which then analyzes them using an emotion engine. The input is the user's facial and voice data, and the output is emotional information. Specifically, facial expressions and voice are captured using the camera and microphone of a smartphone or smart glasses, and emotions are analyzed using emotion recognition software (e.g., EmoPy or Affectiva).

[1461] Step 7:

[1462] The server proposes outfits and new items that match the user's current mood based on the emotional information recognized by the emotion engine. The input is emotional information and existing outfit information, and the output is optimal outfit information based on the emotion and suggested new items. Specifically, a generative AI model is used to propose items that match the user's emotional state.

[1463] Step 8:

[1464] The server sends the generated coordinates and new item suggestions to the device. The input is the generated coordinate information and new item suggestion information, and the output is the coordinates and new item suggestions sent to the device. Specifically, the information is sent to the device via the network, and the device displays it within the app.

[1465] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1466] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1467] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1468] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1469] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1470] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1471] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1472] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1473] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1474] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1475] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1476] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1477] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1478] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1479] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1480] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1481] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1482] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1483] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1484] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1485] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1486] The following is further disclosed regarding the above embodiment.

[1487] (Claim 1)

[1488] means for obtaining digital images of clothing and footwear owned by a user;

[1489] means for analyzing the acquired digital video and extracting category information, color information, and material information of the clothing and footwear;

[1490] A means for generating an optimal coordination based on the extracted information;

[1491] A means for suggesting new items based on the user's budget;

[1492] means for transmitting the generated coordination and new item proposals to a user's terminal;

[1493] A system including:

[1494] (Claim 2)

[1495] a means for further improving the optimal coordination by taking into consideration the user's preferred style and past coordination history;

[1496] 10. The system of claim 1.

[1497] (Claim 3)

[1498] A system including means for obtaining the latest price information from an external shopping information database when suggesting new items;

[1499] 10. The system of claim 1.

[1500] "Example 1"

[1501] (Claim 1)

[1502] means for obtaining digital images of clothing and footwear owned by a user;

[1503] A means for encrypting the acquired digital video and securely transmitting it from the terminal to the server;

[1504] A means for the server to decompress the received digital video and input it into an AI model for video analysis;

[1505] A means of analyzing using an AI model and extracting information on the category, color, pattern, and material of clothing and footwear;

[1506] means for storing the extracted information in a user profile;

[1507] A means for generating an optimal coordination based on the stored information and taking into consideration fashion rules and trend information;

[1508] A means to suggest new items from a shopping API or retailer database based on the user's budget and price information of the new items;

[1509] means for transmitting the generated coordinates and new item suggestions to a user's terminal;

[1510] A system including:

[1511] (Claim 2)

[1512] a means for further improving the optimal coordination by taking into consideration the user's preferred style and past coordination history;

[1513] 10. The system of claim 1.

[1514] (Claim 3)

[1515] including a means for obtaining up-to-date pricing information from external shopping information databases when suggesting new items;

[1516] 10. The system of claim 1.

[1517] "Application Example 1"

[1518] (Claim 1)

[1519] means for acquiring digital images of clothing and footwear owned by a user;

[1520] means for analyzing the acquired digital video and extracting category information, color information, and material information of the clothing and footwear;

[1521] A means for generating an optimal coordination based on the extracted information;

[1522] means for suggesting new items based on the user's budget and style preferences;

[1523] means for transmitting the generated coordinates and new item suggestions to a communication terminal of the user;

[1524] a means for obtaining up-to-date price information from external shopping information databases;

[1525] A system including:

[1526] (Claim 2)

[1527] a means for utilizing the generative AI model to further refine the optimal outfit, taking into account the user's preferred style and past outfit history;

[1528] 10. The system of claim 1.

[1529] (Claim 3)

[1530] When proposing a new item, a means for selecting the item by inputting a prompt sentence into the generative AI model is included.

[1531] 10. The system of claim 1.

[1532] "Example 2: Combining Emotion Engines"

[1533] (Claim 1)

[1534] means for obtaining digital images of clothing and footwear owned by a user;

[1535] means for analyzing the acquired digital video and extracting category information, color information, and material information of the clothing and footwear;

[1536] A means for generating an optimal coordination based on the extracted information;

[1537] A means for suggesting new items based on the user's budget;

[1538] A means for suggesting coordination and new items that match the user's current mood using an emotion engine that recognizes the user's emotions;

[1539] means for transmitting the generated coordination and new item proposals to a user's terminal;

[1540] A system including:

[1541] (Claim 2)

[1542] a means for further improving the optimal coordination by taking into consideration the user's preferred style and past coordination history;

[1543] 10. The system of claim 1.

[1544] (Claim 3)

[1545] including a means for obtaining up-to-date pricing information from external shopping information databases when suggesting new items;

[1546] 10. The system of claim 1.

[1547] "Application example 2 when combining emotion engines"

[1548] (Claim 1)

[1549] means for obtaining digital images of clothing and footwear owned by a user;

[1550] means for analyzing the acquired digital video and extracting category information, color information, and material information of the clothing and footwear;

[1551] A means for generating an optimal coordination based on the extracted information;

[1552] A means for suggesting new items based on the user's budget;

[1553] means for transmitting the generated coordination and new item proposals to a user's terminal;

[1554] a means for acquiring emotional information using an emotion engine that recognizes the emotions of a user, and for suggesting coordination and new items based on the emotional information;

[1555] A system including:

[1556] (Claim 2)

[1557] The system of claim 1, further comprising means for further refining the optimal outfit by taking into consideration the user's preferred style and past outfit history.

[1558] (Claim 3)

[1559] 10. The system of claim 1, further comprising means for obtaining up-to-date price information from an external shopping information database when suggesting new items. [Explanation of symbols]

[1560] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for obtaining digital images of clothing and footwear owned by a user; means for analyzing the acquired digital video and extracting category information, color information, and material information of the clothing and footwear; A means for generating an optimal coordination based on the extracted information; A means for suggesting new items based on the user's budget; means for transmitting the generated coordination and new item proposals to a user's terminal; A system including:

2. A means for further improving the optimal coordination by taking into consideration the user's preferred style and past coordination history, 10. The system of claim 1.

3. A system including means for obtaining the latest price information from an external shopping information database when suggesting new items; 10. The system of claim 1.

Images