system

Abstract

To provide a system. 【Solution means】 Means for receiving personal identification information from an input device, Means for operating a digital image generation device based on the personal identification information to generate a replicated image of a plurality of pieces of clothing being worn, Means for transmitting and presenting the generated replicated image to a display device, Means for securing inventory information of the clothing selected by the user via a communication network, Data analysis means for identifying an optimal physical store, Reservation control means for making a fitting reservation at the physical store, Display means for providing the replicated image generated through virtual fitting in a catalog format, Communication means for enabling the user to complete a fitting reservation before visiting the optimal physical store, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern times, choosing clothes for special events such as coming-of-age ceremonies and weddings has many options, and searching and trying them on requires a great deal of time and effort. In particular, in order to find the most suitable clothes for oneself, it is necessary to visit multiple stores and check the inventory status, which is a great burden for users. Therefore, there is a need for a new method that can efficiently select clothes and find those that suit oneself.

Means for Solving the Problems

[0005] ​​This invention uses AI technology to generate duplicate images of the user wearing various clothes, based on the user's personal identification information, and presents a visual catalog to enable efficient clothing selection. It also instantly checks the inventory information of the selected clothing online and identifies the most suitable physical store. Furthermore, it includes a function to automatically schedule fitting appointments, saving the user time and supporting quick and easy clothing selection and fitting. Such a system allows users to find suitable clothing in a short time, increasing their satisfaction.

[0006] "Personally identifiable information" refers to data that enables the identification of a user, including information such as facial photographs and IDs.

[0007] An "input device" is a device used by a user to provide data to a system, and smartphones and tablets are examples of this.

[0008] A "digital image generation device" is a machine and software system that generates new images using computer graphics based on input data.

[0009] A "duplicate image" is a virtual image of the user wearing different clothes, generated based on the original user's information.

[0010] A "display device" is a device used to visually present generated digital content to a user, and includes displays and monitors.

[0011] A "communication network" is a connection infrastructure for sending and receiving data, and includes the internet, among others.

[0012] "Data analysis means" refers to algorithms and software used to process and interpret data, and to make optimal choices efficiently.

[0013] "Reservation control means" refers to software and communication interfaces for automatically managing and executing the fitting reservation process. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Mode for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention provides a system that allows users to efficiently select clothing and reduce the hassle of trying them on. To implement this system, a terminal for inputting the user's facial photograph and personal identification information, a server for processing data and generating images, and a communication network for checking clothing inventory and making fitting reservations are required.

[0036] The user first inputs a photo of their face using a terminal, and the server receives this photo data and processes it using an image generation device. The image generation device uses AI technology to generate multiple digital images that simulate the user trying on clothes, and sends these to the terminal in catalog format.

[0037] When a user browses this catalog on their device and selects an item of clothing they like, the server checks the inventory of the selected item at multiple stores. During this process, it accesses each store's database via the communication network to collect information such as inventory status and available fitting times.

[0038] To suggest the most suitable store, the server uses data analysis tools to identify the optimal store based on geographical and inventory information. This information is then presented to the user's terminal, and once the user selects a store, a reservation control tool automatically makes a fitting reservation for the specified store.

[0039] As a concrete example, let's consider a case where user A wants to choose a furisode (long-sleeved kimono) for their coming-of-age ceremony. User A takes a photo of their face using their smartphone and sends it via the application. The server then uses AI to generate digital images of user A trying on multiple furisode styles and sends them to the user's device in an easy-to-view catalog format. User A reviews this catalog and selects a red furisode. The server searches a database of nearby rental shops and identifies the best store with the selected garment in stock, suggesting it to user A. When user A chooses to make a fitting reservation at the store, the server uses a reservation control mechanism to confirm the reservation. This embodiment allows users to efficiently select clothing and prepare for a specific event with reduced burden.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The user takes a photo of their face using their device and sends that photo to the server via the application.

[0043] Step 2:

[0044] The server checks the received facial photograph, performs pre-processing such as adjusting the image format, and then prepares it for transmission to the digital image generation device.

[0045] Step 3:

[0046] The server runs an image generation AI to generate digital replicas of the user wearing multiple outfits, based on the user's facial photograph.

[0047] Step 4:

[0048] The server organizes the generated duplicate images into a catalog, transfers them to the user's terminal, and displays them.

[0049] Step 5:

[0050] The user browses a catalog presented through their device and selects the clothing items they like. Once the selection is complete, the device sends that information to the server.

[0051] Step 6:

[0052] The server uses the selection information it receives to query databases of multiple related stores to check the availability of the selected clothing items and the dates and times when they can be tried on.

[0053] Step 7:

[0054] Based on the inventory information obtained by the server, an algorithm selects the optimal store, taking into account the user's location and other factors.

[0055] Step 8:

[0056] The server sends the optimal store information it has selected to the user's terminal and presents the recommended store to the user.

[0057] Step 9:

[0058] When a user approves a fitting appointment at a store suggested through their device, the device transmits that information to the server.

[0059] Step 10:

[0060] The server uses a reservation control mechanism to access the reservation system of the selected store and execute the fitting reservation procedure.

[0061] Step 11:

[0062] Once the server confirms the reservation is complete, it sends the details to the terminal to notify the user of the reservation.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] Traditional clothing selection required users to visit physical stores and try on clothes, which was time-consuming and laborious. Furthermore, checking inventory often required visiting multiple stores, making it inefficient. This resulted in a limited selection of clothing options for users.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes means for receiving personal identification information from a data input device, means for generating try-on images of the user wearing multiple garments using a generation AI model, and means for transmitting and displaying the generated try-on images on a display device. This enables the user to efficiently try on and consider garments. Furthermore, by checking inventory information of selected garments via an information and communication network, identifying the optimal store based on geographical data and inventory status, and automatically making try-on reservations, the effort required to visit physical stores is significantly reduced, allowing the user to smoothly select and reserve desired garments.

[0068] "Personally identifiable information" refers to information used to identify a user, and includes data such as names, facial photographs, and other identifiable data.

[0069] A "data input device" refers to hardware or software used to acquire and transmit personally identifiable information, such as smartphones and tablets.

[0070] A "generative AI model" is a part of artificial intelligence that utilizes machine learning algorithms to generate new digital data based on input data.

[0071] A "try-on image" is a computer-generated image that visually represents what it would look like if a user were wearing various clothes.

[0072] "Display devices" are devices used to visually present digital data, and include monitors and displays.

[0073] An "information and communication network" is a network used to exchange data between multiple devices, and the internet is an example of this.

[0074] "Geographic data" refers to data related to physical locations or places, including GPS information and address data.

[0075] "Inventory information" refers to data that shows how many units of a particular product are currently available.

[0076] "Reservation management methods" refer to the processes and systems used to secure a date and time for using a selected service or product.

[0077] To implement this invention, first, the user inputs personal identification information using a terminal. The terminal uses a data input device such as a smartphone or tablet to take a picture of the user's face through its camera and collect personal identification information. Next, the server receives this information and uses a generation AI model to generate digital images of the user trying on multiple outfits.

[0078] This image generation process utilizes AI software that implements machine learning algorithms. A processor on the server analyzes the received facial photograph, creates a model best suited to the user's face, and applies clothing to it. Specifically, general AI image processing techniques can be used as the generative AI model.

[0079] The generated try-on images are sent from the server to the terminal, where the user can view them in catalog format using a display device. The catalog is designed to allow the user to select their preferred clothing.

[0080] When a user selects clothing on their device, the server checks the inventory information for the selected clothing from multiple stores via the information and communication network. Based on geographical data and inventory status, it identifies the most suitable store and suggests it to the user. To achieve this, the server can utilize geolocation services and store management software.

[0081] Ultimately, the user selects a fitting appointment from the suggested stores on their device. The server uses a reservation management system to confirm the fitting appointment at the specified store. This process eliminates the need for the user to visit physical stores, allowing them to efficiently try on, consider, and select clothing.

[0082] As a concrete example, consider a user who wants to choose a special kimono for their coming-of-age ceremony. The user takes a photo of their face with their smartphone and sends it through the application. The server uses a generative AI model to generate digital images of the user trying on various kimonos and displays them as a catalog. When the user selects a red kimono, the server identifies the most suitable store based on inventory information and geographical data and confirms the fitting reservation. An example of a prompt message could be a request such as, "Please generate images of me trying on a special kimono." Such a system allows users to efficiently try on clothes and prepare for a specific event.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] The user takes a photo of their face using their device and sends this data, along with their personal identification information, to the server. Specifically, they activate the smartphone camera, press the "Capture" button to take a photo, and then press the "Send" button using the application. The system receives the captured photo and personal identification information as input. This data is sent to the server as output.

[0086] Step 2:

[0087] Based on the personal identification information and facial image data received by the server, a generative AI model is used to generate try-on images. Specifically, the server's processor uses an image processing algorithm to generate a body silhouette that fits the user's face and synthesizes different clothing styles. Facial image data is received as input, and multiple try-on images are generated as output.

[0088] Step 3:

[0089] The server converts the generated try-on images into a catalog format and sends them to the terminal. Specifically, the server organizes the images into an easy-to-view layout and remotely transmits them to the user's terminal via the network. Try-on images are received as input, and catalog-formatted image data is sent to the terminal as output.

[0090] Step 4:

[0091] The user browses a catalog on their device and selects the desired clothing item. Specifically, the user operates the touchscreen, scrolls through images, and taps on items. The input is the clothing item selected from the catalog, and the output is information about the selected clothing item sent to the server.

[0092] Step 5:

[0093] The server checks inventory information for the selected clothing item via the information and communication network and identifies the optimal store. Specifically, the server sends queries to multiple pre-registered store databases to retrieve and analyze inventory status and location information. It receives information about the selected clothing item as input and generates a list of optimal stores as output.

[0094] Step 6:

[0095] The server sends and suggests the most suitable store information to the user. Once the user selects a preferred store based on the suggestions, the server confirms the fitting reservation using a reservation management system. Specifically, the reservation information is confirmed when the user taps the selected store, and the schedule is updated on the system. The system receives the most suitable store information as input and generates information confirming the reservation as output.

[0096] (Application Example 1)

[0097] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0098] In traditional consumer goods selection and purchasing processes, it is difficult for individuals to determine whether an item suits them before actually visiting a store, and this is especially true for clothing, where trying things on is time-consuming. Furthermore, there is the problem of difficulty in checking the real-time inventory status of specific products and selecting the most suitable store.

[0099] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0100] In this invention, the server includes means for receiving personal identification information from an input device, means for operating a digital image generation device based on the personal identification information to generate duplicate images of multiple garments being worn, and means for transmitting and presenting the generated duplicate images to a display device. This enables users to select suitable clothing through virtual try-on and efficiently make try-on reservations at the most suitable stores.

[0101] "Personally identifiable information" refers to information used to identify an individual user, and includes biometric data such as facial photographs.

[0102] An "input device" refers to hardware that a user uses to send data to a system, such as a smartphone or tablet.

[0103] A "digital image generation device" is a device that executes a machine learning algorithm to generate virtual try-on images based on the user's personal identification information.

[0104] A "duplicate image" is a digital image generated using AI technology that reproduces how a user looks wearing various types of clothing.

[0105] A "display device" is a device used by users to view generated digital images or catalogs, and is typically a smartphone screen or a computer monitor.

[0106] A "communication network" refers to the infrastructure used to send and receive data, and includes digital communication systems such as the internet.

[0107] "Data analysis means" refers to a method or process for processing information necessary for selecting the optimal physical store and making decisions based on that information.

[0108] "Reservation control means" refers to a method or device for managing and executing fitting reservations at physical stores selected by the user.

[0109] A "catalog format" is a display format that is arranged to make it easy for users to view, compare, and select generated duplicate images.

[0110] "Communication means" refers to the mechanisms and protocols used to send and receive data between a user and a server or system.

[0111] To implement this invention, the user first uses a smartphone as an input device to take a facial photograph and input personal identification information. Upon receiving this information, the server applies a machine learning algorithm, such as a generative AI model (e.g., GANs), using a digital image generation device to generate a duplicate image of the user wearing clothes. This process utilizes machine learning frameworks such as TENSORFLOW®.

[0112] The generated duplicate images are sent from the server to the smartphone's display device and presented to the user in a catalog format. The availability of the clothing selected by the user is checked in real time via the communication network. The server then identifies the optimal physical store based on geographical and inventory information and provides this information to the user. The server further confirms the fitting reservation at the suggested physical store using a reservation control mechanism. A REST API is used for this communication.

[0113] For example, if a user is choosing an outfit for their coming-of-age ceremony, the process can proceed as follows: A prompt message will appear stating, "If you want to choose a stylish kimono for your coming-of-age ceremony, please upload a selfie of your face. We will generate images of you trying on various kimonos and help you choose the one that suits you best!" By following this prompt, the user can efficiently select the appropriate outfit and make a reservation at the most suitable fitting shop.

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] The user takes a photo of their face with their smartphone camera and sends the facial image data to the server through the application. This facial image, as input data, is processed by the server as personally identifiable information. The server receives this data and verifies its reliability and accuracy.

[0117] Step 2:

[0118] The server passes the received personal identification information to a digital image generator. This image generator uses a generation AI model to generate duplicate images that show the user wearing multiple outfits. The data processing performed here involves adding clothing to the user's facial photograph to generate a realistic try-on image. These generated duplicate images are obtained as output.

[0119] Step 3:

[0120] The server sends the generated duplicate images to the smartphone's display device. The device receives these and displays them in a catalog format for the user to browse. The user interacts with this catalog, identifying and selecting clothing items of interest.

[0121] Step 4:

[0122] The user's selected clothing information is sent to the server. The server retrieves real-time inventory information for the selected clothing from multiple stores via the communication network. In this process, it connects to each store's database using a REST API and aggregates the inventory information.

[0123] Step 5:

[0124] Based on collected inventory information and the user's geographical information, the server uses data analysis tools to identify the optimal physical store and propose it to the user. This process involves applying an algorithm to compare multiple candidate stores, ultimately identifying the physical store.

[0125] Step 6:

[0126] After the user selects a suggested physical store, the server uses a reservation control mechanism to make a fitting reservation for the selected store. The reservation data is sent to the store's reservation management system, and a reservation confirmation is output. This prepares the user to efficiently try on clothes at the designated store.

[0127] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0128] This invention constructs a system to provide a more personalized clothing selection experience by combining emotion recognition technology with the process by which users efficiently select clothing. The system includes a terminal for inputting the user's facial photograph and personal identification information, an emotion recognition engine, a digital image generation device, and a communication network for checking clothing inventory and identifying the optimal fitting store.

[0129] The user first uses a device to input their current facial expressions and voice into the system via camera and microphone, along with a photo of their face. The server receives this data and passes the facial image data to an image generator, and the facial expression and voice data to an emotion engine. The emotion engine analyzes the user's emotional state and uses it as an indicator to determine what the user likes and what kind of clothing they are interested in.

[0130] The server then uses image generation AI to create multiple digital outfit images based on the user's facial photograph. These images are compiled into a catalog, taking into account the user's emotional information, and sorted in order of likelihood of user interest. The catalog is transferred to the device, where the user browses and selects their favorite outfits.

[0131] Once the user has made their selection, the server checks the availability of the clothing via the communication network. Simultaneously, the server analyzes the data, taking into account the user's preferences based on an emotion engine, to identify the optimal fitting room. The user's emotional information plays a crucial role in suggesting the best time and location for the user.

[0132] For example, if a user tends to prefer clothing in soothing colors, the system will prioritize displaying clothing in calming tones within the generated catalog. This priority display is achieved through an emotion engine, allowing users to quickly select clothing that better matches their preferences. Once the selection is complete and the user chooses a fitting location, the system automatically makes a fitting reservation and notifies the user of the details. In this way, users can experience an efficient and satisfying clothing selection process.

[0133] The following describes the processing flow.

[0134] Step 1:

[0135] The user uses their device to input a photo of their face, along with facial expressions and voice. During this process, the camera and microphone are activated, and user data is collected.

[0136] Step 2:

[0137] The device sends the facial image data and emotion data it collects to the server. This includes formatting the image data and preparing the audio data for analysis.

[0138] Step 3:

[0139] The server passes a facial image to an image generator and sends emotional data to an emotion engine. The emotion engine uses this data to analyze the user's current emotional state.

[0140] Step 4:

[0141] The server uses image generation AI to generate digital images of the user wearing multiple outfits based on their face. This process incorporates emotional information from an emotion engine and is adjusted to reflect the user's preferences.

[0142] Step 5:

[0143] The server sorts the generated duplicate images based on sentiment and sends them to the terminal as a catalog. The terminal then presents the catalog to the user in this order.

[0144] Step 6:

[0145] The user browses this catalog and selects the clothing items they are interested in. Once the selection is complete, the device sends that information to the server.

[0146] Step 7:

[0147] The server collects inventory information for selected clothing items from multiple stores via a communication network. Simultaneously, it performs data analysis to identify the optimal store based on the results of the emotion engine's analysis.

[0148] Step 8:

[0149] The server sends information about the most suitable fitting room to the user's terminal. Here, the user's preferences are reflected to the greatest extent possible, taking into account geographical conditions and inventory information.

[0150] Step 9:

[0151] The user reviews the store information provided on their device and decides to make a fitting reservation. Once the user approves, the device notifies the server of that information.

[0152] Step 10:

[0153] The server, through the reservation control mechanism, initiates the process of confirming the fitting reservation at the selected store. This information is then confirmed by the user, and the fitting reservation is finalized.

[0154] (Example 2)

[0155] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0156] In recent years, online shopping for clothing has presented numerous options, making it difficult for users to choose the best product based on their preferences and feelings. Furthermore, limited opportunities to try on clothes before purchase often lead to anxiety about whether the items chosen online will actually fit. To address these issues, there is a need for a system that simultaneously provides more personalized clothing selection and appropriate fitting suggestions.

[0157] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0158] In this invention, the server includes means for receiving personal identification information and emotional data from an input device, means for operating a digital image generation device based on the personal identification information and emotional data to generate a replica image of the user wearing the outfit, and means for rearranging the generated replica image taking emotional information into consideration and transmitting it to a display device for presentation. This allows the user to easily select an outfit based on their own emotions and preferences, and also enables the suggestion and reservation of fitting rooms.

[0159] "Personal identification information" refers to information used to identify a specific user, and includes data such as names and facial photographs.

[0160] "Emotional data" refers to information that indicates a user's emotional state, and is obtained through data such as facial expressions and tone of voice.

[0161] A "digital image generation device" is a device that uses digital technology to generate images that represent the user's appearance in a virtual space.

[0162] A "duplicate image" is a digital image created virtually from the user's face and body, dressed in specific clothing.

[0163] A "display device" is a device used to visually present generated digital images or information, and includes screens and displays.

[0164] A "communication network" is an infrastructure for sending and receiving data between different computers, and includes the internet and dedicated lines.

[0165] An "optimal physical store" is a physical store that is most suitable for trying on and purchasing products, based on the user's geographical location and inventory status.

[0166] "Data analysis methods" refer to technical techniques and systems used to process collected data and derive useful conclusions and suggestions for users.

[0167] A "reservation control means" is a mechanism for making reservations at physical stores for trying on clothes or making purchases, and for notifying the user of the results.

[0168] This invention is a system that combines emotion recognition technology and a generative AI model to provide a personalized clothing selection experience in the process of efficiently selecting clothes for the user. First, the user takes a photo of their face using a device and inputs facial expression and voice data through the camera and microphone. The device then sends this data to a server.

[0169] The server performs analysis using the received personal identification information and sentiment data. The personal identification information is passed to a digital image generator, where a generative AI model using a machine learning algorithm generates duplicate images of the user wearing multiple outfits based on the user's facial photograph. In this generation process, the prompt message "Use the generative AI model to generate multiple styles of digital outfit images for the user's facial photograph" can be used.

[0170] The generated duplicate images are aggregated by a server, and the user's emotional data is analyzed through an emotion engine. Then, taking the emotional information into consideration, the images are formatted into a catalog in an order that is likely to interest the user. This catalog is sent to the user's device, allowing the user to select clothing that matches their preferences.

[0171] For example, if a user tends to prefer a certain color scheme or style, the system will prioritize presenting clothing in calmer tones. In this process, the emotion engine analyzes the user's preferences and incorporates them as important information.

[0172] The user's selected clothing information is cross-referenced with inventory data by a server, and the optimal physical store is identified via a communication network. This identification process considers geographical data, inventory information, and even sentiment data to suggest the best location and time for the user to try on the clothes.

[0173] Finally, the server makes a fitting reservation and notifies the user of the reservation details via the terminal. The reservation control system ensures this process runs smoothly, allowing the user to flexibly confirm the date, time, and location of the fitting. In this way, the user can have a faster and more personalized clothing selection experience.

[0174] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0175] Step 1:

[0176] The user uses the device to take a photo of their face with the camera and input voice data through the microphone. The device sends this data to the server in digital format. In this step, the input data consists of a facial photo and voice, and the output data is the digital data sent to the server.

[0177] Step 2:

[0178] The server receives the transmitted facial image data and passes it to the digital image generation device. Simultaneously, it passes facial expression and voice data to the emotion engine. This process takes the received facial image and emotion data as input data and produces personal identification information and analyzed emotion data as output. The server uses the emotion engine to analyze the data and infer the user's emotional state.

[0179] Step 3:

[0180] The server uses a generative AI model to generate multiple digital clothing images based on the user's facial photograph. The prompt used is "Use the generative AI model to generate multiple styles of digital clothing images for the user's facial photograph." In this step, the input data consists of the facial photograph and the generative AI model, and the output is the generated duplicate images.

[0181] Step 4:

[0182] The server aggregates the generated duplicate images and sorts them based on sentiment information to create a catalog. In this process, sentiment information and duplicate images are used as input data, and the output catalog is sorted in an order that is likely to interest the user. The server then sends this organized catalog to the user's terminal.

[0183] Step 5:

[0184] The user browses a catalog on their device and selects clothes they like. The selected data is sent from the device to the server. Here, the input is the catalog and the user's selection, and the output is the information of the selected clothes returned to the server.

[0185] Step 6:

[0186] The server checks the inventory information of the selected clothing item via the communication network and identifies the most suitable physical store. Input for this process includes information about the selected clothing item and the user's geographical data, and the output generates store identification information. The server also takes into account information from the emotion engine to determine the most suitable store for the user.

[0187] Step 7:

[0188] The server makes a fitting reservation at a designated store and notifies the user of the reservation information. In this process, the input data includes information about the fitting store and available time slots. The output includes details of the fitting reservation, which the server sends to the terminal, allowing the user to confirm the schedule.

[0189] (Application Example 2)

[0190] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0191] Today's individual consumers are often overwhelmed by the sheer number of choices available in physical stores and online shops when selecting clothing. However, it is difficult for consumers to quickly choose clothes that suit their personal preferences and current emotional state. Furthermore, they need to be selective about the time of day and store for efficient trying on and purchasing. In this context, providing users with more personalized information is necessary to ensure they have the optimal shopping experience.

[0192] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0193] In this invention, the server includes means for receiving personal identification information and emotional information from an input device, means for operating a digital image generation device based on the personal identification information and emotional information to generate duplicate images of multiple garments being worn, and means for transmitting and presenting the generated duplicate images to a display device. This enables users to efficiently and personally select clothing in physical stores using a visual device.

[0194] "Personally identifiable information" refers to data used to identify a specific individual, including information such as a user's name, ID, and facial photograph.

[0195] "Emotional information" refers to data that indicates a user's emotional state, and is information analyzed from facial expressions, tone of voice, and other similar factors.

[0196] A "digital image generation device" is a device that generates and edits images using digital technology, and it creates images of the user wearing clothes based on their emotional information.

[0197] A "display device" is a device used to visually display electronic data, and includes smart glasses, smartphones, monitors, and other similar devices.

[0198] A "communication network" is the infrastructure used for exchanging information, and includes the internet and dedicated communication lines.

[0199] "Data analysis tools" are algorithms and processes for identifying and extracting useful information from large amounts of data, and have the function of finding the optimal store and time based on geographical data, inventory information, and sentiment data.

[0200] A "reservation control means" refers to a method or system for managing and coordinating reservations for fittings and purchases, ensuring that users can make reservations smoothly.

[0201] "Visual devices" are devices that allow users to acquire visual information, and include smart glasses and AR (augmented reality) headsets.

[0202] The system for realizing this invention involves the cooperation of a user, a terminal, and a server. First, the user provides their personal identification information and emotional information through the terminal's camera and microphone. This allows the terminal to acquire the user's facial image and voice and send them to the server. Upon receiving this data, the server analyzes the user's emotions using an emotion engine. This emotion analysis utilizes machine learning frameworks such as TensorFlow to determine the emotional state from the user's facial expressions and voice.

[0203] Next, the server uses the user's personal identification information and emotional information to operate a digital image generator and generate a replica image of the user wearing the clothes. This process utilizes generative AI models such as GANs (Generative Adversarial Networks) to generate realistic and visually appealing images of the clothing. These generated images are streamed to the user's smart glasses or smartphone and displayed in their field of view in real time.

[0204] When a user selects clothing from a displayed catalog, the server checks the inventory information of that clothing via the communication network and simultaneously suggests the most suitable physical store and visit time. Users can instantly obtain this information using smart glasses and seamlessly make fitting reservations.

[0205] Furthermore, if the user is feeling stressed, the app can prioritize displaying clothing in relaxing colors and play relaxing music. This system utilizes a generated AI model, and by setting prompt examples to "Generate the optimal color coordination for the clothing selected by the user and prioritize displaying items with calming tones," the user experience can be optimized.

[0206] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0207] Step 1:

[0208] The user inputs personal identification information and sentiment information using the device's camera and microphone. In this step, the device captures the user's facial image and voice data. The input data is preprocessed, with the device compressing the facial image and converting the voice to text. The output is sent to the server as personal identification information and sentiment information.

[0209] Step 2:

[0210] The server analyzes the user's emotions using an emotion engine based on the received personal identification and emotion information. The input is the data sent in the previous step, and the server performs emotion analysis using TensorFlow. It determines the user's emotional state from facial features and voice tone and generates an emotion label as output. This is an indicator used in subsequent clothing selection.

[0211] Step 3:

[0212] The server uses personal identification information and emotional information to operate a digital image generator and generate a digital clothing catalog. The input consists of emotional labels and specific user information. A generation AI model is used to generate clothing images suitable for the user and rank them according to the user's preferences. The output is a catalog of digital clothing images.

[0213] Step 4:

[0214] The terminal displays a digital clothing catalog generated on the user's smart glasses or smartphone. Input from the server is image data, which is rendered in real time on the terminal. The user browses the displayed catalog and selects clothing items of interest through the interface. The output is the user's clothing selection information.

[0215] Step 5:

[0216] The server retrieves and analyzes inventory and geographical information of the user's selected clothing via a communication network. Inputs include user selection information and provided store information. Using inventory information and the user's geographical location, the server identifies the optimal store for trying on clothes and the best time to visit. The output is a suggestion of the optimal fitting location and time.

[0217] Step 6:

[0218] The user uses the terminal to check the suggested fitting rooms and appointment times, and makes a fitting reservation if necessary. The input from the server is fitting information, and the terminal, after obtaining user approval, confirms the fitting reservation with the store using the reservation control mechanism. The output is a notification that the fitting reservation has been completed.

[0219] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0220] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0221] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0222] [Second Embodiment]

[0223] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0224] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0225] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0226] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0227] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0228] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0229] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0230] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0231] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0232] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0233] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0234] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0235] This invention provides a system that allows users to efficiently select clothing and reduce the hassle of trying them on. To implement this system, a terminal for inputting the user's facial photograph and personal identification information, a server for processing data and generating images, and a communication network for checking clothing inventory and making fitting reservations are required.

[0236] The user first inputs a photo of their face using a terminal, and the server receives this photo data and processes it using an image generation device. The image generation device uses AI technology to generate multiple digital images that simulate the user trying on clothes, and sends these to the terminal in catalog format.

[0237] When a user browses this catalog on their device and selects an item of clothing they like, the server checks the inventory of the selected item at multiple stores. During this process, it accesses each store's database via the communication network to collect information such as inventory status and available fitting times.

[0238] To suggest the most suitable store, the server uses data analysis tools to identify the optimal store based on geographical and inventory information. This information is then presented to the user's terminal, and once the user selects a store, a reservation control tool automatically makes a fitting reservation for the specified store.

[0239] As a concrete example, let's consider a case where user A wants to choose a furisode (long-sleeved kimono) for their coming-of-age ceremony. User A takes a photo of their face using their smartphone and sends it via the application. The server then uses AI to generate digital images of user A trying on multiple furisode styles and sends them to the user's device in an easy-to-view catalog format. User A reviews this catalog and selects a red furisode. The server searches a database of nearby rental shops and identifies the best store with the selected garment in stock, suggesting it to user A. When user A chooses to make a fitting reservation at the store, the server uses a reservation control mechanism to confirm the reservation. This embodiment allows users to efficiently select clothing and prepare for a specific event with reduced burden.

[0240] The following describes the processing flow.

[0241] Step 1:

[0242] The user takes a photo of their face using their device and sends that photo to the server via the application.

[0243] Step 2:

[0244] The server checks the received facial photograph, performs pre-processing such as adjusting the image format, and then prepares it for transmission to the digital image generation device.

[0245] Step 3:

[0246] The server runs an image generation AI to generate digital replicas of the user wearing multiple outfits, based on the user's facial photograph.

[0247] Step 4:

[0248] The server organizes the generated duplicate images into a catalog, transfers them to the user's terminal, and displays them.

[0249] Step 5:

[0250] The user browses a catalog presented through their device and selects the clothing items they like. Once the selection is complete, the device sends that information to the server.

[0251] Step 6:

[0252] The server uses the selection information it receives to query databases of multiple related stores to check the availability of the selected clothing items and the dates and times when they can be tried on.

[0253] Step 7:

[0254] Based on the inventory information obtained by the server, an algorithm selects the optimal store, taking into account the user's location and other factors.

[0255] Step 8:

[0256] The server sends the optimal store information it has selected to the user's terminal and presents the recommended store to the user.

[0257] Step 9:

[0258] When a user approves a fitting appointment at a store suggested through their device, the device transmits that information to the server.

[0259] Step 10:

[0260] The server uses a reservation control mechanism to access the reservation system of the selected store and execute the fitting reservation procedure.

[0261] Step 11:

[0262] Once the server confirms the reservation is complete, it sends the details to the terminal to notify the user of the reservation.

[0263] (Example 1)

[0264] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0265] Traditional clothing selection required users to visit physical stores and try on clothes, which was time-consuming and laborious. Furthermore, checking inventory often required visiting multiple stores, making it inefficient. This resulted in a limited selection of clothing options for users.

[0266] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0267] In this invention, the server includes means for receiving personal identification information from a data input device, means for generating try-on images of the user wearing multiple garments using a generation AI model, and means for transmitting and displaying the generated try-on images on a display device. This enables the user to efficiently try on and consider garments. Furthermore, by checking inventory information of selected garments via an information and communication network, identifying the optimal store based on geographical data and inventory status, and automatically making try-on reservations, the effort required to visit physical stores is significantly reduced, allowing the user to smoothly select and reserve desired garments.

[0268] "Personally identifiable information" refers to information used to identify a user, and includes data such as names, facial photographs, and other identifiable data.

[0269] A "data input device" refers to hardware or software used to acquire and transmit personally identifiable information, such as smartphones and tablets.

[0270] A "generative AI model" is a part of artificial intelligence that utilizes machine learning algorithms to generate new digital data based on input data.

[0271] A "try-on image" is a computer-generated image that visually represents what it would look like if a user were wearing various clothes.

[0272] "Display devices" are devices used to visually present digital data, and include monitors and displays.

[0273] An "information and communication network" is a network used to exchange data between multiple devices, and the internet is an example of this.

[0274] "Geographic data" refers to data related to physical locations or places, including GPS information and address data.

[0275] "Inventory information" refers to data that shows how many units of a particular product are currently available.

[0276] "Reservation management methods" refer to the processes and systems used to secure a date and time for using a selected service or product.

[0277] To implement this invention, first, the user inputs personal identification information using a terminal. The terminal uses a data input device such as a smartphone or tablet to take a picture of the user's face through its camera and collect personal identification information. Next, the server receives this information and uses a generation AI model to generate digital images of the user trying on multiple outfits.

[0278] This image generation process utilizes AI software that implements machine learning algorithms. A processor on the server analyzes the received facial photograph, creates a model best suited to the user's face, and applies clothing to it. Specifically, general AI image processing techniques can be used as the generative AI model.

[0279] The generated try-on images are sent from the server to the terminal, where the user can view them in catalog format using a display device. The catalog is designed to allow the user to select their preferred clothing.

[0280] When a user selects clothing on their device, the server checks the inventory information for the selected clothing from multiple stores via the information and communication network. Based on geographical data and inventory status, it identifies the most suitable store and suggests it to the user. To achieve this, the server can utilize geolocation services and store management software.

[0281] Ultimately, the user selects a fitting appointment from the suggested stores on their device. The server uses a reservation management system to confirm the fitting appointment at the specified store. This process eliminates the need for the user to visit physical stores, allowing them to efficiently try on, consider, and select clothing.

[0282] As a specific example, consider the case where a user wants to choose a special kimono for an adult ceremony. The user takes a selfie with their smartphone and sends it through an application. The server uses a generative AI model to generate digital images of the user trying on various kimonos and displays them as a catalog. When the user selects a red kimono, the server identifies the optimal store based on inventory information and geographical data and finalizes the fitting reservation. As an example of a prompt sentence, a request like "Please generate a fitting image of a special kimono" can be considered. With such a system, the user can efficiently try on clothes and prepare for a specific event.

[0283] The flow of the specific process in Example 1 will be described using FIG. 11.

[0284] Step 1:

[0285] The user uses the terminal to take a selfie and sends this data, together with personal identification information, to the server. Specifically, the user activates the smartphone camera, presses the "Take Photo" button to take a photo, and then presses the "Send" button using the application. The input is the taken selfie and personal identification information, and the output is that these data are sent to the server.

[0286] Step 2:

[0287] Based on the personal identification information and selfie data received by the server, the generative AI model is used to generate fitting images. As a specific operation, the server's processor uses an image processing algorithm to generate a body silhouette that suits the user's face and synthesizes different clothing styles. The input is the selfie data, and the output is that multiple fitting images are generated. [[ID=No.21]] [[ID=No.22]]

[0288] [[ID=No.23]] [[ID=No.24]]Step 3: [[ID=No.25]] [[ID=No.26]]

[0289] [[ID=No.27]] The server converts the generated try-on images into a catalog format and sends them to the terminal. Specifically, the server organizes the images into an easy-to-view layout and remotely transmits them to the user's terminal via the network. Try-on images are received as input, and catalog-formatted image data is sent to the terminal as output.

[0290] Step 4:

[0291] The user browses a catalog on their device and selects the desired clothing item. Specifically, the user operates the touchscreen, scrolls through images, and taps on items. The input is the clothing item selected from the catalog, and the output is information about the selected clothing item sent to the server.

[0292] Step 5:

[0293] The server checks inventory information for the selected clothing item via the information and communication network and identifies the optimal store. Specifically, the server sends queries to multiple pre-registered store databases to retrieve and analyze inventory status and location information. It receives information about the selected clothing item as input and generates a list of optimal stores as output.

[0294] Step 6:

[0295] The server sends and suggests the most suitable store information to the user. Once the user selects a preferred store based on the suggestions, the server confirms the fitting reservation using a reservation management system. Specifically, the reservation information is confirmed when the user taps the selected store, and the schedule is updated on the system. The system receives the most suitable store information as input and generates information confirming the reservation as output.

[0296] (Application Example 1)

[0297] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0298] In traditional consumer goods selection and purchasing processes, it is difficult for individuals to determine whether an item suits them before actually visiting a store, and this is especially true for clothing, where trying things on is time-consuming. Furthermore, there is the problem of difficulty in checking the real-time inventory status of specific products and selecting the most suitable store.

[0299] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0300] In this invention, the server includes means for receiving personal identification information from an input device, means for operating a digital image generation device based on the personal identification information to generate duplicate images of multiple garments being worn, and means for transmitting and presenting the generated duplicate images to a display device. This enables users to select suitable clothing through virtual try-on and efficiently make try-on reservations at the most suitable stores.

[0301] "Personally identifiable information" refers to information used to identify an individual user, and includes biometric data such as facial photographs.

[0302] An "input device" refers to hardware that a user uses to send data to a system, such as a smartphone or tablet.

[0303] A "digital image generation device" is a device that executes a machine learning algorithm to generate virtual try-on images based on the user's personal identification information.

[0304] A "duplicate image" is a digital image generated using AI technology that reproduces how a user looks wearing various types of clothing.

[0305] A "display device" is a device used by users to view generated digital images or catalogs, and is typically a smartphone screen or a computer monitor.

[0306] The "communication network" is the infrastructure used for transmitting and receiving data, and refers to digital communication systems such as the Internet.

[0307] The "data analysis means" refers to a method or process for processing information necessary for selecting an optimal physical store and making a judgment based on that information.

[0308] The "reservation control means" is a method or device for managing and executing a fitting reservation to the physical store selected by the user.

[0309] The "catalog format" is a display format arranged so that the replicated images generated by the user can be listed, compared, and selected easily.

[0310] The "communication means" refers to a mechanism or protocol for mutually transmitting and receiving data between the user and the server or system.

[0311] To implement this invention, the user first uses an input device such as a smartphone to take a face photo and input personal identification information. When the server receives this information, it applies a generation AI model (e.g., GANs), which is a machine learning algorithm, using a digital image generation device to generate a replicated image of the user wearing clothes. In this process, a machine learning framework such as TensorFlow is utilized.

[0312] The generated replicated image is transmitted from the server to the display device of the smartphone and presented in a catalog format that the user can view. Through the communication network, the inventory of the clothes selected by the user is confirmed in real time. Thereafter, the server identifies the optimal physical store based on the geographical information and the inventory information and provides this information to the user. The server further uses the reservation control means to confirm the fitting reservation at the proposed physical store. For the communication at that time, a REST API is utilized.

[0313] For example, if a user is choosing an outfit for their coming-of-age ceremony, the process can proceed as follows: A prompt message will appear stating, "If you want to choose a stylish kimono for your coming-of-age ceremony, please upload a selfie of your face. We will generate images of you trying on various kimonos and help you choose the one that suits you best!" By following this prompt, the user can efficiently select the appropriate outfit and make a reservation at the most suitable fitting shop.

[0314] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0315] Step 1:

[0316] The user takes a photo of their face with their smartphone camera and sends the facial image data to the server through the application. This facial image, as input data, is processed by the server as personally identifiable information. The server receives this data and verifies its reliability and accuracy.

[0317] Step 2:

[0318] The server passes the received personal identification information to a digital image generator. This image generator uses a generation AI model to generate duplicate images that show the user wearing multiple outfits. The data processing performed here involves adding clothing to the user's facial photograph to generate a realistic try-on image. These generated duplicate images are obtained as output.

[0319] Step 3:

[0320] The server sends the generated duplicate images to the smartphone's display device. The device receives these and displays them in a catalog format for the user to browse. The user interacts with this catalog, identifying and selecting clothing items of interest.

[0321] Step 4:

[0322] The user's selected clothing information is sent to the server. The server retrieves real-time inventory information for the selected clothing from multiple stores via the communication network. In this process, it connects to each store's database using a REST API and aggregates the inventory information.

[0323] Step 5:

[0324] Based on collected inventory information and the user's geographical information, the server uses data analysis tools to identify the optimal physical store and propose it to the user. This process involves applying an algorithm to compare multiple candidate stores, ultimately identifying the physical store.

[0325] Step 6:

[0326] After the user selects a suggested physical store, the server uses a reservation control mechanism to make a fitting reservation for the selected store. The reservation data is sent to the store's reservation management system, and a reservation confirmation is output. This prepares the user to efficiently try on clothes at the designated store.

[0327] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0328] This invention constructs a system to provide a more personalized clothing selection experience by combining emotion recognition technology with the process by which users efficiently select clothing. The system includes a terminal for inputting the user's facial photograph and personal identification information, an emotion recognition engine, a digital image generation device, and a communication network for checking clothing inventory and identifying the optimal fitting store.

[0329] The user first uses a device to input their current facial expressions and voice into the system via camera and microphone, along with a photo of their face. The server receives this data and passes the facial image data to an image generator, and the facial expression and voice data to an emotion engine. The emotion engine analyzes the user's emotional state and uses it as an indicator to determine what the user likes and what kind of clothing they are interested in.

[0330] The server then uses image generation AI to create multiple digital outfit images based on the user's facial photograph. These images are compiled into a catalog, taking into account the user's emotional information, and sorted in order of likelihood of user interest. The catalog is transferred to the device, where the user browses and selects their favorite outfits.

[0331] Once the user has made their selection, the server checks the availability of the clothing via the communication network. Simultaneously, the server analyzes the data, taking into account the user's preferences based on an emotion engine, to identify the optimal fitting room. The user's emotional information plays a crucial role in suggesting the best time and location for the user.

[0332] For example, if a user tends to prefer clothing in soothing colors, the system will prioritize displaying clothing in calming tones within the generated catalog. This priority display is achieved through an emotion engine, allowing users to quickly select clothing that better matches their preferences. Once the selection is complete and the user chooses a fitting location, the system automatically makes a fitting reservation and notifies the user of the details. In this way, users can experience an efficient and satisfying clothing selection process.

[0333] The following describes the processing flow.

[0334] Step 1:

[0335] The user uses their device to input a photo of their face, along with facial expressions and voice. During this process, the camera and microphone are activated, and user data is collected.

[0336] Step 2:

[0337] The device sends the facial image data and emotion data it collects to the server. This includes formatting the image data and preparing the audio data for analysis.

[0338] Step 3:

[0339] The server passes a facial image to an image generator and sends emotional data to an emotion engine. The emotion engine uses this data to analyze the user's current emotional state.

[0340] Step 4:

[0341] The server uses image generation AI to generate digital images of the user wearing multiple outfits based on their face. This process incorporates emotional information from an emotion engine and is adjusted to reflect the user's preferences.

[0342] Step 5:

[0343] The server sorts the generated duplicate images based on sentiment and sends them to the terminal as a catalog. The terminal then presents the catalog to the user in this order.

[0344] Step 6:

[0345] The user browses this catalog and selects the clothing items they are interested in. Once the selection is complete, the device sends that information to the server.

[0346] Step 7:

[0347] The server collects inventory information for selected clothing items from multiple stores via a communication network. Simultaneously, it performs data analysis to identify the optimal store based on the results of the emotion engine's analysis.

[0348] Step 8:

[0349] The server sends information about the most suitable fitting room to the user's terminal. Here, the user's preferences are reflected to the greatest extent possible, taking into account geographical conditions and inventory information.

[0350] Step 9:

[0351] The user reviews the store information provided on their device and decides to make a fitting reservation. Once the user approves, the device notifies the server of that information.

[0352] Step 10:

[0353] The server, through the reservation control mechanism, initiates the process of confirming the fitting reservation at the selected store. This information is then confirmed by the user, and the fitting reservation is finalized.

[0354] (Example 2)

[0355] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0356] In recent years, online shopping for clothing has presented numerous options, making it difficult for users to choose the best product based on their preferences and feelings. Furthermore, limited opportunities to try on clothes before purchase often lead to anxiety about whether the items chosen online will actually fit. To address these issues, there is a need for a system that simultaneously provides more personalized clothing selection and appropriate fitting suggestions.

[0357] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0358] In this invention, the server includes means for receiving personal identification information and emotional data from an input device, means for operating a digital image generation device based on the personal identification information and emotional data to generate a replica image of the user wearing the outfit, and means for rearranging the generated replica image taking emotional information into consideration and transmitting it to a display device for presentation. This allows the user to easily select an outfit based on their own emotions and preferences, and also enables the suggestion and reservation of fitting rooms.

[0359] "Personal identification information" refers to information used to identify a specific user, and includes data such as names and facial photographs.

[0360] "Emotional data" refers to information that indicates a user's emotional state, and is obtained through data such as facial expressions and tone of voice.

[0361] A "digital image generation device" is a device that uses digital technology to generate images that represent the user's appearance in a virtual space.

[0362] A "duplicate image" is a digital image created virtually from the user's face and body, dressed in specific clothing.

[0363] A "display device" is a device used to visually present generated digital images or information, and includes screens and displays.

[0364] A "communication network" is an infrastructure for sending and receiving data between different computers, and includes the internet and dedicated lines.

[0365] An "optimal physical store" is a physical store that is most suitable for trying on and purchasing products, based on the user's geographical location and inventory status.

[0366] "Data analysis methods" refer to technical techniques and systems used to process collected data and derive useful conclusions and suggestions for users.

[0367] A "reservation control means" is a mechanism for making reservations at physical stores for trying on clothes or making purchases, and for notifying the user of the results.

[0368] This invention is a system that combines emotion recognition technology and a generative AI model to provide a personalized clothing selection experience in the process of efficiently selecting clothes for the user. First, the user takes a photo of their face using a device and inputs facial expression and voice data through the camera and microphone. The device then sends this data to a server.

[0369] The server performs analysis using the received personal identification information and sentiment data. The personal identification information is passed to a digital image generator, where a generative AI model using a machine learning algorithm generates duplicate images of the user wearing multiple outfits based on the user's facial photograph. In this generation process, the prompt message "Use the generative AI model to generate multiple styles of digital outfit images for the user's facial photograph" can be used.

[0370] The generated duplicate images are aggregated by a server, and the user's emotional data is analyzed through an emotion engine. Then, taking the emotional information into consideration, the images are formatted into a catalog in an order that is likely to interest the user. This catalog is sent to the user's device, allowing the user to select clothing that matches their preferences.

[0371] For example, if a user tends to prefer a certain color scheme or style, the system will prioritize presenting clothing in calmer tones. In this process, the emotion engine analyzes the user's preferences and incorporates them as important information.

[0372] The user's selected clothing information is cross-referenced with inventory data by a server, and the optimal physical store is identified via a communication network. This identification process considers geographical data, inventory information, and even sentiment data to suggest the best location and time for the user to try on the clothes.

[0373] Finally, the server makes a fitting reservation and notifies the user of the reservation details via the terminal. The reservation control system ensures this process runs smoothly, allowing the user to flexibly confirm the date, time, and location of the fitting. In this way, the user can have a faster and more personalized clothing selection experience.

[0374] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0375] Step 1:

[0376] The user uses the device to take a photo of their face with the camera and input voice data through the microphone. The device sends this data to the server in digital format. In this step, the input data consists of a facial photo and voice, and the output data is the digital data sent to the server.

[0377] Step 2:

[0378] The server receives the transmitted facial image data and passes it to the digital image generation device. Simultaneously, it passes facial expression and voice data to the emotion engine. This process takes the received facial image and emotion data as input data and produces personal identification information and analyzed emotion data as output. The server uses the emotion engine to analyze the data and infer the user's emotional state.

[0379] Step 3:

[0380] The server uses a generative AI model to generate multiple digital clothing images based on the user's facial photograph. The prompt used is "Use the generative AI model to generate multiple styles of digital clothing images for the user's facial photograph." In this step, the input data consists of the facial photograph and the generative AI model, and the output is the generated duplicate images.

[0381] Step 4:

[0382] The server aggregates the generated duplicate images and sorts them based on sentiment information to create a catalog. In this process, sentiment information and duplicate images are used as input data, and the output catalog is sorted in an order that is likely to interest the user. The server then sends this organized catalog to the user's terminal.

[0383] Step 5:

[0384] The user browses a catalog on their device and selects clothes they like. The selected data is sent from the device to the server. Here, the input is the catalog and the user's selection, and the output is the information of the selected clothes returned to the server.

[0385] Step 6:

[0386] The server checks the inventory information of the selected clothing item via the communication network and identifies the most suitable physical store. Input for this process includes information about the selected clothing item and the user's geographical data, and the output generates store identification information. The server also takes into account information from the emotion engine to determine the most suitable store for the user.

[0387] Step 7:

[0388] The server makes a fitting reservation at a designated store and notifies the user of the reservation information. In this process, the input data includes information about the fitting store and available time slots. The output includes details of the fitting reservation, which the server sends to the terminal, allowing the user to confirm the schedule.

[0389] (Application Example 2)

[0390] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0391] Today's individual consumers are often overwhelmed by the sheer number of choices available in physical stores and online shops when selecting clothing. However, it is difficult for consumers to quickly choose clothes that suit their personal preferences and current emotional state. Furthermore, they need to be selective about the time of day and store for efficient trying on and purchasing. In this context, providing users with more personalized information is necessary to ensure they have the optimal shopping experience.

[0392] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0393] In this invention, the server includes means for receiving personal identification information and emotional information from an input device, means for operating a digital image generation device based on the personal identification information and emotional information to generate duplicate images of multiple garments being worn, and means for transmitting and presenting the generated duplicate images to a display device. This enables users to efficiently and personally select clothing in physical stores using a visual device.

[0394] "Personally identifiable information" refers to data used to identify a specific individual, including information such as a user's name, ID, and facial photograph.

[0395] "Emotional information" refers to data that indicates a user's emotional state, and is information analyzed from facial expressions, tone of voice, and other similar factors.

[0396] A "digital image generation device" is a device that generates and edits images using digital technology, and it creates images of the user wearing clothes based on their emotional information.

[0397] A "display device" is a device used to visually display electronic data, and includes smart glasses, smartphones, monitors, and other similar devices.

[0398] A "communication network" is the infrastructure used for exchanging information, and includes the internet and dedicated communication lines.

[0399] "Data analysis tools" are algorithms and processes for identifying and extracting useful information from large amounts of data, and have the function of finding the optimal store and time based on geographical data, inventory information, and sentiment data.

[0400] A "reservation control means" refers to a method or system for managing and coordinating reservations for fittings and purchases, ensuring that users can make reservations smoothly.

[0401] "Visual devices" are devices that allow users to acquire visual information, and include smart glasses and AR (augmented reality) headsets.

[0402] The system for realizing this invention involves the cooperation of a user, a terminal, and a server. First, the user provides their personal identification information and emotional information through the terminal's camera and microphone. This allows the terminal to acquire the user's facial image and voice and send them to the server. Upon receiving this data, the server analyzes the user's emotions using an emotion engine. This emotion analysis utilizes machine learning frameworks such as TensorFlow to determine the emotional state from the user's facial expressions and voice.

[0403] Next, the server uses the user's personal identification information and emotional information to operate a digital image generator and generate a replica image of the user wearing the clothes. This process utilizes generative AI models such as GANs (Generative Adversarial Networks) to generate realistic and visually appealing images of the clothing. These generated images are streamed to the user's smart glasses or smartphone and displayed in their field of view in real time.

[0404] When a user selects clothing from a displayed catalog, the server checks the inventory information of that clothing via the communication network and simultaneously suggests the most suitable physical store and visit time. Users can instantly obtain this information using smart glasses and seamlessly make fitting reservations.

[0405] Furthermore, if the user is feeling stressed, the app can prioritize displaying clothing in relaxing colors and play relaxing music. This system utilizes a generated AI model, and by setting prompt examples to "Generate the optimal color coordination for the clothing selected by the user and prioritize displaying items with calming tones," the user experience can be optimized.

[0406] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0407] Step 1:

[0408] The user inputs personal identification information and sentiment information using the device's camera and microphone. In this step, the device captures the user's facial image and voice data. The input data is preprocessed, with the device compressing the facial image and converting the voice to text. The output is sent to the server as personal identification information and sentiment information.

[0409] Step 2:

[0410] The server analyzes the user's emotions using an emotion engine based on the received personal identification and emotion information. The input is the data sent in the previous step, and the server performs emotion analysis using TensorFlow. It determines the user's emotional state from facial features and voice tone and generates an emotion label as output. This is an indicator used in subsequent clothing selection.

[0411] Step 3:

[0412] The server uses personal identification information and emotional information to operate a digital image generator and generate a digital clothing catalog. The input consists of emotional labels and specific user information. A generation AI model is used to generate clothing images suitable for the user and rank them according to the user's preferences. The output is a catalog of digital clothing images.

[0413] Step 4:

[0414] The terminal displays a digital clothing catalog generated on the user's smart glasses or smartphone. Input from the server is image data, which is rendered in real time on the terminal. The user browses the displayed catalog and selects clothing items of interest through the interface. The output is the user's clothing selection information.

[0415] Step 5:

[0416] The server retrieves and analyzes inventory and geographical information of the user's selected clothing via a communication network. Inputs include user selection information and provided store information. Using inventory information and the user's geographical location, the server identifies the optimal store for trying on clothes and the best time to visit. The output is a suggestion of the optimal fitting location and time.

[0417] Step 6:

[0418] The user uses the terminal to check the suggested fitting rooms and appointment times, and makes a fitting reservation if necessary. The input from the server is fitting information, and the terminal, after obtaining user approval, confirms the fitting reservation with the store using the reservation control mechanism. The output is a notification that the fitting reservation has been completed.

[0419] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0420] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0421] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0422] [Third Embodiment]

[0423] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0424] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0425] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0426] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0427] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0429] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0430] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0431] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0432] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0433] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0434] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0435] This invention provides a system that allows users to efficiently select clothing and reduce the hassle of trying them on. To implement this system, a terminal for inputting the user's facial photograph and personal identification information, a server for processing data and generating images, and a communication network for checking clothing inventory and making fitting reservations are required.

[0436] The user first inputs a photo of their face using a terminal, and the server receives this photo data and processes it using an image generation device. The image generation device uses AI technology to generate multiple digital images that simulate the user trying on clothes, and sends these to the terminal in catalog format.

[0437] When a user browses this catalog on their device and selects an item of clothing they like, the server checks the inventory of the selected item at multiple stores. During this process, it accesses each store's database via the communication network to collect information such as inventory status and available fitting times.

[0438] To suggest the most suitable store, the server uses data analysis tools to identify the optimal store based on geographical and inventory information. This information is then presented to the user's terminal, and once the user selects a store, a reservation control tool automatically makes a fitting reservation for the specified store.

[0439] As a concrete example, let's consider a case where user A wants to choose a furisode (long-sleeved kimono) for their coming-of-age ceremony. User A takes a photo of their face using their smartphone and sends it via the application. The server then uses AI to generate digital images of user A trying on multiple furisode styles and sends them to the user's device in an easy-to-view catalog format. User A reviews this catalog and selects a red furisode. The server searches a database of nearby rental shops and identifies the best store with the selected garment in stock, suggesting it to user A. When user A chooses to make a fitting reservation at the store, the server uses a reservation control mechanism to confirm the reservation. This embodiment allows users to efficiently select clothing and prepare for a specific event with reduced burden.

[0440] The following describes the processing flow.

[0441] Step 1:

[0442] The user takes a photo of their face using their device and sends that photo to the server via the application.

[0443] Step 2:

[0444] The server checks the received facial photograph, performs pre-processing such as adjusting the image format, and then prepares it for transmission to the digital image generation device.

[0445] Step 3:

[0446] The server runs an image generation AI to generate digital replicas of the user wearing multiple outfits, based on the user's facial photograph.

[0447] Step 4:

[0448] The server organizes the generated duplicate images into a catalog, transfers them to the user's terminal, and displays them.

[0449] Step 5:

[0450] The user browses a catalog presented through their device and selects the clothing items they like. Once the selection is complete, the device sends that information to the server.

[0451] Step 6:

[0452] The server uses the selection information it receives to query databases of multiple related stores to check the availability of the selected clothing items and the dates and times when they can be tried on.

[0453] Step 7:

[0454] Based on the inventory information obtained by the server, an algorithm selects the optimal store, taking into account the user's location and other factors.

[0455] Step 8:

[0456] The server sends the optimal store information it has selected to the user's terminal and presents the recommended store to the user.

[0457] Step 9:

[0458] When a user approves a fitting appointment at a store suggested through their device, the device transmits that information to the server.

[0459] Step 10:

[0460] The server uses a reservation control mechanism to access the reservation system of the selected store and execute the fitting reservation procedure.

[0461] Step 11:

[0462] Once the server confirms the reservation is complete, it sends the details to the terminal to notify the user of the reservation.

[0463] (Example 1)

[0464] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0465] Traditional clothing selection required users to visit physical stores and try on clothes, which was time-consuming and laborious. Furthermore, checking inventory often required visiting multiple stores, making it inefficient. This resulted in a limited selection of clothing options for users.

[0466] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0467] In this invention, the server includes means for receiving personal identification information from a data input device, means for generating try-on images of the user wearing multiple garments using a generation AI model, and means for transmitting and displaying the generated try-on images on a display device. This enables the user to efficiently try on and consider garments. Furthermore, by checking inventory information of selected garments via an information and communication network, identifying the optimal store based on geographical data and inventory status, and automatically making try-on reservations, the effort required to visit physical stores is significantly reduced, allowing the user to smoothly select and reserve desired garments.

[0468] "Personally identifiable information" refers to information used to identify a user, and includes data such as names, facial photographs, and other identifiable data.

[0469] A "data input device" refers to hardware or software used to acquire and transmit personally identifiable information, such as smartphones and tablets.

[0470] A "generative AI model" is a part of artificial intelligence that utilizes machine learning algorithms to generate new digital data based on input data.

[0471] A "try-on image" is a computer-generated image that visually represents what it would look like if a user were wearing various clothes.

[0472] "Display devices" are devices used to visually present digital data, and include monitors and displays.

[0473] An "information and communication network" is a network used to exchange data between multiple devices, and the internet is an example of this.

[0474] "Geographic data" refers to data related to physical locations or places, including GPS information and address data.

[0475] "Inventory information" refers to data that shows how many units of a particular product are currently available.

[0476] "Reservation management methods" refer to the processes and systems used to secure a date and time for using a selected service or product.

[0477] To implement this invention, first, the user inputs personal identification information using a terminal. The terminal uses a data input device such as a smartphone or tablet to take a picture of the user's face through its camera and collect personal identification information. Next, the server receives this information and uses a generation AI model to generate digital images of the user trying on multiple outfits.

[0478] This image generation process utilizes AI software that implements machine learning algorithms. A processor on the server analyzes the received facial photograph, creates a model best suited to the user's face, and applies clothing to it. Specifically, general AI image processing techniques can be used as the generative AI model.

[0479] The generated try-on images are sent from the server to the terminal, where the user can view them in catalog format using a display device. The catalog is designed to allow the user to select their preferred clothing.

[0480] When a user selects clothing on their device, the server checks the inventory information for the selected clothing from multiple stores via the information and communication network. Based on geographical data and inventory status, it identifies the most suitable store and suggests it to the user. To achieve this, the server can utilize geolocation services and store management software.

[0481] Ultimately, the user selects a fitting appointment from the suggested stores on their device. The server uses a reservation management system to confirm the fitting appointment at the specified store. This process eliminates the need for the user to visit physical stores, allowing them to efficiently try on, consider, and select clothing.

[0482] As a concrete example, consider a user who wants to choose a special kimono for their coming-of-age ceremony. The user takes a photo of their face with their smartphone and sends it through the application. The server uses a generative AI model to generate digital images of the user trying on various kimonos and displays them as a catalog. When the user selects a red kimono, the server identifies the most suitable store based on inventory information and geographical data and confirms the fitting reservation. An example of a prompt message could be a request such as, "Please generate images of me trying on a special kimono." Such a system allows users to efficiently try on clothes and prepare for a specific event.

[0483] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0484] Step 1:

[0485] The user takes a photo of their face using their device and sends this data, along with their personal identification information, to the server. Specifically, they activate the smartphone camera, press the "Capture" button to take a photo, and then press the "Send" button using the application. The system receives the captured photo and personal identification information as input. This data is sent to the server as output.

[0486] Step 2:

[0487] Based on the personal identification information and facial image data received by the server, a generative AI model is used to generate try-on images. Specifically, the server's processor uses an image processing algorithm to generate a body silhouette that fits the user's face and synthesizes different clothing styles. Facial image data is received as input, and multiple try-on images are generated as output.

[0488] Step 3:

[0489] The server converts the generated try-on images into a catalog format and sends them to the terminal. Specifically, the server organizes the images into an easy-to-view layout and remotely transmits them to the user's terminal via the network. Try-on images are received as input, and catalog-formatted image data is sent to the terminal as output.

[0490] Step 4:

[0491] The user browses a catalog on their device and selects the desired clothing item. Specifically, the user operates the touchscreen, scrolls through images, and taps on items. The input is the clothing item selected from the catalog, and the output is information about the selected clothing item sent to the server.

[0492] Step 5:

[0493] The server checks inventory information for the selected clothing item via the information and communication network and identifies the optimal store. Specifically, the server sends queries to multiple pre-registered store databases to retrieve and analyze inventory status and location information. It receives information about the selected clothing item as input and generates a list of optimal stores as output.

[0494] Step 6:

[0495] The server sends and suggests the most suitable store information to the user. Once the user selects a preferred store based on the suggestions, the server confirms the fitting reservation using a reservation management system. Specifically, the reservation information is confirmed when the user taps the selected store, and the schedule is updated on the system. The system receives the most suitable store information as input and generates information confirming the reservation as output.

[0496] (Application Example 1)

[0497] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0498] In traditional consumer goods selection and purchasing processes, it is difficult for individuals to determine whether an item suits them before actually visiting a store, and this is especially true for clothing, where trying things on is time-consuming. Furthermore, there is the problem of difficulty in checking the real-time inventory status of specific products and selecting the most suitable store.

[0499] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0500] In this invention, the server includes means for receiving personal identification information from an input device, means for operating a digital image generation device based on the personal identification information to generate duplicate images of multiple garments being worn, and means for transmitting and presenting the generated duplicate images to a display device. This enables users to select suitable clothing through virtual try-on and efficiently make try-on reservations at the most suitable stores.

[0501] "Personally identifiable information" refers to information used to identify an individual user, and includes biometric data such as facial photographs.

[0502] An "input device" refers to hardware that a user uses to send data to a system, such as a smartphone or tablet.

[0503] A "digital image generation device" is a device that executes a machine learning algorithm to generate virtual try-on images based on the user's personal identification information.

[0504] A "duplicate image" is a digital image generated using AI technology that reproduces how a user looks wearing various types of clothing.

[0505] A "display device" is a device used by users to view generated digital images or catalogs, and is typically a smartphone screen or a computer monitor.

[0506] A "communication network" refers to the infrastructure used to send and receive data, and includes digital communication systems such as the internet.

[0507] "Data analysis means" refers to a method or process for processing information necessary for selecting the optimal physical store and making decisions based on that information.

[0508] "Reservation control means" refers to a method or device for managing and executing fitting reservations at physical stores selected by the user.

[0509] A "catalog format" is a display format that is arranged to make it easy for users to view, compare, and select generated duplicate images.

[0510] "Communication means" refers to the mechanisms and protocols used to send and receive data between a user and a server or system.

[0511] To implement this invention, the user first uses a smartphone as an input device to take a facial photograph and input personal identification information. Upon receiving this information, the server applies a machine learning algorithm, such as a generative AI model (e.g., GANs), using a digital image generation device to generate a duplicate image of the user wearing clothes. This process utilizes machine learning frameworks such as TensorFlow.

[0512] The generated duplicate images are sent from the server to the smartphone's display device and presented to the user in a catalog format. The availability of the clothing selected by the user is checked in real time via the communication network. The server then identifies the optimal physical store based on geographical and inventory information and provides this information to the user. The server further confirms the fitting reservation at the suggested physical store using a reservation control mechanism. A REST API is used for this communication.

[0513] For example, if a user is choosing an outfit for their coming-of-age ceremony, the process can proceed as follows: A prompt message will appear stating, "If you want to choose a stylish kimono for your coming-of-age ceremony, please upload a selfie of your face. We will generate images of you trying on various kimonos and help you choose the one that suits you best!" By following this prompt, the user can efficiently select the appropriate outfit and make a reservation at the most suitable fitting shop.

[0514] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0515] Step 1:

[0516] The user takes a photo of their face with their smartphone camera and sends the facial image data to the server through the application. This facial image, as input data, is processed by the server as personally identifiable information. The server receives this data and verifies its reliability and accuracy.

[0517] Step 2:

[0518] The server passes the received personal identification information to a digital image generator. This image generator uses a generation AI model to generate duplicate images that show the user wearing multiple outfits. The data processing performed here involves adding clothing to the user's facial photograph to generate a realistic try-on image. These generated duplicate images are obtained as output.

[0519] Step 3:

[0520] The server sends the generated duplicate images to the smartphone's display device. The device receives these and displays them in a catalog format for the user to browse. The user interacts with this catalog, identifying and selecting clothing items of interest.

[0521] Step 4:

[0522] The user's selected clothing information is sent to the server. The server retrieves real-time inventory information for the selected clothing from multiple stores via the communication network. In this process, it connects to each store's database using a REST API and aggregates the inventory information.

[0523] Step 5:

[0524] Based on collected inventory information and the user's geographical information, the server uses data analysis tools to identify the optimal physical store and propose it to the user. This process involves applying an algorithm to compare multiple candidate stores, ultimately identifying the physical store.

[0525] Step 6:

[0526] After the user selects a suggested physical store, the server uses a reservation control mechanism to make a fitting reservation for the selected store. The reservation data is sent to the store's reservation management system, and a reservation confirmation is output. This prepares the user to efficiently try on clothes at the designated store.

[0527] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0528] This invention constructs a system to provide a more personalized clothing selection experience by combining emotion recognition technology with the process by which users efficiently select clothing. The system includes a terminal for inputting the user's facial photograph and personal identification information, an emotion recognition engine, a digital image generation device, and a communication network for checking clothing inventory and identifying the optimal fitting store.

[0529] The user first uses a device to input their current facial expressions and voice into the system via camera and microphone, along with a photo of their face. The server receives this data and passes the facial image data to an image generator, and the facial expression and voice data to an emotion engine. The emotion engine analyzes the user's emotional state and uses it as an indicator to determine what the user likes and what kind of clothing they are interested in.

[0530] The server then uses image generation AI to create multiple digital outfit images based on the user's facial photograph. These images are compiled into a catalog, taking into account the user's emotional information, and sorted in order of likelihood of user interest. The catalog is transferred to the device, where the user browses and selects their favorite outfits.

[0531] Once the user has made their selection, the server checks the availability of the clothing via the communication network. Simultaneously, the server analyzes the data, taking into account the user's preferences based on an emotion engine, to identify the optimal fitting room. The user's emotional information plays a crucial role in suggesting the best time and location for the user.

[0532] For example, if a user tends to prefer clothing in soothing colors, the system will prioritize displaying clothing in calming tones within the generated catalog. This priority display is achieved through an emotion engine, allowing users to quickly select clothing that better matches their preferences. Once the selection is complete and the user chooses a fitting location, the system automatically makes a fitting reservation and notifies the user of the details. In this way, users can experience an efficient and satisfying clothing selection process.

[0533] The following describes the processing flow.

[0534] Step 1:

[0535] The user uses their device to input a photo of their face, along with facial expressions and voice. During this process, the camera and microphone are activated, and user data is collected.

[0536] Step 2:

[0537] The device sends the facial image data and emotion data it collects to the server. This includes formatting the image data and preparing the audio data for analysis.

[0538] Step 3:

[0539] The server passes a facial image to an image generator and sends emotional data to an emotion engine. The emotion engine uses this data to analyze the user's current emotional state.

[0540] Step 4:

[0541] The server uses image generation AI to generate digital images of the user wearing multiple outfits based on their face. This process incorporates emotional information from an emotion engine and is adjusted to reflect the user's preferences.

[0542] Step 5:

[0543] The server sorts the generated duplicate images based on sentiment and sends them to the terminal as a catalog. The terminal then presents the catalog to the user in this order.

[0544] Step 6:

[0545] The user browses this catalog and selects the clothing items they are interested in. Once the selection is complete, the device sends that information to the server.

[0546] Step 7:

[0547] The server collects inventory information for selected clothing items from multiple stores via a communication network. Simultaneously, it performs data analysis to identify the optimal store based on the results of the emotion engine's analysis.

[0548] Step 8:

[0549] The server sends information about the most suitable fitting room to the user's terminal. Here, the user's preferences are reflected to the greatest extent possible, taking into account geographical conditions and inventory information.

[0550] Step 9:

[0551] The user reviews the store information provided on their device and decides to make a fitting reservation. Once the user approves, the device notifies the server of that information.

[0552] Step 10:

[0553] The server, through the reservation control mechanism, initiates the process of confirming the fitting reservation at the selected store. This information is then confirmed by the user, and the fitting reservation is finalized.

[0554] (Example 2)

[0555] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0556] In recent years, online shopping for clothing has presented numerous options, making it difficult for users to choose the best product based on their preferences and feelings. Furthermore, limited opportunities to try on clothes before purchase often lead to anxiety about whether the items chosen online will actually fit. To address these issues, there is a need for a system that simultaneously provides more personalized clothing selection and appropriate fitting suggestions.

[0557] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0558] In this invention, the server includes means for receiving personal identification information and emotional data from an input device, means for operating a digital image generation device based on the personal identification information and emotional data to generate a replica image of the user wearing the outfit, and means for rearranging the generated replica image taking emotional information into consideration and transmitting it to a display device for presentation. This allows the user to easily select an outfit based on their own emotions and preferences, and also enables the suggestion and reservation of fitting rooms.

[0559] "Personal identification information" refers to information used to identify a specific user, and includes data such as names and facial photographs.

[0560] "Emotional data" refers to information that indicates a user's emotional state, and is obtained through data such as facial expressions and tone of voice.

[0561] A "digital image generation device" is a device that uses digital technology to generate images that represent the user's appearance in a virtual space.

[0562] A "duplicate image" is a digital image created virtually from the user's face and body, dressed in specific clothing.

[0563] A "display device" is a device used to visually present generated digital images or information, and includes screens and displays.

[0564] A "communication network" is an infrastructure for sending and receiving data between different computers, and includes the internet and dedicated lines.

[0565] An "optimal physical store" is a physical store that is most suitable for trying on and purchasing products, based on the user's geographical location and inventory status.

[0566] "Data analysis methods" refer to technical techniques and systems used to process collected data and derive useful conclusions and suggestions for users.

[0567] A "reservation control means" is a mechanism for making reservations at physical stores for trying on clothes or making purchases, and for notifying the user of the results.

[0568] This invention is a system that combines emotion recognition technology and a generative AI model to provide a personalized clothing selection experience in the process of efficiently selecting clothes for the user. First, the user takes a photo of their face using a device and inputs facial expression and voice data through the camera and microphone. The device then sends this data to a server.

[0569] The server performs analysis using the received personal identification information and sentiment data. The personal identification information is passed to a digital image generator, where a generative AI model using a machine learning algorithm generates duplicate images of the user wearing multiple outfits based on the user's facial photograph. In this generation process, the prompt message "Use the generative AI model to generate multiple styles of digital outfit images for the user's facial photograph" can be used.

[0570] The generated duplicate images are aggregated by a server, and the user's emotional data is analyzed through an emotion engine. Then, taking the emotional information into consideration, the images are formatted into a catalog in an order that is likely to interest the user. This catalog is sent to the user's device, allowing the user to select clothing that matches their preferences.

[0571] For example, if a user tends to prefer a certain color scheme or style, the system will prioritize presenting clothing in calmer tones. In this process, the emotion engine analyzes the user's preferences and incorporates them as important information.

[0572] The user's selected clothing information is cross-referenced with inventory data by a server, and the optimal physical store is identified via a communication network. This identification process considers geographical data, inventory information, and even sentiment data to suggest the best location and time for the user to try on the clothes.

[0573] Finally, the server makes a fitting reservation and notifies the user of the reservation details via the terminal. The reservation control system ensures this process runs smoothly, allowing the user to flexibly confirm the date, time, and location of the fitting. In this way, the user can have a faster and more personalized clothing selection experience.

[0574] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0575] Step 1:

[0576] The user uses the device to take a photo of their face with the camera and input voice data through the microphone. The device sends this data to the server in digital format. In this step, the input data consists of a facial photo and voice, and the output data is the digital data sent to the server.

[0577] Step 2:

[0578] The server receives the transmitted facial image data and passes it to the digital image generation device. Simultaneously, it passes facial expression and voice data to the emotion engine. This process takes the received facial image and emotion data as input data and produces personal identification information and analyzed emotion data as output. The server uses the emotion engine to analyze the data and infer the user's emotional state.

[0579] Step 3:

[0580] The server uses a generative AI model to generate multiple digital clothing images based on the user's facial photograph. The prompt used is "Use the generative AI model to generate multiple styles of digital clothing images for the user's facial photograph." In this step, the input data consists of the facial photograph and the generative AI model, and the output is the generated duplicate images.

[0581] Step 4:

[0582] The server aggregates the generated duplicate images and sorts them based on sentiment information to create a catalog. In this process, sentiment information and duplicate images are used as input data, and the output catalog is sorted in an order that is likely to interest the user. The server then sends this organized catalog to the user's terminal.

[0583] Step 5:

[0584] The user browses a catalog on their device and selects clothes they like. The selected data is sent from the device to the server. Here, the input is the catalog and the user's selection, and the output is the information of the selected clothes returned to the server.

[0585] Step 6:

[0586] The server checks the inventory information of the selected clothing item via the communication network and identifies the most suitable physical store. Input for this process includes information about the selected clothing item and the user's geographical data, and the output generates store identification information. The server also takes into account information from the emotion engine to determine the most suitable store for the user.

[0587] Step 7:

[0588] The server makes a fitting reservation at a designated store and notifies the user of the reservation information. In this process, the input data includes information about the fitting store and available time slots. The output includes details of the fitting reservation, which the server sends to the terminal, allowing the user to confirm the schedule.

[0589] (Application Example 2)

[0590] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0591] Today's individual consumers are often overwhelmed by the sheer number of choices available in physical stores and online shops when selecting clothing. However, it is difficult for consumers to quickly choose clothes that suit their personal preferences and current emotional state. Furthermore, they need to be selective about the time of day and store for efficient trying on and purchasing. In this context, providing users with more personalized information is necessary to ensure they have the optimal shopping experience.

[0592] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0593] In this invention, the server includes means for receiving personal identification information and emotional information from an input device, means for operating a digital image generation device based on the personal identification information and emotional information to generate duplicate images of multiple garments being worn, and means for transmitting and presenting the generated duplicate images to a display device. This enables users to efficiently and personally select clothing in physical stores using a visual device.

[0594] "Personally identifiable information" refers to data used to identify a specific individual, including information such as a user's name, ID, and facial photograph.

[0595] "Emotional information" refers to data that indicates a user's emotional state, and is information analyzed from facial expressions, tone of voice, and other similar factors.

[0596] A "digital image generation device" is a device that generates and edits images using digital technology, and it creates images of the user wearing clothes based on their emotional information.

[0597] A "display device" is a device used to visually display electronic data, and includes smart glasses, smartphones, monitors, and other similar devices.

[0598] A "communication network" is the infrastructure used for exchanging information, and includes the internet and dedicated communication lines.

[0599] "Data analysis tools" are algorithms and processes for identifying and extracting useful information from large amounts of data, and have the function of finding the optimal store and time based on geographical data, inventory information, and sentiment data.

[0600] A "reservation control means" refers to a method or system for managing and coordinating reservations for fittings and purchases, ensuring that users can make reservations smoothly.

[0601] "Visual devices" are devices that allow users to acquire visual information, and include smart glasses and AR (augmented reality) headsets.

[0602] The system for realizing this invention involves the cooperation of a user, a terminal, and a server. First, the user provides their personal identification information and emotional information through the terminal's camera and microphone. This allows the terminal to acquire the user's facial image and voice and send them to the server. Upon receiving this data, the server analyzes the user's emotions using an emotion engine. This emotion analysis utilizes machine learning frameworks such as TensorFlow to determine the emotional state from the user's facial expressions and voice.

[0603] Next, the server uses the user's personal identification information and emotional information to operate a digital image generator and generate a replica image of the user wearing the clothes. This process utilizes generative AI models such as GANs (Generative Adversarial Networks) to generate realistic and visually appealing images of the clothing. These generated images are streamed to the user's smart glasses or smartphone and displayed in their field of view in real time.

[0604] When a user selects clothing from a displayed catalog, the server checks the inventory information of that clothing via the communication network and simultaneously suggests the most suitable physical store and visit time. Users can instantly obtain this information using smart glasses and seamlessly make fitting reservations.

[0605] Furthermore, if the user is feeling stressed, the app can prioritize displaying clothing in relaxing colors and play relaxing music. This system utilizes a generated AI model, and by setting prompt examples to "Generate the optimal color coordination for the clothing selected by the user and prioritize displaying items with calming tones," the user experience can be optimized.

[0606] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0607] Step 1:

[0608] The user inputs personal identification information and sentiment information using the device's camera and microphone. In this step, the device captures the user's facial image and voice data. The input data is preprocessed, with the device compressing the facial image and converting the voice to text. The output is sent to the server as personal identification information and sentiment information.

[0609] Step 2:

[0610] The server analyzes the user's emotions using an emotion engine based on the received personal identification and emotion information. The input is the data sent in the previous step, and the server performs emotion analysis using TensorFlow. It determines the user's emotional state from facial features and voice tone and generates an emotion label as output. This is an indicator used in subsequent clothing selection.

[0611] Step 3:

[0612] The server uses personal identification information and emotional information to operate a digital image generator and generate a digital clothing catalog. The input consists of emotional labels and specific user information. A generation AI model is used to generate clothing images suitable for the user and rank them according to the user's preferences. The output is a catalog of digital clothing images.

[0613] Step 4:

[0614] The terminal displays a digital clothing catalog generated on the user's smart glasses or smartphone. Input from the server is image data, which is rendered in real time on the terminal. The user browses the displayed catalog and selects clothing items of interest through the interface. The output is the user's clothing selection information.

[0615] Step 5:

[0616] The server retrieves and analyzes inventory and geographical information of the user's selected clothing via a communication network. Inputs include user selection information and provided store information. Using inventory information and the user's geographical location, the server identifies the optimal store for trying on clothes and the best time to visit. The output is a suggestion of the optimal fitting location and time.

[0617] Step 6:

[0618] The user uses the terminal to check the suggested fitting rooms and appointment times, and makes a fitting reservation if necessary. The input from the server is fitting information, and the terminal, after obtaining user approval, confirms the fitting reservation with the store using the reservation control mechanism. The output is a notification that the fitting reservation has been completed.

[0619] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0620] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0621] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0622] [Fourth Embodiment]

[0623] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0624] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0625] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0626] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0627] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0628] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0629] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0630] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0631] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0632] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0633] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0634] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0635] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0636] This invention provides a system that allows users to efficiently select clothing and reduce the hassle of trying them on. To implement this system, a terminal for inputting the user's facial photograph and personal identification information, a server for processing data and generating images, and a communication network for checking clothing inventory and making fitting reservations are required.

[0637] The user first inputs a photo of their face using a terminal, and the server receives this photo data and processes it using an image generation device. The image generation device uses AI technology to generate multiple digital images that simulate the user trying on clothes, and sends these to the terminal in catalog format.

[0638] When a user browses this catalog on their device and selects an item of clothing they like, the server checks the inventory of the selected item at multiple stores. During this process, it accesses each store's database via the communication network to collect information such as inventory status and available fitting times.

[0639] To suggest the most suitable store, the server uses data analysis tools to identify the optimal store based on geographical and inventory information. This information is then presented to the user's terminal, and once the user selects a store, a reservation control tool automatically makes a fitting reservation for the specified store.

[0640] As a concrete example, let's consider a case where user A wants to choose a furisode (long-sleeved kimono) for their coming-of-age ceremony. User A takes a photo of their face using their smartphone and sends it via the application. The server then uses AI to generate digital images of user A trying on multiple furisode styles and sends them to the user's device in an easy-to-view catalog format. User A reviews this catalog and selects a red furisode. The server searches a database of nearby rental shops and identifies the best store with the selected garment in stock, suggesting it to user A. When user A chooses to make a fitting reservation at the store, the server uses a reservation control mechanism to confirm the reservation. This embodiment allows users to efficiently select clothing and prepare for a specific event with reduced burden.

[0641] The following describes the processing flow.

[0642] Step 1:

[0643] The user takes a photo of their face using their device and sends that photo to the server via the application.

[0644] Step 2:

[0645] The server checks the received facial photograph, performs pre-processing such as adjusting the image format, and then prepares it for transmission to the digital image generation device.

[0646] Step 3:

[0647] The server runs an image generation AI to generate digital replicas of the user wearing multiple outfits, based on the user's facial photograph.

[0648] Step 4:

[0649] The server organizes the generated duplicate images into a catalog, transfers them to the user's terminal, and displays them.

[0650] Step 5:

[0651] The user browses a catalog presented through their device and selects the clothing items they like. Once the selection is complete, the device sends that information to the server.

[0652] Step 6:

[0653] The server uses the selection information it receives to query databases of multiple related stores to check the availability of the selected clothing items and the dates and times when they can be tried on.

[0654] Step 7:

[0655] Based on the inventory information obtained by the server, an algorithm selects the optimal store, taking into account the user's location and other factors.

[0656] Step 8:

[0657] The server sends the optimal store information it has selected to the user's terminal and presents the recommended store to the user.

[0658] Step 9:

[0659] When a user approves a fitting appointment at a store suggested through their device, the device transmits that information to the server.

[0660] Step 10:

[0661] The server uses a reservation control mechanism to access the reservation system of the selected store and execute the fitting reservation procedure.

[0662] Step 11:

[0663] Once the server confirms the reservation is complete, it sends the details to the terminal to notify the user of the reservation.

[0664] (Example 1)

[0665] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0666] Traditional clothing selection required users to visit physical stores and try on clothes, which was time-consuming and laborious. Furthermore, checking inventory often required visiting multiple stores, making it inefficient. This resulted in a limited selection of clothing options for users.

[0667] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0668] In this invention, the server includes means for receiving personal identification information from a data input device, means for generating try-on images of the user wearing multiple garments using a generation AI model, and means for transmitting and displaying the generated try-on images on a display device. This enables the user to efficiently try on and consider garments. Furthermore, by checking inventory information of selected garments via an information and communication network, identifying the optimal store based on geographical data and inventory status, and automatically making try-on reservations, the effort required to visit physical stores is significantly reduced, allowing the user to smoothly select and reserve desired garments.

[0669] "Personally identifiable information" refers to information used to identify a user, and includes data such as names, facial photographs, and other identifiable data.

[0670] A "data input device" refers to hardware or software used to acquire and transmit personally identifiable information, such as smartphones and tablets.

[0671] A "generative AI model" is a part of artificial intelligence that utilizes machine learning algorithms to generate new digital data based on input data.

[0672] A "try-on image" is a computer-generated image that visually represents what it would look like if a user were wearing various clothes.

[0673] "Display devices" are devices used to visually present digital data, and include monitors and displays.

[0674] An "information and communication network" is a network used to exchange data between multiple devices, and the internet is an example of this.

[0675] "Geographic data" refers to data related to physical locations or places, including GPS information and address data.

[0676] "Inventory information" refers to data that shows how many units of a particular product are currently available.

[0677] "Reservation management methods" refer to the processes and systems used to secure a date and time for using a selected service or product.

[0678] To implement this invention, first, the user inputs personal identification information using a terminal. The terminal uses a data input device such as a smartphone or tablet to take a picture of the user's face through its camera and collect personal identification information. Next, the server receives this information and uses a generation AI model to generate digital images of the user trying on multiple outfits.

[0679] This image generation process utilizes AI software that implements machine learning algorithms. A processor on the server analyzes the received facial photograph, creates a model best suited to the user's face, and applies clothing to it. Specifically, general AI image processing techniques can be used as the generative AI model.

[0680] The generated try-on images are sent from the server to the terminal, where the user can view them in catalog format using a display device. The catalog is designed to allow the user to select their preferred clothing.

[0681] When a user selects clothing on their device, the server checks the inventory information for the selected clothing from multiple stores via the information and communication network. Based on geographical data and inventory status, it identifies the most suitable store and suggests it to the user. To achieve this, the server can utilize geolocation services and store management software.

[0682] Ultimately, the user selects a fitting appointment from the suggested stores on their device. The server uses a reservation management system to confirm the fitting appointment at the specified store. This process eliminates the need for the user to visit physical stores, allowing them to efficiently try on, consider, and select clothing.

[0683] As a concrete example, consider a user who wants to choose a special kimono for their coming-of-age ceremony. The user takes a photo of their face with their smartphone and sends it through the application. The server uses a generative AI model to generate digital images of the user trying on various kimonos and displays them as a catalog. When the user selects a red kimono, the server identifies the most suitable store based on inventory information and geographical data and confirms the fitting reservation. An example of a prompt message could be a request such as, "Please generate images of me trying on a special kimono." Such a system allows users to efficiently try on clothes and prepare for a specific event.

[0684] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0685] Step 1:

[0686] The user takes a photo of their face using their device and sends this data, along with their personal identification information, to the server. Specifically, they activate the smartphone camera, press the "Capture" button to take a photo, and then press the "Send" button using the application. The system receives the captured photo and personal identification information as input. This data is sent to the server as output.

[0687] Step 2:

[0688] Based on the personal identification information and facial image data received by the server, a generative AI model is used to generate try-on images. Specifically, the server's processor uses an image processing algorithm to generate a body silhouette that fits the user's face and synthesizes different clothing styles. Facial image data is received as input, and multiple try-on images are generated as output.

[0689] Step 3:

[0690] The server converts the generated try-on images into a catalog format and sends them to the terminal. Specifically, the server organizes the images into an easy-to-view layout and remotely transmits them to the user's terminal via the network. Try-on images are received as input, and catalog-formatted image data is sent to the terminal as output.

[0691] Step 4:

[0692] The user browses a catalog on their device and selects the desired clothing item. Specifically, the user operates the touchscreen, scrolls through images, and taps on items. The input is the clothing item selected from the catalog, and the output is information about the selected clothing item sent to the server.

[0693] Step 5:

[0694] The server checks inventory information for the selected clothing item via the information and communication network and identifies the optimal store. Specifically, the server sends queries to multiple pre-registered store databases to retrieve and analyze inventory status and location information. It receives information about the selected clothing item as input and generates a list of optimal stores as output.

[0695] Step 6:

[0696] The server sends and suggests the most suitable store information to the user. Once the user selects a preferred store based on the suggestions, the server confirms the fitting reservation using a reservation management system. Specifically, the reservation information is confirmed when the user taps the selected store, and the schedule is updated on the system. The system receives the most suitable store information as input and generates information confirming the reservation as output.

[0697] (Application Example 1)

[0698] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0699] In traditional consumer goods selection and purchasing processes, it is difficult for individuals to determine whether an item suits them before actually visiting a store, and this is especially true for clothing, where trying things on is time-consuming. Furthermore, there is the problem of difficulty in checking the real-time inventory status of specific products and selecting the most suitable store.

[0700] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0701] In this invention, the server includes means for receiving personal identification information from an input device, means for operating a digital image generation device based on the personal identification information to generate duplicate images of multiple garments being worn, and means for transmitting and presenting the generated duplicate images to a display device. This enables users to select suitable clothing through virtual try-on and efficiently make try-on reservations at the most suitable stores.

[0702] "Personally identifiable information" refers to information used to identify an individual user, and includes biometric data such as facial photographs.

[0703] An "input device" refers to hardware that a user uses to send data to a system, such as a smartphone or tablet.

[0704] A "digital image generation device" is a device that executes a machine learning algorithm to generate virtual try-on images based on the user's personal identification information.

[0705] A "duplicate image" is a digital image generated using AI technology that reproduces how a user looks wearing various types of clothing.

[0706] A "display device" is a device used by users to view generated digital images or catalogs, and is typically a smartphone screen or a computer monitor.

[0707] A "communication network" refers to the infrastructure used to send and receive data, and includes digital communication systems such as the internet.

[0708] "Data analysis means" refers to a method or process for processing information necessary for selecting the optimal physical store and making decisions based on that information.

[0709] "Reservation control means" refers to a method or device for managing and executing fitting reservations at physical stores selected by the user.

[0710] A "catalog format" is a display format that is arranged to make it easy for users to view, compare, and select generated duplicate images.

[0711] "Communication means" refers to the mechanisms and protocols used to send and receive data between a user and a server or system.

[0712] To implement this invention, the user first uses a smartphone as an input device to take a facial photograph and input personal identification information. Upon receiving this information, the server applies a machine learning algorithm, such as a generative AI model (e.g., GANs), using a digital image generation device to generate a duplicate image of the user wearing clothes. This process utilizes machine learning frameworks such as TensorFlow.

[0713] The generated duplicate images are sent from the server to the smartphone's display device and presented to the user in a catalog format. The availability of the clothing selected by the user is checked in real time via the communication network. The server then identifies the optimal physical store based on geographical and inventory information and provides this information to the user. The server further confirms the fitting reservation at the suggested physical store using a reservation control mechanism. A REST API is used for this communication.

[0714] For example, if a user is choosing an outfit for their coming-of-age ceremony, the process can proceed as follows: A prompt message will appear stating, "If you want to choose a stylish kimono for your coming-of-age ceremony, please upload a selfie of your face. We will generate images of you trying on various kimonos and help you choose the one that suits you best!" By following this prompt, the user can efficiently select the appropriate outfit and make a reservation at the most suitable fitting shop.

[0715] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0716] Step 1:

[0717] The user takes a photo of their face with their smartphone camera and sends the facial image data to the server through the application. This facial image, as input data, is processed by the server as personally identifiable information. The server receives this data and verifies its reliability and accuracy.

[0718] Step 2:

[0719] The server passes the received personal identification information to a digital image generator. This image generator uses a generation AI model to generate duplicate images that show the user wearing multiple outfits. The data processing performed here involves adding clothing to the user's facial photograph to generate a realistic try-on image. These generated duplicate images are obtained as output.

[0720] Step 3:

[0721] The server sends the generated duplicate images to the smartphone's display device. The device receives these and displays them in a catalog format for the user to browse. The user interacts with this catalog, identifying and selecting clothing items of interest.

[0722] Step 4:

[0723] The user's selected clothing information is sent to the server. The server retrieves real-time inventory information for the selected clothing from multiple stores via the communication network. In this process, it connects to each store's database using a REST API and aggregates the inventory information.

[0724] Step 5:

[0725] Based on collected inventory information and the user's geographical information, the server uses data analysis tools to identify the optimal physical store and propose it to the user. This process involves applying an algorithm to compare multiple candidate stores, ultimately identifying the physical store.

[0726] Step 6:

[0727] After the user selects a suggested physical store, the server uses a reservation control mechanism to make a fitting reservation for the selected store. The reservation data is sent to the store's reservation management system, and a reservation confirmation is output. This prepares the user to efficiently try on clothes at the designated store.

[0728] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0729] This invention constructs a system to provide a more personalized clothing selection experience by combining emotion recognition technology with the process by which users efficiently select clothing. The system includes a terminal for inputting the user's facial photograph and personal identification information, an emotion recognition engine, a digital image generation device, and a communication network for checking clothing inventory and identifying the optimal fitting store.

[0730] The user first uses a device to input their current facial expressions and voice into the system via camera and microphone, along with a photo of their face. The server receives this data and passes the facial image data to an image generator, and the facial expression and voice data to an emotion engine. The emotion engine analyzes the user's emotional state and uses it as an indicator to determine what the user likes and what kind of clothing they are interested in.

[0731] The server then uses image generation AI to create multiple digital outfit images based on the user's facial photograph. These images are compiled into a catalog, taking into account the user's emotional information, and sorted in order of likelihood of user interest. The catalog is transferred to the device, where the user browses and selects their favorite outfits.

[0732] Once the user has made their selection, the server checks the availability of the clothing via the communication network. Simultaneously, the server analyzes the data, taking into account the user's preferences based on an emotion engine, to identify the optimal fitting room. The user's emotional information plays a crucial role in suggesting the best time and location for the user.

[0733] For example, if a user tends to prefer clothing in soothing colors, the system will prioritize displaying clothing in calming tones within the generated catalog. This priority display is achieved through an emotion engine, allowing users to quickly select clothing that better matches their preferences. Once the selection is complete and the user chooses a fitting location, the system automatically makes a fitting reservation and notifies the user of the details. In this way, users can experience an efficient and satisfying clothing selection process.

[0734] The following describes the processing flow.

[0735] Step 1:

[0736] The user uses their device to input a photo of their face, along with facial expressions and voice. During this process, the camera and microphone are activated, and user data is collected.

[0737] Step 2:

[0738] The device sends the facial image data and emotion data it collects to the server. This includes formatting the image data and preparing the audio data for analysis.

[0739] Step 3:

[0740] The server passes a facial image to an image generator and sends emotional data to an emotion engine. The emotion engine uses this data to analyze the user's current emotional state.

[0741] Step 4:

[0742] The server uses image generation AI to generate digital images of the user wearing multiple outfits based on their face. This process incorporates emotional information from an emotion engine and is adjusted to reflect the user's preferences.

[0743] Step 5:

[0744] The server sorts the generated duplicate images based on sentiment and sends them to the terminal as a catalog. The terminal then presents the catalog to the user in this order.

[0745] Step 6:

[0746] The user browses this catalog and selects the clothing items they are interested in. Once the selection is complete, the device sends that information to the server.

[0747] Step 7:

[0748] The server collects inventory information for selected clothing items from multiple stores via a communication network. Simultaneously, it performs data analysis to identify the optimal store based on the results of the emotion engine's analysis.

[0749] Step 8:

[0750] The server sends information about the most suitable fitting room to the user's terminal. Here, the user's preferences are reflected to the greatest extent possible, taking into account geographical conditions and inventory information.

[0751] Step 9:

[0752] The user reviews the store information provided on their device and decides to make a fitting reservation. Once the user approves, the device notifies the server of that information.

[0753] Step 10:

[0754] The server, through the reservation control mechanism, initiates the process of confirming the fitting reservation at the selected store. This information is then confirmed by the user, and the fitting reservation is finalized.

[0755] (Example 2)

[0756] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0757] In recent years, online shopping for clothing has presented numerous options, making it difficult for users to choose the best product based on their preferences and feelings. Furthermore, limited opportunities to try on clothes before purchase often lead to anxiety about whether the items chosen online will actually fit. To address these issues, there is a need for a system that simultaneously provides more personalized clothing selection and appropriate fitting suggestions.

[0758] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0759] In this invention, the server includes means for receiving personal identification information and emotional data from an input device, means for operating a digital image generation device based on the personal identification information and emotional data to generate a replica image of the user wearing the outfit, and means for rearranging the generated replica image taking emotional information into consideration and transmitting it to a display device for presentation. This allows the user to easily select an outfit based on their own emotions and preferences, and also enables the suggestion and reservation of fitting rooms.

[0760] "Personal identification information" refers to information used to identify a specific user, and includes data such as names and facial photographs.

[0761] "Emotional data" refers to information that indicates a user's emotional state, and is obtained through data such as facial expressions and tone of voice.

[0762] A "digital image generation device" is a device that uses digital technology to generate images that represent the user's appearance in a virtual space.

[0763] A "duplicate image" is a digital image created virtually from the user's face and body, dressed in specific clothing.

[0764] A "display device" is a device used to visually present generated digital images or information, and includes screens and displays.

[0765] A "communication network" is an infrastructure for sending and receiving data between different computers, and includes the internet and dedicated lines.

[0766] An "optimal physical store" is a physical store that is most suitable for trying on and purchasing products, based on the user's geographical location and inventory status.

[0767] "Data analysis methods" refer to technical techniques and systems used to process collected data and derive useful conclusions and suggestions for users.

[0768] A "reservation control means" is a mechanism for making reservations at physical stores for trying on clothes or making purchases, and for notifying the user of the results.

[0769] This invention is a system that combines emotion recognition technology and a generative AI model to provide a personalized clothing selection experience in the process of efficiently selecting clothes for the user. First, the user takes a photo of their face using a device and inputs facial expression and voice data through the camera and microphone. The device then sends this data to a server.

[0770] The server performs analysis using the received personal identification information and sentiment data. The personal identification information is passed to a digital image generator, where a generative AI model using a machine learning algorithm generates duplicate images of the user wearing multiple outfits based on the user's facial photograph. In this generation process, the prompt message "Use the generative AI model to generate multiple styles of digital outfit images for the user's facial photograph" can be used.

[0771] The generated duplicate images are aggregated by a server, and the user's emotional data is analyzed through an emotion engine. Then, taking the emotional information into consideration, the images are formatted into a catalog in an order that is likely to interest the user. This catalog is sent to the user's device, allowing the user to select clothing that matches their preferences.

[0772] For example, if a user tends to prefer a certain color scheme or style, the system will prioritize presenting clothing in calmer tones. In this process, the emotion engine analyzes the user's preferences and incorporates them as important information.

[0773] The user's selected clothing information is cross-referenced with inventory data by a server, and the optimal physical store is identified via a communication network. This identification process considers geographical data, inventory information, and even sentiment data to suggest the best location and time for the user to try on the clothes.

[0774] Finally, the server makes a fitting reservation and notifies the user of the reservation details via the terminal. The reservation control system ensures this process runs smoothly, allowing the user to flexibly confirm the date, time, and location of the fitting. In this way, the user can have a faster and more personalized clothing selection experience.

[0775] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0776] Step 1:

[0777] The user uses the device to take a photo of their face with the camera and input voice data through the microphone. The device sends this data to the server in digital format. In this step, the input data consists of a facial photo and voice, and the output data is the digital data sent to the server.

[0778] Step 2:

[0779] The server receives the transmitted facial image data and passes it to the digital image generation device. Simultaneously, it passes facial expression and voice data to the emotion engine. This process takes the received facial image and emotion data as input data and produces personal identification information and analyzed emotion data as output. The server uses the emotion engine to analyze the data and infer the user's emotional state.

[0780] Step 3:

[0781] The server uses a generative AI model to generate multiple digital clothing images based on the user's facial photograph. The prompt used is "Use the generative AI model to generate multiple styles of digital clothing images for the user's facial photograph." In this step, the input data consists of the facial photograph and the generative AI model, and the output is the generated duplicate images.

[0782] Step 4:

[0783] The server aggregates the generated duplicate images and sorts them based on sentiment information to create a catalog. In this process, sentiment information and duplicate images are used as input data, and the output catalog is sorted in an order that is likely to interest the user. The server then sends this organized catalog to the user's terminal.

[0784] Step 5:

[0785] The user browses a catalog on their device and selects clothes they like. The selected data is sent from the device to the server. Here, the input is the catalog and the user's selection, and the output is the information of the selected clothes returned to the server.

[0786] Step 6:

[0787] The server checks the inventory information of the selected clothing item via the communication network and identifies the most suitable physical store. Input for this process includes information about the selected clothing item and the user's geographical data, and the output generates store identification information. The server also takes into account information from the emotion engine to determine the most suitable store for the user.

[0788] Step 7:

[0789] The server makes a fitting reservation at a designated store and notifies the user of the reservation information. In this process, the input data includes information about the fitting store and available time slots. The output includes details of the fitting reservation, which the server sends to the terminal, allowing the user to confirm the schedule.

[0790] (Application Example 2)

[0791] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0792] Today's individual consumers are often overwhelmed by the sheer number of choices available in physical stores and online shops when selecting clothing. However, it is difficult for consumers to quickly choose clothes that suit their personal preferences and current emotional state. Furthermore, they need to be selective about the time of day and store for efficient trying on and purchasing. In this context, providing users with more personalized information is necessary to ensure they have the optimal shopping experience.

[0793] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0794] In this invention, the server includes means for receiving personal identification information and emotional information from an input device, means for operating a digital image generation device based on the personal identification information and emotional information to generate duplicate images of multiple garments being worn, and means for transmitting and presenting the generated duplicate images to a display device. This enables users to efficiently and personally select clothing in physical stores using a visual device.

[0795] "Personally identifiable information" refers to data used to identify a specific individual, including information such as a user's name, ID, and facial photograph.

[0796] "Emotional information" refers to data that indicates a user's emotional state, and is information analyzed from facial expressions, tone of voice, and other similar factors.

[0797] A "digital image generation device" is a device that generates and edits images using digital technology, and it creates images of the user wearing clothes based on their emotional information.

[0798] A "display device" is a device used to visually display electronic data, and includes smart glasses, smartphones, monitors, and other similar devices.

[0799] A "communication network" is the infrastructure used for exchanging information, and includes the internet and dedicated communication lines.

[0800] "Data analysis tools" are algorithms and processes for identifying and extracting useful information from large amounts of data, and have the function of finding the optimal store and time based on geographical data, inventory information, and sentiment data.

[0801] A "reservation control means" refers to a method or system for managing and coordinating reservations for fittings and purchases, ensuring that users can make reservations smoothly.

[0802] "Visual devices" are devices that allow users to acquire visual information, and include smart glasses and AR (augmented reality) headsets.

[0803] The system for realizing this invention involves the cooperation of a user, a terminal, and a server. First, the user provides their personal identification information and emotional information through the terminal's camera and microphone. This allows the terminal to acquire the user's facial image and voice and send them to the server. Upon receiving this data, the server analyzes the user's emotions using an emotion engine. This emotion analysis utilizes machine learning frameworks such as TensorFlow to determine the emotional state from the user's facial expressions and voice.

[0804] Next, the server uses the user's personal identification information and emotional information to operate a digital image generator and generate a replica image of the user wearing the clothes. This process utilizes generative AI models such as GANs (Generative Adversarial Networks) to generate realistic and visually appealing images of the clothing. These generated images are streamed to the user's smart glasses or smartphone and displayed in their field of view in real time.

[0805] When a user selects clothing from a displayed catalog, the server checks the inventory information of that clothing via the communication network and simultaneously suggests the most suitable physical store and visit time. Users can instantly obtain this information using smart glasses and seamlessly make fitting reservations.

[0806] Furthermore, if the user is feeling stressed, the app can prioritize displaying clothing in relaxing colors and play relaxing music. This system utilizes a generated AI model, and by setting prompt examples to "Generate the optimal color coordination for the clothing selected by the user and prioritize displaying items with calming tones," the user experience can be optimized.

[0807] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0808] Step 1:

[0809] The user inputs personal identification information and sentiment information using the device's camera and microphone. In this step, the device captures the user's facial image and voice data. The input data is preprocessed, with the device compressing the facial image and converting the voice to text. The output is sent to the server as personal identification information and sentiment information.

[0810] Step 2:

[0811] The server analyzes the user's emotions using an emotion engine based on the received personal identification and emotion information. The input is the data sent in the previous step, and the server performs emotion analysis using TensorFlow. It determines the user's emotional state from facial features and voice tone and generates an emotion label as output. This is an indicator used in subsequent clothing selection.

[0812] Step 3:

[0813] The server uses personal identification information and emotional information to operate a digital image generator and generate a digital clothing catalog. The input consists of emotional labels and specific user information. A generation AI model is used to generate clothing images suitable for the user and rank them according to the user's preferences. The output is a catalog of digital clothing images.

[0814] Step 4:

[0815] The terminal displays a digital clothing catalog generated on the user's smart glasses or smartphone. Input from the server is image data, which is rendered in real time on the terminal. The user browses the displayed catalog and selects clothing items of interest through the interface. The output is the user's clothing selection information.

[0816] Step 5:

[0817] The server retrieves and analyzes inventory and geographical information of the user's selected clothing via a communication network. Inputs include user selection information and provided store information. Using inventory information and the user's geographical location, the server identifies the optimal store for trying on clothes and the best time to visit. The output is a suggestion of the optimal fitting location and time.

[0818] Step 6:

[0819] The user uses the terminal to check the suggested fitting rooms and appointment times, and makes a fitting reservation if necessary. The input from the server is fitting information, and the terminal, after obtaining user approval, confirms the fitting reservation with the store using the reservation control mechanism. The output is a notification that the fitting reservation has been completed.

[0820] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0821] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0822] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0823] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0824] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0825] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0826] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0827] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0828] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0829] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0830] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0831] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0832] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0833] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0834] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0835] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0836] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0837] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0838] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0839] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0840] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0841] The following is further disclosed regarding the embodiments described above.

[0842] (Claim 1)

[0843] A means of receiving personally identifiable information from an input device,

[0844] A means for operating a digital image generation device based on the aforementioned personal identification information to generate duplicate images of multiple garments being worn,

[0845] Means for transmitting the generated duplicate image to a display device and presenting it,

[0846] A means of securing inventory information of clothing selected by the user via a communication network,

[0847] Data analysis methods for identifying the optimal physical store,

[0848] A reservation control means for making a fitting reservation at the aforementioned physical store,

[0849] A system that includes this.

[0850] (Claim 2)

[0851] The digital image generation device generates duplicate images using a machine learning algorithm, according to claim 1.

[0852] (Claim 3)

[0853] The system according to claim 1, which takes geographical data and inventory information into consideration in order to identify the optimal physical store.

[0854] "Example 1"

[0855] (Claim 1)

[0856] A means for receiving personal identification information from a data input device,

[0857] A means for generating try-on images of a user wearing multiple garments, using a generation AI model based on the aforementioned personal identification information,

[0858] A means for transmitting the generated fitting image to a display device and displaying it,

[0859] A means of checking and maintaining inventory information of clothing selected by the user via an information and communication network,

[0860] Information analysis tools for identifying the optimal store based on geographical data and inventory status,

[0861] A reservation management means that automatically makes fitting reservations at the aforementioned specified stores,

[0862] A system that includes this.

[0863] (Claim 2)

[0864] The system according to claim 1, which generates try-on images using a generation AI model.

[0865] (Claim 3)

[0866] The system according to claim 1, which identifies the optimal store by combining geographical data and inventory information.

[0867] "Application Example 1"

[0868] (Claim 1)

[0869] A means of receiving personally identifiable information from an input device,

[0870] A means for operating a digital image generation device based on the aforementioned personal identification information to generate duplicate images of multiple garments being worn,

[0871] Means for transmitting the generated duplicate image to a display device and presenting it,

[0872] A means of securing inventory information of clothing selected by the user via a communication network,

[0873] Data analysis methods for identifying the optimal physical store,

[0874] A reservation control means for making a fitting reservation at the aforementioned physical store,

[0875] A display means that provides duplicate images generated through virtual try-on in a catalog format,

[0876] A communication method that allows users to complete fitting reservations before visiting the most suitable physical store,

[0877] A system that includes this.

[0878] (Claim 2)

[0879] The system according to claim 1, wherein the digital image generation device generates duplicate images using a machine learning algorithm and creates a catalog based on virtual try-on.

[0880] (Claim 3)

[0881] The system according to claim 1, which takes geographical data and inventory information into consideration to identify the optimal physical store and suggests an appropriate fitting appointment before the visit.

[0882] "Example 2 of combining an emotion engine"

[0883] (Claim 1)

[0884] Means for receiving personally identifiable information and emotional data from an input device,

[0885] A means for operating a digital image generation device based on the aforementioned personal identification information and emotional data to generate a replica image of the wearer,

[0886] A means for rearranging the generated duplicate images, taking emotional information into consideration, and transmitting them to a display device for presentation,

[0887] A means of securing inventory information of clothing selected by the user via a communication network,

[0888] A data analysis method for identifying the optimal physical store, taking into account emotional data, geographical data, and inventory information,

[0889] A reservation control means for making a fitting reservation at the aforementioned physical store and notifying the details thereof,

[0890] A system that includes this.

[0891] (Claim 2)

[0892] The digital image generation device generates duplicate images using a machine learning algorithm and determines the display priority based on emotion data, according to claim 1.

[0893] (Claim 3)

[0894] The system according to claim 1, which considers emotional data, geographical data, and inventory information to identify the optimal physical store and suggests the best time and location for the user.

[0895] "Application example 2 when combining with an emotional engine"

[0896] (Claim 1)

[0897] Means for receiving personally identifiable information and emotional information from an input device,

[0898] A means for operating a digital image generation device based on the aforementioned personal identification information and emotional information to generate duplicate images of multiple garments being worn,

[0899] Means for transmitting the generated duplicate image to a display device and presenting it,

[0900] A means of securing inventory information of clothing selected by the user via a communication network,

[0901] A data analysis method for identifying the optimal physical store and the optimal time based on user sentiment,

[0902] A reservation control means for making a fitting reservation at the aforementioned physical store,

[0903] A means by which users acquire information in real time using visual devices,

[0904] A system that includes this.

[0905] (Claim 2)

[0906] The system according to claim 1, wherein the digital image generation device generates a duplicate image using a machine learning algorithm and displays the duplicate image taking into account the results of sentiment analysis.

[0907] (Claim 3)

[0908] The system according to claim 1, which takes geographical data, inventory information and sentiment data into consideration in order to identify the optimal physical store. [Explanation of Symbols]

[0909] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving personally identifiable information from an input device, A means for operating a digital image generation device based on the aforementioned personal identification information to generate duplicate images of multiple garments being worn, Means for transmitting the generated duplicate image to a display device and presenting it, A means of securing inventory information of clothing selected by the user via a communication network, Data analysis methods for identifying the optimal physical store, A reservation control means for making a fitting reservation at the aforementioned physical store, A display means that provides duplicate images generated through virtual try-on in a catalog format, A communication method that allows users to complete fitting reservations before visiting the most suitable physical store, A system that includes this.

2. The system according to claim 1, wherein the digital image generation device generates duplicate images using a machine learning algorithm and creates a catalog based on virtual try-on.

3. The system according to claim 1, which takes geographical data and inventory information into consideration to identify the optimal physical store and suggests an appropriate fitting appointment before the visit.

Images