system

Abstract

We provide a system that links sales behavior in physical stores with purchasing behavior in online stores. [Solution] A system for linking sales behavior in a physical store with purchasing behavior in a web store, the system includes: an input means for inputting try-on information about a user who tries on a product in a physical store; a QR code generation means for generating a QR code based on the try-on information; a provision means for providing the QR code to the user; a reading means for reading the QR code when the user purchases the product in the web store; a linking means for acquiring try-on information from the QR code and linking the acquired try-on information with purchase information; an evaluation update means for updating the evaluation of staff linked to the try-on information; and a point awarding means for awarding points based on the evaluation.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the spread of online stores has led to a decline in the sales power of physical stores, forcing an increasing number of companies to withdraw from the market. This situation is due to the fact that, while customers enjoy the benefits of actually trying on, experiencing, and consulting about products, it has become common for them to make their final purchases through online stores. As a result, the efforts of store staff are not appreciated, and the operation of physical stores becomes unsustainable. Therefore, there is a need to provide a system that links physical stores and online stores and properly evaluates the try-on and consultations made in physical stores. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention provides a system for linking sales behavior in a physical store with purchasing behavior in a web store. The system includes an input unit for inputting try-on information about a user who tries on a product in the physical store. It also includes a QR code generation unit for generating a QR code (registered trademark) based on the try-on information and a provision unit for providing the QR code to the user. The system also includes a reading unit for reading the QR code when the user purchases the product in the web store, a linking unit for acquiring try-on information from the QR code, and a linking unit for linking the acquired try-on information with purchase information. The system also includes an evaluation update unit for updating the evaluation of staff linked to the try-on information, and a point awarding unit for awarding points based on the evaluation. This allows try-on and consultation behavior in the physical store to be appropriately reflected in purchasing behavior in the web store, thereby realizing a system in which staff efforts are recognized.

[0006] A "physical store" is a physical sales location where customers can visit in person, touch products, try them on, experience them, and receive consultations.

[0007] "WEB Store" is an online sales platform where you can browse and purchase products via the Internet.

[0008] "User" refers to a consumer who tries on or purchases products at a physical store or online store.

[0009] "Try-on information" is data related to the user's attempt to try on items at a physical store, and includes information such as the product number, try-on date and time, and staff ID.

[0010] "Input means" refers to devices and software used to register try-on information into the system at a physical store.

[0011] A "QR code" is a scannable two-dimensional barcode that digitally encodes try-on information.

[0012] The "QR code generating means" is a device or software that automatically generates a QR code based on the input fitting information.

[0013] "Means of provision" refers to the method of displaying or distributing the generated QR code to the user.

[0014] "Reading means" refers to a device or software that reads the information in the QR code on the web store.

[0015] The "linking means" is a device or software that links the try-on information obtained from the QR code with the purchase information from the web store.

[0016] The "evaluation update means" is a device or software that reflects new evaluations by store staff based on the link between try-on information and purchase information.

[0017] The "point awarding means" is a device or software that awards points to users or staff members according to their ratings and purchasing status. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store. By integrating product try-ons in a physical store with purchases in a web store, this system evaluates the efforts of store staff and creates an environment where users can easily try on products and receive consultations in a physical store.

[0040] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The staff member uses a terminal (tablet or smartphone) to input the try-on information. This try-on information includes the product number, try-on date and time, and the staff member ID in charge. This allows the device to be used as an input means.

[0041] Next, the terminal generates a QR code based on the try-on information. The generated QR code is displayed to the user by the providing means or printed on paper and given to the user. The user can take this QR code home and use it to access the web store at home or elsewhere.

[0042] A user accesses a web store, searches for and selects a product, adds it to their cart, and then enters the QR code. By scanning the QR code into a dedicated input field, they can proceed to the web store's purchasing process. This operation is performed by the scanning means.

[0043] When the QR code is scanned, the server analyzes the information and obtains the try-on information. The analyzed information is automatically linked to the purchase information. Through this linking method, the server links the try-on information from the physical store with the purchase information from the online store.

[0044] The server then updates the store staff's evaluation linked to the fitting information. Using the evaluation update means, the staff's evaluation points are updated on the system. This allows the staff's efforts to be properly evaluated. Points are also awarded based on the evaluation. These points are also awarded to the user and are returned as purchase points. Staff are awarded points for assisting with fitting.

[0045] Furthermore, the server uses AI to analyze the user's purchase history and try-on history, and recommends appropriate products for the next visit. This allows for more detailed and appropriate suggestions to be made to the user. In-store staff are also provided with product suggestion information based on user attributes, enabling more effective customer service.

[0046] As a concrete example, a user tries on a red jacket in a physical store and receives a QR code with that information registered. Later, when the user purchases the red jacket from a web store, they enter the QR code, and the server links the try-on information with the purchase information, updates the staff's evaluation, and awards points to each. This links the physical store and the web store, properly evaluates the efforts of the store staff, and allows the user to enjoy trying on items in the physical store without feeling guilty.

[0047] This invention is expected to improve the purchasing experience and make the management of physical stores more sustainable by combining the advantages of existing physical stores and web stores.

[0048] The processing flow will be explained below.

[0049] Step 1:

[0050] A user visits a store, browses the shelves, and finds an item they want to try on.

[0051] Step 2:

[0052] The user asks the store staff for the product they want to try on. The user picks up the product they want to try on and tells the staff, "I'd like to try this product on."

[0053] Step 3:

[0054] Product information (product number, model number, etc.) is entered into a terminal (a tablet or smartphone held by store staff), and the staff then enters the product information into a dedicated app on the tablet.

[0055] Step 4:

[0056] The terminal registers fitting information (fitting date and time, user ID (optional), staff ID in charge, etc.). The staff member enters the fitting date and time and their own staff ID, and also the user ID if necessary.

[0057] Step 5:

[0058] The device will generate a QR code. The QR code will be automatically generated based on the above information.

[0059] Step 6:

[0060] The terminal shows the QR code to the user or prints it out on paper and hands it over to the user. The staff member shows the QR code on the tablet screen to the user or prints it out on paper and hands it over to the user.

[0061] Step 7:

[0062] A user accesses the web store from their home or other location. A user accesses the web store using a computer or smartphone.

[0063] Step 8:

[0064] A user searches for a product on a web store, tries on the product, and adds it to their cart. A user searches for a red jacket, checks the product detail page, and adds it to their cart.

[0065] Step 9:

[0066] The user scans the QR code into the input field. During the purchase process, the user scans the QR code received at the store into the dedicated QR code input field, or enters the displayed information.

[0067] Step 10:

[0068] The server analyzes the QR code information and obtains the fitting information, such as the product ID, fitting date and time, and staff ID.

[0069] Step 11:

[0070] The server links purchase information from the online store with try-on information from the physical store, matches the purchase data with the try-on data, and records which staff member assisted in trying on which product.

[0071] Step 12:

[0072] The server updates the store staff's evaluation based on the purchase data, and adds the evaluation points of the staff who assisted with trying on the items to the staff evaluation in the system.

[0073] Step 13:

[0074] The server grants the user points for the purchase. Points corresponding to the purchase amount are added to the user's account.

[0075] Step 14:

[0076] The server also gives points to the store staff. The staff also receive points on the system for purchases associated with the fitting assistance.

[0077] Step 15:

[0078] The server uses AI to analyze the user's purchase history and try-on history. The AI ​​analyzes the user's past purchase history, try-on history, and browsing history.

[0079] Step 16:

[0080] The server recommends the most suitable products to the user. Highly relevant products are displayed on the web store based on the user's interests and purchasing trends.

[0081] Step 17:

[0082] The server provides store staff with product suggestion information based on user attributes. Information useful for the next suggestion (recommended products, similar items, etc.) is provided to the store staff's terminal.

[0083] Example 1

[0084] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0085] In recent years, there has been a demand for systems that can provide convenience to consumers and fairly evaluate the efforts of sales representatives at physical stores by integrating product try-on sessions in physical stores with purchases via e-commerce. However, existing systems have not established a method for effectively linking try-on information at physical stores with purchase information via e-commerce. Furthermore, the lack of linkage between try-on information and purchase information has led to problems such as the inability to evaluate sales representatives or make appropriate product recommendations to consumers.

[0086] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0087] In this invention, the server includes a linking means for linking acquired try-on information with purchase information, an evaluation updating means for updating the salesperson's evaluation linked to the try-on information, a recommendation means for analyzing habits and recommending appropriate products to the consumer, and a provision means for providing the salesperson with product suggestion information based on the consumer's attributes. This effectively links the try-on information from the physical store with the purchase information from e-commerce, enabling the salesperson to make fair evaluations and suggest appropriate products to the consumer.

[0088] A "brick and mortar store" is a store that exists in a physical location and where consumers can visit in person to try on and purchase products.

[0089] "Electronic commerce" is the act of buying and selling goods over the Internet.

[0090] "Consumer" refers to an individual who tries on or purchases a product in a physical store or through e-commerce.

[0091] "Try-on information" is data generated when a consumer tries on a product in a physical store, and includes information such as the product number, try-on date and time, and the person in charge's ID.

[0092] "Input means" refers to a device or method for a consumer to input try-on information into a terminal when trying on a product in a physical store.

[0093] "QR code generation means" refers to a device or method for generating a QR code based on input try-on information.

[0094] "Providing means" refers to the method or device for delivering the generated QR code to the consumer.

[0095] "Reading means" refers to a method or device that allows a consumer to read a QR code using a digital input or scanning device when purchasing a product through e-commerce.

[0096] The "linking means" refers to a method or device for linking the try-on information obtained from the QR code with the purchase information in the e-commerce transaction.

[0097] The "rating update means" refers to a method or device for updating the sales representative's rating based on the linked try-on information and purchase information.

[0098] A "scoring means" refers to a method or device for awarding points or scores to consumers or salespeople based on the evaluation.

[0099] "Habits" refer to a consumer's behavioral characteristics determined based on their past purchase history and try-on history.

[0100] A "recommendation means" is a method or device for recommending appropriate products for the next visit based on the analyzed consumer habits.

[0101] A "sales representative" is a staff member who interacts with consumers in a physical store, allowing them to try on products and providing advice.

[0102] "Product proposal information" refers to information provided to sales representatives based on consumer attributes and past behavioral history, which is used to make effective product proposals.

[0103] A "database" is a system that structures, stores, and manages vast amounts of data, such as try-on information, purchase information, and evaluation information.

[0104] The present invention is a system that effectively links product try-ons in physical stores with purchases made through e-commerce, thereby increasing consumer convenience and providing fair recognition for the efforts of sales representatives. Specific embodiments for implementing the system of the present invention are described below.

[0105] First, the user visits a physical store and asks a sales representative for the item they would like to try on. The sales representative uses a device (tablet or smartphone) to enter the try-on information. The software on this device has a function that allows users to enter try-on information such as the product number, try-on date and time, and the staff member ID.

[0106] Next, the device generates a QR code based on the entered fitting information. This QR code is provided to the user and can be displayed on the device screen or printed on paper using a dedicated printer. The user can take this QR code home and use it at a later date.

[0107] When a user accesses an e-commerce site from home or elsewhere, searches for and selects a product, and adds it to their cart, they enter the QR code they received when trying on the item. This QR code is read into a dedicated input field on the e-commerce site. Once the QR code is entered, the server analyzes the information and obtains the try-on information. This analysis automatically links the try-on information and purchase information within the server.

[0108] In the next step of the workflow, the server uses this linked information to update the sales representative's evaluation at the physical store, ensuring that the sales representative's efforts are properly recognized and scored. The system also rewards the consumer with points for their purchase, which can be used toward their next purchase, helping to build consumer loyalty.

[0109] Furthermore, the server uses AI to analyze the consumer's purchase history and try-on history, and recommends appropriate products for the consumer's next visit. This allows the consumer to receive more detailed and appropriate suggestions. Sales representatives are also provided with product suggestion information based on the consumer's attributes, enabling more effective customer service.

[0110] As a concrete example, a user tries on a red jacket in a physical store and receives a QR code with that information registered. Later, when the user purchases the red jacket on an e-commerce site, they enter that QR code. The server links the try-on information with the purchase information, updates the sales representative's evaluation, and awards points to each. This links the physical store and the e-commerce site, properly evaluates the sales representative's efforts, and allows the user to enjoy trying on items in the physical store with ease.

[0111] Example prompts to input to a generative AI model:

[0112] "A user tries on a red jacket in a physical store and receives a QR code with that information registered. When the user then purchases the red jacket on an e-commerce site, they enter the QR code, which links the try-on information with the purchase information. Please explain in detail the system process. Please clearly state the roles of the server, terminal, and user."

[0113] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0114] Step 1:

[0115] A user visits a physical store and asks a salesperson for the product they would like to try on. The salesperson uses a device (tablet or smartphone) to enter the try-on information. This information includes the product number (e.g., "RJ1234"), try-on date and time (e.g., "2023-10-01 14:00"), and the staff member ID (e.g., "ST5678"). The entered try-on information is saved in the device's database and becomes the basis for generating a QR code in the next step.

[0116] Step 2:

[0117] The device generates a QR code based on the entered try-on information. The device's QR code generation function encodes this try-on information and outputs a QR code with the content "RJ1234_20231001_1400_ST5678". The device then provides this generated QR code to the user by displaying it on the screen or printing it out on a dedicated printer and handing it to the user. The user then takes this QR code home. Try-on information is received as input, and a QR code is generated and provided as output.

[0118] Step 3:

[0119] A user accesses an e-commerce site from home or elsewhere, searches for and selects a product. After adding the product to their cart, they proceed to the checkout screen. Here, the user enters the QR code they received when trying on the item by scanning or manually entering it into a dedicated input field. This QR code, used as input data, is used for analysis in the next step.

[0120] Step 4:

[0121] The server receives and analyzes the QR code entered on the e-commerce site. The server decodes the QR code "RJ1234_20231001_1400_ST5678" and obtains the fitting information (product number "RJ1234", fitting date and time "2023-10-01 14:00", staff ID "ST5678"). This fitting information is used in the next step of collaboration. The QR code is received as input, and fitting information is extracted as output.

[0122] Step 5:

[0123] The server links the try-on information with the purchase information. The purchase information includes the product number "RJ1234" and the purchase date and time "2023-10-02 18:00." This information is recorded in the server's database, and the try-on information and purchase information are linked. The server receives the try-on information and purchase information acquired as input, and saves the linked data in the database as output.

[0124] Step 6:

[0125] The server updates the sales representative's evaluation based on the linked information. The system assigns additional evaluation points to the staff member ID "ST5678" associated with the fitting information. For example, 10 points are added to the evaluation. Five points are also assigned to the user. The system receives linked data as input and updates the evaluation points as output.

[0126] Step 7:

[0127] The server uses AI to analyze the user's purchase history and try-on history. Based on past data, it recommends products that the user is likely to be interested in the next time they visit. This recommendation information is not only suggested to the user, but also provided to sales representatives. For example, if a user purchases a red jacket, matching black pants and a white shirt will be recommended the next time they visit. The server receives purchase history and try-on history as input and generates recommended product information as output.

[0128] By following each step of this system in an orderly manner, the integration of physical stores and e-commerce will be realized, consumer convenience will be improved, and the efforts of sales representatives will be properly recognized.

[0129] (Application example 1)

[0130] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0131] The current system for linking physical stores and online stores has the problem that when a customer tries on an item in a physical store and then purchases it on the online store, the entire process is interrupted. This raises concerns that the motivation of store staff will decrease because customer service in the physical store is not properly evaluated. Another problem is that customers cannot receive real-time recommendations for different products while trying on items in the physical store.

[0132] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0133] In this invention, the server includes: an input unit for inputting fitting information about a user trying on a product in a physical store; a QR code generation unit for generating a QR code based on the fitting information; a provision unit for providing the QR code to the user; a reading unit for reading the QR code when the user purchases the product in a web store; a linking unit for acquiring fitting information from the QR code and linking the acquired fitting information with purchase information; an evaluation update unit for updating staff ratings linked to the fitting information; a point awarding unit for awarding points based on the ratings; a unit for checking product suggestions and fitting status in real time using smart glasses; and a recommendation unit using a generative AI model that analyzes the user's past fitting history and purchase history to recommend products suitable for the user's next visit. This seamlessly links fitting behavior in a physical store with purchasing behavior in the web store, enabling fair recognition of the efforts of physical store staff. Furthermore, real-time product suggestions using smart glasses are expected to improve the quality of customer service and increase sales.

[0134] A "brick and mortar store" is a point of sale that exists in a physical location and allows customers to visit in person to purchase or try on products.

[0135] A "web store" is an online sales outlet operated on the Internet where customers can purchase products via the Internet.

[0136] "Try-on information" is detailed information about the product that the user tried on in the physical store, and includes data such as the product number, the try-on date and time, and the staff ID in charge.

[0137] "Input means" refers to a device or interface used to collect try-on information in a physical store and input it into the system. Examples of such devices include tablets and smartphones.

[0138] The "QR code generating means" refers to a device or method for generating a QR code based on the fitting information.

[0139] "Providing means" refers to a method or device for providing the generated QR code to the user. This includes displaying the QR code on a screen or printing it on paper.

[0140] The "reading means" refers to a method or device that reads the QR code when a user purchases the product from the web store. Specific examples include a smartphone camera or a dedicated input field.

[0141] The "linking means" refers to a method or device for acquiring try-on information from a QR code and associating the acquired try-on information with purchase information.

[0142] The "rating update means" refers to a method or device for updating the staff rating linked to the fitting information.

[0143] The "point awarding means" is a method or device for awarding points based on evaluation.

[0144] "Smart glasses" are wearable devices that users can wear to display and input information in real time.

[0145] A "generative AI model" is an artificial intelligence model that learns from large amounts of data and provides information appropriate to the user.

[0146] A "recommendation means" is a method or device that uses a generative AI model to analyze a user's past try-on history and purchase history and suggest appropriate products.

[0147] This invention relates to a system that links product try-on in a physical store with purchasing behavior in a web store. This system provides comprehensive support for users to try on products in a physical store and then smoothly purchase them in a web store.

[0148] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The store staff member uses a tablet or smartphone to input the user's fitting information. This fitting information includes the product number, the date and time of fitting, and the staff member ID in charge. This input method allows the user's fitting behavior in the physical store to be managed as digital data.

[0149] Next, a QR code is generated based on the input try-on information. The QR code generation means provides the generated QR code to the user. The QR code is displayed on the display of the smart glasses or printed on paper, which the user can take home. The QR code contains the try-on information and is used to access the web store later.

[0150] When a user accesses the web store from home or elsewhere and purchases an item they have tried on, they input the QR code. The QR code is read by a reader and sent to a server. The server obtains the try-on information from the QR code and links it to the purchase information. This linking means allows the system to consistently manage which items the user tried on and which items they purchased.

[0151] Furthermore, the server updates the staff evaluation linked to the fitting information. Using the evaluation update means, the staff is awarded points for assisting with fitting. This point system fairly evaluates the efforts of the staff and provides incentives. In addition, purchase points are awarded to the user, and discounts and benefits that can be used on the next purchase are offered.

[0152] In addition, the system uses smart glasses to allow users to check product suggestions and try-on status in real time. This allows store staff to make appropriate product suggestions based on the user's interests. Furthermore, using a generative AI model, the server analyzes the user's past try-on and purchase history and recommends appropriate products for the next visit. This allows users to enjoy a more personalized shopping experience.

[0153] As a concrete example, a user may try on a red jacket in a physical store and receive a QR code based on the fitting information. The user then takes a photo of this QR code with their smartphone and enters it later when purchasing the red jacket in a web store. The server retrieves the fitting information from the QR code, links it to the purchase information, updates the staff evaluation, and awards points. This series of processes seamlessly links the physical store and the web store, improving the purchasing experience.

[0154] Examples of prompts include:

[0155] "We are imagining a scenario where a user tries on a product using smart glasses in a store. The try-on information includes the user ID, staff ID, and product ID, which are generated as a QR code and provided to the user. When the user then enters the QR code when making a purchase on the web store, the try-on information and purchase information are automatically linked in the system, and the staff member's evaluation points are updated. Please explain this process in detail."

[0156] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0157] Step 1:

[0158] A user visits a physical store and asks a store staff member to try on an item. The store staff member uses a tablet or smartphone to enter the user's fitting information. The input information includes data such as the product number, fitting date and time, and the staff member ID in charge. The fitting information is then recorded on the terminal and sent to a database.

[0159] Input: Product information the user wants to try on, try-on date and time, staff ID

[0160] Output: Digital data including fitting information

[0161] Specific operation: The staff member enters the necessary information into the input form on the terminal and sends it to the database.

[0162] Step 2:

[0163] A QR code is generated based on the entered fitting information. The tablet or smartphone application generates a QR code based on the fitting information and displays it on the device. The generated QR code is then displayed to the user using smart glasses or printed on paper.

[0164] Input: Try-on information

[0165] Output: Generated QR code

[0166] Specific operation: The QR code generation application installed on the device acquires the fitting information and generates a QR code, which is then displayed or printed to the user.

[0167] Step 3:

[0168] The user accesses the web store from home or elsewhere, searches for and selects the product they have tried on, and adds it to their cart. When they proceed to the checkout, the QR code is read into the input field. The user can either scan the QR code with their smartphone camera or manually enter the contents of the QR code.

[0169] Input: QR code

[0170] Output: Try-on information obtained from QR code

[0171] Specific operation: The user scans the QR code with their smartphone and enters the information into the dedicated input field on the web store. The system analyzes the QR code and obtains the fitting information.

[0172] Step 4:

[0173] The server analyzes the try-on information obtained from the QR code and automatically links it to purchase information, thereby recording in a database which products the user tried on and which products they purchased.

[0174] Input: Try-on information and purchase information obtained from the QR code

[0175] Output: Linked try-on information and purchase information

[0176] Specific operation: The server analyzes the QR code data and associates the try-on information with the purchase information in the database.

[0177] Step 5:

[0178] The server updates the evaluation of the store staff linked to the fitting information. The evaluation points of the staff are updated on the system using the evaluation update means. This allows the staff's efforts to be properly evaluated.

[0179] Input: Linked try-on information and purchase information

[0180] Output: Updated staff rating points

[0181] Specific operation: The server updates the reputation points corresponding to the staff ID and saves them in the database.

[0182] Step 6:

[0183] Points are awarded to users and staff using a point awarding means. Users are awarded purchase points that can be used for their next purchase, and staff are awarded fitting assistance points.

[0184] Input: Updated staff rating, purchase completion information

[0185] Output: Points awarded

[0186] Specific operation: The server grants the corresponding points to the user's and staff's accounts.

[0187] Step 7:

[0188] The server uses the generated AI model to analyze the user's purchase history and try-on history, and recommends appropriate products for the user's next visit. The recommended information is displayed to the user through the smart glasses.

[0189] Input: User's purchase history, try-on history

[0190] Output: Product information to recommend to the user

[0191] How it works: The generative AI model analyzes the user's past data, calculates the optimal product, and displays it on the smart glasses.

[0192] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0193] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store, and in particular to a system that combines an emotion engine that recognizes user emotions. This system integrates product try-on in a physical store with purchases in a web store, and by recognizing user emotions, it can evaluate the efforts of store staff and create an environment where users can easily try on products and receive consultations in a physical store.

[0194] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The staff member uses a terminal (tablet or smartphone) to input the try-on information. This try-on information includes the product number, try-on date and time, and the staff member ID in charge. This allows the device to be used as an input means.

[0195] Next, the terminal generates a QR code based on the try-on information. The generated QR code is displayed to the user by the providing means or printed on paper and given to the user. The user can take this QR code home and use it to access the web store at home or elsewhere.

[0196] A user accesses a web store, searches for and selects a product, adds it to their cart, and then enters the QR code. By scanning the QR code into a dedicated input field, they can proceed to the web store's purchasing process. This operation is performed by the scanning means.

[0197] When the QR code is scanned, the server analyzes the information and obtains the try-on information. The analyzed information is automatically linked to the purchase information. Through this linking method, the server links the try-on information from the physical store with the purchase information from the online store.

[0198] The server then updates the store staff's evaluation linked to the fitting information. Using the evaluation update means, the staff's evaluation points are updated on the system. This allows the staff's efforts to be properly evaluated. Points are also awarded based on the evaluation. These points are also awarded to the user and are returned as purchase points. Staff are awarded points for assisting with fitting.

[0199] Furthermore, this system uses an emotion engine. The emotion engine analyzes the user's facial expressions and voice when trying on clothes or making a purchase, and acquires emotional information. This emotional information is recorded in association with the try-on information and purchase information. The server uses this emotional information to further analyze the user's purchasing trends and recommend appropriate products.

[0200] For example, if a user tries on a red jacket in a physical store and smiles or shows satisfaction, the emotion engine recognizes this and adds that emotion information to the try-on information. Later, when the user enters the QR code to purchase the red jacket in a web store, the positive emotion expressed during the try-on is reflected and linked to the purchase information. The server can then analyze this emotion information and recommend products that the user is likely to like the next time they visit.

[0201] The server also reflects emotional information in the evaluation of the staff's customer service. For example, if a user expresses positive emotions toward the staff's customer service while trying on clothes, that evaluation will be added to the staff's evaluation points. This allows the quality of the staff's customer service to be evaluated appropriately.

[0202] In this way, the system of the present invention strengthens the connection between physical stores and online stores, and by utilizing an emotion engine, it is possible to improve the user's purchasing experience and properly evaluate the efforts of store staff.

[0203] The processing flow will be explained below.

[0204] Step 1:

[0205] A user visits a store, browses the shelves, and finds an item they want to try on.

[0206] Step 2:

[0207] The user asks the store staff for the product they want to try on. The user picks up the product they want to try on and tells the staff, "I'd like to try this product on."

[0208] Step 3:

[0209] Product information (product number, model number, etc.) is entered into a terminal (a tablet or smartphone held by store staff), and the staff then enters the product information into a dedicated app on the tablet.

[0210] Step 4:

[0211] The terminal registers fitting information (fitting date and time, user ID (optional), staff ID in charge, etc.). The staff member enters the fitting date and time and their own staff ID, and also the user ID if necessary.

[0212] Step 5:

[0213] The emotion engine recognizes the user's emotions by analyzing emotional information (e.g., satisfaction, dissatisfaction, excitement, etc.) from the user's facial expressions and voice.

[0214] Step 6:

[0215] The device generates a QR code, which is automatically generated based on the fitting information and emotional information.

[0216] Step 7:

[0217] The terminal shows the QR code to the user or prints it out on paper and hands it over to the user. The staff member shows the QR code on the tablet screen to the user or prints it out on paper and hands it over to the user.

[0218] Step 8:

[0219] A user accesses the web store from their home or other location. A user accesses the web store using a computer or smartphone.

[0220] Step 9:

[0221] A user searches for a product on a web store, tries on the product, and adds it to their cart. A user searches for a red jacket, checks the product detail page, and adds it to their cart.

[0222] Step 10:

[0223] The user scans the QR code into the input field. During the purchase process, the user scans the QR code received at the store into the dedicated QR code input field, or enters the displayed information.

[0224] Step 11:

[0225] The server analyzes the QR code information and obtains the fitting information and emotion information. Information such as the product ID, fitting date and time, staff ID, and emotion information is analyzed from the QR code.

[0226] Step 12:

[0227] The server links purchase information from the web store with try-on information and emotional information from the physical store. It matches the purchase data with try-on data and emotional data, and records which staff member assisted in trying on which products.

[0228] Step 13:

[0229] The server updates the evaluation of the store staff based on the purchase data. The evaluation points of the staff who assisted with fitting are updated in the system. Evaluation points based on emotional information are also added.

[0230] Step 14:

[0231] The server grants the user points for the purchase. Points corresponding to the purchase amount are added to the user's account.

[0232] Step 15:

[0233] The server also awards points to store staff. Points are also awarded to staff in the system based on emotional information for purchases associated with fitting assistance.

[0234] Step 16:

[0235] The server uses AI to analyze the user's purchase history, try-on history, emotional information, and browsing history.

[0236] Step 17:

[0237] The server recommends the most suitable products to the user. Highly relevant products are displayed on the web store based on the user's interests, purchasing tendencies, and emotional information.

[0238] Step 18:

[0239] The server provides store staff with product suggestion information based on user attributes. Information useful for the next suggestion (recommended products, similar items, etc.) is provided to the store staff's terminal.

[0240] This will strengthen the connection between physical stores and online stores, improve the user's purchasing experience by utilizing the emotion engine, and create a system that properly evaluates the efforts of store staff.

[0241] Example 2

[0242] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0243] In today's commercial environment, there is a lack of coordination between physical and virtual stores, which makes it difficult for users to purchase products they have tried in a physical store in a virtual store. Furthermore, the lack of proper reflection of users' emotions and evaluations of staff service makes it difficult to improve the quality of service.

[0244] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0245] In this invention, the server includes: an input means for inputting trial information about a user who tries out a product at a real store; a two-dimensional code generation means for generating a two-dimensional code based on the trial information; a provision means for providing the two-dimensional code to the user; a reading means for reading the two-dimensional code when the user purchases the product at a virtual store; a linking means for acquiring trial information from the two-dimensional code and linking the acquired trial information with purchase information; an evaluation update means for updating an evaluation of a staff member linked to the trial information; a score assignment means for assigning a score based on the evaluation; an emotion analysis means for analyzing the user's emotions; a means for associating emotion information acquired from the emotion analysis means with the trial information and purchase information; and a recommendation means for recommending products based on the emotion information and trial information. This strengthens the connection between the real store and the virtual store, improves the user's purchasing experience, and enables the customer service efforts of staff to be appropriately evaluated.

[0246] "Brick and mortar store" means a commercial establishment that customers can physically visit.

[0247] "Virtual store" refers to an online store that sells products over the Internet.

[0248] "User" refers to an individual who uses a physical or virtual store to try out or purchase a product.

[0249] "Trial information" is information about the user when they try out a product, and includes the product number, trial date and time, and person in charge ID.

[0250] "Input means" refers to a device or system for inputting trial information at a physical store.

[0251] "Two-dimensional code" refers to a code that contains visual information, such as a QR code.

[0252] The "two-dimensional code generating means" refers to a system or software for generating a two-dimensional code based on input trial information.

[0253] "Providing means" refers to a method for providing the generated two-dimensional code to the user.

[0254] The "reading means" refers to the method used by the user to read the two-dimensional code when making a purchase at the virtual store.

[0255] The "linking means" refers to a method for linking the trial information obtained from the two-dimensional code with the purchase information at the virtual store.

[0256] The "evaluation update means" refers to a function for updating the evaluation of the person in charge on the system based on the trial information.

[0257] The "point giving means" refers to a method for giving points to users and personnel based on the evaluation.

[0258] "Emotion analysis means" refers to a device or software for analyzing a user's facial expressions and voice and acquiring emotional information.

[0259] The "recommendation means" refers to a method for recommending appropriate products to users based on trial information and emotion information.

[0260] MODE FOR CARRYING OUT THE INVENTION

[0261] This invention is a system for improving the quality of service by linking the sales activities of real stores and virtual stores and analyzing user emotions. This system includes the following components.

[0262] Hardware and software used

[0263] Input method: A device such as a tablet, smartphone, or PC is used. A dedicated application is installed on these devices, providing an interface for entering trial information.

[0264] Two-dimensional code generation means: Application software on the terminal generates a two-dimensional code (such as a QR code) based on the trial information.

[0265] Means of presentation: The generated two-dimensional code is displayed on the terminal screen, or a printer is used to print it on paper.

[0266] Reading method: Using a QR code reader built into the virtual store's website or application.

[0267] Linking means: A database management system running on the server side links the trial information with the purchase information.

[0268] Evaluation update means, score assignment means: The evaluation management system that operates on the server side updates the evaluation of the person in charge and assigns a score.

[0269] Emotion analysis tools: Emotion analysis software is used to analyze the user's facial expressions and voice. This data is collected through cameras and microphones in physical stores and webcams and microphones in virtual stores.

[0270] Recommendation means: A recommendation engine running on the server recommends products based on trial information and emotion information.

[0271] Specific methods for data processing and calculation

[0272] Entering trial information: When a user picks up a product in a physical store and tries it out, a staff member uses a dedicated app on a tablet or smartphone to enter the trial information (product number, trial date and time, staff ID, etc.).

[0273] QR Code Generation: After the trial information is entered, the application software generates a 2D code based on that information, which is then displayed on the terminal screen or printed and handed to the user.

[0274] Purchase from the web store: The user accesses the virtual store from home, searches for and selects the product they have tried, and adds it to their cart. They then scan the aforementioned QR code into the dedicated input field to complete the purchase.

[0275] Data analysis and linking: The server analyzes the QR code and obtains trial information. This information is automatically linked to purchase information in the virtual store. This allows for consistent management of trial information in the physical store and purchase information in the virtual store.

[0276] Rating and score update: The server automatically updates the staff's rating based on the linked trial information and assigns appropriate scores. Purchase points are also awarded to the user.

[0277] Emotion analysis: Emotion analysis software analyzes the user's facial expressions and voice during trial and purchase to obtain emotional information, which is then associated with trial and purchase information.

[0278] Providing recommendations: The server analyzes the emotional and trial information to understand the user's purchasing tendencies, and based on this, it can recommend appropriate products for the next visit.

[0279] Specific examples

[0280] For example, if a user tries on a red jacket in a physical store and smiles or shows satisfaction, the emotion analysis software recognizes this and adds positive emotional information to the trial information. Later, when the user enters the QR code to purchase a red jacket in a virtual store, the emotional information from the trial is linked to the purchase information. This allows the software to recommend products the user is likely to like the next time they visit. The software also updates the customer service evaluation of staff based on their positive emotions, and assigns appropriate scores.

[0281] Prompt Sentence Examples

[0282] "A user tries on a red jacket in a physical store and smiles and expresses satisfaction. The emotion analysis software recognizes this and adds the emotion information to the trial information. The user then enters a QR code when purchasing the red jacket in a virtual store. Please explain how the system handles this."

[0283] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0284] Step 1:

[0285] Users try out products in-store.

[0286] Specific operation: A user visits a physical store and tries on a product (e.g., a red jacket). During the trial, a staff member checks the user's facial expression and voice and provides a guide for the trial.

[0287] Step 2:

[0288] Use the terminal to enter your trial information.

[0289] Specific operation: The staff member uses a dedicated application on a tablet or smartphone to enter trial information (product number, trial date and time, staff ID, etc.).

[0290] Input: Product number (e.g., 12345), trial date and time (e.g., 2023-10-01 14:00), staff ID (e.g., 567)

[0291] Output: The entered trial information is saved on the device.

[0292] Step 3:

[0293] The terminal generates a two-dimensional code based on the trial information.

[0294] Specific operation: The terminal application generates a two-dimensional code (QR code) based on the entered trial information.

[0295] Input: Trial information (product number, trial date and time, staff ID)

[0296] Output: The generated 2D code is displayed on the terminal.

[0297] Step 4:

[0298] The two-dimensional code is provided to the user through the providing means.

[0299] Specific operation: Show the 2D code displayed on the terminal screen to the user, or print it out on paper using a printer and hand it to the user.

[0300] Input: Generated 2D code

[0301] Output: 2D code provided by the user

[0302] Step 5:

[0303] A user accesses a virtual store and purchases a product using a two-dimensional code.

[0304] Specific operation: The user accesses a virtual store (e.g., a website) from home, searches for and selects the product they want to try (e.g., a red jacket), adds it to their cart, and then enters the QR code in a dedicated input field.

[0305] Input: The two-dimensional code that the user enters in the virtual store

[0306] Output: Proceed with the purchase in the virtual store.

[0307] Step 6:

[0308] The server analyzes the two-dimensional code and obtains the trial information.

[0309] Specific operation: The server analyzes the two-dimensional code sent from the virtual store and obtains the trial information (product number, trial date and time, staff ID).

[0310] Input: 2D code

[0311] Output: Parsed trial information

[0312] Step 7:

[0313] The server links the trial information with the purchase information.

[0314] Specific operation: The server links the acquired trial information with the purchase information at the virtual store in the database, thereby confirming that the trialed product was purchased.

[0315] Input: Trial information, purchase information

[0316] Output: Linked trial and purchase information

[0317] Step 8:

[0318] Update staff ratings using the rating update means.

[0319] Specific operation: The server updates the staff evaluation on the system based on the trial information. Evaluation points are automatically calculated and updated.

[0320] Input: Trial information (Staff ID)

[0321] Output: Updated staff rating points

[0322] Step 9:

[0323] A point giving means is used to give points to users and staff members.

[0324] Specific operation: The server assigns purchase points to the user and trial support points to the staff based on the trial information and evaluation.

[0325] Input: Rating points

[0326] Output: Purchase number for user, trial support number for staff

[0327] Step 10:

[0328] Emotion analysis means is used to obtain user emotion information.

[0329] How it works: Emotion analysis software analyzes the user's facial expressions and voice during trial and purchase to obtain emotional information.

[0330] Input: User's facial expressions and voice data

[0331] Output: Parsed emotion information

[0332] Step 11:

[0333] Associate sentiment information with trial and purchase information.

[0334] Specific operation: The server associates the emotion information with the trial information and purchase information and stores the information in a database.

[0335] Input: Emotion information, trial information, purchase information

[0336] Output: Associated emotion information and trial / purchase information

[0337] Step 12:

[0338] To recommend appropriate products to a user using a recommendation means.

[0339] Specific operation: The server analyzes the emotional information and trial information, understands the user's purchasing tendencies, and recommends appropriate products the next time the user visits the virtual store.

[0340] Input: Emotion information, trial information

[0341] Output: Products recommended to the user

[0342] (Application example 2)

[0343] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0344] Conventional systems linking physical stores and online stores only link try-on information from the physical store with purchase information from the online store, and do not take into account the user's emotions or purchasing trends. This makes it difficult to recommend the most suitable products to the user, and also lacks a mechanism for properly evaluating the customer service quality of staff. The purpose of this invention is to solve these problems.

[0345] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: an input means for inputting try-on information about a user trying on a product in a physical store; a QR code generation means for generating a QR code based on the try-on information; a provision means for providing the QR code to the user; a reading means for reading the QR code when the user purchases the product in the web store; a linking means for acquiring try-on information from the QR code and linking the acquired try-on information with purchase information; an evaluation update means for updating the evaluation of the staff member linked to the try-on information; a point awarding means for awarding points based on the evaluation; an emotion recognition means for recognizing the user's emotions when trying on products and when purchasing; and a recommendation means for analyzing the user's purchasing tendencies using the emotion information acquired by the emotion recognition means and recommending products. This enables optimal product recommendations that take into account the user's emotions and purchasing tendencies, and an appropriate evaluation of the staff member's customer service quality.

[0346] A "physical store" is a store that exists in a physical location and that consumers can visit in person to purchase goods or receive services.

[0347] "Webstore" means a website or platform that exists on the Internet and allows consumers to access and purchase products online.

[0348] "Try-on information" refers to information when a user tries on a product in a physical store, and includes data such as the product number, the try-on date and time, and the staff ID in charge.

[0349] A "QR code" is a type of two-dimensional barcode, a matrix code that can encode and store information.

[0350] "Emotion recognition means" refers to a technology or device that analyzes emotions from the user's facial expressions, voice, etc., and recognizes a specific emotional state.

[0351] "Purchase information" refers to information when a user purchases a product from a web store, and includes data such as the product name, purchase date and time, and purchase quantity.

[0352] The "point awarding means" is a method or system for awarding points to users or staff members based on evaluations.

[0353] The "linking means" is a method or system that links the try-on information obtained from the QR code with the purchase information at the web store.

[0354] A "recommendation means" is a technology or device that recommends appropriate products to a user based on the user's purchasing tendencies and emotional information.

[0355] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store, and in particular utilizes an emotion engine that recognizes user emotions. This system is realized by a process in which a user tries on products in a physical store, enters the try-on information into a terminal, and generates a QR code.

[0356] Hardware and software used

[0357] Hardware: Smartphones, tablets, web cameras

[0358] Software: Azure (registered trademark) Cognitive Services Emotion API, QR code generation API, Firebase (real-time database)

[0359] Processing flow

[0360] First, the store staff enters the user's fitting information via a smartphone or tablet. This fitting information includes the product number, fitting date and time, and the staff ID in charge. Next, a QR code based on the fitting information is generated using a QR code generation API and provided to the user. The QR code is displayed on the smartphone screen or printed out on paper and given to the user.

[0361] The user takes the QR code home and enters it later when purchasing the product from the web store. The QR code is analyzed using a QR code reading API, and try-on information is obtained. Based on this information, the server analyzes the user's emotions using emotion recognition means (Azure Cognitive Services Emotion API) and obtains the emotional information shown when trying on the product or purchasing it. This emotional information is associated with the try-on information and purchase information and stored in Firebase.

[0362] The purchasing information associated with the emotional information is analyzed by the recommendation means, and appropriate products are recommended for the user's next visit based on the user's purchasing trends. Furthermore, the server updates the staff evaluation to reflect the emotional information at the time of trying on clothes and purchasing, and points are awarded based on the results. This allows the staff's customer service quality to be properly evaluated, leading to improved service for users.

[0363] Specific examples

[0364] For example, if User A tries on a blue dress in a physical store and shows a satisfied expression while trying it on, the emotion recognition API will identify this expression as positive. Later, if User A adds the same dress to her cart in the online store and enters the QR code, the positive emotion data will be referenced and the staff will give it a high rating. Furthermore, the next time User A visits, new blue products will be recommended.

[0365] Prompt Sentence Examples

[0366] "Please tell me the algorithm that allows a user to try on clothes in a physical store, collect the emotions displayed while trying them on using a camera and microphone, and generate and encode a QR code. Please provide specific emotion recognition code and QR code generation code, and explain how to save the try-on information and emotion data in Firebase."

[0367] In this way, the connection between physical stores and online stores is strengthened, and by utilizing the emotion engine, it is possible to improve the user's purchasing experience and properly evaluate the efforts of staff.

[0368] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0369] Step 1:

[0370] Store staff enter the user's fitting information on a smartphone or tablet. The input information includes the product number, fitting date and time, and the staff ID in charge. This information is uploaded to the Firebase real-time database. The input is the user's fitting information, and the output is the fitting information stored in Firebase.

[0371] Step 2:

[0372] A QR code is generated using a QR code generation API based on the try-on information entered on the device. The generated QR code is displayed on the smartphone screen or printed on paper and given to the user. The input is the try-on information, and the output is the QR code provided to the user.

[0373] Step 3:

[0374] A user accesses a web store and adds a product to their cart. At this time, the QR code is read by the device and analyzed by the QR code reading API. Try-on information is obtained from the QR code, and Firebase is accessed to read the try-on information. The QR code is the input, and the analyzed try-on information is obtained as the output.

[0375] Step 4:

[0376] The Azure Cognitive Services Emotion API is used as an emotion recognition method to analyze the user's emotions when trying on clothes and purchasing. Data collected through the camera and microphone is sent to the API, and emotional information is obtained as the analysis result. The obtained emotional information is stored in Firebase. The input is the user's emotional data, and the output is analyzed emotional information.

[0377] Step 5:

[0378] The server links the try-on information obtained from the QR code with the purchase information from the web store. This linking allows for centralized management of purchase information along with related emotional information. The inputs are try-on information and purchase information, and the output is the linked purchase information.

[0379] Step 6:

[0380] The rating update means updates the staff rating linked to the fitting information. If positive emotional information is reflected, the staff rating points increase. This rating is saved in Firebase. The inputs are emotional information and fitting information, and the output is the updated staff rating points.

[0381] Step 7:

[0382] As a recommendation method, the server recommends appropriate products based on the user's purchasing trends and emotional information. It analyzes purchasing history and emotional data to determine which products to recommend to the user on their next visit. The input is the user's purchasing trends data, and the output is the recommended products.

[0383] In this way, each process step improves and improves service for both users and staff.

[0384] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0385] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0386] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0387] [Second embodiment]

[0388] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0389] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0390] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0391] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0392] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0393] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0394] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0395] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0396] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0397] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0398] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0399] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0400] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store. By integrating product try-ons in a physical store with purchases in a web store, this system evaluates the efforts of store staff and creates an environment where users can easily try on products and receive consultations in a physical store.

[0401] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The staff member uses a terminal (tablet or smartphone) to input the try-on information. This try-on information includes the product number, try-on date and time, and the staff member ID in charge. This allows the device to be used as an input means.

[0402] Next, the terminal generates a QR code based on the try-on information. The generated QR code is displayed to the user by the providing means or printed on paper and given to the user. The user can take this QR code home and use it to access the web store at home or elsewhere.

[0403] A user accesses a web store, searches for and selects a product, adds it to their cart, and then enters the QR code. By scanning the QR code into a dedicated input field, they can proceed to the web store's purchasing process. This operation is performed by the scanning means.

[0404] When the QR code is scanned, the server analyzes the information and obtains the try-on information. The analyzed information is automatically linked to the purchase information. Through this linking method, the server links the try-on information from the physical store with the purchase information from the online store.

[0405] The server then updates the store staff's evaluation linked to the fitting information. Using the evaluation update means, the staff's evaluation points are updated on the system. This allows the staff's efforts to be properly evaluated. Points are also awarded based on the evaluation. These points are also awarded to the user and are returned as purchase points. Staff are awarded points for assisting with fitting.

[0406] Furthermore, the server uses AI to analyze the user's purchase history and try-on history, and recommends appropriate products for the next visit. This allows for more detailed and appropriate suggestions to be made to the user. In-store staff are also provided with product suggestion information based on user attributes, enabling more effective customer service.

[0407] As a concrete example, a user tries on a red jacket in a physical store and receives a QR code with that information registered. Later, when the user purchases the red jacket from a web store, they enter the QR code, and the server links the try-on information with the purchase information, updates the staff's evaluation, and awards points to each. This links the physical store and the web store, properly evaluates the efforts of the store staff, and allows the user to enjoy trying on items in the physical store without feeling guilty.

[0408] This invention is expected to improve the purchasing experience and make the management of physical stores more sustainable by combining the advantages of existing physical stores and web stores.

[0409] The processing flow will be explained below.

[0410] Step 1:

[0411] A user visits a store, browses the shelves, and finds an item they want to try on.

[0412] Step 2:

[0413] The user asks the store staff for the product they want to try on. The user picks up the product they want to try on and tells the staff, "I'd like to try this product on."

[0414] Step 3:

[0415] Product information (product number, model number, etc.) is entered into a terminal (a tablet or smartphone held by store staff), and the staff then enters the product information into a dedicated app on the tablet.

[0416] Step 4:

[0417] The terminal registers fitting information (fitting date and time, user ID (optional), staff ID in charge, etc.). The staff member enters the fitting date and time and their own staff ID, and also the user ID if necessary.

[0418] Step 5:

[0419] The device will generate a QR code. The QR code will be automatically generated based on the above information.

[0420] Step 6:

[0421] The terminal shows the QR code to the user or prints it out on paper and hands it over to the user. The staff member shows the QR code on the tablet screen to the user or prints it out on paper and hands it over to the user.

[0422] Step 7:

[0423] A user accesses the web store from their home or other location. A user accesses the web store using a computer or smartphone.

[0424] Step 8:

[0425] A user searches for a product on a web store, tries on the product, and adds it to their cart. A user searches for a red jacket, checks the product detail page, and adds it to their cart.

[0426] Step 9:

[0427] The user scans the QR code into the input field. During the purchase process, the user scans the QR code received at the store into the dedicated QR code input field, or enters the displayed information.

[0428] Step 10:

[0429] The server analyzes the QR code information and obtains the fitting information, such as the product ID, fitting date and time, and staff ID.

[0430] Step 11:

[0431] The server links purchase information from the online store with try-on information from the physical store, matches the purchase data with the try-on data, and records which staff member assisted in trying on which product.

[0432] Step 12:

[0433] The server updates the store staff's evaluation based on the purchase data, and adds the evaluation points of the staff who assisted with trying on the items to the staff evaluation in the system.

[0434] Step 13:

[0435] The server grants the user points for the purchase. Points corresponding to the purchase amount are added to the user's account.

[0436] Step 14:

[0437] The server also gives points to the store staff. The staff also receive points on the system for purchases associated with the fitting assistance.

[0438] Step 15:

[0439] The server uses AI to analyze the user's purchase history and try-on history. The AI ​​analyzes the user's past purchase history, try-on history, and browsing history.

[0440] Step 16:

[0441] The server recommends the most suitable products to the user. Highly relevant products are displayed on the web store based on the user's interests and purchasing trends.

[0442] Step 17:

[0443] The server provides store staff with product suggestion information based on user attributes. Information useful for the next suggestion (recommended products, similar items, etc.) is provided to the store staff's terminal.

[0444] Example 1

[0445] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0446] In recent years, there has been a demand for systems that can provide convenience to consumers and fairly evaluate the efforts of sales representatives at physical stores by integrating product try-on sessions in physical stores with purchases via e-commerce. However, existing systems have not established a method for effectively linking try-on information at physical stores with purchase information via e-commerce. Furthermore, the lack of linkage between try-on information and purchase information has led to problems such as the inability to evaluate sales representatives or make appropriate product recommendations to consumers.

[0447] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0448] In this invention, the server includes a linking means for linking acquired try-on information with purchase information, an evaluation updating means for updating the salesperson's evaluation linked to the try-on information, a recommendation means for analyzing habits and recommending appropriate products to the consumer, and a provision means for providing the salesperson with product suggestion information based on the consumer's attributes. This effectively links the try-on information from the physical store with the purchase information from e-commerce, enabling the salesperson to make fair evaluations and suggest appropriate products to the consumer.

[0449] A "brick and mortar store" is a store that exists in a physical location and where consumers can visit in person to try on and purchase products.

[0450] "Electronic commerce" is the act of buying and selling goods over the Internet.

[0451] "Consumer" refers to an individual who tries on or purchases a product in a physical store or through e-commerce.

[0452] "Try-on information" is data generated when a consumer tries on a product in a physical store, and includes information such as the product number, try-on date and time, and the person in charge's ID.

[0453] "Input means" refers to a device or method for a consumer to input try-on information into a terminal when trying on a product in a physical store.

[0454] "QR code generation means" refers to a device or method for generating a QR code based on input try-on information.

[0455] "Providing means" refers to the method or device for delivering the generated QR code to the consumer.

[0456] "Reading means" refers to a method or device that allows a consumer to read a QR code using a digital input or scanning device when purchasing a product through e-commerce.

[0457] The "linking means" refers to a method or device for linking the try-on information obtained from the QR code with the purchase information in the e-commerce transaction.

[0458] The "rating update means" refers to a method or device for updating the sales representative's rating based on the linked try-on information and purchase information.

[0459] A "scoring means" refers to a method or device for awarding points or scores to consumers or salespeople based on the evaluation.

[0460] "Habits" refer to a consumer's behavioral characteristics determined based on their past purchase history and try-on history.

[0461] A "recommendation means" is a method or device for recommending appropriate products for the next visit based on the analyzed consumer habits.

[0462] A "sales representative" is a staff member who interacts with consumers in a physical store, allowing them to try on products and providing advice.

[0463] "Product proposal information" refers to information provided to sales representatives based on consumer attributes and past behavioral history, which is used to make effective product proposals.

[0464] A "database" is a system that structures, stores, and manages vast amounts of data, such as try-on information, purchase information, and evaluation information.

[0465] The present invention is a system that effectively links product try-ons in physical stores with purchases made through e-commerce, thereby increasing consumer convenience and providing fair recognition for the efforts of sales representatives. Specific embodiments for implementing the system of the present invention are described below.

[0466] First, the user visits a physical store and asks a sales representative for the item they would like to try on. The sales representative uses a device (tablet or smartphone) to enter the try-on information. The software on this device has a function that allows users to enter try-on information such as the product number, try-on date and time, and the staff member ID.

[0467] Next, the device generates a QR code based on the entered fitting information. This QR code is provided to the user and can be displayed on the device screen or printed on paper using a dedicated printer. The user can take this QR code home and use it at a later date.

[0468] When a user accesses an e-commerce site from home or elsewhere, searches for and selects a product, and adds it to their cart, they enter the QR code they received when trying on the item. This QR code is read into a dedicated input field on the e-commerce site. Once the QR code is entered, the server analyzes the information and obtains the try-on information. This analysis automatically links the try-on information and purchase information within the server.

[0469] In the next step of the workflow, the server uses this linked information to update the sales representative's evaluation at the physical store, ensuring that the sales representative's efforts are properly recognized and scored. The system also rewards the consumer with points for their purchase, which can be used toward their next purchase, helping to build consumer loyalty.

[0470] Furthermore, the server uses AI to analyze the consumer's purchase history and try-on history, and recommends appropriate products for the consumer's next visit. This allows the consumer to receive more detailed and appropriate suggestions. Sales representatives are also provided with product suggestion information based on the consumer's attributes, enabling more effective customer service.

[0471] As a concrete example, a user tries on a red jacket in a physical store and receives a QR code with that information registered. Later, when the user purchases the red jacket on an e-commerce site, they enter that QR code. The server links the try-on information with the purchase information, updates the sales representative's evaluation, and awards points to each. This links the physical store and the e-commerce site, properly evaluates the sales representative's efforts, and allows the user to enjoy trying on items in the physical store with ease.

[0472] Example prompts to input to a generative AI model:

[0473] "A user tries on a red jacket in a physical store and receives a QR code with that information registered. When the user then purchases the red jacket on an e-commerce site, they enter the QR code, which links the try-on information with the purchase information. Please explain in detail the system process. Please clearly state the roles of the server, terminal, and user."

[0474] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0475] Step 1:

[0476] A user visits a physical store and asks a salesperson for the product they would like to try on. The salesperson uses a device (tablet or smartphone) to enter the try-on information. This information includes the product number (e.g., "RJ1234"), try-on date and time (e.g., "2023-10-01 14:00"), and the staff member ID (e.g., "ST5678"). The entered try-on information is saved in the device's database and becomes the basis for generating a QR code in the next step.

[0477] Step 2:

[0478] The device generates a QR code based on the entered try-on information. The device's QR code generation function encodes this try-on information and outputs a QR code with the content "RJ1234_20231001_1400_ST5678". The device then provides this generated QR code to the user by displaying it on the screen or printing it out on a dedicated printer and handing it to the user. The user then takes this QR code home. Try-on information is received as input, and a QR code is generated and provided as output.

[0479] Step 3:

[0480] A user accesses an e-commerce site from home or elsewhere, searches for and selects a product. After adding the product to their cart, they proceed to the checkout screen. Here, the user enters the QR code they received when trying on the item by scanning or manually entering it into a dedicated input field. This QR code, used as input data, is used for analysis in the next step.

[0481] Step 4:

[0482] The server receives and analyzes the QR code entered on the e-commerce site. The server decodes the QR code "RJ1234_20231001_1400_ST5678" and obtains the fitting information (product number "RJ1234", fitting date and time "2023-10-01 14:00", staff ID "ST5678"). This fitting information is used in the next step of collaboration. The QR code is received as input, and fitting information is extracted as output.

[0483] Step 5:

[0484] The server links the try-on information with the purchase information. The purchase information includes the product number "RJ1234" and the purchase date and time "2023-10-02 18:00." This information is recorded in the server's database, and the try-on information and purchase information are linked. The server receives the try-on information and purchase information acquired as input, and saves the linked data in the database as output.

[0485] Step 6:

[0486] The server updates the sales representative's evaluation based on the linked information. The system assigns additional evaluation points to the staff member ID "ST5678" associated with the fitting information. For example, 10 points are added to the evaluation. Five points are also assigned to the user. The system receives linked data as input and updates the evaluation points as output.

[0487] Step 7:

[0488] The server uses AI to analyze the user's purchase history and try-on history. Based on past data, it recommends products that the user is likely to be interested in the next time they visit. This recommendation information is not only suggested to the user, but also provided to sales representatives. For example, if a user purchases a red jacket, matching black pants and a white shirt will be recommended the next time they visit. The server receives purchase history and try-on history as input and generates recommended product information as output.

[0489] By following each step of this system in an orderly manner, the integration of physical stores and e-commerce will be realized, consumer convenience will be improved, and the efforts of sales representatives will be properly recognized.

[0490] (Application example 1)

[0491] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0492] The current system for linking physical stores and online stores has the problem that when a customer tries on an item in a physical store and then purchases it on the online store, the entire process is interrupted. This raises concerns that the motivation of store staff will decrease because customer service in the physical store is not properly evaluated. Another problem is that customers cannot receive real-time recommendations for different products while trying on items in the physical store.

[0493] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0494] In this invention, the server includes: an input unit for inputting fitting information about a user trying on a product in a physical store; a QR code generation unit for generating a QR code based on the fitting information; a provision unit for providing the QR code to the user; a reading unit for reading the QR code when the user purchases the product in a web store; a linking unit for acquiring fitting information from the QR code and linking the acquired fitting information with purchase information; an evaluation update unit for updating staff ratings linked to the fitting information; a point awarding unit for awarding points based on the ratings; a unit for checking product suggestions and fitting status in real time using smart glasses; and a recommendation unit using a generative AI model that analyzes the user's past fitting history and purchase history to recommend products suitable for the user's next visit. This seamlessly links fitting behavior in a physical store with purchasing behavior in the web store, enabling fair recognition of the efforts of physical store staff. Furthermore, real-time product suggestions using smart glasses are expected to improve the quality of customer service and increase sales.

[0495] A "brick and mortar store" is a point of sale that exists in a physical location and allows customers to visit in person to purchase or try on products.

[0496] A "web store" is an online sales outlet operated on the Internet where customers can purchase products via the Internet.

[0497] "Try-on information" is detailed information about the product that the user tried on in the physical store, and includes data such as the product number, the try-on date and time, and the staff ID in charge.

[0498] "Input means" refers to a device or interface used to collect try-on information in a physical store and input it into the system. Examples of such devices include tablets and smartphones.

[0499] The "QR code generating means" refers to a device or method for generating a QR code based on the fitting information.

[0500] "Providing means" refers to a method or device for providing the generated QR code to the user. This includes displaying the QR code on a screen or printing it on paper.

[0501] The "reading means" refers to a method or device that reads the QR code when a user purchases the product from the web store. Specific examples include a smartphone camera or a dedicated input field.

[0502] The "linking means" refers to a method or device for acquiring try-on information from a QR code and associating the acquired try-on information with purchase information.

[0503] The "rating update means" refers to a method or device for updating the staff rating linked to the fitting information.

[0504] The "point awarding means" is a method or device for awarding points based on evaluation.

[0505] "Smart glasses" are wearable devices that users can wear to display and input information in real time.

[0506] A "generative AI model" is an artificial intelligence model that learns from large amounts of data and provides information appropriate to the user.

[0507] A "recommendation means" is a method or device that uses a generative AI model to analyze a user's past try-on history and purchase history and suggest appropriate products.

[0508] This invention relates to a system that links product try-on in a physical store with purchasing behavior in a web store. This system provides comprehensive support for users to try on products in a physical store and then smoothly purchase them in a web store.

[0509] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The store staff member uses a tablet or smartphone to input the user's fitting information. This fitting information includes the product number, the date and time of fitting, and the staff member ID in charge. This input method allows the user's fitting behavior in the physical store to be managed as digital data.

[0510] Next, a QR code is generated based on the input try-on information. The QR code generation means provides the generated QR code to the user. The QR code is displayed on the display of the smart glasses or printed on paper, which the user can take home. The QR code contains the try-on information and is used to access the web store later.

[0511] When a user accesses the web store from home or elsewhere and purchases an item they have tried on, they input the QR code. The QR code is read by a reader and sent to a server. The server obtains the try-on information from the QR code and links it to the purchase information. This linking means allows the system to consistently manage which items the user tried on and which items they purchased.

[0512] Furthermore, the server updates the staff evaluation linked to the fitting information. Using the evaluation update means, the staff is awarded points for assisting with fitting. This point system fairly evaluates the efforts of the staff and provides incentives. In addition, purchase points are awarded to the user, and discounts and benefits that can be used on the next purchase are offered.

[0513] In addition, the system uses smart glasses to allow users to check product suggestions and try-on status in real time. This allows store staff to make appropriate product suggestions based on the user's interests. Furthermore, using a generative AI model, the server analyzes the user's past try-on and purchase history and recommends appropriate products for the next visit. This allows users to enjoy a more personalized shopping experience.

[0514] As a concrete example, a user may try on a red jacket in a physical store and receive a QR code based on the fitting information. The user then takes a photo of this QR code with their smartphone and enters it later when purchasing the red jacket in a web store. The server retrieves the fitting information from the QR code, links it to the purchase information, updates the staff evaluation, and awards points. This series of processes seamlessly links the physical store and the web store, improving the purchasing experience.

[0515] Examples of prompts include:

[0516] "We are imagining a scenario where a user tries on a product using smart glasses in a store. The try-on information includes the user ID, staff ID, and product ID, which are generated as a QR code and provided to the user. When the user then enters the QR code when making a purchase on the web store, the try-on information and purchase information are automatically linked in the system, and the staff member's evaluation points are updated. Please explain this process in detail."

[0517] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0518] Step 1:

[0519] A user visits a physical store and asks a store staff member to try on an item. The store staff member uses a tablet or smartphone to enter the user's fitting information. The input information includes data such as the product number, fitting date and time, and the staff member ID in charge. The fitting information is then recorded on the terminal and sent to a database.

[0520] Input: Product information the user wants to try on, try-on date and time, staff ID

[0521] Output: Digital data including fitting information

[0522] Specific operation: The staff member enters the necessary information into the input form on the terminal and sends it to the database.

[0523] Step 2:

[0524] A QR code is generated based on the entered fitting information. The tablet or smartphone application generates a QR code based on the fitting information and displays it on the device. The generated QR code is then displayed to the user using smart glasses or printed on paper.

[0525] Input: Try-on information

[0526] Output: Generated QR code

[0527] Specific operation: The QR code generation application installed on the device acquires the fitting information and generates a QR code, which is then displayed or printed to the user.

[0528] Step 3:

[0529] The user accesses the web store from home or elsewhere, searches for and selects the product they have tried on, and adds it to their cart. When they proceed to the checkout, the QR code is read into the input field. The user can either scan the QR code with their smartphone camera or manually enter the contents of the QR code.

[0530] Input: QR code

[0531] Output: Try-on information obtained from QR code

[0532] Specific operation: The user scans the QR code with their smartphone and enters the information into the dedicated input field on the web store. The system analyzes the QR code and obtains the fitting information.

[0533] Step 4:

[0534] The server analyzes the try-on information obtained from the QR code and automatically links it to purchase information, thereby recording in a database which products the user tried on and which products they purchased.

[0535] Input: Try-on information and purchase information obtained from the QR code

[0536] Output: Linked try-on information and purchase information

[0537] Specific operation: The server analyzes the QR code data and associates the try-on information with the purchase information in the database.

[0538] Step 5:

[0539] The server updates the evaluation of the store staff linked to the fitting information. The evaluation points of the staff are updated on the system using the evaluation update means. This allows the staff's efforts to be properly evaluated.

[0540] Input: Linked try-on information and purchase information

[0541] Output: Updated staff rating points

[0542] Specific operation: The server updates the reputation points corresponding to the staff ID and saves them in the database.

[0543] Step 6:

[0544] Points are awarded to users and staff using a point awarding means. Users are awarded purchase points that can be used for their next purchase, and staff are awarded fitting assistance points.

[0545] Input: Updated staff rating, purchase completion information

[0546] Output: Points awarded

[0547] Specific operation: The server grants the corresponding points to the user's and staff's accounts.

[0548] Step 7:

[0549] The server uses the generated AI model to analyze the user's purchase history and try-on history, and recommends appropriate products for the user's next visit. The recommended information is displayed to the user through the smart glasses.

[0550] Input: User's purchase history, try-on history

[0551] Output: Product information to recommend to the user

[0552] How it works: The generative AI model analyzes the user's past data, calculates the optimal product, and displays it on the smart glasses.

[0553] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0554] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store, and in particular to a system that combines an emotion engine that recognizes user emotions. This system integrates product try-on in a physical store with purchases in a web store, and by recognizing user emotions, it can evaluate the efforts of store staff and create an environment where users can easily try on products and receive consultations in a physical store.

[0555] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The staff member uses a terminal (tablet or smartphone) to input the try-on information. This try-on information includes the product number, try-on date and time, and the staff member ID in charge. This allows the device to be used as an input means.

[0556] Next, the terminal generates a QR code based on the try-on information. The generated QR code is displayed to the user by the providing means or printed on paper and given to the user. The user can take this QR code home and use it to access the web store at home or elsewhere.

[0557] A user accesses a web store, searches for and selects a product, adds it to their cart, and then enters the QR code. By scanning the QR code into a dedicated input field, they can proceed to the web store's purchasing process. This operation is performed by the scanning means.

[0558] When the QR code is scanned, the server analyzes the information and obtains the try-on information. The analyzed information is automatically linked to the purchase information. Through this linking method, the server links the try-on information from the physical store with the purchase information from the online store.

[0559] The server then updates the store staff's evaluation linked to the fitting information. Using the evaluation update means, the staff's evaluation points are updated on the system. This allows the staff's efforts to be properly evaluated. Points are also awarded based on the evaluation. These points are also awarded to the user and are returned as purchase points. Staff are awarded points for assisting with fitting.

[0560] Furthermore, this system uses an emotion engine. The emotion engine analyzes the user's facial expressions and voice when trying on clothes or making a purchase, and acquires emotional information. This emotional information is recorded in association with the try-on information and purchase information. The server uses this emotional information to further analyze the user's purchasing trends and recommend appropriate products.

[0561] For example, if a user tries on a red jacket in a physical store and smiles or shows satisfaction, the emotion engine recognizes this and adds that emotion information to the try-on information. Later, when the user enters the QR code to purchase the red jacket in a web store, the positive emotion expressed during the try-on is reflected and linked to the purchase information. The server can then analyze this emotion information and recommend products that the user is likely to like the next time they visit.

[0562] The server also reflects emotional information in the evaluation of the staff's customer service. For example, if a user expresses positive emotions toward the staff's customer service while trying on clothes, that evaluation will be added to the staff's evaluation points. This allows the quality of the staff's customer service to be evaluated appropriately.

[0563] In this way, the system of the present invention strengthens the connection between physical stores and online stores, and by utilizing an emotion engine, it is possible to improve the user's purchasing experience and properly evaluate the efforts of store staff.

[0564] The processing flow will be explained below.

[0565] Step 1:

[0566] A user visits a store, browses the shelves, and finds an item they want to try on.

[0567] Step 2:

[0568] The user asks the store staff for the product they want to try on. The user picks up the product they want to try on and tells the staff, "I'd like to try this product on."

[0569] Step 3:

[0570] Product information (product number, model number, etc.) is entered into a terminal (a tablet or smartphone held by store staff), and the staff then enters the product information into a dedicated app on the tablet.

[0571] Step 4:

[0572] The terminal registers fitting information (fitting date and time, user ID (optional), staff ID in charge, etc.). The staff member enters the fitting date and time and their own staff ID, and also the user ID if necessary.

[0573] Step 5:

[0574] The emotion engine recognizes the user's emotions by analyzing emotional information (e.g., satisfaction, dissatisfaction, excitement, etc.) from the user's facial expressions and voice.

[0575] Step 6:

[0576] The device generates a QR code, which is automatically generated based on the fitting information and emotional information.

[0577] Step 7:

[0578] The terminal shows the QR code to the user or prints it out on paper and hands it over to the user. The staff member shows the QR code on the tablet screen to the user or prints it out on paper and hands it over to the user.

[0579] Step 8:

[0580] A user accesses the web store from their home or other location. A user accesses the web store using a computer or smartphone.

[0581] Step 9:

[0582] A user searches for a product on a web store, tries on the product, and adds it to their cart. A user searches for a red jacket, checks the product detail page, and adds it to their cart.

[0583] Step 10:

[0584] The user scans the QR code into the input field. During the purchase process, the user scans the QR code received at the store into the dedicated QR code input field, or enters the displayed information.

[0585] Step 11:

[0586] The server analyzes the QR code information and obtains the fitting information and emotion information. Information such as the product ID, fitting date and time, staff ID, and emotion information is analyzed from the QR code.

[0587] Step 12:

[0588] The server links purchase information from the web store with try-on information and emotional information from the physical store. It matches the purchase data with try-on data and emotional data, and records which staff member assisted in trying on which products.

[0589] Step 13:

[0590] The server updates the evaluation of the store staff based on the purchase data. The evaluation points of the staff who assisted with fitting are updated in the system. Evaluation points based on emotional information are also added.

[0591] Step 14:

[0592] The server grants the user points for the purchase. Points corresponding to the purchase amount are added to the user's account.

[0593] Step 15:

[0594] The server also awards points to store staff. Points are also awarded to staff in the system based on emotional information for purchases associated with fitting assistance.

[0595] Step 16:

[0596] The server uses AI to analyze the user's purchase history, try-on history, emotional information, and browsing history.

[0597] Step 17:

[0598] The server recommends the most suitable products to the user. Highly relevant products are displayed on the web store based on the user's interests, purchasing tendencies, and emotional information.

[0599] Step 18:

[0600] The server provides store staff with product suggestion information based on user attributes. Information useful for the next suggestion (recommended products, similar items, etc.) is provided to the store staff's terminal.

[0601] This will strengthen the connection between physical stores and online stores, improve the user's purchasing experience by utilizing the emotion engine, and create a system that properly evaluates the efforts of store staff.

[0602] Example 2

[0603] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0604] In today's commercial environment, there is a lack of coordination between physical and virtual stores, which makes it difficult for users to purchase products they have tried in a physical store in a virtual store. Furthermore, the lack of proper reflection of users' emotions and evaluations of staff service makes it difficult to improve the quality of service.

[0605] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0606] In this invention, the server includes: an input means for inputting trial information about a user who tries out a product at a real store; a two-dimensional code generation means for generating a two-dimensional code based on the trial information; a provision means for providing the two-dimensional code to the user; a reading means for reading the two-dimensional code when the user purchases the product at a virtual store; a linking means for acquiring trial information from the two-dimensional code and linking the acquired trial information with purchase information; an evaluation update means for updating an evaluation of a staff member linked to the trial information; a score assignment means for assigning a score based on the evaluation; an emotion analysis means for analyzing the user's emotions; a means for associating emotion information acquired from the emotion analysis means with the trial information and purchase information; and a recommendation means for recommending products based on the emotion information and trial information. This strengthens the connection between the real store and the virtual store, improves the user's purchasing experience, and enables the customer service efforts of staff to be appropriately evaluated.

[0607] "Brick and mortar store" means a commercial establishment that customers can physically visit.

[0608] "Virtual store" refers to an online store that sells products over the Internet.

[0609] "User" refers to an individual who uses a physical or virtual store to try out or purchase a product.

[0610] "Trial information" is information about the user when they try out a product, and includes the product number, trial date and time, and person in charge ID.

[0611] "Input means" refers to a device or system for inputting trial information at a physical store.

[0612] "Two-dimensional code" refers to a code that contains visual information, such as a QR code.

[0613] The "two-dimensional code generating means" refers to a system or software for generating a two-dimensional code based on input trial information.

[0614] "Providing means" refers to a method for providing the generated two-dimensional code to the user.

[0615] The "reading means" refers to the method used by the user to read the two-dimensional code when making a purchase at the virtual store.

[0616] The "linking means" refers to a method for linking the trial information obtained from the two-dimensional code with the purchase information at the virtual store.

[0617] The "evaluation update means" refers to a function for updating the evaluation of the person in charge on the system based on the trial information.

[0618] The "point giving means" refers to a method for giving points to users and personnel based on the evaluation.

[0619] "Emotion analysis means" refers to a device or software for analyzing a user's facial expressions and voice and acquiring emotional information.

[0620] The "recommendation means" refers to a method for recommending appropriate products to users based on trial information and emotion information.

[0621] MODE FOR CARRYING OUT THE INVENTION

[0622] This invention is a system for improving the quality of service by linking the sales activities of real stores and virtual stores and analyzing user emotions. This system includes the following components.

[0623] Hardware and software used

[0624] Input method: A device such as a tablet, smartphone, or PC is used. A dedicated application is installed on these devices, providing an interface for entering trial information.

[0625] Two-dimensional code generation means: Application software on the terminal generates a two-dimensional code (such as a QR code) based on the trial information.

[0626] Means of presentation: The generated two-dimensional code is displayed on the terminal screen, or a printer is used to print it on paper.

[0627] Reading method: Using a QR code reader built into the virtual store's website or application.

[0628] Linking means: A database management system running on the server side links the trial information with the purchase information.

[0629] Evaluation update means, score assignment means: The evaluation management system that operates on the server side updates the evaluation of the person in charge and assigns a score.

[0630] Emotion analysis tools: Emotion analysis software is used to analyze the user's facial expressions and voice. This data is collected through cameras and microphones in physical stores and webcams and microphones in virtual stores.

[0631] Recommendation means: A recommendation engine running on the server recommends products based on trial information and emotion information.

[0632] Specific methods for data processing and calculation

[0633] Entering trial information: When a user picks up a product in a physical store and tries it out, a staff member uses a dedicated app on a tablet or smartphone to enter the trial information (product number, trial date and time, staff ID, etc.).

[0634] QR Code Generation: After the trial information is entered, the application software generates a 2D code based on that information, which is then displayed on the terminal screen or printed and handed to the user.

[0635] Purchase from the web store: The user accesses the virtual store from home, searches for and selects the product they have tried, and adds it to their cart. They then scan the aforementioned QR code into the dedicated input field to complete the purchase.

[0636] Data analysis and linking: The server analyzes the QR code and obtains trial information. This information is automatically linked to purchase information in the virtual store. This allows for consistent management of trial information in the physical store and purchase information in the virtual store.

[0637] Rating and score update: The server automatically updates the staff's rating based on the linked trial information and assigns appropriate scores. Purchase points are also awarded to the user.

[0638] Emotion analysis: Emotion analysis software analyzes the user's facial expressions and voice during trial and purchase to obtain emotional information, which is then associated with trial and purchase information.

[0639] Providing recommendations: The server analyzes the emotional and trial information to understand the user's purchasing tendencies, and based on this, it can recommend appropriate products for the next visit.

[0640] Specific examples

[0641] For example, if a user tries on a red jacket in a physical store and smiles or shows satisfaction, the emotion analysis software recognizes this and adds positive emotional information to the trial information. Later, when the user enters the QR code to purchase a red jacket in a virtual store, the emotional information from the trial is linked to the purchase information. This allows the software to recommend products the user is likely to like the next time they visit. The software also updates the customer service evaluation of staff based on their positive emotions, and assigns appropriate scores.

[0642] Prompt Sentence Examples

[0643] "A user tries on a red jacket in a physical store and smiles and expresses satisfaction. The emotion analysis software recognizes this and adds the emotion information to the trial information. The user then enters a QR code when purchasing the red jacket in a virtual store. Please explain how the system handles this."

[0644] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0645] Step 1:

[0646] Users try out products in-store.

[0647] Specific operation: A user visits a physical store and tries on a product (e.g., a red jacket). During the trial, a staff member checks the user's facial expression and voice and provides a guide for the trial.

[0648] Step 2:

[0649] Use the terminal to enter your trial information.

[0650] Specific operation: The staff member uses a dedicated application on a tablet or smartphone to enter trial information (product number, trial date and time, staff ID, etc.).

[0651] Input: Product number (e.g., 12345), trial date and time (e.g., 2023-10-01 14:00), staff ID (e.g., 567)

[0652] Output: The entered trial information is saved on the device.

[0653] Step 3:

[0654] The terminal generates a two-dimensional code based on the trial information.

[0655] Specific operation: The terminal application generates a two-dimensional code (QR code) based on the entered trial information.

[0656] Input: Trial information (product number, trial date and time, staff ID)

[0657] Output: The generated 2D code is displayed on the terminal.

[0658] Step 4:

[0659] The two-dimensional code is provided to the user through the providing means.

[0660] Specific operation: Show the 2D code displayed on the terminal screen to the user, or print it out on paper using a printer and hand it to the user.

[0661] Input: Generated 2D code

[0662] Output: 2D code provided by the user

[0663] Step 5:

[0664] A user accesses a virtual store and purchases a product using a two-dimensional code.

[0665] Specific operation: The user accesses a virtual store (e.g., a website) from home, searches for and selects the product they want to try (e.g., a red jacket), adds it to their cart, and then enters the QR code in a dedicated input field.

[0666] Input: The two-dimensional code that the user enters in the virtual store

[0667] Output: Proceed with the purchase in the virtual store.

[0668] Step 6:

[0669] The server analyzes the two-dimensional code and obtains the trial information.

[0670] Specific operation: The server analyzes the two-dimensional code sent from the virtual store and obtains the trial information (product number, trial date and time, staff ID).

[0671] Input: 2D code

[0672] Output: Parsed trial information

[0673] Step 7:

[0674] The server links the trial information with the purchase information.

[0675] Specific operation: The server links the acquired trial information with the purchase information at the virtual store in the database, thereby confirming that the trialed product was purchased.

[0676] Input: Trial information, purchase information

[0677] Output: Linked trial and purchase information

[0678] Step 8:

[0679] Update staff ratings using the rating update means.

[0680] Specific operation: The server updates the staff evaluation on the system based on the trial information. Evaluation points are automatically calculated and updated.

[0681] Input: Trial information (Staff ID)

[0682] Output: Updated staff rating points

[0683] Step 9:

[0684] A point giving means is used to give points to users and staff members.

[0685] Specific operation: The server assigns purchase points to the user and trial support points to the staff based on the trial information and evaluation.

[0686] Input: Rating points

[0687] Output: Purchase number for user, trial support number for staff

[0688] Step 10:

[0689] Emotion analysis means is used to obtain user emotion information.

[0690] How it works: Emotion analysis software analyzes the user's facial expressions and voice during trial and purchase to obtain emotional information.

[0691] Input: User's facial expressions and voice data

[0692] Output: Parsed emotion information

[0693] Step 11:

[0694] Associate sentiment information with trial and purchase information.

[0695] Specific operation: The server associates the emotion information with the trial information and purchase information and stores the information in a database.

[0696] Input: Emotion information, trial information, purchase information

[0697] Output: Associated emotion information and trial / purchase information

[0698] Step 12:

[0699] To recommend appropriate products to a user using a recommendation means.

[0700] Specific operation: The server analyzes the emotional information and trial information, understands the user's purchasing tendencies, and recommends appropriate products the next time the user visits the virtual store.

[0701] Input: Emotion information, trial information

[0702] Output: Products recommended to the user

[0703] (Application example 2)

[0704] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0705] Conventional systems linking physical stores and online stores only link try-on information from the physical store with purchase information from the online store, and do not take into account the user's emotions or purchasing trends. This makes it difficult to recommend the most suitable products to the user, and also lacks a mechanism for properly evaluating the customer service quality of staff. The purpose of this invention is to solve these problems.

[0706] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: an input means for inputting try-on information about a user trying on a product in a physical store; a QR code generation means for generating a QR code based on the try-on information; a provision means for providing the QR code to the user; a reading means for reading the QR code when the user purchases the product in the web store; a linking means for acquiring try-on information from the QR code and linking the acquired try-on information with purchase information; an evaluation update means for updating the evaluation of the staff member linked to the try-on information; a point awarding means for awarding points based on the evaluation; an emotion recognition means for recognizing the user's emotions when trying on products and when purchasing; and a recommendation means for analyzing the user's purchasing tendencies using the emotion information acquired by the emotion recognition means and recommending products. This enables optimal product recommendations that take into account the user's emotions and purchasing tendencies, and an appropriate evaluation of the staff member's customer service quality.

[0707] A "physical store" is a store that exists in a physical location and that consumers can visit in person to purchase goods or receive services.

[0708] "Webstore" means a website or platform that exists on the Internet and allows consumers to access and purchase products online.

[0709] "Try-on information" refers to information when a user tries on a product in a physical store, and includes data such as the product number, the try-on date and time, and the staff ID in charge.

[0710] A "QR code" is a type of two-dimensional barcode, a matrix code that can encode and store information.

[0711] "Emotion recognition means" refers to a technology or device that analyzes emotions from the user's facial expressions, voice, etc., and recognizes a specific emotional state.

[0712] "Purchase information" refers to information when a user purchases a product from a web store, and includes data such as the product name, purchase date and time, and purchase quantity.

[0713] The "point awarding means" is a method or system for awarding points to users or staff members based on evaluations.

[0714] The "linking means" is a method or system that links the try-on information obtained from the QR code with the purchase information at the web store.

[0715] A "recommendation means" is a technology or device that recommends appropriate products to a user based on the user's purchasing tendencies and emotional information.

[0716] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store, and in particular utilizes an emotion engine that recognizes user emotions. This system is realized by a process in which a user tries on products in a physical store, enters the try-on information into a terminal, and generates a QR code.

[0717] Hardware and software used

[0718] Hardware: Smartphones, tablets, web cameras

[0719] Software: Azure Cognitive Services Emotion API, QR code generation API, Firebase (real-time database)

[0720] Processing flow

[0721] First, the store staff enters the user's fitting information via a smartphone or tablet. This fitting information includes the product number, fitting date and time, and the staff ID in charge. Next, a QR code based on the fitting information is generated using a QR code generation API and provided to the user. The QR code is displayed on the smartphone screen or printed out on paper and given to the user.

[0722] The user takes the QR code home and enters it later when purchasing the product from the web store. The QR code is analyzed using a QR code reading API, and try-on information is obtained. Based on this information, the server analyzes the user's emotions using emotion recognition means (Azure Cognitive Services Emotion API) and obtains the emotional information shown when trying on the product or purchasing it. This emotional information is associated with the try-on information and purchase information and stored in Firebase.

[0723] The purchasing information associated with the emotional information is analyzed by the recommendation means, and appropriate products are recommended for the user's next visit based on the user's purchasing trends. Furthermore, the server updates the staff evaluation to reflect the emotional information at the time of trying on clothes and purchasing, and points are awarded based on the results. This allows the staff's customer service quality to be properly evaluated, leading to improved service for users.

[0724] Specific examples

[0725] For example, if User A tries on a blue dress in a physical store and shows a satisfied expression while trying it on, the emotion recognition API will identify this expression as positive. Later, if User A adds the same dress to her cart in the online store and enters the QR code, the positive emotion data will be referenced and the staff will give it a high rating. Furthermore, the next time User A visits, new blue products will be recommended.

[0726] Prompt Sentence Examples

[0727] "Please tell me the algorithm that allows a user to try on clothes in a physical store, collect the emotions displayed while trying them on using a camera and microphone, and generate and encode a QR code. Please provide specific emotion recognition code and QR code generation code, and explain how to save the try-on information and emotion data in Firebase."

[0728] In this way, the connection between physical stores and online stores is strengthened, and by utilizing the emotion engine, it is possible to improve the user's purchasing experience and properly evaluate the efforts of staff.

[0729] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0730] Step 1:

[0731] Store staff enter the user's fitting information on a smartphone or tablet. The input information includes the product number, fitting date and time, and the staff ID in charge. This information is uploaded to the Firebase real-time database. The input is the user's fitting information, and the output is the fitting information stored in Firebase.

[0732] Step 2:

[0733] A QR code is generated using a QR code generation API based on the try-on information entered on the device. The generated QR code is displayed on the smartphone screen or printed on paper and given to the user. The input is the try-on information, and the output is the QR code provided to the user.

[0734] Step 3:

[0735] A user accesses a web store and adds a product to their cart. At this time, the QR code is read by the device and analyzed by the QR code reading API. Try-on information is obtained from the QR code, and Firebase is accessed to read the try-on information. The QR code is the input, and the analyzed try-on information is obtained as the output.

[0736] Step 4:

[0737] The Azure Cognitive Services Emotion API is used as an emotion recognition method to analyze the user's emotions when trying on clothes and purchasing. Data collected through the camera and microphone is sent to the API, and emotional information is obtained as the analysis result. The obtained emotional information is stored in Firebase. The input is the user's emotional data, and the output is analyzed emotional information.

[0738] Step 5:

[0739] The server links the try-on information obtained from the QR code with the purchase information from the web store. This linking allows for centralized management of purchase information along with related emotional information. The inputs are try-on information and purchase information, and the output is the linked purchase information.

[0740] Step 6:

[0741] The rating update means updates the staff rating linked to the fitting information. If positive emotional information is reflected, the staff rating points increase. This rating is saved in Firebase. The inputs are emotional information and fitting information, and the output is the updated staff rating points.

[0742] Step 7:

[0743] As a recommendation method, the server recommends appropriate products based on the user's purchasing trends and emotional information. It analyzes purchasing history and emotional data to determine which products to recommend to the user on their next visit. The input is the user's purchasing trends data, and the output is the recommended products.

[0744] In this way, each process step improves and improves service for both users and staff.

[0745] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0746] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0747] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0748] [Third embodiment]

[0749] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0750] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0751] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0752] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0753] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0754] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0755] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0756] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0757] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0758] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0759] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0760] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0761] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store. By integrating product try-ons in a physical store with purchases in a web store, this system evaluates the efforts of store staff and creates an environment where users can easily try on products and receive consultations in a physical store.

[0762] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The staff member uses a terminal (tablet or smartphone) to input the try-on information. This try-on information includes the product number, try-on date and time, and the staff member ID in charge. This allows the device to be used as an input means.

[0763] Next, the terminal generates a QR code based on the try-on information. The generated QR code is displayed to the user by the providing means or printed on paper and given to the user. The user can take this QR code home and use it to access the web store at home or elsewhere.

[0764] A user accesses a web store, searches for and selects a product, adds it to their cart, and then enters the QR code. By scanning the QR code into a dedicated input field, they can proceed to the web store's purchasing process. This operation is performed by the scanning means.

[0765] When the QR code is scanned, the server analyzes the information and obtains the try-on information. The analyzed information is automatically linked to the purchase information. Through this linking method, the server links the try-on information from the physical store with the purchase information from the online store.

[0766] The server then updates the store staff's evaluation linked to the fitting information. Using the evaluation update means, the staff's evaluation points are updated on the system. This allows the staff's efforts to be properly evaluated. Points are also awarded based on the evaluation. These points are also awarded to the user and are returned as purchase points. Staff are awarded points for assisting with fitting.

[0767] Furthermore, the server uses AI to analyze the user's purchase history and try-on history, and recommends appropriate products for the next visit. This allows for more detailed and appropriate suggestions to be made to the user. In-store staff are also provided with product suggestion information based on user attributes, enabling more effective customer service.

[0768] As a concrete example, a user tries on a red jacket in a physical store and receives a QR code with that information registered. Later, when the user purchases the red jacket from a web store, they enter the QR code, and the server links the try-on information with the purchase information, updates the staff's evaluation, and awards points to each. This links the physical store and the web store, properly evaluates the efforts of the store staff, and allows the user to enjoy trying on items in the physical store without feeling guilty.

[0769] This invention is expected to improve the purchasing experience and make the management of physical stores more sustainable by combining the advantages of existing physical stores and web stores.

[0770] The processing flow will be explained below.

[0771] Step 1:

[0772] A user visits a store, browses the shelves, and finds an item they want to try on.

[0773] Step 2:

[0774] The user asks the store staff for the product they want to try on. The user picks up the product they want to try on and tells the staff, "I'd like to try this product on."

[0775] Step 3:

[0776] Product information (product number, model number, etc.) is entered into a terminal (a tablet or smartphone held by store staff), and the staff then enters the product information into a dedicated app on the tablet.

[0777] Step 4:

[0778] The terminal registers fitting information (fitting date and time, user ID (optional), staff ID in charge, etc.). The staff member enters the fitting date and time and their own staff ID, and also the user ID if necessary.

[0779] Step 5:

[0780] The device will generate a QR code. The QR code will be automatically generated based on the above information.

[0781] Step 6:

[0782] The terminal shows the QR code to the user or prints it out on paper and hands it over to the user. The staff member shows the QR code on the tablet screen to the user or prints it out on paper and hands it over to the user.

[0783] Step 7:

[0784] A user accesses the web store from their home or other location. A user accesses the web store using a computer or smartphone.

[0785] Step 8:

[0786] A user searches for a product on a web store, tries on the product, and adds it to their cart. A user searches for a red jacket, checks the product detail page, and adds it to their cart.

[0787] Step 9:

[0788] The user scans the QR code into the input field. During the purchase process, the user scans the QR code received at the store into the dedicated QR code input field, or enters the displayed information.

[0789] Step 10:

[0790] The server analyzes the QR code information and obtains the fitting information, such as the product ID, fitting date and time, and staff ID.

[0791] Step 11:

[0792] The server links purchase information from the online store with try-on information from the physical store, matches the purchase data with the try-on data, and records which staff member assisted in trying on which product.

[0793] Step 12:

[0794] The server updates the store staff's evaluation based on the purchase data, and adds the evaluation points of the staff who assisted with trying on the items to the staff evaluation in the system.

[0795] Step 13:

[0796] The server grants the user points for the purchase. Points corresponding to the purchase amount are added to the user's account.

[0797] Step 14:

[0798] The server also gives points to the store staff. The staff also receive points on the system for purchases associated with the fitting assistance.

[0799] Step 15:

[0800] The server uses AI to analyze the user's purchase history and try-on history. The AI ​​analyzes the user's past purchase history, try-on history, and browsing history.

[0801] Step 16:

[0802] The server recommends the most suitable products to the user. Highly relevant products are displayed on the web store based on the user's interests and purchasing trends.

[0803] Step 17:

[0804] The server provides store staff with product suggestion information based on user attributes. Information useful for the next suggestion (recommended products, similar items, etc.) is provided to the store staff's terminal.

[0805] Example 1

[0806] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0807] In recent years, there has been a demand for systems that can provide convenience to consumers and fairly evaluate the efforts of sales representatives at physical stores by integrating product try-on sessions in physical stores with purchases via e-commerce. However, existing systems have not established a method for effectively linking try-on information at physical stores with purchase information via e-commerce. Furthermore, the lack of linkage between try-on information and purchase information has led to problems such as the inability to evaluate sales representatives or make appropriate product recommendations to consumers.

[0808] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0809] In this invention, the server includes a linking means for linking acquired try-on information with purchase information, an evaluation updating means for updating the salesperson's evaluation linked to the try-on information, a recommendation means for analyzing habits and recommending appropriate products to the consumer, and a provision means for providing the salesperson with product suggestion information based on the consumer's attributes. This effectively links the try-on information from the physical store with the purchase information from e-commerce, enabling the salesperson to make fair evaluations and suggest appropriate products to the consumer.

[0810] A "brick and mortar store" is a store that exists in a physical location and where consumers can visit in person to try on and purchase products.

[0811] "Electronic commerce" is the act of buying and selling goods over the Internet.

[0812] "Consumer" refers to an individual who tries on or purchases a product in a physical store or through e-commerce.

[0813] "Try-on information" is data generated when a consumer tries on a product in a physical store, and includes information such as the product number, try-on date and time, and the person in charge's ID.

[0814] "Input means" refers to a device or method for a consumer to input try-on information into a terminal when trying on a product in a physical store.

[0815] "QR code generation means" refers to a device or method for generating a QR code based on input try-on information.

[0816] "Providing means" refers to the method or device for delivering the generated QR code to the consumer.

[0817] "Reading means" refers to a method or device that allows a consumer to read a QR code using a digital input or scanning device when purchasing a product through e-commerce.

[0818] The "linking means" refers to a method or device for linking the try-on information obtained from the QR code with the purchase information in the e-commerce transaction.

[0819] The "rating update means" refers to a method or device for updating the sales representative's rating based on the linked try-on information and purchase information.

[0820] A "scoring means" refers to a method or device for awarding points or scores to consumers or salespeople based on the evaluation.

[0821] "Habits" refer to a consumer's behavioral characteristics determined based on their past purchase history and try-on history.

[0822] A "recommendation means" is a method or device for recommending appropriate products for the next visit based on the analyzed consumer habits.

[0823] A "sales representative" is a staff member who interacts with consumers in a physical store, allowing them to try on products and providing advice.

[0824] "Product proposal information" refers to information provided to sales representatives based on consumer attributes and past behavioral history, which is used to make effective product proposals.

[0825] A "database" is a system that structures, stores, and manages vast amounts of data, such as try-on information, purchase information, and evaluation information.

[0826] The present invention is a system that effectively links product try-ons in physical stores with purchases made through e-commerce, thereby increasing consumer convenience and providing fair recognition for the efforts of sales representatives. Specific embodiments for implementing the system of the present invention are described below.

[0827] First, the user visits a physical store and asks a sales representative for the item they would like to try on. The sales representative uses a device (tablet or smartphone) to enter the try-on information. The software on this device has a function that allows users to enter try-on information such as the product number, try-on date and time, and the staff member ID.

[0828] Next, the device generates a QR code based on the entered fitting information. This QR code is provided to the user and can be displayed on the device screen or printed on paper using a dedicated printer. The user can take this QR code home and use it at a later date.

[0829] When a user accesses an e-commerce site from home or elsewhere, searches for and selects a product, and adds it to their cart, they enter the QR code they received when trying on the item. This QR code is read into a dedicated input field on the e-commerce site. Once the QR code is entered, the server analyzes the information and obtains the try-on information. This analysis automatically links the try-on information and purchase information within the server.

[0830] In the next step of the workflow, the server uses this linked information to update the sales representative's evaluation at the physical store, ensuring that the sales representative's efforts are properly recognized and scored. The system also rewards the consumer with points for their purchase, which can be used toward their next purchase, helping to build consumer loyalty.

[0831] Furthermore, the server uses AI to analyze the consumer's purchase history and try-on history, and recommends appropriate products for the consumer's next visit. This allows the consumer to receive more detailed and appropriate suggestions. Sales representatives are also provided with product suggestion information based on the consumer's attributes, enabling more effective customer service.

[0832] As a concrete example, a user tries on a red jacket in a physical store and receives a QR code with that information registered. Later, when the user purchases the red jacket on an e-commerce site, they enter that QR code. The server links the try-on information with the purchase information, updates the sales representative's evaluation, and awards points to each. This links the physical store and the e-commerce site, properly evaluates the sales representative's efforts, and allows the user to enjoy trying on items in the physical store with ease.

[0833] Example prompts to input to a generative AI model:

[0834] "A user tries on a red jacket in a physical store and receives a QR code with that information registered. When the user then purchases the red jacket on an e-commerce site, they enter the QR code, which links the try-on information with the purchase information. Please explain in detail the system process. Please clearly state the roles of the server, terminal, and user."

[0835] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0836] Step 1:

[0837] A user visits a physical store and asks a salesperson for the product they would like to try on. The salesperson uses a device (tablet or smartphone) to enter the try-on information. This information includes the product number (e.g., "RJ1234"), try-on date and time (e.g., "2023-10-01 14:00"), and the staff member ID (e.g., "ST5678"). The entered try-on information is saved in the device's database and becomes the basis for generating a QR code in the next step.

[0838] Step 2:

[0839] The device generates a QR code based on the entered try-on information. The device's QR code generation function encodes this try-on information and outputs a QR code with the content "RJ1234_20231001_1400_ST5678". The device then provides this generated QR code to the user by displaying it on the screen or printing it out on a dedicated printer and handing it to the user. The user then takes this QR code home. Try-on information is received as input, and a QR code is generated and provided as output.

[0840] Step 3:

[0841] A user accesses an e-commerce site from home or elsewhere, searches for and selects a product. After adding the product to their cart, they proceed to the checkout screen. Here, the user enters the QR code they received when trying on the item by scanning or manually entering it into a dedicated input field. This QR code, used as input data, is used for analysis in the next step.

[0842] Step 4:

[0843] The server receives and analyzes the QR code entered on the e-commerce site. The server decodes the QR code "RJ1234_20231001_1400_ST5678" and obtains the fitting information (product number "RJ1234", fitting date and time "2023-10-01 14:00", staff ID "ST5678"). This fitting information is used in the next step of collaboration. The QR code is received as input, and fitting information is extracted as output.

[0844] Step 5:

[0845] The server links the try-on information with the purchase information. The purchase information includes the product number "RJ1234" and the purchase date and time "2023-10-02 18:00." This information is recorded in the server's database, and the try-on information and purchase information are linked. The server receives the try-on information and purchase information acquired as input, and saves the linked data in the database as output.

[0846] Step 6:

[0847] The server updates the sales representative's evaluation based on the linked information. The system assigns additional evaluation points to the staff member ID "ST5678" associated with the fitting information. For example, 10 points are added to the evaluation. Five points are also assigned to the user. The system receives linked data as input and updates the evaluation points as output.

[0848] Step 7:

[0849] The server uses AI to analyze the user's purchase history and try-on history. Based on past data, it recommends products that the user is likely to be interested in the next time they visit. This recommendation information is not only suggested to the user, but also provided to sales representatives. For example, if a user purchases a red jacket, matching black pants and a white shirt will be recommended the next time they visit. The server receives purchase history and try-on history as input and generates recommended product information as output.

[0850] By following each step of this system in an orderly manner, the integration of physical stores and e-commerce will be realized, consumer convenience will be improved, and the efforts of sales representatives will be properly recognized.

[0851] (Application example 1)

[0852] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0853] The current system for linking physical stores and online stores has the problem that when a customer tries on an item in a physical store and then purchases it on the online store, the entire process is interrupted. This raises concerns that the motivation of store staff will decrease because customer service in the physical store is not properly evaluated. Another problem is that customers cannot receive real-time recommendations for different products while trying on items in the physical store.

[0854] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0855] In this invention, the server includes: an input unit for inputting fitting information about a user trying on a product in a physical store; a QR code generation unit for generating a QR code based on the fitting information; a provision unit for providing the QR code to the user; a reading unit for reading the QR code when the user purchases the product in a web store; a linking unit for acquiring fitting information from the QR code and linking the acquired fitting information with purchase information; an evaluation update unit for updating staff ratings linked to the fitting information; a point awarding unit for awarding points based on the ratings; a unit for checking product suggestions and fitting status in real time using smart glasses; and a recommendation unit using a generative AI model that analyzes the user's past fitting history and purchase history to recommend products suitable for the user's next visit. This seamlessly links fitting behavior in a physical store with purchasing behavior in the web store, enabling fair recognition of the efforts of physical store staff. Furthermore, real-time product suggestions using smart glasses are expected to improve the quality of customer service and increase sales.

[0856] A "brick and mortar store" is a point of sale that exists in a physical location and allows customers to visit in person to purchase or try on products.

[0857] A "web store" is an online sales outlet operated on the Internet where customers can purchase products via the Internet.

[0858] "Try-on information" is detailed information about the product that the user tried on in the physical store, and includes data such as the product number, the try-on date and time, and the staff ID in charge.

[0859] "Input means" refers to a device or interface used to collect try-on information in a physical store and input it into the system. Examples of such devices include tablets and smartphones.

[0860] The "QR code generating means" refers to a device or method for generating a QR code based on the fitting information.

[0861] "Providing means" refers to a method or device for providing the generated QR code to the user. This includes displaying the QR code on a screen or printing it on paper.

[0862] The "reading means" refers to a method or device that reads the QR code when a user purchases the product from the web store. Specific examples include a smartphone camera or a dedicated input field.

[0863] The "linking means" refers to a method or device for acquiring try-on information from a QR code and associating the acquired try-on information with purchase information.

[0864] The "rating update means" refers to a method or device for updating the staff rating linked to the fitting information.

[0865] The "point awarding means" is a method or device for awarding points based on evaluation.

[0866] "Smart glasses" are wearable devices that users can wear to display and input information in real time.

[0867] A "generative AI model" is an artificial intelligence model that learns from large amounts of data and provides information appropriate to the user.

[0868] A "recommendation means" is a method or device that uses a generative AI model to analyze a user's past try-on history and purchase history and suggest appropriate products.

[0869] This invention relates to a system that links product try-on in a physical store with purchasing behavior in a web store. This system provides comprehensive support for users to try on products in a physical store and then smoothly purchase them in a web store.

[0870] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The store staff member uses a tablet or smartphone to input the user's fitting information. This fitting information includes the product number, the date and time of fitting, and the staff member ID in charge. This input method allows the user's fitting behavior in the physical store to be managed as digital data.

[0871] Next, a QR code is generated based on the input try-on information. The QR code generation means provides the generated QR code to the user. The QR code is displayed on the display of the smart glasses or printed on paper, which the user can take home. The QR code contains the try-on information and is used to access the web store later.

[0872] When a user accesses the web store from home or elsewhere and purchases an item they have tried on, they input the QR code. The QR code is read by a reader and sent to a server. The server obtains the try-on information from the QR code and links it to the purchase information. This linking means allows the system to consistently manage which items the user tried on and which items they purchased.

[0873] Furthermore, the server updates the staff evaluation linked to the fitting information. Using the evaluation update means, the staff is awarded points for assisting with fitting. This point system fairly evaluates the efforts of the staff and provides incentives. In addition, purchase points are awarded to the user, and discounts and benefits that can be used on the next purchase are offered.

[0874] In addition, the system uses smart glasses to allow users to check product suggestions and try-on status in real time. This allows store staff to make appropriate product suggestions based on the user's interests. Furthermore, using a generative AI model, the server analyzes the user's past try-on and purchase history and recommends appropriate products for the next visit. This allows users to enjoy a more personalized shopping experience.

[0875] As a concrete example, a user may try on a red jacket in a physical store and receive a QR code based on the fitting information. The user then takes a photo of this QR code with their smartphone and enters it later when purchasing the red jacket in a web store. The server retrieves the fitting information from the QR code, links it to the purchase information, updates the staff evaluation, and awards points. This series of processes seamlessly links the physical store and the web store, improving the purchasing experience.

[0876] Examples of prompts include:

[0877] "We are imagining a scenario where a user tries on a product using smart glasses in a store. The try-on information includes the user ID, staff ID, and product ID, which are generated as a QR code and provided to the user. When the user then enters the QR code when making a purchase on the web store, the try-on information and purchase information are automatically linked in the system, and the staff member's evaluation points are updated. Please explain this process in detail."

[0878] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0879] Step 1:

[0880] A user visits a physical store and asks a store staff member to try on an item. The store staff member uses a tablet or smartphone to enter the user's fitting information. The input information includes data such as the product number, fitting date and time, and the staff member ID in charge. The fitting information is then recorded on the terminal and sent to a database.

[0881] Input: Product information the user wants to try on, try-on date and time, staff ID

[0882] Output: Digital data including fitting information

[0883] Specific operation: The staff member enters the necessary information into the input form on the terminal and sends it to the database.

[0884] Step 2:

[0885] A QR code is generated based on the entered fitting information. The tablet or smartphone application generates a QR code based on the fitting information and displays it on the device. The generated QR code is then displayed to the user using smart glasses or printed on paper.

[0886] Input: Try-on information

[0887] Output: Generated QR code

[0888] Specific operation: The QR code generation application installed on the device acquires the fitting information and generates a QR code, which is then displayed or printed to the user.

[0889] Step 3:

[0890] The user accesses the web store from home or elsewhere, searches for and selects the product they have tried on, and adds it to their cart. When they proceed to the checkout, the QR code is read into the input field. The user can either scan the QR code with their smartphone camera or manually enter the contents of the QR code.

[0891] Input: QR code

[0892] Output: Try-on information obtained from QR code

[0893] Specific operation: The user scans the QR code with their smartphone and enters the information into the dedicated input field on the web store. The system analyzes the QR code and obtains the fitting information.

[0894] Step 4:

[0895] The server analyzes the try-on information obtained from the QR code and automatically links it to purchase information, thereby recording in a database which products the user tried on and which products they purchased.

[0896] Input: Try-on information and purchase information obtained from the QR code

[0897] Output: Linked try-on information and purchase information

[0898] Specific operation: The server analyzes the QR code data and associates the try-on information with the purchase information in the database.

[0899] Step 5:

[0900] The server updates the evaluation of the store staff linked to the fitting information. The evaluation points of the staff are updated on the system using the evaluation update means. This allows the staff's efforts to be properly evaluated.

[0901] Input: Linked try-on information and purchase information

[0902] Output: Updated staff rating points

[0903] Specific operation: The server updates the reputation points corresponding to the staff ID and saves them in the database.

[0904] Step 6:

[0905] Points are awarded to users and staff using a point awarding means. Users are awarded purchase points that can be used for their next purchase, and staff are awarded fitting assistance points.

[0906] Input: Updated staff rating, purchase completion information

[0907] Output: Points awarded

[0908] Specific operation: The server grants the corresponding points to the user's and staff's accounts.

[0909] Step 7:

[0910] The server uses the generated AI model to analyze the user's purchase history and try-on history, and recommends appropriate products for the user's next visit. The recommended information is displayed to the user through the smart glasses.

[0911] Input: User's purchase history, try-on history

[0912] Output: Product information to recommend to the user

[0913] How it works: The generative AI model analyzes the user's past data, calculates the optimal product, and displays it on the smart glasses.

[0914] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0915] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store, and in particular to a system that combines an emotion engine that recognizes user emotions. This system integrates product try-on in a physical store with purchases in a web store, and by recognizing user emotions, it can evaluate the efforts of store staff and create an environment where users can easily try on products and receive consultations in a physical store.

[0916] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The staff member uses a terminal (tablet or smartphone) to input the try-on information. This try-on information includes the product number, try-on date and time, and the staff member ID in charge. This allows the device to be used as an input means.

[0917] Next, the terminal generates a QR code based on the try-on information. The generated QR code is displayed to the user by the providing means or printed on paper and given to the user. The user can take this QR code home and use it to access the web store at home or elsewhere.

[0918] A user accesses a web store, searches for and selects a product, adds it to their cart, and then enters the QR code. By scanning the QR code into a dedicated input field, they can proceed to the web store's purchasing process. This operation is performed by the scanning means.

[0919] When the QR code is scanned, the server analyzes the information and obtains the try-on information. The analyzed information is automatically linked to the purchase information. Through this linking method, the server links the try-on information from the physical store with the purchase information from the online store.

[0920] The server then updates the store staff's evaluation linked to the fitting information. Using the evaluation update means, the staff's evaluation points are updated on the system. This allows the staff's efforts to be properly evaluated. Points are also awarded based on the evaluation. These points are also awarded to the user and are returned as purchase points. Staff are awarded points for assisting with fitting.

[0921] Furthermore, this system uses an emotion engine. The emotion engine analyzes the user's facial expressions and voice when trying on clothes or making a purchase, and acquires emotional information. This emotional information is recorded in association with the try-on information and purchase information. The server uses this emotional information to further analyze the user's purchasing trends and recommend appropriate products.

[0922] For example, if a user tries on a red jacket in a physical store and smiles or shows satisfaction, the emotion engine recognizes this and adds that emotion information to the try-on information. Later, when the user enters the QR code to purchase the red jacket in a web store, the positive emotion expressed during the try-on is reflected and linked to the purchase information. The server can then analyze this emotion information and recommend products that the user is likely to like the next time they visit.

[0923] The server also reflects emotional information in the evaluation of the staff's customer service. For example, if a user expresses positive emotions toward the staff's customer service while trying on clothes, that evaluation will be added to the staff's evaluation points. This allows the quality of the staff's customer service to be evaluated appropriately.

[0924] In this way, the system of the present invention strengthens the connection between physical stores and online stores, and by utilizing an emotion engine, it is possible to improve the user's purchasing experience and properly evaluate the efforts of store staff.

[0925] The processing flow will be explained below.

[0926] Step 1:

[0927] A user visits a store, browses the shelves, and finds an item they want to try on.

[0928] Step 2:

[0929] The user asks the store staff for the product they want to try on. The user picks up the product they want to try on and tells the staff, "I'd like to try this product on."

[0930] Step 3:

[0931] Product information (product number, model number, etc.) is entered into a terminal (a tablet or smartphone held by store staff), and the staff then enters the product information into a dedicated app on the tablet.

[0932] Step 4:

[0933] The terminal registers fitting information (fitting date and time, user ID (optional), staff ID in charge, etc.). The staff member enters the fitting date and time and their own staff ID, and also the user ID if necessary.

[0934] Step 5:

[0935] The emotion engine recognizes the user's emotions by analyzing emotional information (e.g., satisfaction, dissatisfaction, excitement, etc.) from the user's facial expressions and voice.

[0936] Step 6:

[0937] The device generates a QR code, which is automatically generated based on the fitting information and emotional information.

[0938] Step 7:

[0939] The terminal shows the QR code to the user or prints it out on paper and hands it over to the user. The staff member shows the QR code on the tablet screen to the user or prints it out on paper and hands it over to the user.

[0940] Step 8:

[0941] A user accesses the web store from their home or other location. A user accesses the web store using a computer or smartphone.

[0942] Step 9:

[0943] A user searches for a product on a web store, tries on the product, and adds it to their cart. A user searches for a red jacket, checks the product detail page, and adds it to their cart.

[0944] Step 10:

[0945] The user scans the QR code into the input field. During the purchase process, the user scans the QR code received at the store into the dedicated QR code input field, or enters the displayed information.

[0946] Step 11:

[0947] The server analyzes the QR code information and obtains the fitting information and emotion information. Information such as the product ID, fitting date and time, staff ID, and emotion information is analyzed from the QR code.

[0948] Step 12:

[0949] The server links purchase information from the web store with try-on information and emotional information from the physical store. It matches the purchase data with try-on data and emotional data, and records which staff member assisted in trying on which products.

[0950] Step 13:

[0951] The server updates the evaluation of the store staff based on the purchase data. The evaluation points of the staff who assisted with fitting are updated in the system. Evaluation points based on emotional information are also added.

[0952] Step 14:

[0953] The server grants the user points for the purchase. Points corresponding to the purchase amount are added to the user's account.

[0954] Step 15:

[0955] The server also awards points to store staff. Points are also awarded to staff in the system based on emotional information for purchases associated with fitting assistance.

[0956] Step 16:

[0957] The server uses AI to analyze the user's purchase history, try-on history, emotional information, and browsing history.

[0958] Step 17:

[0959] The server recommends the most suitable products to the user. Highly relevant products are displayed on the web store based on the user's interests, purchasing tendencies, and emotional information.

[0960] Step 18:

[0961] The server provides store staff with product suggestion information based on user attributes. Information useful for the next suggestion (recommended products, similar items, etc.) is provided to the store staff's terminal.

[0962] This will strengthen the connection between physical stores and online stores, improve the user's purchasing experience by utilizing the emotion engine, and create a system that properly evaluates the efforts of store staff.

[0963] Example 2

[0964] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0965] In today's commercial environment, there is a lack of coordination between physical and virtual stores, which makes it difficult for users to purchase products they have tried in a physical store in a virtual store. Furthermore, the lack of proper reflection of users' emotions and evaluations of staff service makes it difficult to improve the quality of service.

[0966] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0967] In this invention, the server includes: an input means for inputting trial information about a user who tries out a product at a real store; a two-dimensional code generation means for generating a two-dimensional code based on the trial information; a provision means for providing the two-dimensional code to the user; a reading means for reading the two-dimensional code when the user purchases the product at a virtual store; a linking means for acquiring trial information from the two-dimensional code and linking the acquired trial information with purchase information; an evaluation update means for updating an evaluation of a staff member linked to the trial information; a score assignment means for assigning a score based on the evaluation; an emotion analysis means for analyzing the user's emotions; a means for associating emotion information acquired from the emotion analysis means with the trial information and purchase information; and a recommendation means for recommending products based on the emotion information and trial information. This strengthens the connection between the real store and the virtual store, improves the user's purchasing experience, and enables the customer service efforts of staff to be appropriately evaluated.

[0968] "Brick and mortar store" means a commercial establishment that customers can physically visit.

[0969] "Virtual store" refers to an online store that sells products over the Internet.

[0970] "User" refers to an individual who uses a physical or virtual store to try out or purchase a product.

[0971] "Trial information" is information about the user when they try out a product, and includes the product number, trial date and time, and person in charge ID.

[0972] "Input means" refers to a device or system for inputting trial information at a physical store.

[0973] "Two-dimensional code" refers to a code that contains visual information, such as a QR code.

[0974] The "two-dimensional code generating means" refers to a system or software for generating a two-dimensional code based on input trial information.

[0975] "Providing means" refers to a method for providing the generated two-dimensional code to the user.

[0976] The "reading means" refers to the method used by the user to read the two-dimensional code when making a purchase at the virtual store.

[0977] The "linking means" refers to a method for linking the trial information obtained from the two-dimensional code with the purchase information at the virtual store.

[0978] The "evaluation update means" refers to a function for updating the evaluation of the person in charge on the system based on the trial information.

[0979] The "point giving means" refers to a method for giving points to users and personnel based on the evaluation.

[0980] "Emotion analysis means" refers to a device or software for analyzing a user's facial expressions and voice and acquiring emotional information.

[0981] The "recommendation means" refers to a method for recommending appropriate products to users based on trial information and emotion information.

[0982] MODE FOR CARRYING OUT THE INVENTION

[0983] This invention is a system for improving the quality of service by linking the sales activities of real stores and virtual stores and analyzing user emotions. This system includes the following components.

[0984] Hardware and software used

[0985] Input method: A device such as a tablet, smartphone, or PC is used. A dedicated application is installed on these devices, providing an interface for entering trial information.

[0986] Two-dimensional code generation means: Application software on the terminal generates a two-dimensional code (such as a QR code) based on the trial information.

[0987] Means of presentation: The generated two-dimensional code is displayed on the terminal screen, or a printer is used to print it on paper.

[0988] Reading method: Using a QR code reader built into the virtual store's website or application.

[0989] Linking means: A database management system running on the server side links the trial information with the purchase information.

[0990] Evaluation update means, score assignment means: The evaluation management system that operates on the server side updates the evaluation of the person in charge and assigns a score.

[0991] Emotion analysis tools: Emotion analysis software is used to analyze the user's facial expressions and voice. This data is collected through cameras and microphones in physical stores and webcams and microphones in virtual stores.

[0992] Recommendation means: A recommendation engine running on the server recommends products based on trial information and emotion information.

[0993] Specific methods for data processing and calculation

[0994] Entering trial information: When a user picks up a product in a physical store and tries it out, a staff member uses a dedicated app on a tablet or smartphone to enter the trial information (product number, trial date and time, staff ID, etc.).

[0995] QR Code Generation: After the trial information is entered, the application software generates a 2D code based on that information, which is then displayed on the terminal screen or printed and handed to the user.

[0996] Purchase from the web store: The user accesses the virtual store from home, searches for and selects the product they have tried, and adds it to their cart. They then scan the aforementioned QR code into the dedicated input field to complete the purchase.

[0997] Data analysis and linking: The server analyzes the QR code and obtains trial information. This information is automatically linked to purchase information in the virtual store. This allows for consistent management of trial information in the physical store and purchase information in the virtual store.

[0998] Rating and score update: The server automatically updates the staff's rating based on the linked trial information and assigns appropriate scores. Purchase points are also awarded to the user.

[0999] Emotion analysis: Emotion analysis software analyzes the user's facial expressions and voice during trial and purchase to obtain emotional information, which is then associated with trial and purchase information.

[1000] Providing recommendations: The server analyzes the emotional and trial information to understand the user's purchasing tendencies, and based on this, it can recommend appropriate products for the next visit.

[1001] Specific examples

[1002] For example, if a user tries on a red jacket in a physical store and smiles or shows satisfaction, the emotion analysis software recognizes this and adds positive emotional information to the trial information. Later, when the user enters the QR code to purchase a red jacket in a virtual store, the emotional information from the trial is linked to the purchase information. This allows the software to recommend products the user is likely to like the next time they visit. The software also updates the customer service evaluation of staff based on their positive emotions, and assigns appropriate scores.

[1003] Prompt Sentence Examples

[1004] "A user tries on a red jacket in a physical store and smiles and expresses satisfaction. The emotion analysis software recognizes this and adds the emotion information to the trial information. The user then enters a QR code when purchasing the red jacket in a virtual store. Please explain how the system handles this."

[1005] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1006] Step 1:

[1007] Users try out products in-store.

[1008] Specific operation: A user visits a physical store and tries on a product (e.g., a red jacket). During the trial, a staff member checks the user's facial expression and voice and provides a guide for the trial.

[1009] Step 2:

[1010] Use the terminal to enter your trial information.

[1011] Specific operation: The staff member uses a dedicated application on a tablet or smartphone to enter trial information (product number, trial date and time, staff ID, etc.).

[1012] Input: Product number (e.g., 12345), trial date and time (e.g., 2023-10-01 14:00), staff ID (e.g., 567)

[1013] Output: The entered trial information is saved on the device.

[1014] Step 3:

[1015] The terminal generates a two-dimensional code based on the trial information.

[1016] Specific operation: The terminal application generates a two-dimensional code (QR code) based on the entered trial information.

[1017] Input: Trial information (product number, trial date and time, staff ID)

[1018] Output: The generated 2D code is displayed on the terminal.

[1019] Step 4:

[1020] The two-dimensional code is provided to the user through the providing means.

[1021] Specific operation: Show the 2D code displayed on the terminal screen to the user, or print it out on paper using a printer and hand it to the user.

[1022] Input: Generated 2D code

[1023] Output: 2D code provided by the user

[1024] Step 5:

[1025] A user accesses a virtual store and purchases a product using a two-dimensional code.

[1026] Specific operation: The user accesses a virtual store (e.g., a website) from home, searches for and selects the product they want to try (e.g., a red jacket), adds it to their cart, and then enters the QR code in a dedicated input field.

[1027] Input: The two-dimensional code that the user enters in the virtual store

[1028] Output: Proceed with the purchase in the virtual store.

[1029] Step 6:

[1030] The server analyzes the two-dimensional code and obtains the trial information.

[1031] Specific operation: The server analyzes the two-dimensional code sent from the virtual store and obtains the trial information (product number, trial date and time, staff ID).

[1032] Input: 2D code

[1033] Output: Parsed trial information

[1034] Step 7:

[1035] The server links the trial information with the purchase information.

[1036] Specific operation: The server links the acquired trial information with the purchase information at the virtual store in the database, thereby confirming that the trialed product was purchased.

[1037] Input: Trial information, purchase information

[1038] Output: Linked trial and purchase information

[1039] Step 8:

[1040] Update staff ratings using the rating update means.

[1041] Specific operation: The server updates the staff evaluation on the system based on the trial information. Evaluation points are automatically calculated and updated.

[1042] Input: Trial information (Staff ID)

[1043] Output: Updated staff rating points

[1044] Step 9:

[1045] A point giving means is used to give points to users and staff members.

[1046] Specific operation: The server assigns purchase points to the user and trial support points to the staff based on the trial information and evaluation.

[1047] Input: Rating points

[1048] Output: Purchase number for user, trial support number for staff

[1049] Step 10:

[1050] Emotion analysis means is used to obtain user emotion information.

[1051] How it works: Emotion analysis software analyzes the user's facial expressions and voice during trial and purchase to obtain emotional information.

[1052] Input: User's facial expressions and voice data

[1053] Output: Parsed emotion information

[1054] Step 11:

[1055] Associate sentiment information with trial and purchase information.

[1056] Specific operation: The server associates the emotion information with the trial information and purchase information and stores the information in a database.

[1057] Input: Emotion information, trial information, purchase information

[1058] Output: Associated emotion information and trial / purchase information

[1059] Step 12:

[1060] To recommend appropriate products to a user using a recommendation means.

[1061] Specific operation: The server analyzes the emotional information and trial information, understands the user's purchasing tendencies, and recommends appropriate products the next time the user visits the virtual store.

[1062] Input: Emotion information, trial information

[1063] Output: Products recommended to the user

[1064] (Application example 2)

[1065] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1066] Conventional systems linking physical stores and online stores only link try-on information from the physical store with purchase information from the online store, and do not take into account the user's emotions or purchasing trends. This makes it difficult to recommend the most suitable products to the user, and also lacks a mechanism for properly evaluating the customer service quality of staff. The purpose of this invention is to solve these problems.

[1067] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: an input means for inputting try-on information about a user trying on a product in a physical store; a QR code generation means for generating a QR code based on the try-on information; a provision means for providing the QR code to the user; a reading means for reading the QR code when the user purchases the product in the web store; a linking means for acquiring try-on information from the QR code and linking the acquired try-on information with purchase information; an evaluation update means for updating the evaluation of the staff member linked to the try-on information; a point awarding means for awarding points based on the evaluation; an emotion recognition means for recognizing the user's emotions when trying on products and when purchasing; and a recommendation means for analyzing the user's purchasing tendencies using the emotion information acquired by the emotion recognition means and recommending products. This enables optimal product recommendations that take into account the user's emotions and purchasing tendencies, and an appropriate evaluation of the staff member's customer service quality.

[1068] A "physical store" is a store that exists in a physical location and that consumers can visit in person to purchase goods or receive services.

[1069] "Webstore" means a website or platform that exists on the Internet and allows consumers to access and purchase products online.

[1070] "Try-on information" refers to information when a user tries on a product in a physical store, and includes data such as the product number, the try-on date and time, and the staff ID in charge.

[1071] A "QR code" is a type of two-dimensional barcode, a matrix code that can encode and store information.

[1072] "Emotion recognition means" refers to a technology or device that analyzes emotions from the user's facial expressions, voice, etc., and recognizes a specific emotional state.

[1073] "Purchase information" refers to information when a user purchases a product from a web store, and includes data such as the product name, purchase date and time, and purchase quantity.

[1074] The "point awarding means" is a method or system for awarding points to users or staff members based on evaluations.

[1075] The "linking means" is a method or system that links the try-on information obtained from the QR code with the purchase information at the web store.

[1076] A "recommendation means" is a technology or device that recommends appropriate products to a user based on the user's purchasing tendencies and emotional information.

[1077] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store, and in particular utilizes an emotion engine that recognizes user emotions. This system is realized by a process in which a user tries on products in a physical store, enters the try-on information into a terminal, and generates a QR code.

[1078] Hardware and software used

[1079] Hardware: Smartphones, tablets, web cameras

[1080] Software: Azure Cognitive Services Emotion API, QR code generation API, Firebase (real-time database)

[1081] Processing flow

[1082] First, the store staff enters the user's fitting information via a smartphone or tablet. This fitting information includes the product number, fitting date and time, and the staff ID in charge. Next, a QR code based on the fitting information is generated using a QR code generation API and provided to the user. The QR code is displayed on the smartphone screen or printed out on paper and given to the user.

[1083] The user takes the QR code home and enters it later when purchasing the product from the web store. The QR code is analyzed using a QR code reading API, and try-on information is obtained. Based on this information, the server analyzes the user's emotions using emotion recognition means (Azure Cognitive Services Emotion API) and obtains the emotional information shown when trying on the product or purchasing it. This emotional information is associated with the try-on information and purchase information and stored in Firebase.

[1084] The purchasing information associated with the emotional information is analyzed by the recommendation means, and appropriate products are recommended for the user's next visit based on the user's purchasing trends. Furthermore, the server updates the staff evaluation to reflect the emotional information at the time of trying on clothes and purchasing, and points are awarded based on the results. This allows the staff's customer service quality to be properly evaluated, leading to improved service for users.

[1085] Specific examples

[1086] For example, if User A tries on a blue dress in a physical store and shows a satisfied expression while trying it on, the emotion recognition API will identify this expression as positive. Later, if User A adds the same dress to her cart in the online store and enters the QR code, the positive emotion data will be referenced and the staff will give it a high rating. Furthermore, the next time User A visits, new blue products will be recommended.

[1087] Prompt Sentence Examples

[1088] "Please tell me the algorithm that allows a user to try on clothes in a physical store, collect the emotions displayed while trying them on using a camera and microphone, and generate and encode a QR code. Please provide specific emotion recognition code and QR code generation code, and explain how to save the try-on information and emotion data in Firebase."

[1089] In this way, the connection between physical stores and online stores is strengthened, and by utilizing the emotion engine, it is possible to improve the user's purchasing experience and properly evaluate the efforts of staff.

[1090] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1091] Step 1:

[1092] Store staff enter the user's fitting information on a smartphone or tablet. The input information includes the product number, fitting date and time, and the staff ID in charge. This information is uploaded to the Firebase real-time database. The input is the user's fitting information, and the output is the fitting information stored in Firebase.

[1093] Step 2:

[1094] A QR code is generated using a QR code generation API based on the try-on information entered on the device. The generated QR code is displayed on the smartphone screen or printed on paper and given to the user. The input is the try-on information, and the output is the QR code provided to the user.

[1095] Step 3:

[1096] A user accesses a web store and adds a product to their cart. At this time, the QR code is read by the device and analyzed by the QR code reading API. Try-on information is obtained from the QR code, and Firebase is accessed to read the try-on information. The QR code is the input, and the analyzed try-on information is obtained as the output.

[1097] Step 4:

[1098] The Azure Cognitive Services Emotion API is used as an emotion recognition method to analyze the user's emotions when trying on clothes and purchasing. Data collected through the camera and microphone is sent to the API, and emotional information is obtained as the analysis result. The obtained emotional information is stored in Firebase. The input is the user's emotional data, and the output is analyzed emotional information.

[1099] Step 5:

[1100] The server links the try-on information obtained from the QR code with the purchase information from the web store. This linking allows for centralized management of purchase information along with related emotional information. The inputs are try-on information and purchase information, and the output is the linked purchase information.

[1101] Step 6:

[1102] The rating update means updates the staff rating linked to the fitting information. If positive emotional information is reflected, the staff rating points increase. This rating is saved in Firebase. The inputs are emotional information and fitting information, and the output is the updated staff rating points.

[1103] Step 7:

[1104] As a recommendation method, the server recommends appropriate products based on the user's purchasing trends and emotional information. It analyzes purchasing history and emotional data to determine which products to recommend to the user on their next visit. The input is the user's purchasing trends data, and the output is the recommended products.

[1105] In this way, each process step improves and improves service for both users and staff.

[1106] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1107] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1108] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1109] [Fourth embodiment]

[1110] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1111] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1112] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1113] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1114] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1115] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1116] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1117] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1118] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1119] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1120] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1121] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1122] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1123] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store. By integrating product try-ons in a physical store with purchases in a web store, this system evaluates the efforts of store staff and creates an environment where users can easily try on products and receive consultations in a physical store.

[1124] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The staff member uses a terminal (tablet or smartphone) to input the try-on information. This try-on information includes the product number, try-on date and time, and the staff member ID in charge. This allows the device to be used as an input means.

[1125] Next, the terminal generates a QR code based on the try-on information. The generated QR code is displayed to the user by the providing means or printed on paper and given to the user. The user can take this QR code home and use it to access the web store at home or elsewhere.

[1126] A user accesses a web store, searches for and selects a product, adds it to their cart, and then enters the QR code. By scanning the QR code into a dedicated input field, they can proceed to the web store's purchasing process. This operation is performed by the scanning means.

[1127] When the QR code is scanned, the server analyzes the information and obtains the try-on information. The analyzed information is automatically linked to the purchase information. Through this linking method, the server links the try-on information from the physical store with the purchase information from the online store.

[1128] The server then updates the store staff's evaluation linked to the fitting information. Using the evaluation update means, the staff's evaluation points are updated on the system. This allows the staff's efforts to be properly evaluated. Points are also awarded based on the evaluation. These points are also awarded to the user and are returned as purchase points. Staff are awarded points for assisting with fitting.

[1129] Furthermore, the server uses AI to analyze the user's purchase history and try-on history, and recommends appropriate products for the next visit. This allows for more detailed and appropriate suggestions to be made to the user. In-store staff are also provided with product suggestion information based on user attributes, enabling more effective customer service.

[1130] As a concrete example, a user tries on a red jacket in a physical store and receives a QR code with that information registered. Later, when the user purchases the red jacket from a web store, they enter the QR code, and the server links the try-on information with the purchase information, updates the staff's evaluation, and awards points to each. This links the physical store and the web store, properly evaluates the efforts of the store staff, and allows the user to enjoy trying on items in the physical store without feeling guilty.

[1131] This invention is expected to improve the purchasing experience and make the management of physical stores more sustainable by combining the advantages of existing physical stores and web stores.

[1132] The processing flow will be explained below.

[1133] Step 1:

[1134] A user visits a store, browses the shelves, and finds an item they want to try on.

[1135] Step 2:

[1136] The user asks the store staff for the product they want to try on. The user picks up the product they want to try on and tells the staff, "I'd like to try this product on."

[1137] Step 3:

[1138] Product information (product number, model number, etc.) is entered into a terminal (a tablet or smartphone held by store staff), and the staff then enters the product information into a dedicated app on the tablet.

[1139] Step 4:

[1140] The terminal registers fitting information (fitting date and time, user ID (optional), staff ID in charge, etc.). The staff member enters the fitting date and time and their own staff ID, and also the user ID if necessary.

[1141] Step 5:

[1142] The device will generate a QR code. The QR code will be automatically generated based on the above information.

[1143] Step 6:

[1144] The terminal shows the QR code to the user or prints it out on paper and hands it over to the user. The staff member shows the QR code on the tablet screen to the user or prints it out on paper and hands it over to the user.

[1145] Step 7:

[1146] A user accesses the web store from their home or other location. A user accesses the web store using a computer or smartphone.

[1147] Step 8:

[1148] A user searches for a product on a web store, tries on the product, and adds it to their cart. A user searches for a red jacket, checks the product detail page, and adds it to their cart.

[1149] Step 9:

[1150] The user scans the QR code into the input field. During the purchase process, the user scans the QR code received at the store into the dedicated QR code input field, or enters the displayed information.

[1151] Step 10:

[1152] The server analyzes the QR code information and obtains the fitting information, such as the product ID, fitting date and time, and staff ID.

[1153] Step 11:

[1154] The server links purchase information from the online store with try-on information from the physical store, matches the purchase data with the try-on data, and records which staff member assisted in trying on which product.

[1155] Step 12:

[1156] The server updates the store staff's evaluation based on the purchase data, and adds the evaluation points of the staff who assisted with trying on the items to the staff evaluation in the system.

[1157] Step 13:

[1158] The server grants the user points for the purchase. Points corresponding to the purchase amount are added to the user's account.

[1159] Step 14:

[1160] The server also gives points to the store staff. The staff also receive points on the system for purchases associated with the fitting assistance.

[1161] Step 15:

[1162] The server uses AI to analyze the user's purchase history and try-on history. The AI ​​analyzes the user's past purchase history, try-on history, and browsing history.

[1163] Step 16:

[1164] The server recommends the most suitable products to the user. Highly relevant products are displayed on the web store based on the user's interests and purchasing trends.

[1165] Step 17:

[1166] The server provides store staff with product suggestion information based on user attributes. Information useful for the next suggestion (recommended products, similar items, etc.) is provided to the store staff's terminal.

[1167] Example 1

[1168] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1169] In recent years, there has been a demand for systems that can provide convenience to consumers and fairly evaluate the efforts of sales representatives at physical stores by integrating product try-on sessions in physical stores with purchases via e-commerce. However, existing systems have not established a method for effectively linking try-on information at physical stores with purchase information via e-commerce. Furthermore, the lack of linkage between try-on information and purchase information has led to problems such as the inability to evaluate sales representatives or make appropriate product recommendations to consumers.

[1170] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1171] In this invention, the server includes a linking means for linking acquired try-on information with purchase information, an evaluation updating means for updating the salesperson's evaluation linked to the try-on information, a recommendation means for analyzing habits and recommending appropriate products to the consumer, and a provision means for providing the salesperson with product suggestion information based on the consumer's attributes. This effectively links the try-on information from the physical store with the purchase information from e-commerce, enabling the salesperson to make fair evaluations and suggest appropriate products to the consumer.

[1172] A "brick and mortar store" is a store that exists in a physical location and where consumers can visit in person to try on and purchase products.

[1173] "Electronic commerce" is the act of buying and selling goods over the Internet.

[1174] "Consumer" refers to an individual who tries on or purchases a product in a physical store or through e-commerce.

[1175] "Try-on information" is data generated when a consumer tries on a product in a physical store, and includes information such as the product number, try-on date and time, and the person in charge's ID.

[1176] "Input means" refers to a device or method for a consumer to input try-on information into a terminal when trying on a product in a physical store.

[1177] "QR code generation means" refers to a device or method for generating a QR code based on input try-on information.

[1178] "Providing means" refers to the method or device for delivering the generated QR code to the consumer.

[1179] "Reading means" refers to a method or device that allows a consumer to read a QR code using a digital input or scanning device when purchasing a product through e-commerce.

[1180] The "linking means" refers to a method or device for linking the try-on information obtained from the QR code with the purchase information in the e-commerce transaction.

[1181] The "rating update means" refers to a method or device for updating the sales representative's rating based on the linked try-on information and purchase information.

[1182] A "scoring means" refers to a method or device for awarding points or scores to consumers or salespeople based on the evaluation.

[1183] "Habits" refer to a consumer's behavioral characteristics determined based on their past purchase history and try-on history.

[1184] A "recommendation means" is a method or device for recommending appropriate products for the next visit based on the analyzed consumer habits.

[1185] A "sales representative" is a staff member who interacts with consumers in a physical store, allowing them to try on products and providing advice.

[1186] "Product proposal information" refers to information provided to sales representatives based on consumer attributes and past behavioral history, which is used to make effective product proposals.

[1187] A "database" is a system that structures, stores, and manages vast amounts of data, such as try-on information, purchase information, and evaluation information.

[1188] The present invention is a system that effectively links product try-ons in physical stores with purchases made through e-commerce, thereby increasing consumer convenience and providing fair recognition for the efforts of sales representatives. Specific embodiments for implementing the system of the present invention are described below.

[1189] First, the user visits a physical store and asks a sales representative for the item they would like to try on. The sales representative uses a device (tablet or smartphone) to enter the try-on information. The software on this device has a function that allows users to enter try-on information such as the product number, try-on date and time, and the staff member ID.

[1190] Next, the device generates a QR code based on the entered fitting information. This QR code is provided to the user and can be displayed on the device screen or printed on paper using a dedicated printer. The user can take this QR code home and use it at a later date.

[1191] When a user accesses an e-commerce site from home or elsewhere, searches for and selects a product, and adds it to their cart, they enter the QR code they received when trying on the item. This QR code is read into a dedicated input field on the e-commerce site. Once the QR code is entered, the server analyzes the information and obtains the try-on information. This analysis automatically links the try-on information and purchase information within the server.

[1192] In the next step of the workflow, the server uses this linked information to update the sales representative's evaluation at the physical store, ensuring that the sales representative's efforts are properly recognized and scored. The system also rewards the consumer with points for their purchase, which can be used toward their next purchase, helping to build consumer loyalty.

[1193] Furthermore, the server uses AI to analyze the consumer's purchase history and try-on history, and recommends appropriate products for the consumer's next visit. This allows the consumer to receive more detailed and appropriate suggestions. Sales representatives are also provided with product suggestion information based on the consumer's attributes, enabling more effective customer service.

[1194] As a concrete example, a user tries on a red jacket in a physical store and receives a QR code with that information registered. Later, when the user purchases the red jacket on an e-commerce site, they enter that QR code. The server links the try-on information with the purchase information, updates the sales representative's evaluation, and awards points to each. This links the physical store and the e-commerce site, properly evaluates the sales representative's efforts, and allows the user to enjoy trying on items in the physical store with ease.

[1195] Example prompts to input to a generative AI model:

[1196] "A user tries on a red jacket in a physical store and receives a QR code with that information registered. When the user then purchases the red jacket on an e-commerce site, they enter the QR code, which links the try-on information with the purchase information. Please explain in detail the system process. Please clearly state the roles of the server, terminal, and user."

[1197] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1198] Step 1:

[1199] A user visits a physical store and asks a salesperson for the product they would like to try on. The salesperson uses a device (tablet or smartphone) to enter the try-on information. This information includes the product number (e.g., "RJ1234"), try-on date and time (e.g., "2023-10-01 14:00"), and the staff member ID (e.g., "ST5678"). The entered try-on information is saved in the device's database and becomes the basis for generating a QR code in the next step.

[1200] Step 2:

[1201] The device generates a QR code based on the entered try-on information. The device's QR code generation function encodes this try-on information and outputs a QR code with the content "RJ1234_20231001_1400_ST5678". The device then provides this generated QR code to the user by displaying it on the screen or printing it out on a dedicated printer and handing it to the user. The user then takes this QR code home. Try-on information is received as input, and a QR code is generated and provided as output.

[1202] Step 3:

[1203] A user accesses an e-commerce site from home or elsewhere, searches for and selects a product. After adding the product to their cart, they proceed to the checkout screen. Here, the user enters the QR code they received when trying on the item by scanning or manually entering it into a dedicated input field. This QR code, used as input data, is used for analysis in the next step.

[1204] Step 4:

[1205] The server receives and analyzes the QR code entered on the e-commerce site. The server decodes the QR code "RJ1234_20231001_1400_ST5678" and obtains the fitting information (product number "RJ1234", fitting date and time "2023-10-01 14:00", staff ID "ST5678"). This fitting information is used in the next step of collaboration. The QR code is received as input, and fitting information is extracted as output.

[1206] Step 5:

[1207] The server links the try-on information with the purchase information. The purchase information includes the product number "RJ1234" and the purchase date and time "2023-10-02 18:00." This information is recorded in the server's database, and the try-on information and purchase information are linked. The server receives the try-on information and purchase information acquired as input, and saves the linked data in the database as output.

[1208] Step 6:

[1209] The server updates the sales representative's evaluation based on the linked information. The system assigns additional evaluation points to the staff member ID "ST5678" associated with the fitting information. For example, 10 points are added to the evaluation. Five points are also assigned to the user. The system receives linked data as input and updates the evaluation points as output.

[1210] Step 7:

[1211] The server uses AI to analyze the user's purchase history and try-on history. Based on past data, it recommends products that the user is likely to be interested in the next time they visit. This recommendation information is not only suggested to the user, but also provided to sales representatives. For example, if a user purchases a red jacket, matching black pants and a white shirt will be recommended the next time they visit. The server receives purchase history and try-on history as input and generates recommended product information as output.

[1212] By following each step of this system in an orderly manner, the integration of physical stores and e-commerce will be realized, consumer convenience will be improved, and the efforts of sales representatives will be properly recognized.

[1213] (Application example 1)

[1214] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1215] The current system for linking physical stores and online stores has the problem that when a customer tries on an item in a physical store and then purchases it on the online store, the entire process is interrupted. This raises concerns that the motivation of store staff will decrease because customer service in the physical store is not properly evaluated. Another problem is that customers cannot receive real-time recommendations for different products while trying on items in the physical store.

[1216] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1217] In this invention, the server includes: an input unit for inputting fitting information about a user trying on a product in a physical store; a QR code generation unit for generating a QR code based on the fitting information; a provision unit for providing the QR code to the user; a reading unit for reading the QR code when the user purchases the product in a web store; a linking unit for acquiring fitting information from the QR code and linking the acquired fitting information with purchase information; an evaluation update unit for updating staff ratings linked to the fitting information; a point awarding unit for awarding points based on the ratings; a unit for checking product suggestions and fitting status in real time using smart glasses; and a recommendation unit using a generative AI model that analyzes the user's past fitting history and purchase history to recommend products suitable for the user's next visit. This seamlessly links fitting behavior in a physical store with purchasing behavior in the web store, enabling fair recognition of the efforts of physical store staff. Furthermore, real-time product suggestions using smart glasses are expected to improve the quality of customer service and increase sales.

[1218] A "brick and mortar store" is a point of sale that exists in a physical location and allows customers to visit in person to purchase or try on products.

[1219] A "web store" is an online sales outlet operated on the Internet where customers can purchase products via the Internet.

[1220] "Try-on information" is detailed information about the product that the user tried on in the physical store, and includes data such as the product number, the try-on date and time, and the staff ID in charge.

[1221] "Input means" refers to a device or interface used to collect try-on information in a physical store and input it into the system. Examples of such devices include tablets and smartphones.

[1222] The "QR code generating means" refers to a device or method for generating a QR code based on the fitting information.

[1223] "Providing means" refers to a method or device for providing the generated QR code to the user. This includes displaying the QR code on a screen or printing it on paper.

[1224] The "reading means" refers to a method or device that reads the QR code when a user purchases the product from the web store. Specific examples include a smartphone camera or a dedicated input field.

[1225] The "linking means" refers to a method or device for acquiring try-on information from a QR code and associating the acquired try-on information with purchase information.

[1226] The "rating update means" refers to a method or device for updating the staff rating linked to the fitting information.

[1227] The "point awarding means" is a method or device for awarding points based on evaluation.

[1228] "Smart glasses" are wearable devices that users can wear to display and input information in real time.

[1229] A "generative AI model" is an artificial intelligence model that learns from large amounts of data and provides information appropriate to the user.

[1230] A "recommendation means" is a method or device that uses a generative AI model to analyze a user's past try-on history and purchase history and suggest appropriate products.

[1231] This invention relates to a system that links product try-on in a physical store with purchasing behavior in a web store. This system provides comprehensive support for users to try on products in a physical store and then smoothly purchase them in a web store.

[1232] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The store staff member uses a tablet or smartphone to input the user's fitting information. This fitting information includes the product number, the date and time of fitting, and the staff member ID in charge. This input method allows the user's fitting behavior in the physical store to be managed as digital data.

[1233] Next, a QR code is generated based on the input try-on information. The QR code generation means provides the generated QR code to the user. The QR code is displayed on the display of the smart glasses or printed on paper, which the user can take home. The QR code contains the try-on information and is used to access the web store later.

[1234] When a user accesses the web store from home or elsewhere and purchases an item they have tried on, they input the QR code. The QR code is read by a reader and sent to a server. The server obtains the try-on information from the QR code and links it to the purchase information. This linking means allows the system to consistently manage which items the user tried on and which items they purchased.

[1235] Furthermore, the server updates the staff evaluation linked to the fitting information. Using the evaluation update means, the staff is awarded points for assisting with fitting. This point system fairly evaluates the efforts of the staff and provides incentives. In addition, purchase points are awarded to the user, and discounts and benefits that can be used on the next purchase are offered.

[1236] In addition, the system uses smart glasses to allow users to check product suggestions and try-on status in real time. This allows store staff to make appropriate product suggestions based on the user's interests. Furthermore, using a generative AI model, the server analyzes the user's past try-on and purchase history and recommends appropriate products for the next visit. This allows users to enjoy a more personalized shopping experience.

[1237] As a concrete example, a user may try on a red jacket in a physical store and receive a QR code based on the fitting information. The user then takes a photo of this QR code with their smartphone and enters it later when purchasing the red jacket in a web store. The server retrieves the fitting information from the QR code, links it to the purchase information, updates the staff evaluation, and awards points. This series of processes seamlessly links the physical store and the web store, improving the purchasing experience.

[1238] Examples of prompts include:

[1239] "We are imagining a scenario where a user tries on a product using smart glasses in a store. The try-on information includes the user ID, staff ID, and product ID, which are generated as a QR code and provided to the user. When the user then enters the QR code when making a purchase on the web store, the try-on information and purchase information are automatically linked in the system, and the staff member's evaluation points are updated. Please explain this process in detail."

[1240] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1241] Step 1:

[1242] A user visits a physical store and asks a store staff member to try on an item. The store staff member uses a tablet or smartphone to enter the user's fitting information. The input information includes data such as the product number, fitting date and time, and the staff member ID in charge. The fitting information is then recorded on the terminal and sent to a database.

[1243] Input: Product information the user wants to try on, try-on date and time, staff ID

[1244] Output: Digital data including fitting information

[1245] Specific operation: The staff member enters the necessary information into the input form on the terminal and sends it to the database.

[1246] Step 2:

[1247] A QR code is generated based on the entered fitting information. The tablet or smartphone application generates a QR code based on the fitting information and displays it on the device. The generated QR code is then displayed to the user using smart glasses or printed on paper.

[1248] Input: Try-on information

[1249] Output: Generated QR code

[1250] Specific operation: The QR code generation application installed on the device acquires the fitting information and generates a QR code, which is then displayed or printed to the user.

[1251] Step 3:

[1252] The user accesses the web store from home or elsewhere, searches for and selects the product they have tried on, and adds it to their cart. When they proceed to the checkout, the QR code is read into the input field. The user can either scan the QR code with their smartphone camera or manually enter the contents of the QR code.

[1253] Input: QR code

[1254] Output: Try-on information obtained from QR code

[1255] Specific operation: The user scans the QR code with their smartphone and enters the information into the dedicated input field on the web store. The system analyzes the QR code and obtains the fitting information.

[1256] Step 4:

[1257] The server analyzes the try-on information obtained from the QR code and automatically links it to purchase information, thereby recording in a database which products the user tried on and which products they purchased.

[1258] Input: Try-on information and purchase information obtained from the QR code

[1259] Output: Linked try-on information and purchase information

[1260] Specific operation: The server analyzes the QR code data and associates the try-on information with the purchase information in the database.

[1261] Step 5:

[1262] The server updates the evaluation of the store staff linked to the fitting information. The evaluation points of the staff are updated on the system using the evaluation update means. This allows the staff's efforts to be properly evaluated.

[1263] Input: Linked try-on information and purchase information

[1264] Output: Updated staff rating points

[1265] Specific operation: The server updates the reputation points corresponding to the staff ID and saves them in the database.

[1266] Step 6:

[1267] Points are awarded to users and staff using a point awarding means. Users are awarded purchase points that can be used for their next purchase, and staff are awarded fitting assistance points.

[1268] Input: Updated staff rating, purchase completion information

[1269] Output: Points awarded

[1270] Specific operation: The server grants the corresponding points to the user's and staff's accounts.

[1271] Step 7:

[1272] The server uses the generated AI model to analyze the user's purchase history and try-on history, and recommends appropriate products for the user's next visit. The recommended information is displayed to the user through the smart glasses.

[1273] Input: User's purchase history, try-on history

[1274] Output: Product information to recommend to the user

[1275] How it works: The generative AI model analyzes the user's past data, calculates the optimal product, and displays it on the smart glasses.

[1276] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1277] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store, and in particular to a system that combines an emotion engine that recognizes user emotions. This system integrates product try-on in a physical store with purchases in a web store, and by recognizing user emotions, it can evaluate the efforts of store staff and create an environment where users can easily try on products and receive consultations in a physical store.

[1278] First, a user visits a physical store and asks a store staff member for the item they would like to try on. The staff member uses a terminal (tablet or smartphone) to input the try-on information. This try-on information includes the product number, try-on date and time, and the staff member ID in charge. This allows the device to be used as an input means.

[1279] Next, the terminal generates a QR code based on the try-on information. The generated QR code is displayed to the user by the providing means or printed on paper and given to the user. The user can take this QR code home and use it to access the web store at home or elsewhere.

[1280] A user accesses a web store, searches for and selects a product, adds it to their cart, and then enters the QR code. By scanning the QR code into a dedicated input field, they can proceed to the web store's purchasing process. This operation is performed by the scanning means.

[1281] When the QR code is scanned, the server analyzes the information and obtains the try-on information. The analyzed information is automatically linked to the purchase information. Through this linking method, the server links the try-on information from the physical store with the purchase information from the online store.

[1282] The server then updates the store staff's evaluation linked to the fitting information. Using the evaluation update means, the staff's evaluation points are updated on the system. This allows the staff's efforts to be properly evaluated. Points are also awarded based on the evaluation. These points are also awarded to the user and are returned as purchase points. Staff are awarded points for assisting with fitting.

[1283] Furthermore, this system uses an emotion engine. The emotion engine analyzes the user's facial expressions and voice when trying on clothes or making a purchase, and acquires emotional information. This emotional information is recorded in association with the try-on information and purchase information. The server uses this emotional information to further analyze the user's purchasing trends and recommend appropriate products.

[1284] For example, if a user tries on a red jacket in a physical store and smiles or shows satisfaction, the emotion engine recognizes this and adds that emotion information to the try-on information. Later, when the user enters the QR code to purchase the red jacket in a web store, the positive emotion expressed during the try-on is reflected and linked to the purchase information. The server can then analyze this emotion information and recommend products that the user is likely to like the next time they visit.

[1285] The server also reflects emotional information in the evaluation of the staff's customer service. For example, if a user expresses positive emotions toward the staff's customer service while trying on clothes, that evaluation will be added to the staff's evaluation points. This allows the quality of the staff's customer service to be evaluated appropriately.

[1286] In this way, the system of the present invention strengthens the connection between physical stores and online stores, and by utilizing an emotion engine, it is possible to improve the user's purchasing experience and properly evaluate the efforts of store staff.

[1287] The processing flow will be explained below.

[1288] Step 1:

[1289] A user visits a store, browses the shelves, and finds an item they want to try on.

[1290] Step 2:

[1291] The user asks the store staff for the product they want to try on. The user picks up the product they want to try on and tells the staff, "I'd like to try this product on."

[1292] Step 3:

[1293] Product information (product number, model number, etc.) is entered into a terminal (a tablet or smartphone held by store staff), and the staff then enters the product information into a dedicated app on the tablet.

[1294] Step 4:

[1295] The terminal registers fitting information (fitting date and time, user ID (optional), staff ID in charge, etc.). The staff member enters the fitting date and time and their own staff ID, and also the user ID if necessary.

[1296] Step 5:

[1297] The emotion engine recognizes the user's emotions by analyzing emotional information (e.g., satisfaction, dissatisfaction, excitement, etc.) from the user's facial expressions and voice.

[1298] Step 6:

[1299] The device generates a QR code, which is automatically generated based on the fitting information and emotional information.

[1300] Step 7:

[1301] The terminal shows the QR code to the user or prints it out on paper and hands it over to the user. The staff member shows the QR code on the tablet screen to the user or prints it out on paper and hands it over to the user.

[1302] Step 8:

[1303] A user accesses the web store from their home or other location. A user accesses the web store using a computer or smartphone.

[1304] Step 9:

[1305] A user searches for a product on a web store, tries on the product, and adds it to their cart. A user searches for a red jacket, checks the product detail page, and adds it to their cart.

[1306] Step 10:

[1307] The user scans the QR code into the input field. During the purchase process, the user scans the QR code received at the store into the dedicated QR code input field, or enters the displayed information.

[1308] Step 11:

[1309] The server analyzes the QR code information and obtains the fitting information and emotion information. Information such as the product ID, fitting date and time, staff ID, and emotion information is analyzed from the QR code.

[1310] Step 12:

[1311] The server links purchase information from the web store with try-on information and emotional information from the physical store. It matches the purchase data with try-on data and emotional data, and records which staff member assisted in trying on which products.

[1312] Step 13:

[1313] The server updates the evaluation of the store staff based on the purchase data. The evaluation points of the staff who assisted with fitting are updated in the system. Evaluation points based on emotional information are also added.

[1314] Step 14:

[1315] The server grants the user points for the purchase. Points corresponding to the purchase amount are added to the user's account.

[1316] Step 15:

[1317] The server also awards points to store staff. Points are also awarded to staff in the system based on emotional information for purchases associated with fitting assistance.

[1318] Step 16:

[1319] The server uses AI to analyze the user's purchase history, try-on history, emotional information, and browsing history.

[1320] Step 17:

[1321] The server recommends the most suitable products to the user. Highly relevant products are displayed on the web store based on the user's interests, purchasing tendencies, and emotional information.

[1322] Step 18:

[1323] The server provides store staff with product suggestion information based on user attributes. Information useful for the next suggestion (recommended products, similar items, etc.) is provided to the store staff's terminal.

[1324] This will strengthen the connection between physical stores and online stores, improve the user's purchasing experience by utilizing the emotion engine, and create a system that properly evaluates the efforts of store staff.

[1325] Example 2

[1326] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1327] In today's commercial environment, there is a lack of coordination between physical and virtual stores, which makes it difficult for users to purchase products they have tried in a physical store in a virtual store. Furthermore, the lack of proper reflection of users' emotions and evaluations of staff service makes it difficult to improve the quality of service.

[1328] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1329] In this invention, the server includes: an input means for inputting trial information about a user who tries out a product at a real store; a two-dimensional code generation means for generating a two-dimensional code based on the trial information; a provision means for providing the two-dimensional code to the user; a reading means for reading the two-dimensional code when the user purchases the product at a virtual store; a linking means for acquiring trial information from the two-dimensional code and linking the acquired trial information with purchase information; an evaluation update means for updating an evaluation of a staff member linked to the trial information; a score assignment means for assigning a score based on the evaluation; an emotion analysis means for analyzing the user's emotions; a means for associating emotion information acquired from the emotion analysis means with the trial information and purchase information; and a recommendation means for recommending products based on the emotion information and trial information. This strengthens the connection between the real store and the virtual store, improves the user's purchasing experience, and enables the customer service efforts of staff to be appropriately evaluated.

[1330] "Brick and mortar store" means a commercial establishment that customers can physically visit.

[1331] "Virtual store" refers to an online store that sells products over the Internet.

[1332] "User" refers to an individual who uses a physical or virtual store to try out or purchase a product.

[1333] "Trial information" is information about the user when they try out a product, and includes the product number, trial date and time, and person in charge ID.

[1334] "Input means" refers to a device or system for inputting trial information at a physical store.

[1335] "Two-dimensional code" refers to a code that contains visual information, such as a QR code.

[1336] The "two-dimensional code generating means" refers to a system or software for generating a two-dimensional code based on input trial information.

[1337] "Providing means" refers to a method for providing the generated two-dimensional code to the user.

[1338] The "reading means" refers to the method used by the user to read the two-dimensional code when making a purchase at the virtual store.

[1339] The "linking means" refers to a method for linking the trial information obtained from the two-dimensional code with the purchase information at the virtual store.

[1340] The "evaluation update means" refers to a function for updating the evaluation of the person in charge on the system based on the trial information.

[1341] The "point giving means" refers to a method for giving points to users and personnel based on the evaluation.

[1342] "Emotion analysis means" refers to a device or software for analyzing a user's facial expressions and voice and acquiring emotional information.

[1343] The "recommendation means" refers to a method for recommending appropriate products to users based on trial information and emotion information.

[1344] MODE FOR CARRYING OUT THE INVENTION

[1345] This invention is a system for improving the quality of service by linking the sales activities of real stores and virtual stores and analyzing user emotions. This system includes the following components.

[1346] Hardware and software used

[1347] Input method: A device such as a tablet, smartphone, or PC is used. A dedicated application is installed on these devices, providing an interface for entering trial information.

[1348] Two-dimensional code generation means: Application software on the terminal generates a two-dimensional code (such as a QR code) based on the trial information.

[1349] Means of presentation: The generated two-dimensional code is displayed on the terminal screen, or a printer is used to print it on paper.

[1350] Reading method: Using a QR code reader built into the virtual store's website or application.

[1351] Linking means: A database management system running on the server side links the trial information with the purchase information.

[1352] Evaluation update means, score assignment means: The evaluation management system that operates on the server side updates the evaluation of the person in charge and assigns a score.

[1353] Emotion analysis tools: Emotion analysis software is used to analyze the user's facial expressions and voice. This data is collected through cameras and microphones in physical stores and webcams and microphones in virtual stores.

[1354] Recommendation means: A recommendation engine running on the server recommends products based on trial information and emotion information.

[1355] Specific methods for data processing and calculation

[1356] Entering trial information: When a user picks up a product in a physical store and tries it out, a staff member uses a dedicated app on a tablet or smartphone to enter the trial information (product number, trial date and time, staff ID, etc.).

[1357] QR Code Generation: After the trial information is entered, the application software generates a 2D code based on that information, which is then displayed on the terminal screen or printed and handed to the user.

[1358] Purchase from the web store: The user accesses the virtual store from home, searches for and selects the product they have tried, and adds it to their cart. They then scan the aforementioned QR code into the dedicated input field to complete the purchase.

[1359] Data analysis and linking: The server analyzes the QR code and obtains trial information. This information is automatically linked to purchase information in the virtual store. This allows for consistent management of trial information in the physical store and purchase information in the virtual store.

[1360] Rating and score update: The server automatically updates the staff's rating based on the linked trial information and assigns appropriate scores. Purchase points are also awarded to the user.

[1361] Emotion analysis: Emotion analysis software analyzes the user's facial expressions and voice during trial and purchase to obtain emotional information, which is then associated with trial and purchase information.

[1362] Providing recommendations: The server analyzes the emotional and trial information to understand the user's purchasing tendencies, and based on this, it can recommend appropriate products for the next visit.

[1363] Specific examples

[1364] For example, if a user tries on a red jacket in a physical store and smiles or shows satisfaction, the emotion analysis software recognizes this and adds positive emotional information to the trial information. Later, when the user enters the QR code to purchase a red jacket in a virtual store, the emotional information from the trial is linked to the purchase information. This allows the software to recommend products the user is likely to like the next time they visit. The software also updates the customer service evaluation of staff based on their positive emotions, and assigns appropriate scores.

[1365] Prompt Sentence Examples

[1366] "A user tries on a red jacket in a physical store and smiles and expresses satisfaction. The emotion analysis software recognizes this and adds the emotion information to the trial information. The user then enters a QR code when purchasing the red jacket in a virtual store. Please explain how the system handles this."

[1367] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1368] Step 1:

[1369] Users try out products in-store.

[1370] Specific operation: A user visits a physical store and tries on a product (e.g., a red jacket). During the trial, a staff member checks the user's facial expression and voice and provides a guide for the trial.

[1371] Step 2:

[1372] Use the terminal to enter your trial information.

[1373] Specific operation: The staff member uses a dedicated application on a tablet or smartphone to enter trial information (product number, trial date and time, staff ID, etc.).

[1374] Input: Product number (e.g., 12345), trial date and time (e.g., 2023-10-01 14:00), staff ID (e.g., 567)

[1375] Output: The entered trial information is saved on the device.

[1376] Step 3:

[1377] The terminal generates a two-dimensional code based on the trial information.

[1378] Specific operation: The terminal application generates a two-dimensional code (QR code) based on the entered trial information.

[1379] Input: Trial information (product number, trial date and time, staff ID)

[1380] Output: The generated 2D code is displayed on the terminal.

[1381] Step 4:

[1382] The two-dimensional code is provided to the user through the providing means.

[1383] Specific operation: Show the 2D code displayed on the terminal screen to the user, or print it out on paper using a printer and hand it to the user.

[1384] Input: Generated 2D code

[1385] Output: 2D code provided by the user

[1386] Step 5:

[1387] A user accesses a virtual store and purchases a product using a two-dimensional code.

[1388] Specific operation: The user accesses a virtual store (e.g., a website) from home, searches for and selects the product they want to try (e.g., a red jacket), adds it to their cart, and then enters the QR code in a dedicated input field.

[1389] Input: The two-dimensional code that the user enters in the virtual store

[1390] Output: Proceed with the purchase in the virtual store.

[1391] Step 6:

[1392] The server analyzes the two-dimensional code and obtains the trial information.

[1393] Specific operation: The server analyzes the two-dimensional code sent from the virtual store and obtains the trial information (product number, trial date and time, staff ID).

[1394] Input: 2D code

[1395] Output: Parsed trial information

[1396] Step 7:

[1397] The server links the trial information with the purchase information.

[1398] Specific operation: The server links the acquired trial information with the purchase information at the virtual store in the database, thereby confirming that the trialed product was purchased.

[1399] Input: Trial information, purchase information

[1400] Output: Linked trial and purchase information

[1401] Step 8:

[1402] Update staff ratings using the rating update means.

[1403] Specific operation: The server updates the staff evaluation on the system based on the trial information. Evaluation points are automatically calculated and updated.

[1404] Input: Trial information (Staff ID)

[1405] Output: Updated staff rating points

[1406] Step 9:

[1407] A point giving means is used to give points to users and staff members.

[1408] Specific operation: The server assigns purchase points to the user and trial support points to the staff based on the trial information and evaluation.

[1409] Input: Rating points

[1410] Output: Purchase number for user, trial support number for staff

[1411] Step 10:

[1412] Emotion analysis means is used to obtain user emotion information.

[1413] How it works: Emotion analysis software analyzes the user's facial expressions and voice during trial and purchase to obtain emotional information.

[1414] Input: User's facial expressions and voice data

[1415] Output: Parsed emotion information

[1416] Step 11:

[1417] Associate sentiment information with trial and purchase information.

[1418] Specific operation: The server associates the emotion information with the trial information and purchase information and stores the information in a database.

[1419] Input: Emotion information, trial information, purchase information

[1420] Output: Associated emotion information and trial / purchase information

[1421] Step 12:

[1422] To recommend appropriate products to a user using a recommendation means.

[1423] Specific operation: The server analyzes the emotional information and trial information, understands the user's purchasing tendencies, and recommends appropriate products the next time the user visits the virtual store.

[1424] Input: Emotion information, trial information

[1425] Output: Products recommended to the user

[1426] (Application example 2)

[1427] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1428] Conventional systems linking physical stores and online stores only link try-on information from the physical store with purchase information from the online store, and do not take into account the user's emotions or purchasing trends. This makes it difficult to recommend the most suitable products to the user, and also lacks a mechanism for properly evaluating the customer service quality of staff. The purpose of this invention is to solve these problems.

[1429] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: an input means for inputting try-on information about a user trying on a product in a physical store; a QR code generation means for generating a QR code based on the try-on information; a provision means for providing the QR code to the user; a reading means for reading the QR code when the user purchases the product in the web store; a linking means for acquiring try-on information from the QR code and linking the acquired try-on information with purchase information; an evaluation update means for updating the evaluation of the staff member linked to the try-on information; a point awarding means for awarding points based on the evaluation; an emotion recognition means for recognizing the user's emotions when trying on products and when purchasing; and a recommendation means for analyzing the user's purchasing tendencies using the emotion information acquired by the emotion recognition means and recommending products. This enables optimal product recommendations that take into account the user's emotions and purchasing tendencies, and an appropriate evaluation of the staff member's customer service quality.

[1430] A "physical store" is a store that exists in a physical location and that consumers can visit in person to purchase goods or receive services.

[1431] "Webstore" means a website or platform that exists on the Internet and allows consumers to access and purchase products online.

[1432] "Try-on information" refers to information when a user tries on a product in a physical store, and includes data such as the product number, the try-on date and time, and the staff ID in charge.

[1433] A "QR code" is a type of two-dimensional barcode, a matrix code that can encode and store information.

[1434] "Emotion recognition means" refers to a technology or device that analyzes emotions from the user's facial expressions, voice, etc., and recognizes a specific emotional state.

[1435] "Purchase information" refers to information when a user purchases a product from a web store, and includes data such as the product name, purchase date and time, and purchase quantity.

[1436] The "point awarding means" is a method or system for awarding points to users or staff members based on evaluations.

[1437] The "linking means" is a method or system that links the try-on information obtained from the QR code with the purchase information at the web store.

[1438] A "recommendation means" is a technology or device that recommends appropriate products to a user based on the user's purchasing tendencies and emotional information.

[1439] This invention relates to a system that links sales behavior in a physical store with purchasing behavior in a web store, and in particular utilizes an emotion engine that recognizes user emotions. This system is realized by a process in which a user tries on products in a physical store, enters the try-on information into a terminal, and generates a QR code.

[1440] Hardware and software used

[1441] Hardware: Smartphones, tablets, web cameras

[1442] Software: Azure Cognitive Services Emotion API, QR code generation API, Firebase (real-time database)

[1443] Processing flow

[1444] First, the store staff enters the user's fitting information via a smartphone or tablet. This fitting information includes the product number, fitting date and time, and the staff ID in charge. Next, a QR code based on the fitting information is generated using a QR code generation API and provided to the user. The QR code is displayed on the smartphone screen or printed out on paper and given to the user.

[1445] The user takes the QR code home and enters it later when purchasing the product from the web store. The QR code is analyzed using a QR code reading API, and try-on information is obtained. Based on this information, the server analyzes the user's emotions using emotion recognition means (Azure Cognitive Services Emotion API) and obtains the emotional information shown when trying on the product or purchasing it. This emotional information is associated with the try-on information and purchase information and stored in Firebase.

[1446] The purchasing information associated with the emotional information is analyzed by the recommendation means, and appropriate products are recommended for the user's next visit based on the user's purchasing trends. Furthermore, the server updates the staff evaluation to reflect the emotional information at the time of trying on clothes and purchasing, and points are awarded based on the results. This allows the staff's customer service quality to be properly evaluated, leading to improved service for users.

[1447] Specific examples

[1448] For example, if User A tries on a blue dress in a physical store and shows a satisfied expression while trying it on, the emotion recognition API will identify this expression as positive. Later, if User A adds the same dress to her cart in the online store and enters the QR code, the positive emotion data will be referenced and the staff will give it a high rating. Furthermore, the next time User A visits, new blue products will be recommended.

[1449] Prompt Sentence Examples

[1450] "Please tell me the algorithm that allows a user to try on clothes in a physical store, collect the emotions displayed while trying them on using a camera and microphone, and generate and encode a QR code. Please provide specific emotion recognition code and QR code generation code, and explain how to save the try-on information and emotion data in Firebase."

[1451] In this way, the connection between physical stores and online stores is strengthened, and by utilizing the emotion engine, it is possible to improve the user's purchasing experience and properly evaluate the efforts of staff.

[1452] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1453] Step 1:

[1454] Store staff enter the user's fitting information on a smartphone or tablet. The input information includes the product number, fitting date and time, and the staff ID in charge. This information is uploaded to the Firebase real-time database. The input is the user's fitting information, and the output is the fitting information stored in Firebase.

[1455] Step 2:

[1456] A QR code is generated using a QR code generation API based on the try-on information entered on the device. The generated QR code is displayed on the smartphone screen or printed on paper and given to the user. The input is the try-on information, and the output is the QR code provided to the user.

[1457] Step 3:

[1458] A user accesses a web store and adds a product to their cart. At this time, the QR code is read by the device and analyzed by the QR code reading API. Try-on information is obtained from the QR code, and Firebase is accessed to read the try-on information. The QR code is the input, and the analyzed try-on information is obtained as the output.

[1459] Step 4:

[1460] The Azure Cognitive Services Emotion API is used as an emotion recognition method to analyze the user's emotions when trying on clothes and purchasing. Data collected through the camera and microphone is sent to the API, and emotional information is obtained as the analysis result. The obtained emotional information is stored in Firebase. The input is the user's emotional data, and the output is analyzed emotional information.

[1461] Step 5:

[1462] The server links the try-on information obtained from the QR code with the purchase information from the web store. This linking allows for centralized management of purchase information along with related emotional information. The inputs are try-on information and purchase information, and the output is the linked purchase information.

[1463] Step 6:

[1464] The rating update means updates the staff rating linked to the fitting information. If positive emotional information is reflected, the staff rating points increase. This rating is saved in Firebase. The inputs are emotional information and fitting information, and the output is the updated staff rating points.

[1465] Step 7:

[1466] As a recommendation method, the server recommends appropriate products based on the user's purchasing trends and emotional information. It analyzes purchasing history and emotional data to determine which products to recommend to the user on their next visit. The input is the user's purchasing trends data, and the output is the recommended products.

[1467] In this way, each process step improves and improves service for both users and staff.

[1468] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1469] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1470] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1471] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1472] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1473] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1474] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1475] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1476] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1477] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1478] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1479] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1480] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1481] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1482] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1483] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1484] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1485] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1486] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1487] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1488] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1489] The following is further disclosed regarding the above embodiment.

[1490] (Claim 1)

[1491] A system for linking sales behavior in physical stores with purchasing behavior in web stores,

[1492] an input means for inputting try-on information about a user who tries on a product in a physical store;

[1493] a QR code generating means for generating a QR code based on the fitting information;

[1494] providing means for providing the QR code to a user;

[1495] a reading means for reading the QR code when a user purchases the product at the web store;

[1496] a linking means for acquiring try-on information from the QR code and linking the acquired try-on information with purchase information;

[1497] evaluation update means for updating the staff evaluation linked to the fitting information;

[1498] point awarding means for awarding points based on the evaluation;

[1499] A system including:

[1500] (Claim 2)

[1501] 2. The system according to claim 1, wherein the point giving means includes a point giving means for giving purchase points to the user, and a point giving means for giving fitting assistance points to the staff.

[1502] (Claim 3)

[1503] 2. The system according to claim 1, wherein the evaluation update means calculates staff evaluations based on try-on information at the physical store and purchase information at the web store, and records the evaluations in a staff evaluation database.

[1504] "Example 1"

[1505] (Claim 1)

[1506] A system for linking sales behavior in physical stores with purchasing behavior in e-commerce,

[1507] an input means for inputting try-on information about a consumer who tries on a product in a physical store;

[1508] a QR code generating means for generating a QR code based on the fitting information;

[1509] A providing means for providing the QR code to a consumer;

[1510] a reading means for reading the QR code when a consumer purchases the product through electronic commerce;

[1511] a linking means for acquiring try-on information from the QR code and linking the acquired try-on information with purchase information;

[1512] evaluation update means for updating the evaluation of the sales representative linked to the fitting information;

[1513] a score assigning means for assigning a score based on the evaluation;

[1514] A recommendation tool that analyzes habits and recommends appropriate products to consumers;

[1515] A means for providing product proposal information based on consumer attributes to sales representatives;

[1516] A system including:

[1517] (Claim 2)

[1518] 2. The system according to claim 1, wherein the score awarding means includes a point awarding means for awarding purchase points to the consumer and a point awarding means for awarding fitting assistance points to the salesperson.

[1519] (Claim 3)

[1520] 2. The system according to claim 1, wherein the evaluation update means calculates an evaluation of the salesperson based on try-on information at the physical store and purchase information through e-commerce, and records the evaluation in the salesperson evaluation database.

[1521] "Application Example 1"

[1522] (Claim 1)

[1523] A system for linking sales behavior in physical stores with purchasing behavior in web stores,

[1524] an input means for inputting try-on information about a user who tries on a product in a physical store;

[1525] a QR code generating means for generating a QR code based on the fitting information;

[1526] providing means for providing the QR code to a user;

[1527] a reading means for reading the QR code when a user purchases the product at the web store;

[1528] a linking means for acquiring try-on information from the QR code and linking the acquired try-on information with purchase information;

[1529] evaluation update means for updating the staff evaluation linked to the fitting information;

[1530] point awarding means for awarding points based on the evaluation;

[1531] A means of checking product suggestions and fitting situations in real time using smart glasses;

[1532] A recommendation method using a generative AI model that analyzes a user's past try-on and purchase history to recommend suitable products for the next visit;

[1533] A system including:

[1534] (Claim 2)

[1535] 2. The system according to claim 1, wherein the point giving means includes a point giving means for giving purchase points to the user, and a point giving means for giving fitting assistance points to the staff.

[1536] (Claim 3)

[1537] 2. The system according to claim 1, wherein the evaluation update means calculates staff evaluations based on try-on information at the physical store and purchase information at the web store, and records the evaluations in a staff evaluation database.

[1538] "Example 2: Combining Emotion Engines"

[1539] (Claim 1)

[1540] A system for linking sales behavior in a real store with purchase behavior in a virtual store,

[1541] an input means for inputting trial information relating to a user who tries out a product in a physical store;

[1542] a two-dimensional code generating means for generating a two-dimensional code based on the trial information;

[1543] providing means for providing the two-dimensional code to a user;

[1544] a reading means for reading the two-dimensional code when a user purchases the product in the virtual store;

[1545] a linking means for acquiring trial information from the two-dimensional code and linking the acquired trial information with purchase information;

[1546] an evaluation update means for updating an evaluation of a person in charge linked to the trial information;

[1547] a score assigning means for assigning a score based on the evaluation;

[1548] emotion analysis means for analyzing the emotions of a user;

[1549] means for associating the emotion information acquired from the emotion analysis means with trial information and purchase information;

[1550] a recommendation means for recommending a product based on the emotion information and trial information;

[1551] A system including:

[1552] (Claim 2)

[1553] 2. The system according to claim 1, wherein the point awarding means includes a point awarding means for awarding purchase points to the user, and a point awarding means for awarding trial assistance points to the person in charge.

[1554] (Claim 3)

[1555] 2. The system according to claim 1, wherein the evaluation update means calculates the evaluation of the salesperson based on trial information in the physical store and purchase information in the virtual store, and records the evaluation in the salesperson evaluation database.

[1556] "Application example 2 when combining emotion engines"

[1557] (Claim 1)

[1558] A system for linking sales behavior in physical stores with purchasing behavior in...

Claims

1. A system for linking sales behavior in physical stores with purchasing behavior in web stores, an input means for inputting try-on information about a user who tries on a product in a physical store; a QR code generating means for generating a QR code based on the fitting information; providing means for providing the QR code to a user; a reading means for reading the QR code when a user purchases the product at the web store; a linking means for acquiring try-on information from the QR code and linking the acquired try-on information with purchase information; evaluation update means for updating the staff evaluation linked to the fitting information; point awarding means for awarding points based on the evaluation; A system including:

2. 2. The system according to claim 1, wherein the point-giving means includes a point-giving means for giving purchase points to the user and a point-giving means for giving fitting assistance points to the staff.

3. The system according to claim 1, wherein the evaluation update means calculates staff evaluations based on try-on information at the physical store and purchase information at the web store, and records the evaluations in a staff evaluation database.

Images