System

Abstract

A system is provided.SOLUTION: A system includes a means for acquiring line-of-sight data, a means for acquiring sales data, a means for analyzing the line-of-sight data and the sales data and simulating an optimum arrangement for each commodity, and a means for providing a recommended layout based on the optimum arrangement.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In retail stores, inventory and display space are limited, so it is necessary to efficiently sell products that have the potential to sell. However, conventional sales management systems can only collect data on whether a product was sold, and are unable to obtain information on product visibility or the effectiveness of product placement. Furthermore, rearranging products for each store is labor-intensive and inefficient. To solve these issues, a system is needed that can comprehensively analyze customer gaze data and sales data and automatically propose optimal product placement. [Means for solving the problem]

[0005] This invention provides a system that includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data to simulate the optimal placement of each product, and a means for providing a recommended layout based on the optimal placement. This system can automatically analyze the optimal placement of products based on the gaze trends of customers in a store and sales data, and provide a recommended layout to the manager. This improves sales efficiency and reduces the effort required for store management.

[0006] "Gaze data" is data that indicates information about a customer's gaze directed at a product or a specific area in a store.

[0007] "Sales data" refers to data that records the sales performance of each product over a certain period of time.

[0008] "Simulation" is a process that predicts the optimal placement of products using gaze data and sales data.

[0009] A "recommended layout" is a proposed in-store layout plan for optimal product placement based on the results of the simulation.

[0010] "Analysis" is the process of integrating gaze data and sales data to evaluate the effectiveness of product placement. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0012] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0013] First, the terms used in the following description will be explained.

[0014] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0015] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0016] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0017] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0019] [First embodiment]

[0020] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0021] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0024] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0027] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0031] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0032] ---

[0033] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, and store interior data. Specifically, the system includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data to simulate optimal placement for each product, and a means for providing a recommended layout based on the optimal placement.

[0034] System configuration

[0035] This system consists of the following main elements:

[0036] 1. Method of acquiring gaze data:

[0037] Device: In-store cameras capture customer gaze data in real time, allowing us to measure which products and areas attract customers' attention.

[0038] 2. Sales data acquisition method:

[0039] Terminal: The POS system acquires sales data for each product and periodically sends it to the server.

[0040] 3. Data analysis methods:

[0041] Server: Analyzes the collected gaze data and sales data. Based on the information obtained from the gaze data, it identifies products and areas that customers pay attention to, and integrates this with sales data to evaluate the effectiveness of product placement.

[0042] 4. Simulation Method:

[0043] Server: Utilizing AI models, it analyzes gaze data and sales data to simulate optimal product placement. Based on this simulation, it maximizes product visibility and placement effectiveness.

[0044] 5. Recommended layout methods:

[0045] Server: Calculates the recommended placement from the simulation results and notifies the store manager's terminal.

[0046] Program processing

[0047] This system is implemented through the following series of processes.

[0048] Data collection

[0049] server

[0050] Receives real-time data from in-store cameras and POS systems.

[0051] The received gaze data and sales data are stored in a database.

[0052] Terminal

[0053] A camera is placed to acquire gaze data and data is collected continuously.

[0054] Sales data from the POS system is periodically sent to the server.

[0055] Data analysis and simulation

[0056] server

[0057] Integrate collected gaze data with sales data to identify areas of customer interest.

[0058] Using an AI model, we analyze the correlation between the number of views and sales.

[0059] Simulate optimal product placement.

[0060] The recommended layout is sent to the administrator's device.

[0061] Implementing the recommended layout

[0062] User

[0063] Store staff adjust product placement based on the recommended layout.

[0064] Review gaze and sales data and make further adjustments as needed.

[0065] Specific examples

[0066] As an example, consider the case where new product A is placed in a store.

[0067] Terminal

[0068] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[0069] The POS system sends sales data for new product A to the server.

[0070] server

[0071] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[0072] By integrating sales data and gaze data, we simulate which area new product A should be placed in to maximize its sales.

[0073] The recommended layout is notified to the administrator's device, and an example is shown: "Moving to the shelf near the entrance will increase the number of views by 45%."

[0074] User

[0075] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[0076] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[0077] In this way, gaze data and sales data can be effectively utilized to achieve optimal product placement.

[0078] The processing flow will be explained below.

[0079] Step 1:

[0080] Terminal

[0081] Cameras are installed in appropriate locations in the store to capture customer gaze data in real time.

[0082] The gaze data collected by the camera is sent to a server.

[0083] Sales data for each product is periodically extracted from the POS system and sent to the server.

[0084] Step 2:

[0085] server

[0086] The received gaze data is stored in a database.

[0087] Store sales data received from the POS system in a database.

[0088] Store interior data is stored in a database and any updates are reflected.

[0089] Step 3:

[0090] server

[0091] Preprocessing of gaze data and sales data is performed.

[0092] Synchronize gaze data and sales data based on timestamps.

[0093] Link gaze data and sales data based on product ID.

[0094] Step 4:

[0095] server

[0096] Using an AI model, we analyze the correlation between the number of customer gazes and sales figures.

[0097] Quantify the correlation between the number of views and sales of a specific product.

[0098] Identify customer interest areas using gaze data.

[0099] Step 5:

[0100] server

[0101] Based on an AI model, gaze data and sales data are analyzed to simulate optimal product placement.

[0102] Calculate the optimal combination of view counts and sales for each product.

[0103] It works in conjunction with the store floor map to generate recommended layouts.

[0104] Step 6:

[0105] server

[0106] Create a recommended layout based on the simulation results.

[0107] The optimal product placement is illustrated and sent to the administrator's terminal.

[0108] Step 7:

[0109] Terminal

[0110] Review the recommended layout provided by your administrator.

[0111] Instruct store staff to reposition products to recommended locations.

[0112] Step 8:

[0113] User

[0114] Store staff arranges products based on the recommended layout.

[0115] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[0116] Example 1

[0117] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0118] Conventional store layouts lack a means to comprehensively analyze customer gaze data and sales data and simulate optimal product placement. This has led to the problem that product placement effects cannot be maximized and sales cannot be expected to increase.

[0119] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0120] In this invention, the server includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data and simulating the optimal placement for each product, a means for analyzing the correlation between the gaze data and sales data using a generative AI model, and a means for providing a recommended layout based on the optimal placement. This makes it possible to effectively utilize the gaze data and sales data, maximize the effectiveness of product placement, and increase sales.

[0121] "Gaze data" is data that indicates in which direction and at which products customers are looking in a store.

[0122] "Sales data" is data that records the sales status of each product, and includes the product name, unit price, number of units sold, date and time of sale, etc.

[0123] A "generative AI model" is an artificial intelligence model used to analyze the correlation between gaze data and sales data and optimize product placement.

[0124] A "recommended layout" is a layout that shows the optimal way to arrange products, calculated based on gaze data and sales data.

[0125] The "in-store camera" is a camera device installed in the store to acquire customer line-of-sight data.

[0126] A "sales management system" is a system that manages the sales situation of a store and records and transmits sales data for each product.

[0127] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, and store interior data. Specifically, the system includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data to simulate optimal placement for each product, and a means for providing a recommended layout based on the optimal placement.

[0128] The system consists of the following main elements:

[0129] Eye gaze data acquisition method

[0130] Terminal

[0131] Cameras installed in the store capture customer gaze data in real time, making it possible to measure which products and areas attract customers' attention.

[0132] Sales data acquisition method

[0133] Terminal

[0134] The sales management system acquires sales data for each product and periodically sends it to the server. The sales data includes the product name, unit price, number of units sold, and the date and time of the sale.

[0135] Data Analysis Methods

[0136] server

[0137] The server analyzes the collected gaze data and sales data. Based on the information obtained from the gaze data, it identifies products and areas that customers are paying attention to, and integrates this with sales data to evaluate the effectiveness of product arrangements.

[0138] Simulation Method

[0139] server

[0140] Using a generative AI model, it analyzes gaze data and sales data to simulate the optimal placement of each product, thereby maximizing product visibility and placement effectiveness.

[0141] Layout recommendation method

[0142] server

[0143] The system calculates the recommended placement from the simulation results and notifies the store manager's terminal, including the reasons for the recommended placement and specific numerical results.

[0144] Specific examples

[0145] As a specific example, consider the case where new product A is placed in a store.

[0146] Terminal

[0147] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[0148] The sales management system sends sales data for new product A to the server.

[0149] server

[0150] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[0151] By integrating sales data and gaze data, we simulate which area new product A should be placed in to maximize its sales.

[0152] The recommended layout is notified to the administrator's device, and an example layout is presented that suggests, "Moving shelves closer to the entrance will increase the number of views by 45% and increase sales by 20%."

[0153] User

[0154] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[0155] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[0156] Example prompts to input to the generative AI model

[0157] "Please simulate the impact on the number of glances and sales if new product A is moved to a shelf near the entrance."

[0158] This allows for the effective use of gaze data and sales data to achieve optimal product placement.

[0159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0160] Step 1:

[0161] Data collection

[0162] Terminal

[0163] In-store cameras installed on the terminals capture customer gaze data in real time. The cameras track customer eye movements and record which areas and products their gazes are focused on. The input is the camera's video data, and the output is each customer's gaze data.

[0164] The sales management system acquires sales data for each product in real time and sends it to the server at regular intervals. The input is sales records and the output is sales data.

[0165] server

[0166] The server stores the gaze data and sales data sent from the in-store cameras and the sales management system in a database.

[0167] Step 2:

[0168] Data Integration

[0169] server

[0170] Gaze data and sales data are retrieved from the database and integrated. The gaze data and sales data are aligned on the same time axis to analyze the level of interest customers show in products and how this affects sales. The input is gaze data and sales data, and the output is an integrated dataset.

[0171] Specific operations include converting the data format and complementing missing data.

[0172] Step 3:

[0173] Correlation analysis

[0174] server

[0175] The server uses a generative AI model to analyze the correlation between gaze counts and sales numbers. It applies machine learning algorithms to find patterns and trends between gaze data and sales data. The input is the integrated dataset, and the output is the result of the correlation analysis.

[0176] Specific operations include preprocessing, feature extraction, and model application.

[0177] Step 4:

[0178] Simulation of optimal layout

[0179] server

[0180] The server simulates the optimal placement for each product based on the results of the correlation analysis. It uses an AI model to evaluate various placement scenarios based on gaze data and sales data to identify the most effective placement. The input is the result of the correlation analysis, and the output is the simulation result of the optimal placement.

[0181] Specifically, multiple scenarios are generated and evaluated.

[0182] Step 5:

[0183] Providing recommended layouts

[0184] server

[0185] The system calculates the recommended layout from the simulation results and notifies the store manager's terminal. The notification includes the reasons for the recommended layout and specific numerical results. The input is the simulation result of the optimal layout, and the output is a notification of the recommended layout.

[0186] Step 6:

[0187] Implementing the recommended layout

[0188] User

[0189] Store staff adjust product placement based on the recommended layout they receive. For example, move new product A to a shelf closer to the store entrance. The input is the recommended layout notification, and the output is the new in-store layout.

[0190] Specific operations include moving and rearranging product shelves.

[0191] Step 7:

[0192] Monitoring the effectiveness

[0193] Terminal

[0194] Gaze data and sales data are collected again to continuously monitor the effectiveness of the new placement. Data for improvement is collected. The input is the gaze data and sales data after the change, and the output is data to evaluate the effectiveness.

[0195] By following the above steps, it is possible to effectively utilize gaze data and sales data to achieve optimal product placement.

[0196] (Application example 1)

[0197] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0198] Conventional methods make it difficult to efficiently and effectively optimize product placement in stores. In particular, the lack of a system that collects customer behavior and gaze data in real time and integrates and analyzes it with sales data makes it difficult to quickly evaluate the effectiveness of product placement and provide an optimal layout. As a result, stores are often slow to make appropriate decisions to maximize sales.

[0199] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0200] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for analyzing the gaze data and sales data and simulating optimal product placement, means for providing a recommended layout based on the optimal placement, means for collecting gaze data in real time using a smart device, means for performing data analysis and simulation using an AI model, and means for displaying the recommended layout on the interface of the smart device in real time. This makes it possible to effectively analyze customer behavior data and sales data and quickly and appropriately achieve optimal product placement.

[0201] "Gaze data" is data that measures a customer's eye movements and points of gaze and records them in digital format.

[0202] "Sales data" refers to data including sales information, sales quantity, sales date and time, etc., for each product.

[0203] "Simulation" is the process of recreating virtual environments and situations based on collected data and attempting optimal placement and operation.

[0204] The "recommended layout" is the optimal product placement pattern derived based on the analysis and simulation results.

[0205] "Smart devices" is a general term for wearable devices and mobile terminals that can connect to the Internet and are capable of advanced calculations and analysis.

[0206] An "AI model" is a collection of algorithms and programs designed to solve a specific problem using artificial intelligence technology.

[0207] An "interface" is a means or mechanism by which systems, devices, and software exchange information with each other.

[0208] This invention relates to a system that performs an integrated analysis of customer gaze data, sales data, and store interior data to automatically simulate optimal product placement. To specifically implement this system, the following hardware and software are used.

[0209] Hardware and software used

[0210] 1. Hardware

[0211] Smart Device: A device used to collect customer gaze data in real time. A specific example is smart glasses (e.g., Google Glass).

[0212] Server: The main hardware used to analyze large amounts of data and simulate optimal product placement.

[0213] POS system: A device that collects product sales data and sends it to a server.

[0214] 2. Software

[0215] Data Analysis Platform: Software used to analyze gaze data and sales data, specifically using Python and TensorFlow.

[0216] Smart Device Interface: An interface for displaying recommended layouts on smart glasses, specifically using the Android Wear platform.

[0217] Explanation of program processing

[0218] 1. Data Collection

[0219] Terminal: The smart glasses constantly collect customer gaze data and send it to the server in real time. The POS system periodically sends sales data for each product to the server.

[0220] Server: The server receives the gaze data and sales data and stores them in a database.

[0221] 2. Data analysis and simulation

[0222] Server: Analyzes collected gaze data and sales data using an AI model using Python and TensorFlow. Through this analysis, areas of customer interest are identified and the correlation between gaze counts and sales is evaluated. Based on this, optimal product placement is simulated.

[0223] 3. Display recommended layout

[0224] Server: Calculates the recommended layout based on the simulation results and sends it to the smart device interface in real time.

[0225] Terminal: A recommended layout is displayed on the smart glasses interface, and staff adjusts product placement according to instructions.

[0226] Specific example explanation

[0227] As an example, consider the case where new product B is introduced into a store.

[0228] 1. Data collection: Staff wearing smart glasses walk around the shelves near new product B to collect customer gaze data. Sales data for new product B is also obtained from the POS system.

[0229] 2. Data analysis and simulation: The server analyzes the collected gaze data and sales data to determine how much customer interest New Product B is attracting. Using an AI model, it simulates optimal placement based on the sales and gaze data.

[0230] 3. Presentation of recommended layout: The recommended layout information is displayed on the smart glasses, and the staff is instructed to "move new product B closer to the entrance, and the number of gazes will increase by 30%."

[0231] 4. Implementation: Staff move new product B to the designated location and monitor gaze and sales data again to confirm the effectiveness of the new placement.

[0232] Prompt Sentence Examples

[0233] "In order to optimize the layout based on gaze data and sales data when new product B is moved closer to the entrance, please simulate which area should be placed to maximize the number of gazes."

[0234] In this way, stores can efficiently analyze customer behavior data and optimize product placement in real time.

[0235] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0236] Step 1:

[0237] Data collection (terminal)

[0238] The smart glasses constantly collect customer gaze data and send it to a server via Wi-Fi. The POS system periodically sends sales data for each product to the server. The input is customer gaze data and sales data, and the output is the data sent to the server.

[0239] Step 2:

[0240] Receiving and storing data (server)

[0241] The server receives the gaze data and sales data and stores them in a database. The input is gaze data and sales data, and the output is raw data stored in the database. Specifically, the received data is converted into an appropriate format and inserted into the corresponding table in the database.

[0242] Step 3:

[0243] Data preprocessing (server)

[0244] The server preprocesses the stored gaze data and sales data. This includes imputing missing values, scaling, and removing unnecessary data. The input is the raw data stored in the database, and the output is the preprocessed data. Specifically, the data is reformatted using Python's pandas and NumPy.

[0245] Step 4:

[0246] Data analysis (server)

[0247] An AI model (e.g., using TensorFlow) analyzes the preprocessed gaze data and sales data to evaluate the correlation between the number of gazes and the number of sales. The input is the preprocessed gaze data and sales data, and the output is the analysis results. Specifically, a regression model is used to analyze the correlation between the number of gazes and the number of sales, and important parameters are extracted.

[0248] Step 5:

[0249] Simulation (server)

[0250] The server simulates the optimal product placement based on the analysis results. The input is the analysis results, and the output is the simulation result of the optimal placement. In concrete terms, a generative AI model is used for the simulation, and prompts are used to evaluate the effectiveness of different placement patterns. An example of a prompt is, "To optimize the layout based on gaze data and sales data when new product B is moved closer to the entrance, please simulate which area should it be placed in to maximize the number of gazes."

[0251] Step 6:

[0252] Generate recommended layout (server)

[0253] The server derives the optimal product placement from the simulation results and generates a recommended layout. The input is the simulation results and the output is the recommended layout. Specifically, the server analyzes the simulation results and selects the most effective placement pattern. Recommended layout information is generated.

[0254] Step 7:

[0255] Display recommended layout (device)

[0256] The server sends the recommended layout to the smart glasses interface in real time and presents it to the staff. The input is the recommended layout information, and the output is the recommended layout displayed on the smart glasses. Specifically, the data is sent to the smart glasses via Wi-Fi and visually displayed on the interface.

[0257] Step 8:

[0258] Product placement adjustment (user)

[0259] Staff members arrange products in designated locations within the store based on the recommended layout. The input is the recommended layout displayed on the smart glasses, and the output is the actual in-store layout with the adjusted layout. Specifically, staff members move the product locations according to instructions, and gaze data and sales data are collected again.

[0260] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0261] ---

[0262] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, emotion data, and store interior data. Specifically, the system includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data using an emotion engine, means for analyzing the gaze data, sales data, and emotion data to simulate optimal placement for each product, and means for providing a recommended layout based on the optimal placement.

[0263] System configuration

[0264] This system consists of the following main elements:

[0265] 1. Method of acquiring gaze data:

[0266] Device: In-store cameras capture customer gaze data in real time, allowing us to measure which products and areas attract customers' attention.

[0267] 2. Sales data acquisition method:

[0268] Terminal: The POS system acquires sales data for each product and periodically sends it to the server.

[0269] 3. Emotion data acquisition method:

[0270] Terminal: Cameras and microphones installed in the store are used to recognize customers' faces and analyze their voices, and an emotion engine is used to obtain customer emotion data.

[0271] 4. Data analysis methods:

[0272] Server: Analyzes the collected gaze data, sales data, and emotion data. Based on information obtained from gaze data, it identifies products and areas that customers pay attention to, takes emotion data into account, and integrates it with sales data to evaluate the effectiveness of product arrangements.

[0273] 5. Simulation Method:

[0274] Server: Utilizing AI models, it analyzes gaze data, sales data, and emotion data to simulate optimal product placement. Based on this simulation, it maximizes product visibility and placement effectiveness.

[0275] 6. Recommended layout methods:

[0276] Server: Calculates the recommended placement from the simulation results and notifies the store manager's terminal.

[0277] Program processing

[0278] This system is implemented through the following series of processes.

[0279] Data collection

[0280] server

[0281] Receives real-time data from in-store cameras and POS systems.

[0282] The received gaze data, emotion data, and sales data are stored in a database.

[0283] Terminal

[0284] A camera is placed to acquire gaze data and data is collected continuously.

[0285] Cameras and microphones are installed to capture emotional data, and customers' facial expressions and voices are analyzed in real time.

[0286] Sales data from the POS system is periodically sent to the server.

[0287] Data analysis and simulation

[0288] server

[0289] The collected gaze data, emotion data and sales data are integrated to identify the customer's areas of interest and emotional state.

[0290] Using an AI model, the correlation between gaze counts, emotional data, and sales figures is analyzed.

[0291] Simulate optimal product placement.

[0292] The recommended layout is sent to the administrator's device.

[0293] Implementing the recommended layout

[0294] User

[0295] Store staff adjust product placement based on the recommended layout.

[0296] Gaze data, emotion data, and sales data will be monitored again to assess the effectiveness of the new placement.

[0297] Specific examples

[0298] As an example, consider the case where new product A is placed in a store.

[0299] Terminal

[0300] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[0301] Customers' facial expressions and voices are collected using cameras and microphones to capture emotional data.

[0302] The POS system sends sales data for new product A to the server.

[0303] server

[0304] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[0305] Based on emotional data, analyze how customers feel about new product A.

[0306] By integrating sales data, gaze data, and emotion data, we simulate which area new product A should be placed in to maximize its sales.

[0307] The recommended layout is sent to the administrator's device, and an example is shown: "Moving the shelves closer to the entrance will increase the number of glances by 45%, and customer preference will also increase."

[0308] User

[0309] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[0310] Gaze data, emotion data and sales data will be monitored again to assess the effectiveness of the new placement.

[0311] In this way, gaze data, sales data, and emotion data can be effectively utilized to achieve optimal product placement.

[0312] The processing flow will be explained below.

[0313] Step 1:

[0314] Terminal

[0315] Cameras are installed in appropriate locations in the store to capture customer gaze data in real time.

[0316] Gaze data is sent to a server in real time.

[0317] Cameras and microphones in the store are used to collect customers' facial expressions and voices, obtaining emotional data in real time.

[0318] Emotion data is sent to the server in real time.

[0319] Sales data for each product is periodically extracted from the POS system and sent to the server.

[0320] Step 2:

[0321] server

[0322] The received gaze data, emotion data, and sales data are stored in a database.

[0323] Store interior data is stored in a database and any updates are reflected.

[0324] Step 3:

[0325] server

[0326] Preprocessing of gaze data, emotion data and sales data is performed.

[0327] Gaze data, sales data, and emotion data are synchronized based on timestamps.

[0328] Link gaze data, emotion data and sales data based on product ID.

[0329] Step 4:

[0330] server

[0331] Using an AI model, the correlation between customer gaze count, emotional data, and sales figures is analyzed.

[0332] Quantify the correlation between glances, customer sentiment, and sales for specific products.

[0333] Identify customer interest areas using gaze data and emotion data.

[0334] Step 5:

[0335] server

[0336] Based on an AI model, gaze data, emotion data, and sales data are analyzed to simulate optimal product placement.

[0337] The optimal combination of gaze count, emotional data, and sales figures is calculated for each product.

[0338] It works in conjunction with the store floor map to generate recommended layouts.

[0339] Step 6:

[0340] server

[0341] Create a recommended layout based on the simulation results.

[0342] The optimal product placement is illustrated and sent to the administrator's terminal.

[0343] Step 7:

[0344] Terminal

[0345] Review the recommended layout provided by your administrator.

[0346] Instruct store staff to reposition products to recommended locations.

[0347] Step 8:

[0348] User

[0349] Store staff arranges products based on the recommended layout.

[0350] Gaze data, emotion data, and sales data will be monitored again to assess the effectiveness of the new placement.

[0351] Example 2

[0352] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0353] Conventional systems have optimized product placement based on sales data and gaze data, but have not performed simulations that take into account the emotional state of customers. As a result, product placement is sometimes not optimized, posing challenges in not fully maximizing sales or improving customer satisfaction. Furthermore, there was a lack of a means to comprehensively analyze this data and recommend the optimal placement for each product.

[0354] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0355] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data, means for analyzing the gaze data, sales data, and emotion data to simulate optimal product placement, and means for providing a recommended layout based on the optimal placement. This enables comprehensive analysis including gaze data and emotion data, and enables precise simulation of optimal product placement, thereby maximizing sales and improving customer satisfaction.

[0356] "Gaze data" is data that represents information on which product or area a customer is looking at.

[0357] "Sales data" refers to data that represents information related to sales and sales volume of each product.

[0358] "Emotion data" is data that represents the emotional state of a customer extracted from their facial expressions and voice.

[0359] "Means for acquiring gaze data" refers to a device or system for detecting the gaze movements of customers and collecting that data.

[0360] The "means for acquiring sales data" refers to a device or system for collecting sales information for each product.

[0361] The "means for acquiring emotional data" refers to a device or system for detecting the emotional state of a customer and collecting that data.

[0362] "Simulation" refers to the process of calculating the optimal placement of each product based on collected data.

[0363] "Recommended layout" refers to a proposal for how to arrange products in a store, derived from the simulation results.

[0364] The term "photography device" refers to a device for acquiring gaze data and emotion data in the form of images or videos.

[0365] A "sales management system" is a system for managing the sales status of products and collecting that data.

[0366] The present invention relates to a system that performs an integrated analysis of customer gaze data, sales data, emotion data, and store interior data, and automatically simulates optimal product placement.

[0367] System configuration

[0368] This system consists of the following main elements:

[0369] 1. Method of acquiring gaze data

[0370] Device: Cameras installed in the store capture customer gaze data in real time. For example, an Azure Kinect camera can be used.

[0371] 2. Sales data acquisition method

[0372] Terminal: The sales management system acquires sales data for each product and periodically sends it to the server. The sales management system used includes Oracle's POS system.

[0373] 3. Means of acquiring emotional data

[0374] Terminal: Cameras and microphones installed in the store are used to recognize customers' faces and analyze their voices, and an emotion engine is used to obtain customer emotion data. Emotion analysis is performed using Affectiva's software.

[0375] 4. Data Analysis Methods

[0376] Server: Analyzes the collected gaze data, sales data, and emotion data. Based on the information obtained from the gaze data, it identifies products and areas that customers pay attention to, takes emotion data into account, and integrates it with sales data to evaluate the effectiveness of product arrangements. The Python pandas library is used for analysis.

[0377] 5. Simulation Methods

[0378] Server: Utilizing an AI model, it analyzes gaze data, sales data, and emotion data to simulate the optimal placement of each product. This simulation uses TensorFlow, which maximizes product visibility and placement effectiveness.

[0379] 6. Methods for Providing Recommended Layouts

[0380] Server: Calculates recommended placements based on the simulation results and notifies the store manager's device via email or a dedicated application.

[0381] Specific examples

[0382] As an example, consider the case where new product A is placed in a store.

[0383] Terminal

[0384] An in-store camera (Azure Kinect) is placed facing the shelf of new product A, and gaze data collection begins.

[0385] Cameras and microphones are used to capture emotion data, and customers' facial expressions and voices are collected. Affectiva's software analyzes the customer's emotions.

[0386] The sales management system (Oracle POS system) periodically sends sales data for new product A to the server.

[0387] server

[0388] Based on the collected gaze data, we analyze how much customer gaze New Product A is attracting. We use Python's pandas library to calculate the frequency of gazes.

[0389] Using emotional data, analyze how customers feel about new product A. Affectiva's software calculates an emotional score to determine whether there are many positive emotions.

[0390] Sales data, gaze data, and emotion data are integrated to perform a simulation and calculate which area should be placed to maximize sales of new product A. A TensorFlow model is used.

[0391] The system notifies the store manager of the recommended layout and makes specific suggestions, such as, "By moving new product A to a shelf closer to the entrance, the number of glances will increase by 45%, and customer favorability will also increase."

[0392] User

[0393] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[0394] To evaluate the effectiveness of the new placement, gaze data, emotion data, and sales data are again collected and monitored. This iterative process allows for continuous optimization of the placement.

[0395] Prompt Sentence Examples

[0396] "We conducted a simulation to determine in which area new product A should be placed to maximize sales, and analysis of gaze data revealed that the shelf near the entrance was optimal. If new product A is placed on a shelf near the entrance, the number of gazes will increase by 45%, and it is predicted that customer favorability will also increase. We will notify the store manager of the recommended layout and monitor the effectiveness of the new placement."

[0397] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0398] Step 1:

[0399] Data collection (input: real-time gaze data, sales data, emotion data, output: integrated data)

[0400] Terminal

[0401] In-store cameras capture customer gaze data in real time, specifically the Azure Kinect camera tracks and collects customer gaze data.

[0402] Cameras and microphones installed in the store analyze customers' facial expressions and voices, and an emotion engine captures emotional data using Affectiva's software.

[0403] A sales management system (for example, an Oracle POS system) periodically collects sales data for each product and sends it to a server.

[0404] Step 2:

[0405] Data accumulation and preprocessing (input: integrated data, output: analyzed data)

[0406] server

[0407] The gaze data, emotion data, and sales data sent from the device are stored in a database. An SQL-based database (e.g., MySQL) is used for integrated management.

[0408] Preprocessing various data. Specifically, completing incomplete data, deleting duplicate data, and formatting it into a form suitable for analysis. Cleaning the data using Python's pandas library.

[0409] Step 3:

[0410] Data analysis (input: analysis data, output: analysis results)

[0411] server

[0412] By integrating and analyzing gaze data, emotion data, and sales data, the system identifies products and areas that customers are paying attention to, as well as changes in emotions. Specifically, it calculates the customer's attention time from gaze data and calculates the ratio of positive to negative emotions from emotion data.

[0413] This data is integrated to calculate the popularity and sentiment scores for each product, using the Python pandas library and scikit-learn analysis tools.

[0414] Step 4:

[0415] Simulation (input: analysis results, output: optimal layout plan)

[0416] server

[0417] Based on the analysis results, an AI model is used to simulate optimal product placement, and TensorFlow is used to analyze correlations between gaze data, emotion data, and sales data to identify optimal placement patterns.

[0418] From the simulation results, specific placement plans are created that show which areas will attract the most attention to a particular product and improve customer sentiment scores.

[0419] Step 5:

[0420] Notification of recommended layout (Input: Optimal layout plan, Output: Recommended layout notification)

[0421] server

[0422] Based on the simulation results, the system calculates a recommended layout and notifies the store manager via email or a dedicated application.

[0423] As a specific suggestion, they provide information such as, "Moving new product A to a shelf closer to the entrance will increase the number of glances by 45% and improve customer favorability."

[0424] Step 6:

[0425] Implementation of recommended layout (input: notification of recommended layout, output: data verifying effectiveness after implementation)

[0426] User

[0427] Store staff adjust product placement based on the recommended layout, for example, moving new product A to a shelf closer to the entrance.

[0428] After changing the placement, gaze data, emotion data, and sales data will be collected again to monitor the effects. The effectiveness of the placement will be verified based on the re-collected data.

[0429] (Application example 2)

[0430] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0431] Modern brick-and-mortar stores lack a means for comprehensively analyzing customer gaze, emotion, and sales data to effectively determine optimal product placement. Existing methods struggle to specifically and efficiently reflect product sales growth and customer interest. Furthermore, conventional systems struggle with real-time data collection and analysis, requiring significant time and effort to optimize product placement. Therefore, the present invention aims to solve these problems by providing an integrated, real-time analysis system for optimizing product placement in brick-and-mortar stores.

[0432] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0433] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data, means for analyzing the gaze data, sales data, and emotion data and simulating an optimal layout for each product, means for providing a recommended layout, and means for providing a terminal for displaying the recommended layout to the user. This makes it possible to efficiently determine the optimal product layout through real-time data collection and analysis, maximizing customer interest and increasing sales.

[0434] Definitions of important words

[0435] The "gaze data acquisition means" is a means for acquiring the direction of a customer's gaze and point of gaze in real time using devices such as in-store cameras and surveillance cameras.

[0436] The "sales data acquisition means" is a means for collecting data on the sales of each product in cooperation with an information management system or a POS system.

[0437] The "emotion data acquisition means" is a means for analyzing the customer's facial expressions and voice data using a voice recognition device or an emotion analysis engine, and acquiring the customer's emotional state as data.

[0438] The "data analysis means" is a means for comprehensively analyzing gaze data, sales data, and emotion data on a server, and evaluating the placement effectiveness of each product.

[0439] The "optimal placement simulation means" is a means of simulating the optimal placement of each product using an AI model based on information obtained from the data analysis means.

[0440] The "means for providing recommended layout" is a means for proposing changes to the product layout based on the results of the optimum layout simulation and notifying the user of this information.

[0441] A "recommended layout display terminal" is a terminal that provides recommended layout information to a user via a smartphone, tablet, or the like.

[0442] MODE FOR CARRYING OUT THE INVENTION

[0443] System configuration overview

[0444] The present invention relates to a system for optimizing product placement in a physical store, and is composed of the following main components:

[0445] 1. Method of acquiring gaze data: Customer gaze data is acquired in real time using in-store cameras and surveillance cameras.

[0446] 2. Sales data acquisition method: Collect sales data for each product in cooperation with the POS system.

[0447] 3. Emotion data acquisition means: Includes a voice recognition device that analyzes the customer's facial expressions and voice data to acquire emotion data.

[0448] 4. Data analysis means: The gaze data, sales data, and emotion data collected on the server are integrated and analyzed.

[0449] 5. Optimal placement simulation method: Using an AI model, the optimal placement of each product is simulated based on the analyzed data.

[0450] 6. Layout recommendation method: A layout recommendation is provided based on the results of the optimal layout simulation.

[0451] 7. Devices that display recommended layouts: Display recommended layouts to users via smartphones and tablets.

[0452] System program and processing details

[0453] Data Collection Module

[0454] Hardware: surveillance cameras, microphones, POS systems

[0455] Software: Camera control software, voice recognition software, POS system data integration API

[0456] The server collects real-time gaze and emotion data from surveillance cameras and voice recognition devices installed in the store, and receives sales data from the POS system. This data is stored in a database and used for subsequent analysis.

[0457] Data Analysis Module

[0458] Hardware: Server

[0459] Software: Python, Pandas, Scikit-learn, TensorFlow

[0460] The server integrates the gaze data, sales data, and emotional data stored in the database, performs statistical analysis of gaze direction, and evaluates the emotional state of customers. An AI model (e.g., a neural network using TensorFlow) is used to analyze the correlation between gaze data and sales data and evaluate the effectiveness of product arrangements.

[0461] Simulation Module

[0462] Hardware: Server

[0463] Software: PyTorch, Matplotlib

[0464] The server uses an AI model to simulate optimal product placement based on collected data, thereby calculating the optimal placement that maximizes product visibility and placement effectiveness.

[0465] Recommended layout module

[0466] Hardware: Smartphones, tablets

[0467] Software: React Native, Firebase

[0468] The server then sends the recommended layout information based on the simulation results to a smartphone or tablet, where users can check and implement the optimal layout information.

[0469] Examples and prompts

[0470] Example: Optimizing the placement of new product B

[0471] For example, if new product B is introduced to a store, the system operates as follows:

[0472] Gaze data collection: A surveillance camera is aimed at the display shelf of new product B to capture customer gaze data.

[0473] Emotional data collection: A voice recognition device analyzes the customer's voice and collects emotional data.

[0474] Sales data collection: The POS system sends sales data for new product B to the server.

[0475] Data analysis and simulation: The server analyzes gaze data, emotion data, and sales data, and simulates optimal placement.

[0476] Layout recommendation: The application informs, "If new product B is placed on the left shelf near the entrance, it will receive 40% more views and increase sales by 20%."

[0477] Example prompts for generative AI models

[0478] Generate the desired layout simulation results. Please integrate and analyze the following data: gaze data (coordinates, time), emotion data (customer facial expressions, voice), and sales data (sales by product). The final output format should be a layout proposal such as "Place product X in position Y." Example: New product B, on the left shelf near the entrance, increases gazes by 40%, increases sales by 20%.

[0479] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0480] Program processing flow

[0481] Step 1: Data collection

[0482] The device captures gaze data in real time using a surveillance camera, collects emotion data using a voice recognition device, and periodically acquires sales data from a POS system, which are then sent to a server and stored in a database.

[0483] Input: Gaze data from surveillance cameras, emotion data from voice recognition devices, sales data from POS systems

[0484] Output: Gaze data, emotion data, sales data stored in a database

[0485] Step 2: Data Preprocessing

[0486] The server cleans the gaze, emotion, and sales data stored in the database and converts it into an analyzable format: gaze data is organized by coordinates and time, emotion data is tagged as the customer's emotional state, and sales data is aggregated by product.

[0487] Input: Gaze data, emotion data, sales data stored in the database

[0488] Output: Preprocessed gaze data, emotion data, sales data

[0489] Step 3: Data analysis

[0490] The server uses the pre-processed data to evaluate the correlation between gaze data and sales data, and analyzes the emotional state of customers, using statistical analysis tools with Python and Pandas, and machine learning models with Scikit-learn.

[0491] Input: Preprocessed gaze data, emotion data, sales data

[0492] Output: Correlation results of analyzed data, customer emotional state

[0493] Step 4: Optimal layout simulation

[0494] Based on the analysis results, the server uses an AI model using PyTorch and TensorFlow to simulate optimal product placement, for example, calculating where to place a new product to attract the most customer attention and increase sales.

[0495] Input: Correlation results of analyzed data, customer emotional state

[0496] Output: Simulation results of optimal product placement

[0497] Step 5: Generate a recommended layout

[0498] The server generates specific product placement proposals based on the simulation results, including the extent to which the number of views and sales will increase by placing which products in which positions.

[0499] Input: Optimal product placement simulation results

[0500] Output: Recommended layout suggestions

[0501] Step 6: Notification of recommended layout

[0502] The layout recommendations are sent to devices such as smartphones and tablets, where users can review and implement them. For example, the application might notify users, "Placing new product B on the left shelf near the entrance will increase the number of views by 40% and increase sales by 20%."

[0503] Input: Recommended layout suggestions

[0504] Output: The recommended layout displayed on the user's device

[0505] Through these steps, the system collects data in real time and optimizes product placement to maximize customer interest and increase sales.

[0506] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0507] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0508] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0509] [Second embodiment]

[0510] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0511] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0512] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0513] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0514] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0515] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0516] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0517] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0518] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0519] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0520] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0521] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0522] ---

[0523] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, and store interior data. Specifically, the system includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data to simulate optimal placement for each product, and a means for providing a recommended layout based on the optimal placement.

[0524] System configuration

[0525] This system consists of the following main elements:

[0526] 1. Method of acquiring gaze data:

[0527] Device: In-store cameras capture customer gaze data in real time, allowing us to measure which products and areas attract customers' attention.

[0528] 2. Sales data acquisition method:

[0529] Terminal: The POS system acquires sales data for each product and periodically sends it to the server.

[0530] 3. Data analysis methods:

[0531] Server: Analyzes the collected gaze data and sales data. Based on the information obtained from the gaze data, it identifies products and areas that customers pay attention to, and integrates this with sales data to evaluate the effectiveness of product placement.

[0532] 4. Simulation Method:

[0533] Server: Utilizing AI models, it analyzes gaze data and sales data to simulate optimal product placement. Based on this simulation, it maximizes product visibility and placement effectiveness.

[0534] 5. Recommended layout methods:

[0535] Server: Calculates the recommended placement from the simulation results and notifies the store manager's terminal.

[0536] Program processing

[0537] This system is implemented through the following series of processes.

[0538] Data collection

[0539] server

[0540] Receives real-time data from in-store cameras and POS systems.

[0541] The received gaze data and sales data are stored in a database.

[0542] Terminal

[0543] A camera is placed to acquire gaze data and data is collected continuously.

[0544] Sales data from the POS system is periodically sent to the server.

[0545] Data analysis and simulation

[0546] server

[0547] Integrate collected gaze data with sales data to identify areas of customer interest.

[0548] Using an AI model, we analyze the correlation between the number of views and sales.

[0549] Simulate optimal product placement.

[0550] The recommended layout is sent to the administrator's device.

[0551] Implementing the recommended layout

[0552] User

[0553] Store staff adjust product placement based on the recommended layout.

[0554] Review gaze and sales data and make further adjustments as needed.

[0555] Specific examples

[0556] As an example, consider the case where new product A is placed in a store.

[0557] Terminal

[0558] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[0559] The POS system sends sales data for new product A to the server.

[0560] server

[0561] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[0562] By integrating sales data and gaze data, we simulate which area new product A should be placed in to maximize its sales.

[0563] The recommended layout is notified to the administrator's device, and an example is shown: "Moving to the shelf near the entrance will increase the number of views by 45%."

[0564] User

[0565] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[0566] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[0567] In this way, gaze data and sales data can be effectively utilized to achieve optimal product placement.

[0568] The processing flow will be explained below.

[0569] Step 1:

[0570] Terminal

[0571] Cameras are installed in appropriate locations in the store to capture customer gaze data in real time.

[0572] The gaze data collected by the camera is sent to a server.

[0573] Sales data for each product is periodically extracted from the POS system and sent to the server.

[0574] Step 2:

[0575] server

[0576] The received gaze data is stored in a database.

[0577] Store sales data received from the POS system in a database.

[0578] Store interior data is stored in a database and any updates are reflected.

[0579] Step 3:

[0580] server

[0581] Preprocessing of gaze data and sales data is performed.

[0582] Synchronize gaze data and sales data based on timestamps.

[0583] Link gaze data and sales data based on product ID.

[0584] Step 4:

[0585] server

[0586] Using an AI model, we analyze the correlation between the number of customer gazes and sales figures.

[0587] Quantify the correlation between the number of views and sales of a specific product.

[0588] Identify customer interest areas using gaze data.

[0589] Step 5:

[0590] server

[0591] Based on an AI model, gaze data and sales data are analyzed to simulate optimal product placement.

[0592] Calculate the optimal combination of view counts and sales for each product.

[0593] It works in conjunction with the store floor map to generate recommended layouts.

[0594] Step 6:

[0595] server

[0596] Create a recommended layout based on the simulation results.

[0597] The optimal product placement is illustrated and sent to the administrator's terminal.

[0598] Step 7:

[0599] Terminal

[0600] Review the recommended layout provided by your administrator.

[0601] Instruct store staff to reposition products to recommended locations.

[0602] Step 8:

[0603] User

[0604] Store staff arranges products based on the recommended layout.

[0605] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[0606] Example 1

[0607] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0608] Conventional store layouts lack a means to comprehensively analyze customer gaze data and sales data and simulate optimal product placement. This has led to the problem that product placement effects cannot be maximized and sales cannot be expected to increase.

[0609] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0610] In this invention, the server includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data and simulating the optimal placement for each product, a means for analyzing the correlation between the gaze data and sales data using a generative AI model, and a means for providing a recommended layout based on the optimal placement. This makes it possible to effectively utilize the gaze data and sales data, maximize the effectiveness of product placement, and increase sales.

[0611] "Gaze data" is data that indicates in which direction and at which products customers are looking in a store.

[0612] "Sales data" is data that records the sales status of each product, and includes the product name, unit price, number of units sold, date and time of sale, etc.

[0613] A "generative AI model" is an artificial intelligence model used to analyze the correlation between gaze data and sales data and optimize product placement.

[0614] A "recommended layout" is a layout that shows the optimal way to arrange products, calculated based on gaze data and sales data.

[0615] The "in-store camera" is a camera device installed in the store to acquire customer line-of-sight data.

[0616] A "sales management system" is a system that manages the sales situation of a store and records and transmits sales data for each product.

[0617] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, and store interior data. Specifically, the system includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data to simulate optimal placement for each product, and a means for providing a recommended layout based on the optimal placement.

[0618] The system consists of the following main elements:

[0619] Eye gaze data acquisition method

[0620] Terminal

[0621] Cameras installed in the store capture customer gaze data in real time, making it possible to measure which products and areas attract customers' attention.

[0622] Sales data acquisition method

[0623] Terminal

[0624] The sales management system acquires sales data for each product and periodically sends it to the server. The sales data includes the product name, unit price, number of units sold, and the date and time of the sale.

[0625] Data Analysis Methods

[0626] server

[0627] The server analyzes the collected gaze data and sales data. Based on the information obtained from the gaze data, it identifies products and areas that customers are paying attention to, and integrates this with sales data to evaluate the effectiveness of product arrangements.

[0628] Simulation Method

[0629] server

[0630] Using a generative AI model, it analyzes gaze data and sales data to simulate the optimal placement of each product, thereby maximizing product visibility and placement effectiveness.

[0631] Layout recommendation method

[0632] server

[0633] The system calculates the recommended placement from the simulation results and notifies the store manager's terminal, including the reasons for the recommended placement and specific numerical results.

[0634] Specific examples

[0635] As a specific example, consider the case where new product A is placed in a store.

[0636] Terminal

[0637] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[0638] The sales management system sends sales data for new product A to the server.

[0639] server

[0640] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[0641] By integrating sales data and gaze data, we simulate which area new product A should be placed in to maximize its sales.

[0642] The recommended layout is notified to the administrator's device, and an example layout is presented that suggests, "Moving shelves closer to the entrance will increase the number of views by 45% and increase sales by 20%."

[0643] User

[0644] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[0645] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[0646] Example prompts to input to the generative AI model

[0647] "Please simulate the impact on the number of glances and sales if new product A is moved to a shelf near the entrance."

[0648] This allows for the effective use of gaze data and sales data to achieve optimal product placement.

[0649] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0650] Step 1:

[0651] Data collection

[0652] Terminal

[0653] In-store cameras installed on the terminals capture customer gaze data in real time. The cameras track customer eye movements and record which areas and products their gazes are focused on. The input is the camera's video data, and the output is each customer's gaze data.

[0654] The sales management system acquires sales data for each product in real time and sends it to the server at regular intervals. The input is sales records and the output is sales data.

[0655] server

[0656] The server stores the gaze data and sales data sent from the in-store cameras and the sales management system in a database.

[0657] Step 2:

[0658] Data Integration

[0659] server

[0660] Gaze data and sales data are retrieved from the database and integrated. The gaze data and sales data are aligned on the same time axis to analyze the level of interest customers show in products and how this affects sales. The input is gaze data and sales data, and the output is an integrated dataset.

[0661] Specific operations include converting the data format and complementing missing data.

[0662] Step 3:

[0663] Correlation analysis

[0664] server

[0665] The server uses a generative AI model to analyze the correlation between gaze counts and sales numbers. It applies machine learning algorithms to find patterns and trends between gaze data and sales data. The input is the integrated dataset, and the output is the result of the correlation analysis.

[0666] Specific operations include preprocessing, feature extraction, and model application.

[0667] Step 4:

[0668] Simulation of optimal layout

[0669] server

[0670] The server simulates the optimal placement for each product based on the results of the correlation analysis. It uses an AI model to evaluate various placement scenarios based on gaze data and sales data to identify the most effective placement. The input is the result of the correlation analysis, and the output is the simulation result of the optimal placement.

[0671] Specifically, multiple scenarios are generated and evaluated.

[0672] Step 5:

[0673] Providing recommended layouts

[0674] server

[0675] The system calculates the recommended layout from the simulation results and notifies the store manager's terminal. The notification includes the reasons for the recommended layout and specific numerical results. The input is the simulation result of the optimal layout, and the output is a notification of the recommended layout.

[0676] Step 6:

[0677] Implementing the recommended layout

[0678] User

[0679] Store staff adjust product placement based on the recommended layout they receive. For example, move new product A to a shelf closer to the store entrance. The input is the recommended layout notification, and the output is the new in-store layout.

[0680] Specific operations include moving and rearranging product shelves.

[0681] Step 7:

[0682] Monitoring the effectiveness

[0683] Terminal

[0684] Gaze data and sales data are collected again to continuously monitor the effectiveness of the new placement. Data for improvement is collected. The input is the gaze data and sales data after the change, and the output is data to evaluate the effectiveness.

[0685] By following the above steps, it is possible to effectively utilize gaze data and sales data to achieve optimal product placement.

[0686] (Application example 1)

[0687] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0688] Conventional methods make it difficult to efficiently and effectively optimize product placement in stores. In particular, the lack of a system that collects customer behavior and gaze data in real time and integrates and analyzes it with sales data makes it difficult to quickly evaluate the effectiveness of product placement and provide an optimal layout. As a result, stores are often slow to make appropriate decisions to maximize sales.

[0689] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0690] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for analyzing the gaze data and sales data and simulating optimal product placement, means for providing a recommended layout based on the optimal placement, means for collecting gaze data in real time using a smart device, means for performing data analysis and simulation using an AI model, and means for displaying the recommended layout on the interface of the smart device in real time. This makes it possible to effectively analyze customer behavior data and sales data and quickly and appropriately achieve optimal product placement.

[0691] "Gaze data" is data that measures a customer's eye movements and points of gaze and records them in digital format.

[0692] "Sales data" refers to data including sales information, sales quantity, sales date and time, etc., for each product.

[0693] "Simulation" is the process of recreating virtual environments and situations based on collected data and attempting optimal placement and operation.

[0694] The "recommended layout" is the optimal product placement pattern derived based on the analysis and simulation results.

[0695] "Smart devices" is a general term for wearable devices and mobile terminals that can connect to the Internet and are capable of advanced calculations and analysis.

[0696] An "AI model" is a collection of algorithms and programs designed to solve a specific problem using artificial intelligence technology.

[0697] An "interface" is a means or mechanism by which systems, devices, and software exchange information with each other.

[0698] This invention relates to a system that performs an integrated analysis of customer gaze data, sales data, and store interior data to automatically simulate optimal product placement. To specifically implement this system, the following hardware and software are used.

[0699] Hardware and software used

[0700] 1. Hardware

[0701] Smart Device: A device used to collect customer gaze data in real time. A specific example is smart glasses (e.g., Google Glass).

[0702] Server: The main hardware used to analyze large amounts of data and simulate optimal product placement.

[0703] POS system: A device that collects product sales data and sends it to a server.

[0704] 2. Software

[0705] Data Analysis Platform: Software used to analyze gaze data and sales data, specifically using Python and TensorFlow.

[0706] Smart Device Interface: An interface for displaying recommended layouts on smart glasses, specifically using the Android Wear platform.

[0707] Explanation of program processing

[0708] 1. Data Collection

[0709] Terminal: The smart glasses constantly collect customer gaze data and send it to the server in real time. The POS system periodically sends sales data for each product to the server.

[0710] Server: The server receives the gaze data and sales data and stores them in a database.

[0711] 2. Data analysis and simulation

[0712] Server: Analyzes collected gaze data and sales data using an AI model using Python and TensorFlow. Through this analysis, areas of customer interest are identified and the correlation between gaze counts and sales is evaluated. Based on this, optimal product placement is simulated.

[0713] 3. Display recommended layout

[0714] Server: Calculates the recommended layout based on the simulation results and sends it to the smart device interface in real time.

[0715] Terminal: A recommended layout is displayed on the smart glasses interface, and staff adjusts product placement according to instructions.

[0716] Specific example explanation

[0717] As an example, consider the case where new product B is introduced into a store.

[0718] 1. Data collection: Staff wearing smart glasses walk around the shelves near new product B to collect customer gaze data. Sales data for new product B is also obtained from the POS system.

[0719] 2. Data analysis and simulation: The server analyzes the collected gaze data and sales data to determine how much customer interest New Product B is attracting. Using an AI model, it simulates optimal placement based on the sales and gaze data.

[0720] 3. Presentation of recommended layout: The recommended layout information is displayed on the smart glasses, and the staff is instructed to "move new product B closer to the entrance, and the number of gazes will increase by 30%."

[0721] 4. Implementation: Staff move new product B to the designated location and monitor gaze and sales data again to confirm the effectiveness of the new placement.

[0722] Prompt Sentence Examples

[0723] "In order to optimize the layout based on gaze data and sales data when new product B is moved closer to the entrance, please simulate which area should be placed to maximize the number of gazes."

[0724] In this way, stores can efficiently analyze customer behavior data and optimize product placement in real time.

[0725] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0726] Step 1:

[0727] Data collection (terminal)

[0728] The smart glasses constantly collect customer gaze data and send it to a server via Wi-Fi. The POS system periodically sends sales data for each product to the server. The input is customer gaze data and sales data, and the output is the data sent to the server.

[0729] Step 2:

[0730] Receiving and storing data (server)

[0731] The server receives the gaze data and sales data and stores them in a database. The input is gaze data and sales data, and the output is raw data stored in the database. Specifically, the received data is converted into an appropriate format and inserted into the corresponding table in the database.

[0732] Step 3:

[0733] Data preprocessing (server)

[0734] The server preprocesses the stored gaze data and sales data. This includes imputing missing values, scaling, and removing unnecessary data. The input is the raw data stored in the database, and the output is the preprocessed data. Specifically, the data is reformatted using Python's pandas and NumPy.

[0735] Step 4:

[0736] Data analysis (server)

[0737] An AI model (e.g., using TensorFlow) analyzes the preprocessed gaze data and sales data to evaluate the correlation between the number of gazes and the number of sales. The input is the preprocessed gaze data and sales data, and the output is the analysis results. Specifically, a regression model is used to analyze the correlation between the number of gazes and the number of sales, and important parameters are extracted.

[0738] Step 5:

[0739] Simulation (server)

[0740] The server simulates the optimal product placement based on the analysis results. The input is the analysis results, and the output is the simulation result of the optimal placement. In concrete terms, a generative AI model is used for the simulation, and prompts are used to evaluate the effectiveness of different placement patterns. An example of a prompt is, "To optimize the layout based on gaze data and sales data when new product B is moved closer to the entrance, please simulate which area should it be placed in to maximize the number of gazes."

[0741] Step 6:

[0742] Generate recommended layout (server)

[0743] The server derives the optimal product placement from the simulation results and generates a recommended layout. The input is the simulation results and the output is the recommended layout. Specifically, the server analyzes the simulation results and selects the most effective placement pattern. Recommended layout information is generated.

[0744] Step 7:

[0745] Display recommended layout (device)

[0746] The server sends the recommended layout to the smart glasses interface in real time and presents it to the staff. The input is the recommended layout information, and the output is the recommended layout displayed on the smart glasses. Specifically, the data is sent to the smart glasses via Wi-Fi and visually displayed on the interface.

[0747] Step 8:

[0748] Product placement adjustment (user)

[0749] Staff members arrange products in designated locations within the store based on the recommended layout. The input is the recommended layout displayed on the smart glasses, and the output is the actual in-store layout with the adjusted layout. Specifically, staff members move the product locations according to instructions, and gaze data and sales data are collected again.

[0750] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0751] ---

[0752] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, emotion data, and store interior data. Specifically, the system includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data using an emotion engine, means for analyzing the gaze data, sales data, and emotion data to simulate optimal placement for each product, and means for providing a recommended layout based on the optimal placement.

[0753] System configuration

[0754] This system consists of the following main elements:

[0755] 1. Method of acquiring gaze data:

[0756] Device: In-store cameras capture customer gaze data in real time, allowing us to measure which products and areas attract customers' attention.

[0757] 2. Sales data acquisition method:

[0758] Terminal: The POS system acquires sales data for each product and periodically sends it to the server.

[0759] 3. Emotion data acquisition method:

[0760] Terminal: Cameras and microphones installed in the store are used to recognize customers' faces and analyze their voices, and an emotion engine is used to obtain customer emotion data.

[0761] 4. Data analysis methods:

[0762] Server: Analyzes the collected gaze data, sales data, and emotion data. Based on information obtained from gaze data, it identifies products and areas that customers pay attention to, takes emotion data into account, and integrates it with sales data to evaluate the effectiveness of product arrangements.

[0763] 5. Simulation Method:

[0764] Server: Utilizing AI models, it analyzes gaze data, sales data, and emotion data to simulate optimal product placement. Based on this simulation, it maximizes product visibility and placement effectiveness.

[0765] 6. Recommended layout methods:

[0766] Server: Calculates the recommended placement from the simulation results and notifies the store manager's terminal.

[0767] Program processing

[0768] This system is implemented through the following series of processes.

[0769] Data collection

[0770] server

[0771] Receives real-time data from in-store cameras and POS systems.

[0772] The received gaze data, emotion data, and sales data are stored in a database.

[0773] Terminal

[0774] A camera is placed to acquire gaze data and data is collected continuously.

[0775] Cameras and microphones are installed to capture emotional data, and customers' facial expressions and voices are analyzed in real time.

[0776] Sales data from the POS system is periodically sent to the server.

[0777] Data analysis and simulation

[0778] server

[0779] The collected gaze data, emotion data and sales data are integrated to identify the customer's areas of interest and emotional state.

[0780] Using an AI model, the correlation between gaze counts, emotional data, and sales figures is analyzed.

[0781] Simulate optimal product placement.

[0782] The recommended layout is sent to the administrator's device.

[0783] Implementing the recommended layout

[0784] User

[0785] Store staff adjust product placement based on the recommended layout.

[0786] Gaze data, emotion data, and sales data will be monitored again to assess the effectiveness of the new placement.

[0787] Specific examples

[0788] As an example, consider the case where new product A is placed in a store.

[0789] Terminal

[0790] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[0791] Customers' facial expressions and voices are collected using cameras and microphones to capture emotional data.

[0792] The POS system sends sales data for new product A to the server.

[0793] server

[0794] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[0795] Based on emotional data, analyze how customers feel about new product A.

[0796] By integrating sales data, gaze data, and emotion data, we simulate which area new product A should be placed in to maximize its sales.

[0797] The recommended layout is sent to the administrator's device, and an example is shown: "Moving the shelves closer to the entrance will increase the number of glances by 45%, and customer preference will also increase."

[0798] User

[0799] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[0800] Gaze data, emotion data and sales data will be monitored again to assess the effectiveness of the new placement.

[0801] In this way, gaze data, sales data, and emotion data can be effectively utilized to achieve optimal product placement.

[0802] The processing flow will be explained below.

[0803] Step 1:

[0804] Terminal

[0805] Cameras are installed in appropriate locations in the store to capture customer gaze data in real time.

[0806] Gaze data is sent to a server in real time.

[0807] Cameras and microphones in the store are used to collect customers' facial expressions and voices, obtaining emotional data in real time.

[0808] Emotion data is sent to the server in real time.

[0809] Sales data for each product is periodically extracted from the POS system and sent to the server.

[0810] Step 2:

[0811] server

[0812] The received gaze data, emotion data, and sales data are stored in a database.

[0813] Store interior data is stored in a database and any updates are reflected.

[0814] Step 3:

[0815] server

[0816] Preprocessing of gaze data, emotion data and sales data is performed.

[0817] Gaze data, sales data, and emotion data are synchronized based on timestamps.

[0818] Link gaze data, emotion data and sales data based on product ID.

[0819] Step 4:

[0820] server

[0821] Using an AI model, the correlation between customer gaze count, emotional data, and sales figures is analyzed.

[0822] Quantify the correlation between glances, customer sentiment, and sales for specific products.

[0823] Identify customer interest areas using gaze data and emotion data.

[0824] Step 5:

[0825] server

[0826] Based on an AI model, gaze data, emotion data, and sales data are analyzed to simulate optimal product placement.

[0827] The optimal combination of gaze count, emotional data, and sales figures is calculated for each product.

[0828] It works in conjunction with the store floor map to generate recommended layouts.

[0829] Step 6:

[0830] server

[0831] Create a recommended layout based on the simulation results.

[0832] The optimal product placement is illustrated and sent to the administrator's terminal.

[0833] Step 7:

[0834] Terminal

[0835] Review the recommended layout provided by your administrator.

[0836] Instruct store staff to reposition products to recommended locations.

[0837] Step 8:

[0838] User

[0839] Store staff arranges products based on the recommended layout.

[0840] Gaze data, emotion data, and sales data will be monitored again to assess the effectiveness of the new placement.

[0841] Example 2

[0842] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0843] Conventional systems have optimized product placement based on sales data and gaze data, but have not performed simulations that take into account the emotional state of customers. As a result, product placement is sometimes not optimized, posing challenges in not fully maximizing sales or improving customer satisfaction. Furthermore, there was a lack of a means to comprehensively analyze this data and recommend the optimal placement for each product.

[0844] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0845] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data, means for analyzing the gaze data, sales data, and emotion data to simulate optimal product placement, and means for providing a recommended layout based on the optimal placement. This enables comprehensive analysis including gaze data and emotion data, and enables precise simulation of optimal product placement, thereby maximizing sales and improving customer satisfaction.

[0846] "Gaze data" is data that represents information on which product or area a customer is looking at.

[0847] "Sales data" refers to data that represents information related to sales and sales volume of each product.

[0848] "Emotion data" is data that represents the emotional state of a customer extracted from their facial expressions and voice.

[0849] "Means for acquiring gaze data" refers to a device or system for detecting the gaze movements of customers and collecting that data.

[0850] The "means for acquiring sales data" refers to a device or system for collecting sales information for each product.

[0851] The "means for acquiring emotional data" refers to a device or system for detecting the emotional state of a customer and collecting that data.

[0852] "Simulation" refers to the process of calculating the optimal placement of each product based on collected data.

[0853] "Recommended layout" refers to a proposal for how to arrange products in a store, derived from the simulation results.

[0854] The term "photography device" refers to a device for acquiring gaze data and emotion data in the form of images or videos.

[0855] A "sales management system" is a system for managing the sales status of products and collecting that data.

[0856] The present invention relates to a system that performs an integrated analysis of customer gaze data, sales data, emotion data, and store interior data, and automatically simulates optimal product placement.

[0857] System configuration

[0858] This system consists of the following main elements:

[0859] 1. Method of acquiring gaze data

[0860] Device: Cameras installed in the store capture customer gaze data in real time. For example, an Azure Kinect camera can be used.

[0861] 2. Sales data acquisition method

[0862] Terminal: The sales management system acquires sales data for each product and periodically sends it to the server. The sales management system used includes Oracle's POS system.

[0863] 3. Means of acquiring emotional data

[0864] Terminal: Cameras and microphones installed in the store are used to recognize customers' faces and analyze their voices, and an emotion engine is used to obtain customer emotion data. Emotion analysis is performed using Affectiva's software.

[0865] 4. Data Analysis Methods

[0866] Server: Analyzes the collected gaze data, sales data, and emotion data. Based on the information obtained from the gaze data, it identifies products and areas that customers pay attention to, takes emotion data into account, and integrates it with sales data to evaluate the effectiveness of product arrangements. The Python pandas library is used for analysis.

[0867] 5. Simulation Methods

[0868] Server: Utilizing an AI model, it analyzes gaze data, sales data, and emotion data to simulate the optimal placement of each product. This simulation uses TensorFlow, which maximizes product visibility and placement effectiveness.

[0869] 6. Methods for Providing Recommended Layouts

[0870] Server: Calculates recommended placements based on the simulation results and notifies the store manager's device via email or a dedicated application.

[0871] Specific examples

[0872] As an example, consider the case where new product A is placed in a store.

[0873] Terminal

[0874] An in-store camera (Azure Kinect) is placed facing the shelf of new product A, and gaze data collection begins.

[0875] Cameras and microphones are used to capture emotion data, and customers' facial expressions and voices are collected. Affectiva's software analyzes the customer's emotions.

[0876] The sales management system (Oracle POS system) periodically sends sales data for new product A to the server.

[0877] server

[0878] Based on the collected gaze data, we analyze how much customer gaze New Product A is attracting. We use Python's pandas library to calculate the frequency of gazes.

[0879] Using emotional data, analyze how customers feel about new product A. Affectiva's software calculates an emotional score to determine whether there are many positive emotions.

[0880] Sales data, gaze data, and emotion data are integrated to perform a simulation and calculate which area should be placed to maximize sales of new product A. A TensorFlow model is used.

[0881] The system notifies the store manager of the recommended layout and makes specific suggestions, such as, "By moving new product A to a shelf closer to the entrance, the number of glances will increase by 45%, and customer favorability will also increase."

[0882] User

[0883] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[0884] To evaluate the effectiveness of the new placement, gaze data, emotion data, and sales data are again collected and monitored. This iterative process allows for continuous optimization of the placement.

[0885] Prompt Sentence Examples

[0886] "We conducted a simulation to determine in which area new product A should be placed to maximize sales, and analysis of gaze data revealed that the shelf near the entrance was optimal. If new product A is placed on a shelf near the entrance, the number of gazes will increase by 45%, and it is predicted that customer favorability will also increase. We will notify the store manager of the recommended layout and monitor the effectiveness of the new placement."

[0887] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0888] Step 1:

[0889] Data collection (input: real-time gaze data, sales data, emotion data, output: integrated data)

[0890] Terminal

[0891] In-store cameras capture customer gaze data in real time, specifically the Azure Kinect camera tracks and collects customer gaze data.

[0892] Cameras and microphones installed in the store analyze customers' facial expressions and voices, and an emotion engine captures emotional data using Affectiva's software.

[0893] A sales management system (for example, an Oracle POS system) periodically collects sales data for each product and sends it to a server.

[0894] Step 2:

[0895] Data accumulation and preprocessing (input: integrated data, output: analyzed data)

[0896] server

[0897] The gaze data, emotion data, and sales data sent from the device are stored in a database. An SQL-based database (e.g., MySQL) is used for integrated management.

[0898] Preprocessing various data. Specifically, completing incomplete data, deleting duplicate data, and formatting it into a form suitable for analysis. Cleaning the data using Python's pandas library.

[0899] Step 3:

[0900] Data analysis (input: analysis data, output: analysis results)

[0901] server

[0902] By integrating and analyzing gaze data, emotion data, and sales data, the system identifies products and areas that customers are paying attention to, as well as changes in emotions. Specifically, it calculates the customer's attention time from gaze data and calculates the ratio of positive to negative emotions from emotion data.

[0903] This data is integrated to calculate the popularity and sentiment scores for each product, using the Python pandas library and scikit-learn analysis tools.

[0904] Step 4:

[0905] Simulation (input: analysis results, output: optimal layout plan)

[0906] server

[0907] Based on the analysis results, an AI model is used to simulate optimal product placement, and TensorFlow is used to analyze correlations between gaze data, emotion data, and sales data to identify optimal placement patterns.

[0908] From the simulation results, specific placement plans are created that show which areas will attract the most attention to a particular product and improve customer sentiment scores.

[0909] Step 5:

[0910] Notification of recommended layout (Input: Optimal layout plan, Output: Recommended layout notification)

[0911] server

[0912] Based on the simulation results, the system calculates a recommended layout and notifies the store manager via email or a dedicated application.

[0913] As a specific suggestion, they provide information such as, "Moving new product A to a shelf closer to the entrance will increase the number of glances by 45% and improve customer favorability."

[0914] Step 6:

[0915] Implementation of recommended layout (input: notification of recommended layout, output: data verifying effectiveness after implementation)

[0916] User

[0917] Store staff adjust product placement based on the recommended layout, for example, moving new product A to a shelf closer to the entrance.

[0918] After changing the placement, gaze data, emotion data, and sales data will be collected again to monitor the effects. The effectiveness of the placement will be verified based on the re-collected data.

[0919] (Application example 2)

[0920] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0921] Modern brick-and-mortar stores lack a means for comprehensively analyzing customer gaze, emotion, and sales data to effectively determine optimal product placement. Existing methods struggle to specifically and efficiently reflect product sales growth and customer interest. Furthermore, conventional systems struggle with real-time data collection and analysis, requiring significant time and effort to optimize product placement. Therefore, the present invention aims to solve these problems by providing an integrated, real-time analysis system for optimizing product placement in brick-and-mortar stores.

[0922] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0923] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data, means for analyzing the gaze data, sales data, and emotion data and simulating an optimal layout for each product, means for providing a recommended layout, and means for providing a terminal for displaying the recommended layout to the user. This makes it possible to efficiently determine the optimal product layout through real-time data collection and analysis, maximizing customer interest and increasing sales.

[0924] Definitions of important words

[0925] The "gaze data acquisition means" is a means for acquiring the direction of a customer's gaze and point of gaze in real time using devices such as in-store cameras and surveillance cameras.

[0926] The "sales data acquisition means" is a means for collecting data on the sales of each product in cooperation with an information management system or a POS system.

[0927] The "emotion data acquisition means" is a means for analyzing the customer's facial expressions and voice data using a voice recognition device or an emotion analysis engine, and acquiring the customer's emotional state as data.

[0928] The "data analysis means" is a means for comprehensively analyzing gaze data, sales data, and emotion data on a server, and evaluating the placement effectiveness of each product.

[0929] The "optimal placement simulation means" is a means of simulating the optimal placement of each product using an AI model based on information obtained from the data analysis means.

[0930] The "means for providing recommended layout" is a means for proposing changes to the product layout based on the results of the optimum layout simulation and notifying the user of this information.

[0931] A "recommended layout display terminal" is a terminal that provides recommended layout information to a user via a smartphone, tablet, or the like.

[0932] MODE FOR CARRYING OUT THE INVENTION

[0933] System configuration overview

[0934] The present invention relates to a system for optimizing product placement in a physical store, and is composed of the following main components:

[0935] 1. Method of acquiring gaze data: Customer gaze data is acquired in real time using in-store cameras and surveillance cameras.

[0936] 2. Sales data acquisition method: Collect sales data for each product in cooperation with the POS system.

[0937] 3. Emotion data acquisition means: Includes a voice recognition device that analyzes the customer's facial expressions and voice data to acquire emotion data.

[0938] 4. Data analysis means: The gaze data, sales data, and emotion data collected on the server are integrated and analyzed.

[0939] 5. Optimal placement simulation method: Using an AI model, the optimal placement of each product is simulated based on the analyzed data.

[0940] 6. Layout recommendation method: A layout recommendation is provided based on the results of the optimal layout simulation.

[0941] 7. Devices that display recommended layouts: Display recommended layouts to users via smartphones and tablets.

[0942] System program and processing details

[0943] Data Collection Module

[0944] Hardware: surveillance cameras, microphones, POS systems

[0945] Software: Camera control software, voice recognition software, POS system data integration API

[0946] The server collects real-time gaze and emotion data from surveillance cameras and voice recognition devices installed in the store, and receives sales data from the POS system. This data is stored in a database and used for subsequent analysis.

[0947] Data Analysis Module

[0948] Hardware: Server

[0949] Software: Python, Pandas, Scikit-learn, TensorFlow

[0950] The server integrates the gaze data, sales data, and emotional data stored in the database, performs statistical analysis of gaze direction, and evaluates the emotional state of customers. An AI model (e.g., a neural network using TensorFlow) is used to analyze the correlation between gaze data and sales data and evaluate the effectiveness of product arrangements.

[0951] Simulation Module

[0952] Hardware: Server

[0953] Software: PyTorch, Matplotlib

[0954] The server uses an AI model to simulate optimal product placement based on collected data, thereby calculating the optimal placement that maximizes product visibility and placement effectiveness.

[0955] Recommended layout module

[0956] Hardware: Smartphones, tablets

[0957] Software: React Native, Firebase

[0958] The server then sends the recommended layout information based on the simulation results to a smartphone or tablet, where users can check and implement the optimal layout information.

[0959] Examples and prompts

[0960] Example: Optimizing the placement of new product B

[0961] For example, if new product B is introduced to a store, the system operates as follows:

[0962] Gaze data collection: A surveillance camera is aimed at the display shelf of new product B to capture customer gaze data.

[0963] Emotional data collection: A voice recognition device analyzes the customer's voice and collects emotional data.

[0964] Sales data collection: The POS system sends sales data for new product B to the server.

[0965] Data analysis and simulation: The server analyzes gaze data, emotion data, and sales data, and simulates optimal placement.

[0966] Layout recommendation: The application informs, "If new product B is placed on the left shelf near the entrance, it will receive 40% more views and increase sales by 20%."

[0967] Example prompts for generative AI models

[0968] Generate the desired layout simulation results. Please integrate and analyze the following data: gaze data (coordinates, time), emotion data (customer facial expressions, voice), and sales data (sales by product). The final output format should be a layout proposal such as "Place product X in position Y." Example: New product B, on the left shelf near the entrance, increases gazes by 40%, increases sales by 20%.

[0969] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0970] Program processing flow

[0971] Step 1: Data collection

[0972] The device captures gaze data in real time using a surveillance camera, collects emotion data using a voice recognition device, and periodically acquires sales data from a POS system, which are then sent to a server and stored in a database.

[0973] Input: Gaze data from surveillance cameras, emotion data from voice recognition devices, sales data from POS systems

[0974] Output: Gaze data, emotion data, sales data stored in a database

[0975] Step 2: Data Preprocessing

[0976] The server cleans the gaze, emotion, and sales data stored in the database and converts it into an analyzable format: gaze data is organized by coordinates and time, emotion data is tagged as the customer's emotional state, and sales data is aggregated by product.

[0977] Input: Gaze data, emotion data, sales data stored in the database

[0978] Output: Preprocessed gaze data, emotion data, sales data

[0979] Step 3: Data analysis

[0980] The server uses the pre-processed data to evaluate the correlation between gaze data and sales data, and analyzes the emotional state of customers, using statistical analysis tools with Python and Pandas, and machine learning models with Scikit-learn.

[0981] Input: Preprocessed gaze data, emotion data, sales data

[0982] Output: Correlation results of analyzed data, customer emotional state

[0983] Step 4: Optimal layout simulation

[0984] Based on the analysis results, the server uses an AI model using PyTorch and TensorFlow to simulate optimal product placement, for example, calculating where to place a new product to attract the most customer attention and increase sales.

[0985] Input: Correlation results of analyzed data, customer emotional state

[0986] Output: Simulation results of optimal product placement

[0987] Step 5: Generate a recommended layout

[0988] The server generates specific product placement proposals based on the simulation results, including the extent to which the number of views and sales will increase by placing which products in which positions.

[0989] Input: Optimal product placement simulation results

[0990] Output: Recommended layout suggestions

[0991] Step 6: Notification of recommended layout

[0992] The layout recommendations are sent to devices such as smartphones and tablets, where users can review and implement them. For example, the application might notify users, "Placing new product B on the left shelf near the entrance will increase the number of views by 40% and increase sales by 20%."

[0993] Input: Recommended layout suggestions

[0994] Output: The recommended layout displayed on the user's device

[0995] Through these steps, the system collects data in real time and optimizes product placement to maximize customer interest and increase sales.

[0996] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0997] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0998] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0999] [Third embodiment]

[1000] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1001] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1002] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1003] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1004] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1005] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1006] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1007] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1008] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1009] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1010] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1011] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1012] ---

[1013] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, and store interior data. Specifically, the system includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data to simulate optimal placement for each product, and a means for providing a recommended layout based on the optimal placement.

[1014] System configuration

[1015] This system consists of the following main elements:

[1016] 1. Method of acquiring gaze data:

[1017] Device: In-store cameras capture customer gaze data in real time, allowing us to measure which products and areas attract customers' attention.

[1018] 2. Sales data acquisition method:

[1019] Terminal: The POS system acquires sales data for each product and periodically sends it to the server.

[1020] 3. Data analysis methods:

[1021] Server: Analyzes the collected gaze data and sales data. Based on the information obtained from the gaze data, it identifies products and areas that customers pay attention to, and integrates this with sales data to evaluate the effectiveness of product placement.

[1022] 4. Simulation Method:

[1023] Server: Utilizing AI models, it analyzes gaze data and sales data to simulate optimal product placement. Based on this simulation, it maximizes product visibility and placement effectiveness.

[1024] 5. Recommended layout methods:

[1025] Server: Calculates the recommended placement from the simulation results and notifies the store manager's terminal.

[1026] Program processing

[1027] This system is implemented through the following series of processes.

[1028] Data collection

[1029] server

[1030] Receives real-time data from in-store cameras and POS systems.

[1031] The received gaze data and sales data are stored in a database.

[1032] Terminal

[1033] A camera is placed to acquire gaze data and data is collected continuously.

[1034] Sales data from the POS system is periodically sent to the server.

[1035] Data analysis and simulation

[1036] server

[1037] Integrate collected gaze data with sales data to identify areas of customer interest.

[1038] Using an AI model, we analyze the correlation between the number of views and sales.

[1039] Simulate optimal product placement.

[1040] The recommended layout is sent to the administrator's device.

[1041] Implementing the recommended layout

[1042] User

[1043] Store staff adjust product placement based on the recommended layout.

[1044] Review gaze and sales data and make further adjustments as needed.

[1045] Specific examples

[1046] As an example, consider the case where new product A is placed in a store.

[1047] Terminal

[1048] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[1049] The POS system sends sales data for new product A to the server.

[1050] server

[1051] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[1052] By integrating sales data and gaze data, we simulate which area new product A should be placed in to maximize its sales.

[1053] The recommended layout is notified to the administrator's device, and an example is shown: "Moving to the shelf near the entrance will increase the number of views by 45%."

[1054] User

[1055] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[1056] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[1057] In this way, gaze data and sales data can be effectively utilized to achieve optimal product placement.

[1058] The processing flow will be explained below.

[1059] Step 1:

[1060] Terminal

[1061] Cameras are installed in appropriate locations in the store to capture customer gaze data in real time.

[1062] The gaze data collected by the camera is sent to a server.

[1063] Sales data for each product is periodically extracted from the POS system and sent to the server.

[1064] Step 2:

[1065] server

[1066] The received gaze data is stored in a database.

[1067] Store sales data received from the POS system in a database.

[1068] Store interior data is stored in a database and any updates are reflected.

[1069] Step 3:

[1070] server

[1071] Preprocessing of gaze data and sales data is performed.

[1072] Synchronize gaze data and sales data based on timestamps.

[1073] Link gaze data and sales data based on product ID.

[1074] Step 4:

[1075] server

[1076] Using an AI model, we analyze the correlation between the number of customer gazes and sales figures.

[1077] Quantify the correlation between the number of views and sales of a specific product.

[1078] Identify customer interest areas using gaze data.

[1079] Step 5:

[1080] server

[1081] Based on an AI model, gaze data and sales data are analyzed to simulate optimal product placement.

[1082] Calculate the optimal combination of view counts and sales for each product.

[1083] It works in conjunction with the store floor map to generate recommended layouts.

[1084] Step 6:

[1085] server

[1086] Create a recommended layout based on the simulation results.

[1087] The optimal product placement is illustrated and sent to the administrator's terminal.

[1088] Step 7:

[1089] Terminal

[1090] Review the recommended layout provided by your administrator.

[1091] Instruct store staff to reposition products to recommended locations.

[1092] Step 8:

[1093] User

[1094] Store staff arranges products based on the recommended layout.

[1095] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[1096] Example 1

[1097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1098] Conventional store layouts lack a means to comprehensively analyze customer gaze data and sales data and simulate optimal product placement. This has led to the problem that product placement effects cannot be maximized and sales cannot be expected to increase.

[1099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1100] In this invention, the server includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data and simulating the optimal placement for each product, a means for analyzing the correlation between the gaze data and sales data using a generative AI model, and a means for providing a recommended layout based on the optimal placement. This makes it possible to effectively utilize the gaze data and sales data, maximize the effectiveness of product placement, and increase sales.

[1101] "Gaze data" is data that indicates in which direction and at which products customers are looking in a store.

[1102] "Sales data" is data that records the sales status of each product, and includes the product name, unit price, number of units sold, date and time of sale, etc.

[1103] A "generative AI model" is an artificial intelligence model used to analyze the correlation between gaze data and sales data and optimize product placement.

[1104] A "recommended layout" is a layout that shows the optimal way to arrange products, calculated based on gaze data and sales data.

[1105] The "in-store camera" is a camera device installed in the store to acquire customer line-of-sight data.

[1106] A "sales management system" is a system that manages the sales situation of a store and records and transmits sales data for each product.

[1107] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, and store interior data. Specifically, the system includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data to simulate optimal placement for each product, and a means for providing a recommended layout based on the optimal placement.

[1108] The system consists of the following main elements:

[1109] Eye gaze data acquisition method

[1110] Terminal

[1111] Cameras installed in the store capture customer gaze data in real time, making it possible to measure which products and areas attract customers' attention.

[1112] Sales data acquisition method

[1113] Terminal

[1114] The sales management system acquires sales data for each product and periodically sends it to the server. The sales data includes the product name, unit price, number of units sold, and the date and time of the sale.

[1115] Data Analysis Methods

[1116] server

[1117] The server analyzes the collected gaze data and sales data. Based on the information obtained from the gaze data, it identifies products and areas that customers are paying attention to, and integrates this with sales data to evaluate the effectiveness of product arrangements.

[1118] Simulation Method

[1119] server

[1120] Using a generative AI model, it analyzes gaze data and sales data to simulate the optimal placement of each product, thereby maximizing product visibility and placement effectiveness.

[1121] Layout recommendation method

[1122] server

[1123] The system calculates the recommended placement from the simulation results and notifies the store manager's terminal, including the reasons for the recommended placement and specific numerical results.

[1124] Specific examples

[1125] As a specific example, consider the case where new product A is placed in a store.

[1126] Terminal

[1127] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[1128] The sales management system sends sales data for new product A to the server.

[1129] server

[1130] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[1131] By integrating sales data and gaze data, we simulate which area new product A should be placed in to maximize its sales.

[1132] The recommended layout is notified to the administrator's device, and an example layout is presented that suggests, "Moving shelves closer to the entrance will increase the number of views by 45% and increase sales by 20%."

[1133] User

[1134] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[1135] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[1136] Example prompts to input to the generative AI model

[1137] "Please simulate the impact on the number of glances and sales if new product A is moved to a shelf near the entrance."

[1138] This allows for the effective use of gaze data and sales data to achieve optimal product placement.

[1139] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1140] Step 1:

[1141] Data collection

[1142] Terminal

[1143] In-store cameras installed on the terminals capture customer gaze data in real time. The cameras track customer eye movements and record which areas and products their gazes are focused on. The input is the camera's video data, and the output is each customer's gaze data.

[1144] The sales management system acquires sales data for each product in real time and sends it to the server at regular intervals. The input is sales records and the output is sales data.

[1145] server

[1146] The server stores the gaze data and sales data sent from the in-store cameras and the sales management system in a database.

[1147] Step 2:

[1148] Data Integration

[1149] server

[1150] Gaze data and sales data are retrieved from the database and integrated. The gaze data and sales data are aligned on the same time axis to analyze the level of interest customers show in products and how this affects sales. The input is gaze data and sales data, and the output is an integrated dataset.

[1151] Specific operations include converting the data format and complementing missing data.

[1152] Step 3:

[1153] Correlation analysis

[1154] server

[1155] The server uses a generative AI model to analyze the correlation between gaze counts and sales numbers. It applies machine learning algorithms to find patterns and trends between gaze data and sales data. The input is the integrated dataset, and the output is the result of the correlation analysis.

[1156] Specific operations include preprocessing, feature extraction, and model application.

[1157] Step 4:

[1158] Simulation of optimal layout

[1159] server

[1160] The server simulates the optimal placement for each product based on the results of the correlation analysis. It uses an AI model to evaluate various placement scenarios based on gaze data and sales data to identify the most effective placement. The input is the result of the correlation analysis, and the output is the simulation result of the optimal placement.

[1161] Specifically, multiple scenarios are generated and evaluated.

[1162] Step 5:

[1163] Providing recommended layouts

[1164] server

[1165] The system calculates the recommended layout from the simulation results and notifies the store manager's terminal. The notification includes the reasons for the recommended layout and specific numerical results. The input is the simulation result of the optimal layout, and the output is a notification of the recommended layout.

[1166] Step 6:

[1167] Implementing the recommended layout

[1168] User

[1169] Store staff adjust product placement based on the recommended layout they receive. For example, move new product A to a shelf closer to the store entrance. The input is the recommended layout notification, and the output is the new in-store layout.

[1170] Specific operations include moving and rearranging product shelves.

[1171] Step 7:

[1172] Monitoring the effectiveness

[1173] Terminal

[1174] Gaze data and sales data are collected again to continuously monitor the effectiveness of the new placement. Data for improvement is collected. The input is the gaze data and sales data after the change, and the output is data to evaluate the effectiveness.

[1175] By following the above steps, it is possible to effectively utilize gaze data and sales data to achieve optimal product placement.

[1176] (Application example 1)

[1177] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1178] Conventional methods make it difficult to efficiently and effectively optimize product placement in stores. In particular, the lack of a system that collects customer behavior and gaze data in real time and integrates and analyzes it with sales data makes it difficult to quickly evaluate the effectiveness of product placement and provide an optimal layout. As a result, stores are often slow to make appropriate decisions to maximize sales.

[1179] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1180] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for analyzing the gaze data and sales data and simulating optimal product placement, means for providing a recommended layout based on the optimal placement, means for collecting gaze data in real time using a smart device, means for performing data analysis and simulation using an AI model, and means for displaying the recommended layout on the interface of the smart device in real time. This makes it possible to effectively analyze customer behavior data and sales data and quickly and appropriately achieve optimal product placement.

[1181] "Gaze data" is data that measures a customer's eye movements and points of gaze and records them in digital format.

[1182] "Sales data" refers to data including sales information, sales quantity, sales date and time, etc., for each product.

[1183] "Simulation" is the process of recreating virtual environments and situations based on collected data and attempting optimal placement and operation.

[1184] The "recommended layout" is the optimal product placement pattern derived based on the analysis and simulation results.

[1185] "Smart devices" is a general term for wearable devices and mobile terminals that can connect to the Internet and are capable of advanced calculations and analysis.

[1186] An "AI model" is a collection of algorithms and programs designed to solve a specific problem using artificial intelligence technology.

[1187] An "interface" is a means or mechanism by which systems, devices, and software exchange information with each other.

[1188] This invention relates to a system that performs an integrated analysis of customer gaze data, sales data, and store interior data to automatically simulate optimal product placement. To specifically implement this system, the following hardware and software are used.

[1189] Hardware and software used

[1190] 1. Hardware

[1191] Smart Device: A device used to collect customer gaze data in real time. A specific example is smart glasses (e.g., Google Glass).

[1192] Server: The main hardware used to analyze large amounts of data and simulate optimal product placement.

[1193] POS system: A device that collects product sales data and sends it to a server.

[1194] 2. Software

[1195] Data Analysis Platform: Software used to analyze gaze data and sales data, specifically using Python and TensorFlow.

[1196] Smart Device Interface: An interface for displaying recommended layouts on smart glasses, specifically using the Android Wear platform.

[1197] Explanation of program processing

[1198] 1. Data Collection

[1199] Terminal: The smart glasses constantly collect customer gaze data and send it to the server in real time. The POS system periodically sends sales data for each product to the server.

[1200] Server: The server receives the gaze data and sales data and stores them in a database.

[1201] 2. Data analysis and simulation

[1202] Server: Analyzes collected gaze data and sales data using an AI model using Python and TensorFlow. Through this analysis, areas of customer interest are identified and the correlation between gaze counts and sales is evaluated. Based on this, optimal product placement is simulated.

[1203] 3. Display recommended layout

[1204] Server: Calculates the recommended layout based on the simulation results and sends it to the smart device interface in real time.

[1205] Terminal: A recommended layout is displayed on the smart glasses interface, and staff adjusts product placement according to instructions.

[1206] Specific example explanation

[1207] As an example, consider the case where new product B is introduced into a store.

[1208] 1. Data collection: Staff wearing smart glasses walk around the shelves near new product B to collect customer gaze data. Sales data for new product B is also obtained from the POS system.

[1209] 2. Data analysis and simulation: The server analyzes the collected gaze data and sales data to determine how much customer interest New Product B is attracting. Using an AI model, it simulates optimal placement based on the sales and gaze data.

[1210] 3. Presentation of recommended layout: The recommended layout information is displayed on the smart glasses, and the staff is instructed to "move new product B closer to the entrance, and the number of gazes will increase by 30%."

[1211] 4. Implementation: Staff move new product B to the designated location and monitor gaze and sales data again to confirm the effectiveness of the new placement.

[1212] Prompt Sentence Examples

[1213] "In order to optimize the layout based on gaze data and sales data when new product B is moved closer to the entrance, please simulate which area should be placed to maximize the number of gazes."

[1214] In this way, stores can efficiently analyze customer behavior data and optimize product placement in real time.

[1215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1216] Step 1:

[1217] Data collection (terminal)

[1218] The smart glasses constantly collect customer gaze data and send it to a server via Wi-Fi. The POS system periodically sends sales data for each product to the server. The input is customer gaze data and sales data, and the output is the data sent to the server.

[1219] Step 2:

[1220] Receiving and storing data (server)

[1221] The server receives the gaze data and sales data and stores them in a database. The input is gaze data and sales data, and the output is raw data stored in the database. Specifically, the received data is converted into an appropriate format and inserted into the corresponding table in the database.

[1222] Step 3:

[1223] Data preprocessing (server)

[1224] The server preprocesses the stored gaze data and sales data. This includes imputing missing values, scaling, and removing unnecessary data. The input is the raw data stored in the database, and the output is the preprocessed data. Specifically, the data is reformatted using Python's pandas and NumPy.

[1225] Step 4:

[1226] Data analysis (server)

[1227] An AI model (e.g., using TensorFlow) analyzes the preprocessed gaze data and sales data to evaluate the correlation between the number of gazes and the number of sales. The input is the preprocessed gaze data and sales data, and the output is the analysis results. Specifically, a regression model is used to analyze the correlation between the number of gazes and the number of sales, and important parameters are extracted.

[1228] Step 5:

[1229] Simulation (server)

[1230] The server simulates the optimal product placement based on the analysis results. The input is the analysis results, and the output is the simulation result of the optimal placement. In concrete terms, a generative AI model is used for the simulation, and prompts are used to evaluate the effectiveness of different placement patterns. An example of a prompt is, "To optimize the layout based on gaze data and sales data when new product B is moved closer to the entrance, please simulate which area should it be placed in to maximize the number of gazes."

[1231] Step 6:

[1232] Generate recommended layout (server)

[1233] The server derives the optimal product placement from the simulation results and generates a recommended layout. The input is the simulation results and the output is the recommended layout. Specifically, the server analyzes the simulation results and selects the most effective placement pattern. Recommended layout information is generated.

[1234] Step 7:

[1235] Display recommended layout (device)

[1236] The server sends the recommended layout to the smart glasses interface in real time and presents it to the staff. The input is the recommended layout information, and the output is the recommended layout displayed on the smart glasses. Specifically, the data is sent to the smart glasses via Wi-Fi and visually displayed on the interface.

[1237] Step 8:

[1238] Product placement adjustment (user)

[1239] Staff members arrange products in designated locations within the store based on the recommended layout. The input is the recommended layout displayed on the smart glasses, and the output is the actual in-store layout with the adjusted layout. Specifically, staff members move the product locations according to instructions, and gaze data and sales data are collected again.

[1240] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1241] ---

[1242] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, emotion data, and store interior data. Specifically, the system includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data using an emotion engine, means for analyzing the gaze data, sales data, and emotion data to simulate optimal placement for each product, and means for providing a recommended layout based on the optimal placement.

[1243] System configuration

[1244] This system consists of the following main elements:

[1245] 1. Method of acquiring gaze data:

[1246] Device: In-store cameras capture customer gaze data in real time, allowing us to measure which products and areas attract customers' attention.

[1247] 2. Sales data acquisition method:

[1248] Terminal: The POS system acquires sales data for each product and periodically sends it to the server.

[1249] 3. Emotion data acquisition method:

[1250] Terminal: Cameras and microphones installed in the store are used to recognize customers' faces and analyze their voices, and an emotion engine is used to obtain customer emotion data.

[1251] 4. Data analysis methods:

[1252] Server: Analyzes the collected gaze data, sales data, and emotion data. Based on information obtained from gaze data, it identifies products and areas that customers pay attention to, takes emotion data into account, and integrates it with sales data to evaluate the effectiveness of product arrangements.

[1253] 5. Simulation Method:

[1254] Server: Utilizing AI models, it analyzes gaze data, sales data, and emotion data to simulate optimal product placement. Based on this simulation, it maximizes product visibility and placement effectiveness.

[1255] 6. Recommended layout methods:

[1256] Server: Calculates the recommended placement from the simulation results and notifies the store manager's terminal.

[1257] Program processing

[1258] This system is implemented through the following series of processes.

[1259] Data collection

[1260] server

[1261] Receives real-time data from in-store cameras and POS systems.

[1262] The received gaze data, emotion data, and sales data are stored in a database.

[1263] Terminal

[1264] A camera is placed to acquire gaze data and data is collected continuously.

[1265] Cameras and microphones are installed to capture emotional data, and customers' facial expressions and voices are analyzed in real time.

[1266] Sales data from the POS system is periodically sent to the server.

[1267] Data analysis and simulation

[1268] server

[1269] The collected gaze data, emotion data and sales data are integrated to identify the customer's areas of interest and emotional state.

[1270] Using an AI model, the correlation between gaze counts, emotional data, and sales figures is analyzed.

[1271] Simulate optimal product placement.

[1272] The recommended layout is sent to the administrator's device.

[1273] Implementing the recommended layout

[1274] User

[1275] Store staff adjust product placement based on the recommended layout.

[1276] Gaze data, emotion data, and sales data will be monitored again to assess the effectiveness of the new placement.

[1277] Specific examples

[1278] As an example, consider the case where new product A is placed in a store.

[1279] Terminal

[1280] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[1281] Customers' facial expressions and voices are collected using cameras and microphones to capture emotional data.

[1282] The POS system sends sales data for new product A to the server.

[1283] server

[1284] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[1285] Based on emotional data, analyze how customers feel about new product A.

[1286] By integrating sales data, gaze data, and emotion data, we simulate which area new product A should be placed in to maximize its sales.

[1287] The recommended layout is sent to the administrator's device, and an example is shown: "Moving the shelves closer to the entrance will increase the number of glances by 45%, and customer preference will also increase."

[1288] User

[1289] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[1290] Gaze data, emotion data and sales data will be monitored again to assess the effectiveness of the new placement.

[1291] In this way, gaze data, sales data, and emotion data can be effectively utilized to achieve optimal product placement.

[1292] The processing flow will be explained below.

[1293] Step 1:

[1294] Terminal

[1295] Cameras are installed in appropriate locations in the store to capture customer gaze data in real time.

[1296] Gaze data is sent to a server in real time.

[1297] Cameras and microphones in the store are used to collect customers' facial expressions and voices, obtaining emotional data in real time.

[1298] Emotion data is sent to the server in real time.

[1299] Sales data for each product is periodically extracted from the POS system and sent to the server.

[1300] Step 2:

[1301] server

[1302] The received gaze data, emotion data, and sales data are stored in a database.

[1303] Store interior data is stored in a database and any updates are reflected.

[1304] Step 3:

[1305] server

[1306] Preprocessing of gaze data, emotion data and sales data is performed.

[1307] Gaze data, sales data, and emotion data are synchronized based on timestamps.

[1308] Link gaze data, emotion data and sales data based on product ID.

[1309] Step 4:

[1310] server

[1311] Using an AI model, the correlation between customer gaze count, emotional data, and sales figures is analyzed.

[1312] Quantify the correlation between glances, customer sentiment, and sales for specific products.

[1313] Identify customer interest areas using gaze data and emotion data.

[1314] Step 5:

[1315] server

[1316] Based on an AI model, gaze data, emotion data, and sales data are analyzed to simulate optimal product placement.

[1317] The optimal combination of gaze count, emotional data, and sales figures is calculated for each product.

[1318] It works in conjunction with the store floor map to generate recommended layouts.

[1319] Step 6:

[1320] server

[1321] Create a recommended layout based on the simulation results.

[1322] The optimal product placement is illustrated and sent to the administrator's terminal.

[1323] Step 7:

[1324] Terminal

[1325] Review the recommended layout provided by your administrator.

[1326] Instruct store staff to reposition products to recommended locations.

[1327] Step 8:

[1328] User

[1329] Store staff arranges products based on the recommended layout.

[1330] Gaze data, emotion data, and sales data will be monitored again to assess the effectiveness of the new placement.

[1331] Example 2

[1332] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1333] Conventional systems have optimized product placement based on sales data and gaze data, but have not performed simulations that take into account the emotional state of customers. As a result, product placement is sometimes not optimized, posing challenges in not fully maximizing sales or improving customer satisfaction. Furthermore, there was a lack of a means to comprehensively analyze this data and recommend the optimal placement for each product.

[1334] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1335] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data, means for analyzing the gaze data, sales data, and emotion data to simulate optimal product placement, and means for providing a recommended layout based on the optimal placement. This enables comprehensive analysis including gaze data and emotion data, and enables precise simulation of optimal product placement, thereby maximizing sales and improving customer satisfaction.

[1336] "Gaze data" is data that represents information on which product or area a customer is looking at.

[1337] "Sales data" refers to data that represents information related to sales and sales volume of each product.

[1338] "Emotion data" is data that represents the emotional state of a customer extracted from their facial expressions and voice.

[1339] "Means for acquiring gaze data" refers to a device or system for detecting the gaze movements of customers and collecting that data.

[1340] The "means for acquiring sales data" refers to a device or system for collecting sales information for each product.

[1341] The "means for acquiring emotional data" refers to a device or system for detecting the emotional state of a customer and collecting that data.

[1342] "Simulation" refers to the process of calculating the optimal placement of each product based on collected data.

[1343] "Recommended layout" refers to a proposal for how to arrange products in a store, derived from the simulation results.

[1344] The term "photography device" refers to a device for acquiring gaze data and emotion data in the form of images or videos.

[1345] A "sales management system" is a system for managing the sales status of products and collecting that data.

[1346] The present invention relates to a system that performs an integrated analysis of customer gaze data, sales data, emotion data, and store interior data, and automatically simulates optimal product placement.

[1347] System configuration

[1348] This system consists of the following main elements:

[1349] 1. Method of acquiring gaze data

[1350] Device: Cameras installed in the store capture customer gaze data in real time. For example, an Azure Kinect camera can be used.

[1351] 2. Sales data acquisition method

[1352] Terminal: The sales management system acquires sales data for each product and periodically sends it to the server. The sales management system used includes Oracle's POS system.

[1353] 3. Means of acquiring emotional data

[1354] Terminal: Cameras and microphones installed in the store are used to recognize customers' faces and analyze their voices, and an emotion engine is used to obtain customer emotion data. Emotion analysis is performed using Affectiva's software.

[1355] 4. Data Analysis Methods

[1356] Server: Analyzes the collected gaze data, sales data, and emotion data. Based on the information obtained from the gaze data, it identifies products and areas that customers pay attention to, takes emotion data into account, and integrates it with sales data to evaluate the effectiveness of product arrangements. The Python pandas library is used for analysis.

[1357] 5. Simulation Methods

[1358] Server: Utilizing an AI model, it analyzes gaze data, sales data, and emotion data to simulate the optimal placement of each product. This simulation uses TensorFlow, which maximizes product visibility and placement effectiveness.

[1359] 6. Methods for Providing Recommended Layouts

[1360] Server: Calculates recommended placements based on the simulation results and notifies the store manager's device via email or a dedicated application.

[1361] Specific examples

[1362] As an example, consider the case where new product A is placed in a store.

[1363] Terminal

[1364] An in-store camera (Azure Kinect) is placed facing the shelf of new product A, and gaze data collection begins.

[1365] Cameras and microphones are used to capture emotion data, and customers' facial expressions and voices are collected. Affectiva's software analyzes the customer's emotions.

[1366] The sales management system (Oracle POS system) periodically sends sales data for new product A to the server.

[1367] server

[1368] Based on the collected gaze data, we analyze how much customer gaze New Product A is attracting. We use Python's pandas library to calculate the frequency of gazes.

[1369] Using emotional data, analyze how customers feel about new product A. Affectiva's software calculates an emotional score to determine whether there are many positive emotions.

[1370] Sales data, gaze data, and emotion data are integrated to perform a simulation and calculate which area should be placed to maximize sales of new product A. A TensorFlow model is used.

[1371] The system notifies the store manager of the recommended layout and makes specific suggestions, such as, "By moving new product A to a shelf closer to the entrance, the number of glances will increase by 45%, and customer favorability will also increase."

[1372] User

[1373] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[1374] To evaluate the effectiveness of the new placement, gaze data, emotion data, and sales data are again collected and monitored. This iterative process allows for continuous optimization of the placement.

[1375] Prompt Sentence Examples

[1376] "We conducted a simulation to determine in which area new product A should be placed to maximize sales, and analysis of gaze data revealed that the shelf near the entrance was optimal. If new product A is placed on a shelf near the entrance, the number of gazes will increase by 45%, and it is predicted that customer favorability will also increase. We will notify the store manager of the recommended layout and monitor the effectiveness of the new placement."

[1377] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1378] Step 1:

[1379] Data collection (input: real-time gaze data, sales data, emotion data, output: integrated data)

[1380] Terminal

[1381] In-store cameras capture customer gaze data in real time, specifically the Azure Kinect camera tracks and collects customer gaze data.

[1382] Cameras and microphones installed in the store analyze customers' facial expressions and voices, and an emotion engine captures emotional data using Affectiva's software.

[1383] A sales management system (for example, an Oracle POS system) periodically collects sales data for each product and sends it to a server.

[1384] Step 2:

[1385] Data accumulation and preprocessing (input: integrated data, output: analyzed data)

[1386] server

[1387] The gaze data, emotion data, and sales data sent from the device are stored in a database. An SQL-based database (e.g., MySQL) is used for integrated management.

[1388] Preprocessing various data. Specifically, completing incomplete data, deleting duplicate data, and formatting it into a form suitable for analysis. Cleaning the data using Python's pandas library.

[1389] Step 3:

[1390] Data analysis (input: analysis data, output: analysis results)

[1391] server

[1392] By integrating and analyzing gaze data, emotion data, and sales data, the system identifies products and areas that customers are paying attention to, as well as changes in emotions. Specifically, it calculates the customer's attention time from gaze data and calculates the ratio of positive to negative emotions from emotion data.

[1393] This data is integrated to calculate the popularity and sentiment scores for each product, using the Python pandas library and scikit-learn analysis tools.

[1394] Step 4:

[1395] Simulation (input: analysis results, output: optimal layout plan)

[1396] server

[1397] Based on the analysis results, an AI model is used to simulate optimal product placement, and TensorFlow is used to analyze correlations between gaze data, emotion data, and sales data to identify optimal placement patterns.

[1398] From the simulation results, specific placement plans are created that show which areas will attract the most attention to a particular product and improve customer sentiment scores.

[1399] Step 5:

[1400] Notification of recommended layout (Input: Optimal layout plan, Output: Recommended layout notification)

[1401] server

[1402] Based on the simulation results, the system calculates a recommended layout and notifies the store manager via email or a dedicated application.

[1403] As a specific suggestion, they provide information such as, "Moving new product A to a shelf closer to the entrance will increase the number of glances by 45% and improve customer favorability."

[1404] Step 6:

[1405] Implementation of recommended layout (input: notification of recommended layout, output: data verifying effectiveness after implementation)

[1406] User

[1407] Store staff adjust product placement based on the recommended layout, for example, moving new product A to a shelf closer to the entrance.

[1408] After changing the placement, gaze data, emotion data, and sales data will be collected again to monitor the effects. The effectiveness of the placement will be verified based on the re-collected data.

[1409] (Application example 2)

[1410] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1411] Modern brick-and-mortar stores lack a means for comprehensively analyzing customer gaze, emotion, and sales data to effectively determine optimal product placement. Existing methods struggle to specifically and efficiently reflect product sales growth and customer interest. Furthermore, conventional systems struggle with real-time data collection and analysis, requiring significant time and effort to optimize product placement. Therefore, the present invention aims to solve these problems by providing an integrated, real-time analysis system for optimizing product placement in brick-and-mortar stores.

[1412] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1413] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data, means for analyzing the gaze data, sales data, and emotion data and simulating an optimal layout for each product, means for providing a recommended layout, and means for providing a terminal for displaying the recommended layout to the user. This makes it possible to efficiently determine the optimal product layout through real-time data collection and analysis, maximizing customer interest and increasing sales.

[1414] Definitions of important words

[1415] The "gaze data acquisition means" is a means for acquiring the direction of a customer's gaze and point of gaze in real time using devices such as in-store cameras and surveillance cameras.

[1416] The "sales data acquisition means" is a means for collecting data on the sales of each product in cooperation with an information management system or a POS system.

[1417] The "emotion data acquisition means" is a means for analyzing the customer's facial expressions and voice data using a voice recognition device or an emotion analysis engine, and acquiring the customer's emotional state as data.

[1418] The "data analysis means" is a means for comprehensively analyzing gaze data, sales data, and emotion data on a server, and evaluating the placement effectiveness of each product.

[1419] The "optimal placement simulation means" is a means of simulating the optimal placement of each product using an AI model based on information obtained from the data analysis means.

[1420] The "means for providing recommended layout" is a means for proposing changes to the product layout based on the results of the optimum layout simulation and notifying the user of this information.

[1421] A "recommended layout display terminal" is a terminal that provides recommended layout information to a user via a smartphone, tablet, or the like.

[1422] MODE FOR CARRYING OUT THE INVENTION

[1423] System configuration overview

[1424] The present invention relates to a system for optimizing product placement in a physical store, and is composed of the following main components:

[1425] 1. Method of acquiring gaze data: Customer gaze data is acquired in real time using in-store cameras and surveillance cameras.

[1426] 2. Sales data acquisition method: Collect sales data for each product in cooperation with the POS system.

[1427] 3. Emotion data acquisition means: Includes a voice recognition device that analyzes the customer's facial expressions and voice data to acquire emotion data.

[1428] 4. Data analysis means: The gaze data, sales data, and emotion data collected on the server are integrated and analyzed.

[1429] 5. Optimal placement simulation method: Using an AI model, the optimal placement of each product is simulated based on the analyzed data.

[1430] 6. Layout recommendation method: A layout recommendation is provided based on the results of the optimal layout simulation.

[1431] 7. Devices that display recommended layouts: Display recommended layouts to users via smartphones and tablets.

[1432] System program and processing details

[1433] Data Collection Module

[1434] Hardware: surveillance cameras, microphones, POS systems

[1435] Software: Camera control software, voice recognition software, POS system data integration API

[1436] The server collects real-time gaze and emotion data from surveillance cameras and voice recognition devices installed in the store, and receives sales data from the POS system. This data is stored in a database and used for subsequent analysis.

[1437] Data Analysis Module

[1438] Hardware: Server

[1439] Software: Python, Pandas, Scikit-learn, TensorFlow

[1440] The server integrates the gaze data, sales data, and emotional data stored in the database, performs statistical analysis of gaze direction, and evaluates the emotional state of customers. An AI model (e.g., a neural network using TensorFlow) is used to analyze the correlation between gaze data and sales data and evaluate the effectiveness of product arrangements.

[1441] Simulation Module

[1442] Hardware: Server

[1443] Software: PyTorch, Matplotlib

[1444] The server uses an AI model to simulate optimal product placement based on collected data, thereby calculating the optimal placement that maximizes product visibility and placement effectiveness.

[1445] Recommended layout module

[1446] Hardware: Smartphones, tablets

[1447] Software: React Native, Firebase

[1448] The server then sends the recommended layout information based on the simulation results to a smartphone or tablet, where users can check and implement the optimal layout information.

[1449] Examples and prompts

[1450] Example: Optimizing the placement of new product B

[1451] For example, if new product B is introduced to a store, the system operates as follows:

[1452] Gaze data collection: A surveillance camera is aimed at the display shelf of new product B to capture customer gaze data.

[1453] Emotional data collection: A voice recognition device analyzes the customer's voice and collects emotional data.

[1454] Sales data collection: The POS system sends sales data for new product B to the server.

[1455] Data analysis and simulation: The server analyzes gaze data, emotion data, and sales data, and simulates optimal placement.

[1456] Layout recommendation: The application informs, "If new product B is placed on the left shelf near the entrance, it will receive 40% more views and increase sales by 20%."

[1457] Example prompts for generative AI models

[1458] Generate the desired layout simulation results. Please integrate and analyze the following data: gaze data (coordinates, time), emotion data (customer facial expressions, voice), and sales data (sales by product). The final output format should be a layout proposal such as "Place product X in position Y." Example: New product B, on the left shelf near the entrance, increases gazes by 40%, increases sales by 20%.

[1459] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1460] Program processing flow

[1461] Step 1: Data collection

[1462] The device captures gaze data in real time using a surveillance camera, collects emotion data using a voice recognition device, and periodically acquires sales data from a POS system, which are then sent to a server and stored in a database.

[1463] Input: Gaze data from surveillance cameras, emotion data from voice recognition devices, sales data from POS systems

[1464] Output: Gaze data, emotion data, sales data stored in a database

[1465] Step 2: Data Preprocessing

[1466] The server cleans the gaze, emotion, and sales data stored in the database and converts it into an analyzable format: gaze data is organized by coordinates and time, emotion data is tagged as the customer's emotional state, and sales data is aggregated by product.

[1467] Input: Gaze data, emotion data, sales data stored in the database

[1468] Output: Preprocessed gaze data, emotion data, sales data

[1469] Step 3: Data analysis

[1470] The server uses the pre-processed data to evaluate the correlation between gaze data and sales data, and analyzes the emotional state of customers, using statistical analysis tools with Python and Pandas, and machine learning models with Scikit-learn.

[1471] Input: Preprocessed gaze data, emotion data, sales data

[1472] Output: Correlation results of analyzed data, customer emotional state

[1473] Step 4: Optimal layout simulation

[1474] Based on the analysis results, the server uses an AI model using PyTorch and TensorFlow to simulate optimal product placement, for example, calculating where to place a new product to attract the most customer attention and increase sales.

[1475] Input: Correlation results of analyzed data, customer emotional state

[1476] Output: Simulation results of optimal product placement

[1477] Step 5: Generate a recommended layout

[1478] The server generates specific product placement proposals based on the simulation results, including the extent to which the number of views and sales will increase by placing which products in which positions.

[1479] Input: Optimal product placement simulation results

[1480] Output: Recommended layout suggestions

[1481] Step 6: Notification of recommended layout

[1482] The layout recommendations are sent to devices such as smartphones and tablets, where users can review and implement them. For example, the application might notify users, "Placing new product B on the left shelf near the entrance will increase the number of views by 40% and increase sales by 20%."

[1483] Input: Recommended layout suggestions

[1484] Output: The recommended layout displayed on the user's device

[1485] Through these steps, the system collects data in real time and optimizes product placement to maximize customer interest and increase sales.

[1486] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1487] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1488] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1489] [Fourth embodiment]

[1490] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1491] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1492] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1493] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1494] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1495] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1496] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1497] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1498] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1499] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1500] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1501] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1502] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1503] ---

[1504] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, and store interior data. Specifically, the system includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data to simulate optimal placement for each product, and a means for providing a recommended layout based on the optimal placement.

[1505] System configuration

[1506] This system consists of the following main elements:

[1507] 1. Method of acquiring gaze data:

[1508] Device: In-store cameras capture customer gaze data in real time, allowing us to measure which products and areas attract customers' attention.

[1509] 2. Sales data acquisition method:

[1510] Terminal: The POS system acquires sales data for each product and periodically sends it to the server.

[1511] 3. Data analysis methods:

[1512] Server: Analyzes the collected gaze data and sales data. Based on the information obtained from the gaze data, it identifies products and areas that customers pay attention to, and integrates this with sales data to evaluate the effectiveness of product placement.

[1513] 4. Simulation Method:

[1514] Server: Utilizing AI models, it analyzes gaze data and sales data to simulate optimal product placement. Based on this simulation, it maximizes product visibility and placement effectiveness.

[1515] 5. Recommended layout methods:

[1516] Server: Calculates the recommended placement from the simulation results and notifies the store manager's terminal.

[1517] Program processing

[1518] This system is implemented through the following series of processes.

[1519] Data collection

[1520] server

[1521] Receives real-time data from in-store cameras and POS systems.

[1522] The received gaze data and sales data are stored in a database.

[1523] Terminal

[1524] A camera is placed to acquire gaze data and data is collected continuously.

[1525] Sales data from the POS system is periodically sent to the server.

[1526] Data analysis and simulation

[1527] server

[1528] Integrate collected gaze data with sales data to identify areas of customer interest.

[1529] Using an AI model, we analyze the correlation between the number of views and sales.

[1530] Simulate optimal product placement.

[1531] The recommended layout is sent to the administrator's device.

[1532] Implementing the recommended layout

[1533] User

[1534] Store staff adjust product placement based on the recommended layout.

[1535] Review gaze and sales data and make further adjustments as needed.

[1536] Specific examples

[1537] As an example, consider the case where new product A is placed in a store.

[1538] Terminal

[1539] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[1540] The POS system sends sales data for new product A to the server.

[1541] server

[1542] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[1543] By integrating sales data and gaze data, we simulate which area new product A should be placed in to maximize its sales.

[1544] The recommended layout is notified to the administrator's device, and an example is shown: "Moving to the shelf near the entrance will increase the number of views by 45%."

[1545] User

[1546] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[1547] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[1548] In this way, gaze data and sales data can be effectively utilized to achieve optimal product placement.

[1549] The processing flow will be explained below.

[1550] Step 1:

[1551] Terminal

[1552] Cameras are installed in appropriate locations in the store to capture customer gaze data in real time.

[1553] The gaze data collected by the camera is sent to a server.

[1554] Sales data for each product is periodically extracted from the POS system and sent to the server.

[1555] Step 2:

[1556] server

[1557] The received gaze data is stored in a database.

[1558] Store sales data received from the POS system in a database.

[1559] Store interior data is stored in a database and any updates are reflected.

[1560] Step 3:

[1561] server

[1562] Preprocessing of gaze data and sales data is performed.

[1563] Synchronize gaze data and sales data based on timestamps.

[1564] Link gaze data and sales data based on product ID.

[1565] Step 4:

[1566] server

[1567] Using an AI model, we analyze the correlation between the number of customer gazes and sales figures.

[1568] Quantify the correlation between the number of views and sales of a specific product.

[1569] Identify customer interest areas using gaze data.

[1570] Step 5:

[1571] server

[1572] Based on an AI model, gaze data and sales data are analyzed to simulate optimal product placement.

[1573] Calculate the optimal combination of view counts and sales for each product.

[1574] It works in conjunction with the store floor map to generate recommended layouts.

[1575] Step 6:

[1576] server

[1577] Create a recommended layout based on the simulation results.

[1578] The optimal product placement is illustrated and sent to the administrator's terminal.

[1579] Step 7:

[1580] Terminal

[1581] Review the recommended layout provided by your administrator.

[1582] Instruct store staff to reposition products to recommended locations.

[1583] Step 8:

[1584] User

[1585] Store staff arranges products based on the recommended layout.

[1586] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[1587] Example 1

[1588] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1589] Conventional store layouts lack a means to comprehensively analyze customer gaze data and sales data and simulate optimal product placement. This has led to the problem that product placement effects cannot be maximized and sales cannot be expected to increase.

[1590] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1591] In this invention, the server includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data and simulating the optimal placement for each product, a means for analyzing the correlation between the gaze data and sales data using a generative AI model, and a means for providing a recommended layout based on the optimal placement. This makes it possible to effectively utilize the gaze data and sales data, maximize the effectiveness of product placement, and increase sales.

[1592] "Gaze data" is data that indicates in which direction and at which products customers are looking in a store.

[1593] "Sales data" is data that records the sales status of each product, and includes the product name, unit price, number of units sold, date and time of sale, etc.

[1594] A "generative AI model" is an artificial intelligence model used to analyze the correlation between gaze data and sales data and optimize product placement.

[1595] A "recommended layout" is a layout that shows the optimal way to arrange products, calculated based on gaze data and sales data.

[1596] The "in-store camera" is a camera device installed in the store to acquire customer line-of-sight data.

[1597] A "sales management system" is a system that manages the sales situation of a store and records and transmits sales data for each product.

[1598] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, and store interior data. Specifically, the system includes a means for acquiring gaze data, a means for acquiring sales data, a means for analyzing the gaze data and sales data to simulate optimal placement for each product, and a means for providing a recommended layout based on the optimal placement.

[1599] The system consists of the following main elements:

[1600] Eye gaze data acquisition method

[1601] Terminal

[1602] Cameras installed in the store capture customer gaze data in real time, making it possible to measure which products and areas attract customers' attention.

[1603] Sales data acquisition method

[1604] Terminal

[1605] The sales management system acquires sales data for each product and periodically sends it to the server. The sales data includes the product name, unit price, number of units sold, and the date and time of the sale.

[1606] Data Analysis Methods

[1607] server

[1608] The server analyzes the collected gaze data and sales data. Based on the information obtained from the gaze data, it identifies products and areas that customers are paying attention to, and integrates this with sales data to evaluate the effectiveness of product arrangements.

[1609] Simulation Method

[1610] server

[1611] Using a generative AI model, it analyzes gaze data and sales data to simulate the optimal placement of each product, thereby maximizing product visibility and placement effectiveness.

[1612] Layout recommendation method

[1613] server

[1614] The system calculates the recommended placement from the simulation results and notifies the store manager's terminal, including the reasons for the recommended placement and specific numerical results.

[1615] Specific examples

[1616] As a specific example, consider the case where new product A is placed in a store.

[1617] Terminal

[1618] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[1619] The sales management system sends sales data for new product A to the server.

[1620] server

[1621] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[1622] By integrating sales data and gaze data, we simulate which area new product A should be placed in to maximize its sales.

[1623] The recommended layout is notified to the administrator's device, and an example layout is presented that suggests, "Moving shelves closer to the entrance will increase the number of views by 45% and increase sales by 20%."

[1624] User

[1625] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[1626] Gaze data and sales data will be monitored again to assess the effectiveness of the new placement.

[1627] Example prompts to input to the generative AI model

[1628] "Please simulate the impact on the number of glances and sales if new product A is moved to a shelf near the entrance."

[1629] This allows for the effective use of gaze data and sales data to achieve optimal product placement.

[1630] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1631] Step 1:

[1632] Data collection

[1633] Terminal

[1634] In-store cameras installed on the terminals capture customer gaze data in real time. The cameras track customer eye movements and record which areas and products their gazes are focused on. The input is the camera's video data, and the output is each customer's gaze data.

[1635] The sales management system acquires sales data for each product in real time and sends it to the server at regular intervals. The input is sales records and the output is sales data.

[1636] server

[1637] The server stores the gaze data and sales data sent from the in-store cameras and the sales management system in a database.

[1638] Step 2:

[1639] Data Integration

[1640] server

[1641] Gaze data and sales data are retrieved from the database and integrated. The gaze data and sales data are aligned on the same time axis to analyze the level of interest customers show in products and how this affects sales. The input is gaze data and sales data, and the output is an integrated dataset.

[1642] Specific operations include converting the data format and complementing missing data.

[1643] Step 3:

[1644] Correlation analysis

[1645] server

[1646] The server uses a generative AI model to analyze the correlation between gaze counts and sales numbers. It applies machine learning algorithms to find patterns and trends between gaze data and sales data. The input is the integrated dataset, and the output is the result of the correlation analysis.

[1647] Specific operations include preprocessing, feature extraction, and model application.

[1648] Step 4:

[1649] Simulation of optimal layout

[1650] server

[1651] The server simulates the optimal placement for each product based on the results of the correlation analysis. It uses an AI model to evaluate various placement scenarios based on gaze data and sales data to identify the most effective placement. The input is the result of the correlation analysis, and the output is the simulation result of the optimal placement.

[1652] Specifically, multiple scenarios are generated and evaluated.

[1653] Step 5:

[1654] Providing recommended layouts

[1655] server

[1656] The system calculates the recommended layout from the simulation results and notifies the store manager's terminal. The notification includes the reasons for the recommended layout and specific numerical results. The input is the simulation result of the optimal layout, and the output is a notification of the recommended layout.

[1657] Step 6:

[1658] Implementing the recommended layout

[1659] User

[1660] Store staff adjust product placement based on the recommended layout they receive. For example, move new product A to a shelf closer to the store entrance. The input is the recommended layout notification, and the output is the new in-store layout.

[1661] Specific operations include moving and rearranging product shelves.

[1662] Step 7:

[1663] Monitoring the effectiveness

[1664] Terminal

[1665] Gaze data and sales data are collected again to continuously monitor the effectiveness of the new placement. Data for improvement is collected. The input is the gaze data and sales data after the change, and the output is data to evaluate the effectiveness.

[1666] By following the above steps, it is possible to effectively utilize gaze data and sales data to achieve optimal product placement.

[1667] (Application example 1)

[1668] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1669] Conventional methods make it difficult to efficiently and effectively optimize product placement in stores. In particular, the lack of a system that collects customer behavior and gaze data in real time and integrates and analyzes it with sales data makes it difficult to quickly evaluate the effectiveness of product placement and provide an optimal layout. As a result, stores are often slow to make appropriate decisions to maximize sales.

[1670] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1671] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for analyzing the gaze data and sales data and simulating optimal product placement, means for providing a recommended layout based on the optimal placement, means for collecting gaze data in real time using a smart device, means for performing data analysis and simulation using an AI model, and means for displaying the recommended layout on the interface of the smart device in real time. This makes it possible to effectively analyze customer behavior data and sales data and quickly and appropriately achieve optimal product placement.

[1672] "Gaze data" is data that measures a customer's eye movements and points of gaze and records them in digital format.

[1673] "Sales data" refers to data including sales information, sales quantity, sales date and time, etc., for each product.

[1674] "Simulation" is the process of recreating virtual environments and situations based on collected data and attempting optimal placement and operation.

[1675] The "recommended layout" is the optimal product placement pattern derived based on the analysis and simulation results.

[1676] "Smart devices" is a general term for wearable devices and mobile terminals that can connect to the Internet and are capable of advanced calculations and analysis.

[1677] An "AI model" is a collection of algorithms and programs designed to solve a specific problem using artificial intelligence technology.

[1678] An "interface" is a means or mechanism by which systems, devices, and software exchange information with each other.

[1679] This invention relates to a system that performs an integrated analysis of customer gaze data, sales data, and store interior data to automatically simulate optimal product placement. To specifically implement this system, the following hardware and software are used.

[1680] Hardware and software used

[1681] 1. Hardware

[1682] Smart Device: A device used to collect customer gaze data in real time. A specific example is smart glasses (e.g., Google Glass).

[1683] Server: The main hardware used to analyze large amounts of data and simulate optimal product placement.

[1684] POS system: A device that collects product sales data and sends it to a server.

[1685] 2. Software

[1686] Data Analysis Platform: Software used to analyze gaze data and sales data, specifically using Python and TensorFlow.

[1687] Smart Device Interface: An interface for displaying recommended layouts on smart glasses, specifically using the Android Wear platform.

[1688] Explanation of program processing

[1689] 1. Data Collection

[1690] Terminal: The smart glasses constantly collect customer gaze data and send it to the server in real time. The POS system periodically sends sales data for each product to the server.

[1691] Server: The server receives the gaze data and sales data and stores them in a database.

[1692] 2. Data analysis and simulation

[1693] Server: Analyzes collected gaze data and sales data using an AI model using Python and TensorFlow. Through this analysis, areas of customer interest are identified and the correlation between gaze counts and sales is evaluated. Based on this, optimal product placement is simulated.

[1694] 3. Display recommended layout

[1695] Server: Calculates the recommended layout based on the simulation results and sends it to the smart device interface in real time.

[1696] Terminal: A recommended layout is displayed on the smart glasses interface, and staff adjusts product placement according to instructions.

[1697] Specific example explanation

[1698] As an example, consider the case where new product B is introduced into a store.

[1699] 1. Data collection: Staff wearing smart glasses walk around the shelves near new product B to collect customer gaze data. Sales data for new product B is also obtained from the POS system.

[1700] 2. Data analysis and simulation: The server analyzes the collected gaze data and sales data to determine how much customer interest New Product B is attracting. Using an AI model, it simulates optimal placement based on the sales and gaze data.

[1701] 3. Presentation of recommended layout: The recommended layout information is displayed on the smart glasses, and the staff is instructed to "move new product B closer to the entrance, and the number of gazes will increase by 30%."

[1702] 4. Implementation: Staff move new product B to the designated location and monitor gaze and sales data again to confirm the effectiveness of the new placement.

[1703] Prompt Sentence Examples

[1704] "In order to optimize the layout based on gaze data and sales data when new product B is moved closer to the entrance, please simulate which area should be placed to maximize the number of gazes."

[1705] In this way, stores can efficiently analyze customer behavior data and optimize product placement in real time.

[1706] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1707] Step 1:

[1708] Data collection (terminal)

[1709] The smart glasses constantly collect customer gaze data and send it to a server via Wi-Fi. The POS system periodically sends sales data for each product to the server. The input is customer gaze data and sales data, and the output is the data sent to the server.

[1710] Step 2:

[1711] Receiving and storing data (server)

[1712] The server receives the gaze data and sales data and stores them in a database. The input is gaze data and sales data, and the output is raw data stored in the database. Specifically, the received data is converted into an appropriate format and inserted into the corresponding table in the database.

[1713] Step 3:

[1714] Data preprocessing (server)

[1715] The server preprocesses the stored gaze data and sales data. This includes imputing missing values, scaling, and removing unnecessary data. The input is the raw data stored in the database, and the output is the preprocessed data. Specifically, the data is reformatted using Python's pandas and NumPy.

[1716] Step 4:

[1717] Data analysis (server)

[1718] An AI model (e.g., using TensorFlow) analyzes the preprocessed gaze data and sales data to evaluate the correlation between the number of gazes and the number of sales. The input is the preprocessed gaze data and sales data, and the output is the analysis results. Specifically, a regression model is used to analyze the correlation between the number of gazes and the number of sales, and important parameters are extracted.

[1719] Step 5:

[1720] Simulation (server)

[1721] The server simulates the optimal product placement based on the analysis results. The input is the analysis results, and the output is the simulation result of the optimal placement. In concrete terms, a generative AI model is used for the simulation, and prompts are used to evaluate the effectiveness of different placement patterns. An example of a prompt is, "To optimize the layout based on gaze data and sales data when new product B is moved closer to the entrance, please simulate which area should it be placed in to maximize the number of gazes."

[1722] Step 6:

[1723] Generate recommended layout (server)

[1724] The server derives the optimal product placement from the simulation results and generates a recommended layout. The input is the simulation results and the output is the recommended layout. Specifically, the server analyzes the simulation results and selects the most effective placement pattern. Recommended layout information is generated.

[1725] Step 7:

[1726] Display recommended layout (device)

[1727] The server sends the recommended layout to the smart glasses interface in real time and presents it to the staff. The input is the recommended layout information, and the output is the recommended layout displayed on the smart glasses. Specifically, the data is sent to the smart glasses via Wi-Fi and visually displayed on the interface.

[1728] Step 8:

[1729] Product placement adjustment (user)

[1730] Staff members arrange products in designated locations within the store based on the recommended layout. The input is the recommended layout displayed on the smart glasses, and the output is the actual in-store layout with the adjusted layout. Specifically, staff members move the product locations according to instructions, and gaze data and sales data are collected again.

[1731] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1732] ---

[1733] The present invention relates to a system that automatically simulates optimal product placement by comprehensively analyzing customer gaze data, sales data, emotion data, and store interior data. Specifically, the system includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data using an emotion engine, means for analyzing the gaze data, sales data, and emotion data to simulate optimal placement for each product, and means for providing a recommended layout based on the optimal placement.

[1734] System configuration

[1735] This system consists of the following main elements:

[1736] 1. Method of acquiring gaze data:

[1737] Device: In-store cameras capture customer gaze data in real time, allowing us to measure which products and areas attract customers' attention.

[1738] 2. Sales data acquisition method:

[1739] Terminal: The POS system acquires sales data for each product and periodically sends it to the server.

[1740] 3. Emotion data acquisition method:

[1741] Terminal: Cameras and microphones installed in the store are used to recognize customers' faces and analyze their voices, and an emotion engine is used to obtain customer emotion data.

[1742] 4. Data analysis methods:

[1743] Server: Analyzes the collected gaze data, sales data, and emotion data. Based on information obtained from gaze data, it identifies products and areas that customers pay attention to, takes emotion data into account, and integrates it with sales data to evaluate the effectiveness of product arrangements.

[1744] 5. Simulation Method:

[1745] Server: Utilizing AI models, it analyzes gaze data, sales data, and emotion data to simulate optimal product placement. Based on this simulation, it maximizes product visibility and placement effectiveness.

[1746] 6. Recommended layout methods:

[1747] Server: Calculates the recommended placement from the simulation results and notifies the store manager's terminal.

[1748] Program processing

[1749] This system is implemented through the following series of processes.

[1750] Data collection

[1751] server

[1752] Receives real-time data from in-store cameras and POS systems.

[1753] The received gaze data, emotion data, and sales data are stored in a database.

[1754] Terminal

[1755] A camera is placed to acquire gaze data and data is collected continuously.

[1756] Cameras and microphones are installed to capture emotional data, and customers' facial expressions and voices are analyzed in real time.

[1757] Sales data from the POS system is periodically sent to the server.

[1758] Data analysis and simulation

[1759] server

[1760] The collected gaze data, emotion data and sales data are integrated to identify the customer's areas of interest and emotional state.

[1761] Using an AI model, the correlation between gaze counts, emotional data, and sales figures is analyzed.

[1762] Simulate optimal product placement.

[1763] The recommended layout is sent to the administrator's device.

[1764] Implementing the recommended layout

[1765] User

[1766] Store staff adjust product placement based on the recommended layout.

[1767] Gaze data, emotion data, and sales data will be monitored again to assess the effectiveness of the new placement.

[1768] Specific examples

[1769] As an example, consider the case where new product A is placed in a store.

[1770] Terminal

[1771] The in-store camera is placed facing the shelf of new product A and gaze data collection begins.

[1772] Customers' facial expressions and voices are collected using cameras and microphones to capture emotional data.

[1773] The POS system sends sales data for new product A to the server.

[1774] server

[1775] Based on the collected gaze data, we analyze the extent to which new product A attracts customer attention.

[1776] Based on emotional data, analyze how customers feel about new product A.

[1777] By integrating sales data, gaze data, and emotion data, we simulate which area new product A should be placed in to maximize its sales.

[1778] The recommended layout is sent to the administrator's device, and an example is shown: "Moving the shelves closer to the entrance will increase the number of glances by 45%, and customer preference will also increase."

[1779] User

[1780] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[1781] Gaze data, emotion data and sales data will be monitored again to assess the effectiveness of the new placement.

[1782] In this way, gaze data, sales data, and emotion data can be effectively utilized to achieve optimal product placement.

[1783] The processing flow will be explained below.

[1784] Step 1:

[1785] Terminal

[1786] Cameras are installed in appropriate locations in the store to capture customer gaze data in real time.

[1787] Gaze data is sent to a server in real time.

[1788] Cameras and microphones in the store are used to collect customers' facial expressions and voices, obtaining emotional data in real time.

[1789] Emotion data is sent to the server in real time.

[1790] Sales data for each product is periodically extracted from the POS system and sent to the server.

[1791] Step 2:

[1792] server

[1793] The received gaze data, emotion data, and sales data are stored in a database.

[1794] Store interior data is stored in a database and any updates are reflected.

[1795] Step 3:

[1796] server

[1797] Preprocessing of gaze data, emotion data and sales data is performed.

[1798] Gaze data, sales data, and emotion data are synchronized based on timestamps.

[1799] Link gaze data, emotion data and sales data based on product ID.

[1800] Step 4:

[1801] server

[1802] Using an AI model, the correlation between customer gaze count, emotional data, and sales figures is analyzed.

[1803] Quantify the correlation between glances, customer sentiment, and sales for specific products.

[1804] Identify customer interest areas using gaze data and emotion data.

[1805] Step 5:

[1806] server

[1807] Based on an AI model, gaze data, emotion data, and sales data are analyzed to simulate optimal product placement.

[1808] The optimal combination of gaze count, emotional data, and sales figures is calculated for each product.

[1809] It works in conjunction with the store floor map to generate recommended layouts.

[1810] Step 6:

[1811] server

[1812] Create a recommended layout based on the simulation results.

[1813] The optimal product placement is illustrated and sent to the administrator's terminal.

[1814] Step 7:

[1815] Terminal

[1816] Review the recommended layout provided by your administrator.

[1817] Instruct store staff to reposition products to recommended locations.

[1818] Step 8:

[1819] User

[1820] Store staff arranges products based on the recommended layout.

[1821] Gaze data, emotion data, and sales data will be monitored again to assess the effectiveness of the new placement.

[1822] Example 2

[1823] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1824] Conventional systems have optimized product placement based on sales data and gaze data, but have not performed simulations that take into account the emotional state of customers. As a result, product placement is sometimes not optimized, posing challenges in not fully maximizing sales or improving customer satisfaction. Furthermore, there was a lack of a means to comprehensively analyze this data and recommend the optimal placement for each product.

[1825] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1826] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data, means for analyzing the gaze data, sales data, and emotion data to simulate optimal product placement, and means for providing a recommended layout based on the optimal placement. This enables comprehensive analysis including gaze data and emotion data, and enables precise simulation of optimal product placement, thereby maximizing sales and improving customer satisfaction.

[1827] "Gaze data" is data that represents information on which product or area a customer is looking at.

[1828] "Sales data" refers to data that represents information related to sales and sales volume of each product.

[1829] "Emotion data" is data that represents the emotional state of a customer extracted from their facial expressions and voice.

[1830] "Means for acquiring gaze data" refers to a device or system for detecting the gaze movements of customers and collecting that data.

[1831] The "means for acquiring sales data" refers to a device or system for collecting sales information for each product.

[1832] The "means for acquiring emotional data" refers to a device or system for detecting the emotional state of a customer and collecting that data.

[1833] "Simulation" refers to the process of calculating the optimal placement of each product based on collected data.

[1834] "Recommended layout" refers to a proposal for how to arrange products in a store, derived from the simulation results.

[1835] The term "photography device" refers to a device for acquiring gaze data and emotion data in the form of images or videos.

[1836] A "sales management system" is a system for managing the sales status of products and collecting that data.

[1837] The present invention relates to a system that performs an integrated analysis of customer gaze data, sales data, emotion data, and store interior data, and automatically simulates optimal product placement.

[1838] System configuration

[1839] This system consists of the following main elements:

[1840] 1. Method of acquiring gaze data

[1841] Device: Cameras installed in the store capture customer gaze data in real time. For example, an Azure Kinect camera can be used.

[1842] 2. Sales data acquisition method

[1843] Terminal: The sales management system acquires sales data for each product and periodically sends it to the server. The sales management system used includes Oracle's POS system.

[1844] 3. Means of acquiring emotional data

[1845] Terminal: Cameras and microphones installed in the store are used to recognize customers' faces and analyze their voices, and an emotion engine is used to obtain customer emotion data. Emotion analysis is performed using Affectiva's software.

[1846] 4. Data Analysis Methods

[1847] Server: Analyzes the collected gaze data, sales data, and emotion data. Based on the information obtained from the gaze data, it identifies products and areas that customers pay attention to, takes emotion data into account, and integrates it with sales data to evaluate the effectiveness of product arrangements. The Python pandas library is used for analysis.

[1848] 5. Simulation Methods

[1849] Server: Utilizing an AI model, it analyzes gaze data, sales data, and emotion data to simulate the optimal placement of each product. This simulation uses TensorFlow, which maximizes product visibility and placement effectiveness.

[1850] 6. Methods for Providing Recommended Layouts

[1851] Server: Calculates recommended placements based on the simulation results and notifies the store manager's device via email or a dedicated application.

[1852] Specific examples

[1853] As an example, consider the case where new product A is placed in a store.

[1854] Terminal

[1855] An in-store camera (Azure Kinect) is placed facing the shelf of new product A, and gaze data collection begins.

[1856] Cameras and microphones are used to capture emotion data, and customers' facial expressions and voices are collected. Affectiva's software analyzes the customer's emotions.

[1857] The sales management system (Oracle POS system) periodically sends sales data for new product A to the server.

[1858] server

[1859] Based on the collected gaze data, we analyze how much customer gaze New Product A is attracting. We use Python's pandas library to calculate the frequency of gazes.

[1860] Using emotional data, analyze how customers feel about new product A. Affectiva's software calculates an emotional score to determine whether there are many positive emotions.

[1861] Sales data, gaze data, and emotion data are integrated to perform a simulation and calculate which area should be placed to maximize sales of new product A. A TensorFlow model is used.

[1862] The system notifies the store manager of the recommended layout and makes specific suggestions, such as, "By moving new product A to a shelf closer to the entrance, the number of glances will increase by 45%, and customer favorability will also increase."

[1863] User

[1864] Based on the recommended layout, the store staff moves new product A to the shelf near the entrance.

[1865] To evaluate the effectiveness of the new placement, gaze data, emotion data, and sales data are again collected and monitored. This iterative process allows for continuous optimization of the placement.

[1866] Prompt Sentence Examples

[1867] "We conducted a simulation to determine in which area new product A should be placed to maximize sales, and analysis of gaze data revealed that the shelf near the entrance was optimal. If new product A is placed on a shelf near the entrance, the number of gazes will increase by 45%, and it is predicted that customer favorability will also increase. We will notify the store manager of the recommended layout and monitor the effectiveness of the new placement."

[1868] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1869] Step 1:

[1870] Data collection (input: real-time gaze data, sales data, emotion data, output: integrated data)

[1871] Terminal

[1872] In-store cameras capture customer gaze data in real time, specifically the Azure Kinect camera tracks and collects customer gaze data.

[1873] Cameras and microphones installed in the store analyze customers' facial expressions and voices, and an emotion engine captures emotional data using Affectiva's software.

[1874] A sales management system (for example, an Oracle POS system) periodically collects sales data for each product and sends it to a server.

[1875] Step 2:

[1876] Data accumulation and preprocessing (input: integrated data, output: analyzed data)

[1877] server

[1878] The gaze data, emotion data, and sales data sent from the device are stored in a database. An SQL-based database (e.g., MySQL) is used for integrated management.

[1879] Preprocessing various data. Specifically, completing incomplete data, deleting duplicate data, and formatting it into a form suitable for analysis. Cleaning the data using Python's pandas library.

[1880] Step 3:

[1881] Data analysis (input: analysis data, output: analysis results)

[1882] server

[1883] By integrating and analyzing gaze data, emotion data, and sales data, the system identifies products and areas that customers are paying attention to, as well as changes in emotions. Specifically, it calculates the customer's attention time from gaze data and calculates the ratio of positive to negative emotions from emotion data.

[1884] This data is integrated to calculate the popularity and sentiment scores for each product, using the Python pandas library and scikit-learn analysis tools.

[1885] Step 4:

[1886] Simulation (input: analysis results, output: optimal layout plan)

[1887] server

[1888] Based on the analysis results, an AI model is used to simulate optimal product placement, and TensorFlow is used to analyze correlations between gaze data, emotion data, and sales data to identify optimal placement patterns.

[1889] From the simulation results, specific placement plans are created that show which areas will attract the most attention to a particular product and improve customer sentiment scores.

[1890] Step 5:

[1891] Notification of recommended layout (Input: Optimal layout plan, Output: Recommended layout notification)

[1892] server

[1893] Based on the simulation results, the system calculates a recommended layout and notifies the store manager via email or a dedicated application.

[1894] As a specific suggestion, they provide information such as, "Moving new product A to a shelf closer to the entrance will increase the number of glances by 45% and improve customer favorability."

[1895] Step 6:

[1896] Implementation of recommended layout (input: notification of recommended layout, output: data verifying effectiveness after implementation)

[1897] User

[1898] Store staff adjust product placement based on the recommended layout, for example, moving new product A to a shelf closer to the entrance.

[1899] After changing the placement, gaze data, emotion data, and sales data will be collected again to monitor the effects. The effectiveness of the placement will be verified based on the re-collected data.

[1900] (Application example 2)

[1901] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1902] Modern brick-and-mortar stores lack a means for comprehensively analyzing customer gaze, emotion, and sales data to effectively determine optimal product placement. Existing methods struggle to specifically and efficiently reflect product sales growth and customer interest. Furthermore, conventional systems struggle with real-time data collection and analysis, requiring significant time and effort to optimize product placement. Therefore, the present invention aims to solve these problems by providing an integrated, real-time analysis system for optimizing product placement in brick-and-mortar stores.

[1903] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1904] In this invention, the server includes means for acquiring gaze data, means for acquiring sales data, means for acquiring emotion data, means for analyzing the gaze data, sales data, and emotion data and simulating an optimal layout for each product, means for providing a recommended layout, and means for providing a terminal for displaying the recommended layout to the user. This makes it possible to efficiently determine the optimal product layout through real-time data collection and analysis, maximizing customer interest and increasing sales.

[1905] Definitions of important words

[1906] The "gaze data acquisition means" is a means for acquiring the direction of a customer's gaze and point of gaze in real time using devices such as in-store cameras and surveillance cameras.

[1907] The "sales data acquisition means" is a means for collecting data on the sales of each product in cooperation with an information management system or a POS system.

[1908] The "emotion data acquisition means" is a means for analyzing the customer's facial expressions and voice data using a voice recognition device or an emotion analysis engine, and acquiring the customer's emotional state as data.

[1909] The "data analysis means" is a means for comprehensively analyzing gaze data, sales data, and emotion data on a server, and evaluating the placement effectiveness of each product.

[1910] The "optimal placement simulation means" is a means of simulating the optimal placement of each product using an AI model based on information obtained from the data analysis means.

[1911] The "means for providing recommended layout" is a means for proposing changes to the product layout based on the results of the optimum layout simulation and notifying the user of this information.

[1912] A "recommended layout display terminal" is a terminal that provides recommended layout information to a user via a smartphone, tablet, or the like.

[1913] MODE FOR CARRYING OUT THE INVENTION

[1914] System configuration overview

[1915] The present invention relates to a system for optimizing product placement in a physical store, and is composed of the following main components:

[1916] 1. Method of acquiring gaze data: Customer gaze data is acquired in real time using in-store cameras and surveillance cameras.

[1917] 2. Sales data acquisition method: Collect sales data for each product in cooperation with the POS system.

[1918] 3. Emotion data acquisition means: Includes a voice recognition device that analyzes the customer's facial expressions and voice data to acquire emotion data.

[1919] 4. Data analysis means: The gaze data, sales data, and emotion data collected on the server are integrated and analyzed.

[1920] 5. Optimal placement simulation method: Using an AI model, the optimal placement of each product is simulated based on the analyzed data.

[1921] 6. Layout recommendation method: A layout recommendation is provided based on the results of the optimal layout simulation.

[1922] 7. Devices that display recommended layouts: Display recommended layouts to users via smartphones and tablets.

[1923] System program and processing details

[1924] Data Collection Module

[1925] Hardware: surveillance cameras, microphones, POS systems

[1926] Software: Camera control software, voice recognition software, POS system data integration API

[1927] The server collects real-time gaze and emotion data from surveillance cameras and voice recognition devices installed in the store, and receives sales data from the POS system. This data is stored in a database and used for subsequent analysis.

[1928] Data Analysis Module

[1929] Hardware: Server

[1930] Software: Python, Pandas, Scikit-learn, TensorFlow

[1931] The server integrates the gaze data, sales data, and emotional data stored in the database, performs statistical analysis of gaze direction, and evaluates the emotional state of customers. An AI model (e.g., a neural network using TensorFlow) is used to analyze the correlation between gaze data and sales data and evaluate the effectiveness of product arrangements.

[1932] Simulation Module

[1933] Hardware: Server

[1934] Software: PyTorch, Matplotlib

[1935] The server uses an AI model to simulate optimal product placement based on collected data, thereby calculating the optimal placement that maximizes product visibility and placement effectiveness.

[1936] Recommended layout module

[1937] Hardware: Smartphones, tablets

[1938] Software: React Native, Firebase

[1939] The server then sends the recommended layout information based on the simulation results to a smartphone or tablet, where users can check and implement the optimal layout information.

[1940] Examples and prompts

[1941] Example: Optimizing the placement of new product B

[1942] For example, if new product B is introduced to a store, the system operates as follows:

[1943] Gaze data collection: A surveillance camera is aimed at the display shelf of new product B to capture customer gaze data.

[1944] Emotional data collection: A voice recognition device analyzes the customer's voice and collects emotional data.

[1945] Sales data collection: The POS system sends sales data for new product B to the server.

[1946] Data analysis and simulation: The server analyzes gaze data, emotion data, and sales data, and simulates optimal placement.

[1947] Layout recommendation: The application informs, "If new product B is placed on the left shelf near the entrance, it will receive 40% more views and increase sales by 20%."

[1948] Example prompts for generative AI models

[1949] Generate the desired layout simulation results. Please integrate and analyze the following data: gaze data (coordinates, time), emotion data (customer facial expressions, voice), and sales data (sales by product). The final output format should be a layout proposal such as "Place product X in position Y." Example: New product B, on the left shelf near the entrance, increases gazes by 40%, increases sales by 20%.

[1950] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1951] Program processing flow

[1952] Step 1: Data collection

[1953] The device captures gaze data in real time using a surveillance camera, collects emotion data using a voice recognition device, and periodically acquires sales data from a POS system, which are then sent to a server and stored in a database.

[1954] Input: Gaze data from surveillance cameras, emotion data from voice recognition devices, sales data from POS systems

[1955] Output: Gaze data, emotion data, sales data stored in a database

[1956] Step 2: Data Preprocessing

[1957] The server cleans the gaze, emotion, and sales data stored in the database and converts it into an analyzable format: gaze data is organized by coordinates and time, emotion data is tagged as the customer's emotional state, and sales data is aggregated by product.

[1958] Input: Gaze data, emotion data, sales data stored in the database

[1959] Output: Preprocessed gaze data, emotion data, sales data

[1960] Step 3: Data analysis

[1961] The server uses the pre-processed data to evaluate the correlation between gaze data and sales data, and analyzes the emotional state of customers, using statistical analysis tools with Python and Pandas, and machine learning models with Scikit-learn.

[1962] Input: Preprocessed gaze data, emotion data, sales data

[1963] Output: Correlation results of analyzed data, customer emotional state

[1964] Step 4: Optimal layout simulation

[1965] Based on the analysis results, the server uses an AI model using PyTorch and TensorFlow to simulate optimal product placement, for example, calculating where to place a new product to attract the most customer attention and increase sales.

[1966] Input: Correlation results of analyzed data, customer emotional state

[1967] Output: Simulation results of optimal product placement

[1968] Step 5: Generate a recommended layout

[1969] The server generates specific product placement proposals based on the simulation results, including the extent to which the number of views and sales will increase by placing which products in which positions.

[1970] Input: Optimal product placement simulation results

[1971] Output: Recommended layout suggestions

[1972] Step 6: Notification of recommended layout

[1973] The layout recommendations are sent to devices such as smartphones and tablets, where users can review and implement them. For example, the application might notify users, "Placing new product B on the left shelf near the entrance will increase the number of views by 40% and increase sales by 20%."

[1974] Input: Recommended layout suggestions

[1975] Output: The recommended layout displayed on the user's device

[1976] Through these steps, the system collects data in real time and optimizes product placement to maximize customer interest and increase sales.

[1977] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1978] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1979] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1980] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1981] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1982] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1983] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1984] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1985] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1986] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1987] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1988] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1989] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1990] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1991] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1992] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1993] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1994] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1995] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1996] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1997] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1998] The following is further disclosed regarding the above embodiment.

[1999] (Claim 1)

[2000] A means for acquiring gaze data;

[2001] a means for obtaining sales data;

[2002] A method for analyzing gaze data and sales data to simulate the optimal placement of each product,

[2003] The system includes a means for providing a recommended layout based on the optimal placement.

[2004] (Claim 2)

[2005] 2. The system of claim 1, wherein the means for acquiring gaze data is an in-store camera.

[2006] (Claim 3)

[2007] 2. The system according to claim 1, wherein the means for acquiring sales data is linked to a sales management system.

[2008] "Example 1"

[2009] (Claim 1)

[2010] A means for acquiring gaze data;

[2011] a means for obtaining sales data;

[2012] A method for analyzing gaze data and sales data to simulate the optimal placement of each product,

[2013] A means of analyzing the correlation between gaze data and sales data using a generative AI model;

[2014] The system includes a means for providing a recommended layout based on the optimal placement.

[2015] (Claim 2)

[2016] 2. The system of claim 1, wherein the means for acquiring gaze data is an in-store camera.

[2017] (Claim 3)

[2018] 2. The system according to claim 1, wherein the means for acquiring sales data is linked to a sales management system.

[2019] "Application Example 1"

[2020] (Claim 1)

[2021] A means for acquiring gaze data;

[2022] a means for obtaining sales data;

[2023] A method for analyzing gaze data and sales data to simulate the optimal placement of each product,

[2024] a means for providing a recommended layout based on the optimal placement;

[2025] a means for collecting gaze data in real time using a smart device;

[2026] A means of performing data analysis and simulation using AI models;

[2027] The system includes means for displaying the recommended layout in real time on the interface of the smart device.

[2028] (Claim 2)

[2029] 2. The system of claim 1, wherein the means for acquiring gaze data is an in-store camera or a smart device.

[2030] (Claim 3)

[2031] 2. The system according to claim 1, wherein the means for acquiring sales data is linked to a sales management system.

[2032] "Example 2: Combining Emotion Engines"

[2033] (Claim 1)

[2034] A means for acquiring gaze data;

[2035] a means for obtaining sales data;

[2036] A means for acquiring emotion data;

[2037] A means for analyzing gaze data, sales data, and emotion data to simulate the optimal placement of each product;

[2038] The system includes a means for providing a recommended layout based on the optimal placement.

[2039] (Claim 2)

[2040] 2. The system according to claim 1, wherein the means for acquiring the line of sight data is a photographing device.

[2041] (Claim 3)

[2042] 2. The system according to claim 1, wherein the means for acquiring sales data is linked to a sales management system.

[2043] "Application example 2 when combining emotion engines"

[2044] Rewritten claims

[2045] (Claim 1)

[2046] A means for acquiring gaze data;

[2047] a means for obtaining sales data;

[2048] A means for acquiring emotion data;

[2049] A means for analyzing gaze data, sales data, and emotion data and simulating the optimal placement of each product;

[2050] a means for providing a recommended layout based on the optimal placement;

[2051] A system including means for providing a terminal to a user that displays a recommended layout.

[2052] (Claim 2)

[2053] 2. The system according to claim 1, wherein the means for acquiring gaze data is a surveillance camera, and the means for analyzing emotion data includes a voice recognition device.

[2054] (Claim 3)

[2055] 2. The system according to claim 1, wherein the means for acquiring sales data is linked to an information management system. [Explanation of symbols]

[2056] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring gaze data; a means for obtaining sales data; A method for analyzing gaze data and sales data to simulate the optimal placement of each product, The system includes a means for providing a recommended layout based on the optimal placement.

2. 2. The system of claim 1, wherein the means for acquiring gaze data is an in-store camera.

3. 2. The system according to claim 1, wherein the means for acquiring sales data is linked to a sales management system.

Images