System

Abstract

A system is provided.SOLUTION: A system, comprising: means for obtaining price update information; means for generating an announcement sentence based on the obtained price update information using generative artificial intelligence technology; means for audibly providing the generated announcement sentence to a customer in a store; means for monitoring customer behavior and product inventory in the store using object detection technology; and means for notifying an employee of out-of-stock information based on the monitored product inventory information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional supermarkets, implementing limited-time sales and special offers requires a great deal of effort and time. Announcements of special sale information must be made manually, and inventory management is also done manually, resulting in inefficiency and lower customer satisfaction. Furthermore, it is difficult to respond quickly when an item runs out, increasing the risk of missed sales. There is a need to solve these issues and improve the efficiency of limited-time sales and the consumer experience. [Means for solving the problem]

[0005] The present invention provides a system that combines multiple means to improve the efficiency of limited-time sales in stores and the consumer experience. Specifically, the system includes a means for acquiring price update information, a means for generating announcements based on the acquired price update information using generative artificial intelligence technology, a means for providing the generated announcements to customers in the store via voice, a means for monitoring customer behavior and product inventory in the store using object detection technology, and a means for notifying employees of out-of-stock information based on the monitored product inventory information. This system enables the rapid and effective communication of limited-time sale information, automated inventory management, and immediate response to out-of-stock issues, thereby improving the efficiency of store operations and customer satisfaction.

[0006] "Price Update Information" refers to the latest changes in the prices of products sold in stores.

[0007] "Generative AI technology" refers to technology that automatically generates text or speech based on input data.

[0008] "Announcement text" refers to the content of the voice message provided to customers in the store.

[0009] "Object detection technology" refers to technology that uses cameras, sensors, etc. to detect and recognize objects and their movements within a store.

[0010] "Customer behavior" refers to the movements of customers within the store and the actions they take when picking up products.

[0011] "Product inventory" refers to the quantity and condition of products stored in a store.

[0012] "Out-of-stock information" refers to information regarding a shortage or lack of stock of a particular product.

[0013] "Notifying employees" refers to communicating specific information to staff or employees via voice or message.

[0014] "Bone conduction earphones" are earphones that use bone conduction to transmit sound, allowing you to hear outside sounds without blocking your ears. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] To implement the present invention, an in-store system is constructed as follows.

[0037] System Overview

[0038] To streamline time sales in stores and improve the consumer experience, the system retrieves price updates, generates and delivers announcements, monitors customer behavior and product inventory using object detection, and notifies staff of out-of-stock information.

[0039] System configuration

[0040] 1. Get price updates

[0041] The server periodically retrieves the latest price updates from the price update system.

[0042] Example: The server collects new price information such as "eggs 200 yen → 180 yen" from a supermarket price update database.

[0043] 2. Announcement Generation

[0044] The server generates announcement text based on the obtained price update information using generative artificial intelligence techniques.

[0045] Example: The server generates an announcement such as "Egg sale, only 180 yen today!"

[0046] 3. Providing announcement text

[0047] The server transmits the generated announcement to a terminal that controls a speaker in the store, and provides it to the customer by voice.

[0048] Example: The terminal announces through the in-store speaker, "We're currently having a special sale on eggs. They're 180 yen per pack. Please buy some."

[0049] 4. Object Detection Monitoring

[0050] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[0051] Example: The server uses data from in-store cameras to capture when a customer picks up an egg and updates the inventory.

[0052] 5. Notification of out-of-stock information

[0053] The server checks the inventory status of the detected product and notifies staff if there is a shortage.

[0054] Example: Generate a notification such as "You're low on eggs. Please get more from the warehouse."

[0055] Users (staff) receive information about out-of-stock items in real time through bone conduction earphones and can quickly replenish products.

[0056] Example: A staff member receives a notification via bone conduction earphones saying, "We are out of eggs. Please replenish them," and restocks the stock from the warehouse.

[0057] A natural language description of the process

[0058] Get price updates

[0059] The server periodically retrieves the latest price update information from the price update system, which is price change information for the product and may fluctuate in real time.

[0060] Announcement generation

[0061] The server uses generative artificial intelligence technology to generate announcements based on the acquired price update information. The generated announcements effectively communicate information about special sales and limited-time sales to customers.

[0062] Providing announcement text

[0063] The server sends the generated announcement text to a terminal in the store, which then provides the content to customers by voice through the in-store speaker, allowing customers to receive special sale information in real time.

[0064] Object detection surveillance

[0065] The server uses object detection technology to monitor customer behavior and product inventory in the store. For example, if a customer picks up a sale item, the server detects that behavior and updates inventory information accordingly.

[0066] Notification of out-of-stock information

[0067] If a shortage occurs based on the monitored inventory information, the server notifies the staff of the shortage via bone conduction earphones, allowing the staff to immediately take action to deal with the shortage.

[0068] This system will streamline the management of time sales, improve the quality of service to customers, and ensure smooth store operations.

[0069] The processing flow will be explained below.

[0070] Step 1:

[0071] The server periodically retrieves the latest price updates from the price update system.

[0072] Specific operation: Refer to the price update database to get new price information. This information includes "product name" and "new price".

[0073] Example: Get the data "Eggs 200 yen → 180 yen".

[0074] Step 2:

[0075] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[0076] Specific operation: The acquired price update information is passed as input data to the AI ​​model, and an appropriate announcement text is automatically generated.

[0077] Example: Generate an announcement for price update information: "Eggs on sale for 180 yen today only!"

[0078] Step 3:

[0079] The server sends the generated announcement to the terminal (in-store speaker) and provides it as audio.

[0080] Specific operation: The generated announcement text is converted into an audio file and sent to the in-store speaker terminal.

[0081] Example: "We're currently having a special sale on eggs. They're 180 yen per pack. Please come and buy some." is announced over the store's speakers.

[0082] Step 4:

[0083] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[0084] Specific actions: Data from cameras and sensors installed in the store is analyzed to detect when a customer picks up a specific product.

[0085] Example: Recognizing when a customer picks up a carton of eggs from the shelf and reducing stock levels.

[0086] Step 5:

[0087] The server checks the inventory status of the detected product and checks for out-of-stock information.

[0088] What it does: Looks up the updated inventory database and checks if a particular product is below a certain threshold.

[0089] Example: If there are only 2 packs of eggs left in stock, an out-of-stock alert will be issued.

[0090] Step 6:

[0091] If the server detects out-of-stock information, it generates a message to notify staff.

[0092] Specific operation: Based on the out-of-stock information, a message requesting replenishment is generated for staff.

[0093] Example: Generate the message "Low eggs left. Please get more from the warehouse."

[0094] Step 7:

[0095] The server delivers the generated notification message to the staff member via bone conduction earphones.

[0096] Specific operation: Convert notification messages into audio signals and send them to staff through bone conduction earphones.

[0097] Example: A staff member receives a notification that "We are out of eggs. Please restock."

[0098] Step 8:

[0099] After receiving the notification, the user (staff member) promptly replenishes the stock.

[0100] Specific actions: Go to the warehouse, retrieve out-of-stock items, and replenish them on the sales floor.

[0101] Example: A staff member takes cartons of eggs from the warehouse and restocks them on the sales shelves.

[0102] Through each step, a system is built that improves the efficiency of time sales and the consumer experience.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] In traditional store operations, managing special sales and time sales required a lot of manual effort, resulting in inefficiency. Because processes such as providing sales information to consumers, managing inventory, and notifying of out-of-stock items were all done manually, it was difficult to update information in real time, which resulted in reduced customer satisfaction and lost sales opportunities.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative AI technology, means for providing the generated announcements to customers in the store via voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for periodically accessing a price update system to acquire price update information, means for generating announcements by sending prompts to a generative AI model based on the acquired price update information, means for receiving video data from in-store cameras and detecting customer behavior using object detection technology, means for transmitting the generated announcements to in-store terminals, and means for using bone conduction earphones worn by staff to notify out-of-stock information. This enables effective management of limited-time sales, real-time product inventory monitoring, and rapid out-of-stock response.

[0108] "Price update information" is data provided when the price of a product in a store is changed, and includes both the old and new price information.

[0109] "Generative AI technology" is an AI technology for text generation and natural language processing, which generates appropriate output text based on specified input data.

[0110] An "announcement" is a sentence created to inform customers of information such as special sales and limited-time sales, and is a text for providing the content of the announcement by voice.

[0111] "Object detection technology" is a technology for recognizing and detecting specific objects from image data, and is used to monitor customer behavior and merchandise movement within a store.

[0112] "Out-of-stock information" is information that is notified when the product inventory in the store falls below a certain amount, and is information that encourages additional purchases or replenishment.

[0113] A "server" is a computer system that provides a specific service over a network. In this system, it is a device that performs functions such as obtaining price updates, generating announcements, and monitoring using object detection.

[0114] A "terminal" is a device that receives announcements sent from the server and provides them to customers in the store by voice, and is usually connected to a speaker system.

[0115] A "prompt" is an instruction given to a generative AI model, containing hints and instructions for generating a specific output.

[0116] A "generative AI model" is an artificial intelligence model that generates appropriate text based on a given prompt sentence, and in the present invention is primarily used to generate announcement sentences.

[0117] Bone conduction earphones are devices that transmit sound through the bones to the inner ear, allowing employees to receive voice notifications without using their hands.

[0118] The present invention relates to a system for streamlining in-store sales and time-limited sales to improve the consumer experience by obtaining price updates, generating and providing announcements, monitoring customer behavior and product inventory using object detection, and notifying staff of out-of-stock information.

[0119] System configuration

[0120] Get price updates

[0121] The server periodically accesses the price update system to retrieve the latest price updates. This process involves using HTTP requests to retrieve data in JSON format from the API. The retrieved data includes information such as the product name, old price, and new price. For example, the server receives price updates such as "eggs from 200 yen to 180 yen" from a supermarket's price update database.

[0122] Announcement generation

[0123] The server sends a prompt message to the generative AI model based on the obtained price update information. This prompt message contains information about which product's price has changed and how. The generative AI model (e.g., GPT-4) generates an announcement message based on this prompt. For example, a prompt message might be created saying, "The price of the product has changed. Eggs: 200 yen → 180 yen. Please generate a sale announcement message based on this information." The generative AI model would then generate an announcement message saying, "Eggs on sale for 180 yen today only!"

[0124] Providing announcement text

[0125] The server sends the generated announcement to a terminal in the store. The terminal receives the announcement and provides it audibly through the store's speakers. This allows customers to receive sale information in real time. For example, the terminal may broadcast an announcement over the store's speakers saying, "We're currently having a sale on eggs. They're 180 yen a pack. Please buy some."

[0126] Object detection surveillance

[0127] The server uses object detection technology to monitor customer behavior and product inventory in the store. YOLO (You Only Look Once) is one example of the object detection technology. This technology is used to analyze video data from cameras in the store, and if a specific action (such as a customer picking up an egg) is detected, inventory information is updated. For example, the server receives video data from cameras in the store, detects the action of a customer picking up an egg, and updates inventory information accordingly.

[0128] Notification of out-of-stock information

[0129] The server monitors inventory information and generates out-of-stock information when a certain number of items falls below a certain level. This information is notified to staff via bone conduction earphones. For example, the server detects that inventory is running low and generates a notification saying, "We have only a few eggs left. Please replenish them from the warehouse." The staff (users) receive the notification via bone conduction earphones saying, "We are out of eggs. Please replenish them," and quickly replenish the items.

[0130] Hardware and software used

[0131] Price update system: A product management database system used by stores

[0132] Generative AI model: GPT-4 (OpenAI)

[0133] Object detection technology: Image recognition technology such as YOLO (You Only Look Once)

[0134] Notification device: Bone conduction earphones

[0135] Examples and prompts

[0136] Below are some examples of prompt sentences.

[0137] "The price of the product has changed. Eggs: 200 yen → 180 yen. Please generate a sale announcement based on this information."

[0138] "Price update information: Eggs 200 yen → 180 yen. Please use this data to create a sale announcement for your store. For example, please use a format like, 'Egg sale, 180 yen today only!'"

[0139] This system will streamline store operations, improve the quality of service to customers, and provide information in real time.

[0140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0141] Step 1: Get price updates

[0142] The server periodically accesses the price update system to obtain the latest price update information. At this time, an HTTP request is used as input and JSON format data is received from the API endpoint. Specifically, the server sends a request to the price update system and obtains price update information such as "Eggs 200 yen → 180 yen." The data includes the product name, old price, new price, etc.

[0143] Step 2: Generate announcement text

[0144] The server sends a prompt message to the generative AI model based on the acquired price update information. It uses the price update information as input and obtains an announcement message as output. Specifically, it creates a prompt message saying, "The price of the product has changed. Eggs have gone from 200 yen to 180 yen. Please generate a sale announcement message based on this information." and sends it to the generative AI model (e.g., GPT-4). The generative AI model then generates a response message saying, "Eggs on sale for 180 yen today only!"

[0145] Step 3: Provide the announcement

[0146] The server sends the generated announcement to a terminal in the store. The generated announcement is used as input and is provided to the customer as audio output. The terminal receives this announcement and provides it to the customer through the in-store speaker. For example, the announcement may say, "We're currently having a special sale on eggs. One pack is 180 yen. Please buy some."

[0147] Step 4: Object detection surveillance

[0148] The server uses object detection technology to monitor customer behavior and product inventory in the store. It uses video data from in-store cameras as input and obtains customer behavior and inventory information as output. Specifically, it analyzes the video from the in-store cameras and detects customer behavior using object detection technology (e.g., YOLO). For example, it detects the behavior of a customer picking up an egg and updates inventory information accordingly.

[0149] Step 5: Out-of-stock notification

[0150] The server uses the monitored inventory information to notify staff when a stockout occurs. It uses the updated inventory information as input and generates a stockout notification as output. Specifically, when inventory falls below a certain amount (e.g., there are fewer than 10 eggs left), a notification is generated. A notification stating "Only a few eggs left. Please add more from the warehouse" is created and sent to staff via bone conduction earphones. The user (staff member) receives this notification and replenishes the product from the warehouse.

[0151] (Application example 1)

[0152] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0153] With conventional time sale management systems, it was difficult to obtain real-time in-store price update information and provide it to customers efficiently. Furthermore, monitoring of customer behavior and product inventory management were insufficient, and responding quickly when an item was out of stock was also an issue. Furthermore, the lack of a push notification function to provide customers with timely information limited the ability to maximize sales.

[0154] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0155] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative artificial intelligence technology, means for providing the generated announcements to customers in the store by voice and push notification, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for notifying employees of out-of-stock information using bone conduction earphones, and means for generating prompts for generating announcements based on the price update information using generative artificial intelligence technology. This enables efficient and real-time time sale management, improving customer experience and maximizing sales.

[0156] "Price update information" refers to the latest price information for products in stores, which is periodically obtained from a database or price update system.

[0157] "Generative artificial intelligence technology" is a technology that uses an AI model to generate text, images, etc. based on input data.

[0158] An "announcement" is a sentence generated to effectively convey information about special sales or limited-time sales.

[0159] "Voice and push notifications" refers to providing information through in-store speakers and notifications to customers' smart devices.

[0160] "Object detection technology" is a technology that uses cameras and sensors to monitor customer behavior and product inventory in stores in real time.

[0161] "Product inventory" refers to the current number of products held in the store.

[0162] "Out-of-stock information" is notification information that is generated when the stock of a specific product falls below a certain level.

[0163] "Employees" refers to staff working in the store.

[0164] Bone conduction earphones are special earphones that transmit sound through the bones rather than the ears, and are worn by employees.

[0165] A "generative prompt" is input text for generating an announcement using generative artificial intelligence technology.

[0166] The present invention provides a system for increasing the efficiency of limited-time sales in stores and improving consumer experiences. Specific embodiments for realizing this system will be described below.

[0167] Overall system configuration

[0168] The system is designed to efficiently manage time-limited sales and special offers in stores and provide them to customers in real time. Its main components include a server, smartphones, bone conduction earphones, in-store cameras, object detection technology, generative artificial intelligence technology (generative AI model), and a push notification system.

[0169] Hardware and Software Configuration

[0170] 1. Server

[0171] The server plays the important role of obtaining price update information, generating announcements using generative artificial intelligence technology (generative AI model), and sending them to in-store smartphones and voice systems. Specifically, the server was built using Python and Node.js, and Firebase Firestore was used as the database. The generative AI model used was OpenAI GPT-4.

[0172] 2. Smartphone

[0173] Smartphones are devices that provide information to customers and employees. Developed cross-platform using Flutter, they provide information through push notifications and in-app messages.

[0174] 3. Bone conduction earphones

[0175] The bone conduction earphones worn by employees are devices used to notify them of stock shortages in real time.

[0176] 4. In-store cameras and object detection technology

[0177] In-store cameras monitor customer behavior and product inventory using object detection technology (using OpenCV), which allows for real-time tracking of customer pick-up behavior and shelf inventory.

[0178] Main processes and data flow

[0179] 1. Get price updates

[0180] The server periodically retrieves the latest price information from an external price update system, which is then stored in Firebase Firestore.

[0181] 2. Announcement Generation

[0182] The server generates an announcement using a generative AI model (OpenAI GPT-4) based on the acquired price information. An example of a prompt is, "Please generate special sale information for eggs. Today's special sale price is 180 yen."

[0183] 3. Providing announcement text

[0184] The generated announcement text is sent from the server to the smartphone via push notification and in-store speakers. For example, a notification might be sent to the customer's smartphone saying, "We're currently having a special sale on eggs. One pack costs 180 yen."

[0185] 4. Object Detection Monitoring

[0186] The video data captured by the in-store cameras is sent to a server and analyzed in real time using OpenCV. The analysis results are used to understand what products customers have picked up and changes in inventory levels.

[0187] 5. Notification of out-of-stock information

[0188] Using object detection technology, if inventory falls below a certain threshold, the server will notify employees of the shortage via bone conduction earphones, and will also send a notification to their smartphone, such as "There are only a few eggs left. Please add more from the warehouse."

[0189] This enables efficient and real-time time sale management, improving customer experience and maximizing sales.

[0190] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0191] Step 1:

[0192] The server retrieves the latest price information from an external price update system. To do this, it uses an HTTP request to retrieve price update data, parses the data, and saves it in JSON format. The specific input is the response data from the price update API, and the output is the internal data structure of the parsed price information. Example: Retrieving price information such as "Eggs 200 yen → 180 yen."

[0193] Step 2:

[0194] The server generates an announcement using a generative AI model (OpenAI GPT-4) based on the acquired price information. Here, the input is the acquired price information (e.g., the price of eggs has changed to 180 yen), and the output is the announcement generated by the generative AI model. Specifically, the prompt "Please generate egg sale information. Today's sale price is 180 yen" is input into the generative AI model, which generates the announcement "Egg sale, 180 yen for today only!"

[0195] Step 3:

[0196] The server sends the generated announcement to the smartphone application and the in-store speaker. The input is the generated announcement (e.g., "Eggs on sale for just 180 yen today!"), and the output is a push notification and a voice broadcast. Specifically, it uses Firebase Cloud Messaging (FCM) to send a push notification to the customer's smart device and controls the speaker system to broadcast the announcement throughout the store.

[0197] Step 4:

[0198] The terminal uses in-store cameras to monitor customer behavior and product inventory. The input is video data from the cameras, and the output is customer behavior and inventory information analyzed using an object detection algorithm (using OpenCV). Specifically, it analyzes the video captured by the cameras in real time to capture customer behavior when picking up products and changes in inventory levels.

[0199] Step 5:

[0200] The server notifies employees when an item is out of stock based on inventory information obtained using object detection technology. The input is inventory information obtained and analyzed using object detection technology, and the output is an out-of-stock notification message. Specifically, when the inventory level falls below a certain threshold, the generative AI model generates an out-of-stock notification message and sends the notification to bone conduction earphones and smartphones. For example, a message such as "There are only a few eggs left. Please add more from the warehouse" is generated.

[0201] Step 6:

[0202] The user (employee) receives the out-of-stock notification via bone conduction earphones and a smartphone and responds promptly. The input is the out-of-stock notification message from the server, and the output is the replenishment of stock when the user actually takes action. Specifically, an employee receives a notification saying, "Eggs are out of stock. Please replenish them," and takes action to replenish the product from the warehouse.

[0203] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0204] To put the present invention into practice, an emotion engine is combined with an in-store system to recognize the user's emotions and make announcements or notify staff accordingly.

[0205] System Overview

[0206] In order to improve the efficiency of store operations and the consumer experience, this system acquires price update information, generates and provides announcements, monitors using object detection, notifies of out-of-stock information, and also includes an emotion engine that recognizes user emotions.

[0207] System configuration

[0208] 1. Get price updates

[0209] The server periodically retrieves the latest price updates from the price update system.

[0210] Example: A server collects price change information from a supermarket database.

[0211] 2. Announcement Generation

[0212] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[0213] Example: Generate an announcement such as "Eggs on sale for just 180 yen today only!"

[0214] 3. Providing announcement text

[0215] The server transmits the generated announcement text to the terminal and provides it through a speaker in the store.

[0216] Example: Broadcast an announcement saying, "We're currently having a special sale on eggs. One pack costs 180 yen."

[0217] 4. Object Detection Monitoring

[0218] The server uses object detection technology to monitor customer behavior and product inventory within the store.

[0219] Example: Detecting the behavior of a customer picking up a sale item and updating inventory information.

[0220] 5. Notification of out-of-stock information

[0221] The server checks inventory information and notifies staff when an item is out of stock.

[0222] Example: Generate a notification saying "You're low on eggs. Please add more from the warehouse."

[0223] 6. User Emotion Recognition

[0224] The server uses an emotion engine to detect the emotions of customers in the store and bases further processing on that information.

[0225] Example: An emotion engine recognizes emotions such as "interest" or "anxiety" from a customer's facial expressions and behavior.

[0226] 7. Emotion-based announcement generation

[0227] The server generates an announcement sentence based on the recognized emotion and transmits it to the terminal.

[0228] Example: If a customer expresses interest in a sale, add an announcement saying, "Thank you for your interest in our sale."

[0229] 8. Staff Notification of Emotional Information

[0230] The server uses the recognized emotion information to notify staff and assist them in handling customers.

[0231] Example: Sending a message saying, "Customer A has expressed interest. Please provide additional support."

[0232] The user (staff member) receives the emotional information and responds appropriately to the customer.

[0233] Example: Store staff provides additional support to Customer A to facilitate a purchase.

[0234] A natural language description of the process

[0235] Get price updates

[0236] The server periodically retrieves the latest price updates from the price update system, which are price changes for products and are often updated in real time.

[0237] Announcement generation

[0238] The server uses generative artificial intelligence technology to generate announcements based on the price update information. The generated announcements effectively communicate information about special sales and limited-time sales to customers.

[0239] Providing announcement text

[0240] The server sends the generated announcement text to a terminal in the store, which then provides the content to customers by voice through the in-store speaker, allowing customers to receive special sale information in real time.

[0241] Object detection surveillance

[0242] The server uses object detection technology to monitor customer behavior and product inventory in the store. For example, if a customer picks up a sale item, the server detects that behavior and updates inventory information accordingly.

[0243] Notification of out-of-stock information

[0244] If a shortage occurs based on the monitored inventory information, the server notifies the staff of the shortage via bone conduction earphones, allowing the staff to immediately take action to deal with the shortage.

[0245] User Emotion Recognition

[0246] The server uses an emotion engine to detect the user's emotions from their facial expressions and behavior, enabling more detailed customer service.

[0247] Emotion-based announcement generation

[0248] The server generates special announcements based on the recognized emotions, allowing the information provided to customers to match their current emotions.

[0249] Staff notification of emotional information

[0250] The server notifies the staff of the user's emotional information and assists them in responding to the customer. Based on the information received, the staff responds quickly and appropriately, improving customer satisfaction.

[0251] In this way, through each step, a system is created that streamlines the management of time sales and improves the consumer experience.

[0252] The processing flow will be explained below.

[0253] Step 1:

[0254] The server periodically retrieves the latest price updates from the price update system.

[0255] Specific operation: Refer to the price update database to get new price information. This information includes "product name" and "new price".

[0256] Example: Get the data "Eggs 200 yen → 180 yen".

[0257] Step 2:

[0258] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[0259] Specific operation: The acquired price update information is passed as input data to the AI ​​model, and an appropriate announcement text is automatically generated.

[0260] Example: Generate an announcement saying "Eggs on sale for just 180 yen today only!"

[0261] Step 3:

[0262] The server sends the generated announcement to the terminal (in-store speaker) and provides it as audio.

[0263] Specific operation: The generated announcement text is converted into an audio file and sent to the in-store speaker terminal.

[0264] Example: "We're currently having a special sale on eggs. They're 180 yen per pack. Please come and buy some." is announced over the store's speakers.

[0265] Step 4:

[0266] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[0267] Specific actions: Data from cameras and sensors installed in the store is analyzed to detect when a customer picks up a specific product.

[0268] Example: Recognizing when a customer picks up a carton of eggs from the shelf and reducing stock levels.

[0269] Step 5:

[0270] The server checks the inventory status of the detected product and checks for out-of-stock information.

[0271] What it does: Looks up the updated inventory database and checks if a particular product is below a certain threshold.

[0272] Example: If there are only 2 packs of eggs left in stock, an out-of-stock alert will be issued.

[0273] Step 6:

[0274] If the server detects out-of-stock information, it generates a message to notify staff.

[0275] Specific operation: Based on the out-of-stock information, a message requesting replenishment is generated for staff.

[0276] Example: Generate the message "Low eggs left. Please get more from the warehouse."

[0277] Step 7:

[0278] The server delivers the generated notification message to the staff member via bone conduction earphones.

[0279] Specific operation: Convert notification messages into audio signals and send them to staff through bone conduction earphones.

[0280] Example: A staff member receives a notification that "We are out of eggs. Please restock."

[0281] Step 8:

[0282] The server uses an emotion engine to recognize the user's emotions.

[0283] Specific operation: The emotion engine analyzes the user's facial expressions and behavioral patterns based on data from cameras and sensors installed in the store.

[0284] Example: Detecting emotions such as "interest" or "anxiety" from a customer's facial expression.

[0285] Step 9:

[0286] The server generates an announcement sentence based on the recognized emotion and transmits it to the terminal.

[0287] Specific operation: Based on the detected emotion data, a new announcement is generated and sent to the in-store speaker terminal.

[0288] Example: If a customer is interested in a sale item, add the announcement, "Thank you for your interest in our sale item."

[0289] Step 10:

[0290] The server notifies the staff of the recognized emotion information to assist them in dealing with customers.

[0291] Specific operation: Based on the emotional information detected by the emotion engine, a message is generated for the staff and sent via bone conduction earphones.

[0292] Example: Send a message saying, "Customer A has expressed interest. Please provide additional support."

[0293] Step 11:

[0294] After receiving the notification, the user (staff member) will promptly respond to the customer.

[0295] What happens: Staff receive the notification and provide additional support to the customer, increasing satisfaction.

[0296] Example: A staff member explains the product and makes suggestions to Customer A.

[0297] In this way, a system is realized that streamlines the management of time sales through a multi-stage processing flow and improves the customer experience.

[0298] Example 2

[0299] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0300] While conventional store management systems have basic functions such as obtaining updated price information and managing inventory, they lack the ability to recognize customer emotions and provide personalized responses based on those emotions. This results in a lack of concrete measures to improve customer satisfaction, making efficient store management difficult. Furthermore, even when receiving timely notifications about out-of-stock items or special sales, staff responses are delayed, resulting in reduced productivity.

[0301] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0302] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using artificial intelligence generative technology, means for providing the generated announcements to customers in the store by voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for detecting emotions of customers in the store using an emotion recognition engine, means for generating announcements based on the detected emotions, and means for notifying employees of the detected emotion information. This enables individual responses based on customer emotions, contributing to improved customer satisfaction as well as enabling more efficient store operations and faster staff responses.

[0303] "Price updates" refers to the latest price information for products sold in stores, which is often updated in real time.

[0304] "Generative AI technology" refers to artificial intelligence technology for generating natural language text and speech based on acquired data, and specifically includes generative AI models.

[0305] An "announcement" is a voice message provided to customers in a store, and effectively conveys price information, special sale information, and the like.

[0306] "Object detection technology" is a technology that uses cameras and sensors to monitor and recognize customer behavior and the status of products in a store in real time.

[0307] "Product inventory" refers to the quantity and condition of products sold in a store, and is quantitatively grasped by an inventory management system.

[0308] "Out-of-stock information" is information that notifies you that a particular product is out of stock or in short supply.

[0309] An "emotion recognition engine" is a technology that analyzes emotions from a customer's facial expressions and behavior, and recognizes them as states such as "interest," "satisfaction," or "anxiety."

[0310] "Employees" refers to staff in the store who are responsible for tasks such as product management and customer service.

[0311] "Notification" refers to the transmission of information by the system, including messages to encourage employees to take action.

[0312] The "in-store speaker" and "terminal" refer to audio equipment and its control device for playing the generated announcement text as audio.

[0313] The present invention is a system that aims to improve the efficiency of store operations and customer satisfaction. This system operates mainly around three components: a server, a terminal, and a user. Specific embodiments of this system are described in detail below.

[0314] Get price updates

[0315] The server periodically communicates with the price update system to obtain new price information. This communication is performed via an API, and for example, at 9:00 AM and 3:00 PM, the server downloads the day's special prices and new prices from the database. This data includes the product ID, product name, new price, etc.

[0316] Announcement generation

[0317] The server generates an announcement using generative artificial intelligence technology based on the acquired price update information. A generative AI model (e.g., GPT-3) is used to generate this announcement. For example, the price update information is input as a prompt, and the generated text is "Today's sale information! Fresh eggs are 180 yen." An example of a prompt from a generative AI model is as follows:

[0318] Prompt: "Generate an announcement based on the sale information. Example: Eggs, new price 180 yen."

[0319] Generated announcement: "Today's special offer! Fresh eggs for 180 yen."

[0320] Providing announcement text

[0321] The generated announcement is sent from the server to the terminal. The terminal is connected to the store's speaker system, and the received announcement is provided to customers as audio. For example, at 11:00 a.m. and 4:00 p.m., an announcement such as "Today's special: Fresh eggs for 180 yen. Don't miss out!" is played in the store.

[0322] Object detection surveillance

[0323] The server uses object detection technology to monitor customer behavior and product inventory through cameras and sensors installed in the store. For example, a camera monitors the sale shelves and detects the moment a customer picks up an egg. At that time, inventory information is automatically updated based on the image data.

[0324] Notification of out-of-stock information

[0325] Based on inventory information monitored by object detection technology, the server immediately notifies users (store employees) when an item is about to run out. This notification is sent to bone conduction earphones worn by employees, and for example, a message such as "Eggs are running low. Please order more from the warehouse."

[0326] User Emotion Recognition

[0327] The server uses an emotion recognition engine to collect data on customers' facial expressions and behavior from cameras and sensors installed in the store, and analyzes and recognizes their emotions. For example, a customer's smile when looking at a sale item can be recognized as "interest." This emotion data is updated in real time.

[0328] Emotion-based announcement generation

[0329] The server generates a specific announcement based on the recognized emotion information. For example, if a customer is interested in a special sale item, the server creates an announcement saying, "Thank you for your interest in our special sale items! Please take a look."

[0330] Staff notification of emotional information

[0331] The server notifies the user (store employee) of the recognized emotion information. The employee receives this notification and responds to the customer based on the content, for example, "Customer B is interested in a special sale item. Please provide additional information or support." The staff member approaches the customer and kindly provides product details and other special sale information.

[0332] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0333] Step 1:

[0334] The server retrieves price updates.

[0335] Input: Schedule information, authentication information to make API calls.

[0336] Specific operation: Sends an API request at 9:00 AM and 3:00 PM to retrieve the latest price information (product ID, product name, new price, etc.) from the store's database.

[0337] Output: A list of retrieved price updates.

[0338] Step 2:

[0339] The server generates an announcement based on the acquired price update information.

[0340] Input: Retrieved price update information.

[0341] Specific operation: Create a prompt to input price update information into the generated AI model (e.g., GPT-3) and input it into the AI ​​model. For example, enter the product ID and new price information as a prompt: "Eggs, new price 180 yen."

[0342] Output: The generated announcement (e.g., "Today's special offer! Fresh eggs for 180 yen.").

[0343] Step 3:

[0344] The server sends the generated announcement text to the terminal and provides it as audio from speakers inside the store.

[0345] Input: The generated announcement text.

[0346] Specific operation: The generated announcement text is sent to the terminal in JSON format, and the terminal sends the received data to a speech synthesis engine, which then plays the announcement over the in-store speakers.

[0347] Output: Announcement audio broadcast in the store.

[0348] Step 4:

[0349] The server uses object detection technology to monitor customer behavior and product inventory.

[0350] Input: Video data from cameras and sensors in the store.

[0351] Specific actions: Real-time video data analysis detects the moment a specific product is picked up, for example, when a customer reaches out in front of an egg shelf.

[0352] Output: Updated inventory information.

[0353] Step 5:

[0354] The server notifies staff of out-of-stock information based on the monitored inventory information.

[0355] Input: Updated inventory information.

[0356] Specific operation: When the inventory level falls below a set threshold, a notification method (e.g., bone conduction earphones) is selected and a notification message is generated. For example, a message such as "Egg inventory has fallen below 10 units. Please replenish." is created and sent to staff.

[0357] Output: Staff notification message.

[0358] Step 6:

[0359] The server uses an emotion recognition engine to recognize the emotion of the customer.

[0360] Input: Video data from cameras and sensors in the store.

[0361] Specific behavior: Video data is analyzed in real time to recognize emotions such as "interest," "satisfaction," and "anxiety" from facial expressions and behavior. For example, if a customer smiles when looking at a sale item, this is recognized as "interest."

[0362] Output: Recognized emotion information.

[0363] Step 7:

[0364] The server generates an announcement based on the recognized emotion.

[0365] Input: Recognized emotion information.

[0366] Specific operation: Emotional information is input into the generative AI model to generate appropriate announcements. For example, if a customer expresses interest, the system will generate an announcement such as, "Thank you for your interest in our special sale items! Please take a look."

[0367] Output: The generated announcement.

[0368] Step 8:

[0369] The server notifies the staff of the recognized emotion information.

[0370] Input: Recognized emotion information.

[0371] Specific operation: Generate a notification message to staff based on the emotion information. For example, generate a message saying, "Customer B is interested in a special sale item. Please provide additional guidance or support." and send it to staff.

[0372] Output: Staff notification message.

[0373] (Application example 2)

[0374] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0375] Conventional store management systems lacked the ability to recognize customer emotions and respond appropriately. This made it difficult for employees to properly understand the emotions of individual customers and provide services that match those emotions. Furthermore, while efficient operations are required in terms of inventory management and the provision of special sale information, conventional systems were unable to fully meet these needs.

[0376] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative artificial intelligence technology, means for providing the generated announcements to customers in the store by voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for recognizing user emotions, means for generating and providing announcements based on the recognized emotions, and means for notifying employees of the recognized emotion information. This enables responses and announcements tailored to individual customers' emotions, efficient inventory management, and the provision of special sale information.

[0377] "Price update information" is the latest price change information for products and services, and is data used in store operations.

[0378] "Generative AI technology" is a technology that allows computers to learn independently and imitate creative tasks performed by humans, and is used for text generation and data analysis.

[0379] "Means for generating announcements" refers to a function that automatically generates announcements to provide to customers based on price update information and other data.

[0380] "Means for providing the created announcement text to customers in the store by voice" refers to a function for informing customers of the created announcement text as voice through a speaker or other audio output device.

[0381] "Object detection technology" is a technology that uses cameras and sensors to identify specific objects or human movements and acquire them as data.

[0382] "Means for monitoring customer behavior and product inventory" refers to a function that uses object detection technology to check customer movements within the store and product inventory status, and collects and analyzes the data.

[0383] "Means for notifying employees of out-of-stock information" refers to a function that notifies employees when a product is sold out or when inventory is low.

[0384] "Means for recognizing user emotions" refers to technology that analyzes data such as a customer's facial expressions and tone of voice to identify their emotional state.

[0385] "Means for generating and providing announcements based on recognized emotions" refers to a function that automatically creates appropriate guidance messages for individual customers based on the user's emotional data and provides them in voice or text format.

[0386] The "means of notifying employees of recognized emotional information" is a function that conveys the results of analyzing the user's emotions to employees and encourages them to take appropriate action.

[0387] The present invention relates to the implementation of a system for improving operational efficiency and customer experience in a physical store. Specific embodiments for carrying out the present invention are described below.

[0388] System Overview

[0389] The system includes functions to obtain price update information, generate announcements using generative artificial intelligence technology and provide them in-store, monitor customer behavior and product inventory using object detection technology, notify employees of out-of-stock information, recognize customer emotions and generate announcements based on that information, and notify employees of emotional information.

[0390] Hardware and software used

[0391] Hardware:

[0392] Webcam: Capture your customer's facial expressions in real time.

[0393] Head-mounted display: worn by employees to receive information in real time.

[0394] Speaker: An audio output device for providing announcements within the store.

[0395] software:

[0396] OpenCV: Processes image data obtained from the camera and performs object detection.

[0397] DeepFace: A facial recognition library for analyzing customer emotions.

[0398] Google Cloud Text-to-Speech: Converts the generated announcement text into audio.

[0399] Specific operation of the system

[0400] 1. Get price updates:

[0401] The server periodically retrieves the latest price update information from the price update system. This information is the latest price change information for products and services, and is important data for store operations.

[0402] 2. Announcement generation:

[0403] The server uses generative artificial intelligence technology to generate announcements based on price update information. This technology automatically creates announcements that effectively communicate sales and special offers to customers.

[0404] 3. In-store announcements:

[0405] The server sends the generated announcement to the speakers in the store and provides it by voice, so that all customers in the store can receive the latest sales information in real time.

[0406] 4. Object detection surveillance:

[0407] The server uses OpenCV to monitor customer behavior and product inventory in the store. When a customer picks up a specific product, the behavior is detected and the inventory information is automatically updated.

[0408] 5. Out-of-stock notification:

[0409] When inventory is low, the server notifies employees of the shortage, which is displayed on a head-mounted display worn by the employee, allowing them to immediately take action to replenish the stock.

[0410] 6. Customer Emotion Recognition:

[0411] The server uses DeepFace to analyze the video captured by the webcam and recognize the customer's emotions. The recognized emotions are collected and analyzed as data in real time.

[0412] 7. Emotion-based announcement generation:

[0413] The server generates special announcements based on the recognized emotions and delivers them through speakers and displays in the store, enabling guidance optimized for the customer's current emotions.

[0414] 8. Employee Notification of Emotional Information:

[0415] The server then notifies employees of the recognized emotional information. For example, if a customer is confused, the information is conveyed to employees, who can provide support promptly.

[0416] Examples of specific examples and prompts

[0417] Examples:

[0418] If a customer is smiling while looking at products in the sale section, the server generates and provides a voice announcement saying, "Thank you for your interest in our sale. Please choose your favorite product." If the customer looks confused, the server provides a voice announcement saying, "Is there anything I can help you with? Our staff is here to help you."

[0419] Example prompt sentence:

[0420] "Create a program that analyzes the facial expressions of customers in a store and generates announcements based on their emotions. If the emotion is 'happy', announce, 'Thank you for your interest in our sale. Please choose your favorite product.' If the emotion is 'sad', announce, 'Is there anything I can help you with? Our staff is here to help you.'"

[0421] This invention provides an embodiment of a new system that combines emotion recognition and AI to achieve a high level of automation in store operations and improve customer satisfaction.

[0422] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0423] Step 1:

[0424] The server periodically retrieves the latest price update information from the price update system. The input is data from the price update system, and the output is the latest price update information. This information is the latest price change information for products and services, and is data necessary for store operations.

[0425] Step 2:

[0426] Based on the price update information acquired by the server, announcement text is generated using generative artificial intelligence technology. The input is the price update information, and the output is the generated announcement text. The generated announcement text effectively communicates sales and special offers to customers.

[0427] Step 3:

[0428] The server sends the generated announcement to a terminal in the store and provides it audibly through a speaker. The input is the generated announcement, and the output is a voice announcement played through the in-store speaker. This allows customers in the store to receive special sale information in real time.

[0429] Step 4:

[0430] The server uses OpenCV to detect objects in the image data acquired from the webcam, and monitors customer behavior and product inventory in the store. The input is video data from the webcam, and the output is customer behavior data and inventory information. This allows the detection of actions such as customers picking up sale items.

[0431] Step 5:

[0432] The server uses object detection technology to collect inventory information and notifies employees of out-of-stock information. The input is product inventory information, and the output is a shortage notification sent to bone conduction earphones or a head-mounted display. This allows employees to immediately understand shortages and take action.

[0433] Step 6:

[0434] The server uses DeepFace to analyze video data captured by a webcam and recognize customer emotions. The input is video data from the webcam, and the output is the recognized customer emotion data. This data is obtained from the customer's facial expressions and behavior.

[0435] Step 7:

[0436] The server generates special announcements based on the recognized emotion data and provides them through speakers and displays in the store. The input is the recognized customer emotion data, and the output is the announcement provided through speakers and displays in the store. This makes it possible to provide information that matches the customer's current emotion.

[0437] Step 8:

[0438] The server notifies employees of the recognized emotion information and assists them in handling customers. The input is the recognized emotion data, and the output is a notification message sent to employees. This allows employees to take appropriate action based on the customer's emotion.

[0439] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0440] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0441] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0442] [Second embodiment]

[0443] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0444] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0445] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0446] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0447] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0448] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0449] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0450] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0451] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0452] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0453] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0454] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0455] To implement the present invention, an in-store system is constructed as follows.

[0456] System Overview

[0457] To streamline time sales in stores and improve the consumer experience, the system retrieves price updates, generates and delivers announcements, monitors customer behavior and product inventory using object detection, and notifies staff of out-of-stock information.

[0458] System configuration

[0459] 1. Get price updates

[0460] The server periodically retrieves the latest price updates from the price update system.

[0461] Example: The server collects new price information such as "eggs 200 yen → 180 yen" from a supermarket price update database.

[0462] 2. Announcement Generation

[0463] The server generates announcement text based on the obtained price update information using generative artificial intelligence techniques.

[0464] Example: The server generates an announcement such as "Egg sale, only 180 yen today!"

[0465] 3. Providing announcement text

[0466] The server transmits the generated announcement to a terminal that controls a speaker in the store, and provides it to the customer by voice.

[0467] Example: The terminal announces through the in-store speaker, "We're currently having a special sale on eggs. They're 180 yen per pack. Please buy some."

[0468] 4. Object Detection Monitoring

[0469] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[0470] Example: The server uses data from in-store cameras to capture when a customer picks up an egg and updates the inventory.

[0471] 5. Notification of out-of-stock information

[0472] The server checks the inventory status of the detected product and notifies staff if there is a shortage.

[0473] Example: Generate a notification such as "You're low on eggs. Please get more from the warehouse."

[0474] Users (staff) receive information about out-of-stock items in real time through bone conduction earphones and can quickly replenish products.

[0475] Example: A staff member receives a notification via bone conduction earphones saying, "We are out of eggs. Please replenish them," and restocks the stock from the warehouse.

[0476] A natural language description of the process

[0477] Get price updates

[0478] The server periodically retrieves the latest price update information from the price update system, which is price change information for the product and may fluctuate in real time.

[0479] Announcement generation

[0480] The server uses generative artificial intelligence technology to generate announcements based on the acquired price update information. The generated announcements effectively communicate information about special sales and limited-time sales to customers.

[0481] Providing announcement text

[0482] The server sends the generated announcement text to a terminal in the store, which then provides the content to customers by voice through the in-store speaker, allowing customers to receive special sale information in real time.

[0483] Object detection surveillance

[0484] The server uses object detection technology to monitor customer behavior and product inventory in the store. For example, if a customer picks up a sale item, the server detects that behavior and updates inventory information accordingly.

[0485] Notification of out-of-stock information

[0486] If a shortage occurs based on the monitored inventory information, the server notifies the staff of the shortage via bone conduction earphones, allowing the staff to immediately take action to deal with the shortage.

[0487] This system will streamline the management of time sales, improve the quality of service to customers, and ensure smooth store operations.

[0488] The processing flow will be explained below.

[0489] Step 1:

[0490] The server periodically retrieves the latest price updates from the price update system.

[0491] Specific operation: Refer to the price update database to get new price information. This information includes "product name" and "new price".

[0492] Example: Get the data "Eggs 200 yen → 180 yen".

[0493] Step 2:

[0494] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[0495] Specific operation: The acquired price update information is passed as input data to the AI ​​model, and an appropriate announcement text is automatically generated.

[0496] Example: Generate an announcement for price update information: "Eggs on sale for 180 yen today only!"

[0497] Step 3:

[0498] The server sends the generated announcement to the terminal (in-store speaker) and provides it as audio.

[0499] Specific operation: The generated announcement text is converted into an audio file and sent to the in-store speaker terminal.

[0500] Example: "We're currently having a special sale on eggs. They're 180 yen per pack. Please come and buy some." is announced over the store's speakers.

[0501] Step 4:

[0502] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[0503] Specific actions: Data from cameras and sensors installed in the store is analyzed to detect when a customer picks up a specific product.

[0504] Example: Recognizing when a customer picks up a carton of eggs from the shelf and reducing stock levels.

[0505] Step 5:

[0506] The server checks the inventory status of the detected product and checks for out-of-stock information.

[0507] What it does: Looks up the updated inventory database and checks if a particular product is below a certain threshold.

[0508] Example: If there are only 2 packs of eggs left in stock, an out-of-stock alert will be issued.

[0509] Step 6:

[0510] If the server detects out-of-stock information, it generates a message to notify staff.

[0511] Specific operation: Based on the out-of-stock information, a message requesting replenishment is generated for staff.

[0512] Example: Generate the message "Low eggs left. Please get more from the warehouse."

[0513] Step 7:

[0514] The server delivers the generated notification message to the staff member via bone conduction earphones.

[0515] Specific operation: Convert notification messages into audio signals and send them to staff through bone conduction earphones.

[0516] Example: A staff member receives a notification that "We are out of eggs. Please restock."

[0517] Step 8:

[0518] After receiving the notification, the user (staff member) promptly replenishes the stock.

[0519] Specific actions: Go to the warehouse, retrieve out-of-stock items, and replenish them on the sales floor.

[0520] Example: A staff member takes cartons of eggs from the warehouse and restocks them on the sales shelves.

[0521] Through each step, a system is built that improves the efficiency of time sales and the consumer experience.

[0522] Example 1

[0523] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0524] In traditional store operations, managing special sales and time sales required a lot of manual effort, resulting in inefficiency. Because processes such as providing sales information to consumers, managing inventory, and notifying of out-of-stock items were all done manually, it was difficult to update information in real time, which resulted in reduced customer satisfaction and lost sales opportunities.

[0525] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0526] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative AI technology, means for providing the generated announcements to customers in the store via voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for periodically accessing a price update system to acquire price update information, means for generating announcements by sending prompts to a generative AI model based on the acquired price update information, means for receiving video data from in-store cameras and detecting customer behavior using object detection technology, means for transmitting the generated announcements to in-store terminals, and means for using bone conduction earphones worn by staff to notify out-of-stock information. This enables effective management of limited-time sales, real-time product inventory monitoring, and rapid out-of-stock response.

[0527] "Price update information" is data provided when the price of a product in a store is changed, and includes both the old and new price information.

[0528] "Generative AI technology" is an AI technology for text generation and natural language processing, which generates appropriate output text based on specified input data.

[0529] An "announcement" is a sentence created to inform customers of information such as special sales and limited-time sales, and is a text for providing the content of the announcement by voice.

[0530] "Object detection technology" is a technology for recognizing and detecting specific objects from image data, and is used to monitor customer behavior and merchandise movement within a store.

[0531] "Out-of-stock information" is information that is notified when the product inventory in the store falls below a certain amount, and is information that encourages additional purchases or replenishment.

[0532] A "server" is a computer system that provides a specific service over a network. In this system, it is a device that performs functions such as obtaining price updates, generating announcements, and monitoring using object detection.

[0533] A "terminal" is a device that receives announcements sent from the server and provides them to customers in the store by voice, and is usually connected to a speaker system.

[0534] A "prompt" is an instruction given to a generative AI model, containing hints and instructions for generating a specific output.

[0535] A "generative AI model" is an artificial intelligence model that generates appropriate text based on a given prompt sentence, and in the present invention is primarily used to generate announcement sentences.

[0536] Bone conduction earphones are devices that transmit sound through the bones to the inner ear, allowing employees to receive voice notifications without using their hands.

[0537] The present invention relates to a system for streamlining in-store sales and time-limited sales to improve the consumer experience by obtaining price updates, generating and providing announcements, monitoring customer behavior and product inventory using object detection, and notifying staff of out-of-stock information.

[0538] System configuration

[0539] Get price updates

[0540] The server periodically accesses the price update system to retrieve the latest price updates. This process involves using HTTP requests to retrieve data in JSON format from the API. The retrieved data includes information such as the product name, old price, and new price. For example, the server receives price updates such as "eggs from 200 yen to 180 yen" from a supermarket's price update database.

[0541] Announcement generation

[0542] The server sends a prompt message to the generative AI model based on the obtained price update information. This prompt message contains information about which product's price has changed and how. The generative AI model (e.g., GPT-4) generates an announcement message based on this prompt. For example, a prompt message might be created saying, "The price of the product has changed. Eggs: 200 yen → 180 yen. Please generate a sale announcement message based on this information." The generative AI model would then generate an announcement message saying, "Eggs on sale for 180 yen today only!"

[0543] Providing announcement text

[0544] The server sends the generated announcement to a terminal in the store. The terminal receives the announcement and provides it audibly through the store's speakers. This allows customers to receive sale information in real time. For example, the terminal may broadcast an announcement over the store's speakers saying, "We're currently having a sale on eggs. They're 180 yen a pack. Please buy some."

[0545] Object detection surveillance

[0546] The server uses object detection technology to monitor customer behavior and product inventory in the store. YOLO (You Only Look Once) is one example of the object detection technology. This technology is used to analyze video data from cameras in the store, and if a specific action (such as a customer picking up an egg) is detected, inventory information is updated. For example, the server receives video data from cameras in the store, detects the action of a customer picking up an egg, and updates inventory information accordingly.

[0547] Notification of out-of-stock information

[0548] The server monitors inventory information and generates out-of-stock information when a certain number of items falls below a certain level. This information is notified to staff via bone conduction earphones. For example, the server detects that inventory is running low and generates a notification saying, "We have only a few eggs left. Please replenish them from the warehouse." The staff (users) receive the notification via bone conduction earphones saying, "We are out of eggs. Please replenish them," and quickly replenish the items.

[0549] Hardware and software used

[0550] Price update system: A product management database system used by stores

[0551] Generative AI model: GPT-4 (OpenAI)

[0552] Object detection technology: Image recognition technology such as YOLO (You Only Look Once)

[0553] Notification device: Bone conduction earphones

[0554] Examples and prompts

[0555] Below are some examples of prompt sentences.

[0556] "The price of the product has changed. Eggs: 200 yen → 180 yen. Please generate a sale announcement based on this information."

[0557] "Price update information: Eggs 200 yen → 180 yen. Please use this data to create a sale announcement for your store. For example, please use a format like, 'Egg sale, 180 yen today only!'"

[0558] This system will streamline store operations, improve the quality of service to customers, and provide information in real time.

[0559] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0560] Step 1: Get price updates

[0561] The server periodically accesses the price update system to obtain the latest price update information. At this time, an HTTP request is used as input and JSON format data is received from the API endpoint. Specifically, the server sends a request to the price update system and obtains price update information such as "Eggs 200 yen → 180 yen." The data includes the product name, old price, new price, etc.

[0562] Step 2: Generate announcement text

[0563] The server sends a prompt message to the generative AI model based on the acquired price update information. It uses the price update information as input and obtains an announcement message as output. Specifically, it creates a prompt message saying, "The price of the product has changed. Eggs have gone from 200 yen to 180 yen. Please generate a sale announcement message based on this information." and sends it to the generative AI model (e.g., GPT-4). The generative AI model then generates a response message saying, "Eggs on sale for 180 yen today only!"

[0564] Step 3: Provide the announcement

[0565] The server sends the generated announcement to a terminal in the store. The generated announcement is used as input and is provided to the customer as audio output. The terminal receives this announcement and provides it to the customer through the in-store speaker. For example, the announcement may say, "We're currently having a special sale on eggs. One pack is 180 yen. Please buy some."

[0566] Step 4: Object detection surveillance

[0567] The server uses object detection technology to monitor customer behavior and product inventory in the store. It uses video data from in-store cameras as input and obtains customer behavior and inventory information as output. Specifically, it analyzes the video from the in-store cameras and detects customer behavior using object detection technology (e.g., YOLO). For example, it detects the behavior of a customer picking up an egg and updates inventory information accordingly.

[0568] Step 5: Out-of-stock notification

[0569] The server uses the monitored inventory information to notify staff when a stockout occurs. It uses the updated inventory information as input and generates a stockout notification as output. Specifically, when inventory falls below a certain amount (e.g., there are fewer than 10 eggs left), a notification is generated. A notification stating "Only a few eggs left. Please add more from the warehouse" is created and sent to staff via bone conduction earphones. The user (staff member) receives this notification and replenishes the product from the warehouse.

[0570] (Application example 1)

[0571] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0572] With conventional time sale management systems, it was difficult to obtain real-time in-store price update information and provide it to customers efficiently. Furthermore, monitoring of customer behavior and product inventory management were insufficient, and responding quickly when an item was out of stock was also an issue. Furthermore, the lack of a push notification function to provide customers with timely information limited the ability to maximize sales.

[0573] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0574] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative artificial intelligence technology, means for providing the generated announcements to customers in the store by voice and push notification, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for notifying employees of out-of-stock information using bone conduction earphones, and means for generating prompts for generating announcements based on the price update information using generative artificial intelligence technology. This enables efficient and real-time time sale management, improving customer experience and maximizing sales.

[0575] "Price update information" refers to the latest price information for products in stores, which is periodically obtained from a database or price update system.

[0576] "Generative artificial intelligence technology" is a technology that uses an AI model to generate text, images, etc. based on input data.

[0577] An "announcement" is a sentence generated to effectively convey information about special sales or limited-time sales.

[0578] "Voice and push notifications" refers to providing information through in-store speakers and notifications to customers' smart devices.

[0579] "Object detection technology" is a technology that uses cameras and sensors to monitor customer behavior and product inventory in stores in real time.

[0580] "Product inventory" refers to the current number of products held in the store.

[0581] "Out-of-stock information" is notification information that is generated when the stock of a specific product falls below a certain level.

[0582] "Employees" refers to staff working in the store.

[0583] Bone conduction earphones are special earphones that transmit sound through the bones rather than the ears, and are worn by employees.

[0584] A "generative prompt" is input text for generating an announcement using generative artificial intelligence technology.

[0585] The present invention provides a system for increasing the efficiency of limited-time sales in stores and improving consumer experiences. Specific embodiments for realizing this system will be described below.

[0586] Overall system configuration

[0587] The system is designed to efficiently manage time-limited sales and special offers in stores and provide them to customers in real time. Its main components include a server, smartphones, bone conduction earphones, in-store cameras, object detection technology, generative artificial intelligence technology (generative AI model), and a push notification system.

[0588] Hardware and Software Configuration

[0589] 1. Server

[0590] The server plays the important role of obtaining price update information, generating announcements using generative artificial intelligence technology (generative AI model), and sending them to in-store smartphones and voice systems. Specifically, the server was built using Python and Node.js, and Firebase Firestore was used as the database. The generative AI model used was OpenAI GPT-4.

[0591] 2. Smartphone

[0592] Smartphones are devices that provide information to customers and employees. Developed cross-platform using Flutter, they provide information through push notifications and in-app messages.

[0593] 3. Bone conduction earphones

[0594] The bone conduction earphones worn by employees are devices used to notify them of stock shortages in real time.

[0595] 4. In-store cameras and object detection technology

[0596] In-store cameras monitor customer behavior and product inventory using object detection technology (using OpenCV), which allows for real-time tracking of customer pick-up behavior and shelf inventory.

[0597] Main processes and data flow

[0598] 1. Get price updates

[0599] The server periodically retrieves the latest price information from an external price update system, which is then stored in Firebase Firestore.

[0600] 2. Announcement Generation

[0601] The server generates an announcement using a generative AI model (OpenAI GPT-4) based on the acquired price information. An example of a prompt is, "Please generate special sale information for eggs. Today's special sale price is 180 yen."

[0602] 3. Providing announcement text

[0603] The generated announcement text is sent from the server to the smartphone via push notification and in-store speakers. For example, a notification might be sent to the customer's smartphone saying, "We're currently having a special sale on eggs. One pack costs 180 yen."

[0604] 4. Object Detection Monitoring

[0605] The video data captured by the in-store cameras is sent to a server and analyzed in real time using OpenCV. The analysis results are used to understand what products customers have picked up and changes in inventory levels.

[0606] 5. Notification of out-of-stock information

[0607] Using object detection technology, if inventory falls below a certain threshold, the server will notify employees of the shortage via bone conduction earphones, and will also send a notification to their smartphone, such as "There are only a few eggs left. Please add more from the warehouse."

[0608] This enables efficient and real-time time sale management, improving customer experience and maximizing sales.

[0609] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0610] Step 1:

[0611] The server retrieves the latest price information from an external price update system. To do this, it uses an HTTP request to retrieve price update data, parses the data, and saves it in JSON format. The specific input is the response data from the price update API, and the output is the internal data structure of the parsed price information. Example: Retrieving price information such as "Eggs 200 yen → 180 yen."

[0612] Step 2:

[0613] The server generates an announcement using a generative AI model (OpenAI GPT-4) based on the acquired price information. Here, the input is the acquired price information (e.g., the price of eggs has changed to 180 yen), and the output is the announcement generated by the generative AI model. Specifically, the prompt "Please generate egg sale information. Today's sale price is 180 yen" is input into the generative AI model, which generates the announcement "Egg sale, 180 yen for today only!"

[0614] Step 3:

[0615] The server sends the generated announcement to the smartphone application and the in-store speaker. The input is the generated announcement (e.g., "Eggs on sale for just 180 yen today!"), and the output is a push notification and a voice broadcast. Specifically, it uses Firebase Cloud Messaging (FCM) to send a push notification to the customer's smart device and controls the speaker system to broadcast the announcement throughout the store.

[0616] Step 4:

[0617] The terminal uses in-store cameras to monitor customer behavior and product inventory. The input is video data from the cameras, and the output is customer behavior and inventory information analyzed using an object detection algorithm (using OpenCV). Specifically, it analyzes the video captured by the cameras in real time to capture customer behavior when picking up products and changes in inventory levels.

[0618] Step 5:

[0619] The server notifies employees when an item is out of stock based on inventory information obtained using object detection technology. The input is inventory information obtained and analyzed using object detection technology, and the output is an out-of-stock notification message. Specifically, when the inventory level falls below a certain threshold, the generative AI model generates an out-of-stock notification message and sends the notification to bone conduction earphones and smartphones. For example, a message such as "There are only a few eggs left. Please add more from the warehouse" is generated.

[0620] Step 6:

[0621] The user (employee) receives the out-of-stock notification via bone conduction earphones and a smartphone and responds promptly. The input is the out-of-stock notification message from the server, and the output is the replenishment of stock when the user actually takes action. Specifically, an employee receives a notification saying, "Eggs are out of stock. Please replenish them," and takes action to replenish the product from the warehouse.

[0622] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0623] To put the present invention into practice, an emotion engine is combined with an in-store system to recognize the user's emotions and make announcements or notify staff accordingly.

[0624] System Overview

[0625] In order to improve the efficiency of store operations and the consumer experience, this system acquires price update information, generates and provides announcements, monitors using object detection, notifies of out-of-stock information, and also includes an emotion engine that recognizes user emotions.

[0626] System configuration

[0627] 1. Get price updates

[0628] The server periodically retrieves the latest price updates from the price update system.

[0629] Example: A server collects price change information from a supermarket database.

[0630] 2. Announcement Generation

[0631] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[0632] Example: Generate an announcement such as "Eggs on sale for just 180 yen today only!"

[0633] 3. Providing announcement text

[0634] The server transmits the generated announcement text to the terminal and provides it through a speaker in the store.

[0635] Example: Broadcast an announcement saying, "We're currently having a special sale on eggs. One pack costs 180 yen."

[0636] 4. Object Detection Monitoring

[0637] The server uses object detection technology to monitor customer behavior and product inventory within the store.

[0638] Example: Detecting the behavior of a customer picking up a sale item and updating inventory information.

[0639] 5. Notification of out-of-stock information

[0640] The server checks inventory information and notifies staff when an item is out of stock.

[0641] Example: Generate a notification saying "You're low on eggs. Please add more from the warehouse."

[0642] 6. User Emotion Recognition

[0643] The server uses an emotion engine to detect the emotions of customers in the store and bases further processing on that information.

[0644] Example: An emotion engine recognizes emotions such as "interest" or "anxiety" from a customer's facial expressions and behavior.

[0645] 7. Emotion-based announcement generation

[0646] The server generates an announcement sentence based on the recognized emotion and transmits it to the terminal.

[0647] Example: If a customer expresses interest in a sale, add an announcement saying, "Thank you for your interest in our sale."

[0648] 8. Staff Notification of Emotional Information

[0649] The server uses the recognized emotion information to notify staff and assist them in handling customers.

[0650] Example: Sending a message saying, "Customer A has expressed interest. Please provide additional support."

[0651] The user (staff member) receives the emotional information and responds appropriately to the customer.

[0652] Example: Store staff provides additional support to Customer A to facilitate a purchase.

[0653] A natural language description of the process

[0654] Get price updates

[0655] The server periodically retrieves the latest price updates from the price update system, which are price changes for products and are often updated in real time.

[0656] Announcement generation

[0657] The server uses generative artificial intelligence technology to generate announcements based on the price update information. The generated announcements effectively communicate information about special sales and limited-time sales to customers.

[0658] Providing announcement text

[0659] The server sends the generated announcement text to a terminal in the store, which then provides the content to customers by voice through the in-store speaker, allowing customers to receive special sale information in real time.

[0660] Object detection surveillance

[0661] The server uses object detection technology to monitor customer behavior and product inventory in the store. For example, if a customer picks up a sale item, the server detects that behavior and updates inventory information accordingly.

[0662] Notification of out-of-stock information

[0663] If a shortage occurs based on the monitored inventory information, the server notifies the staff of the shortage via bone conduction earphones, allowing the staff to immediately take action to deal with the shortage.

[0664] User Emotion Recognition

[0665] The server uses an emotion engine to detect the user's emotions from their facial expressions and behavior, enabling more detailed customer service.

[0666] Emotion-based announcement generation

[0667] The server generates special announcements based on the recognized emotions, allowing the information provided to customers to match their current emotions.

[0668] Staff notification of emotional information

[0669] The server notifies the staff of the user's emotional information and assists them in responding to the customer. Based on the information received, the staff responds quickly and appropriately, improving customer satisfaction.

[0670] In this way, through each step, a system is created that streamlines the management of time sales and improves the consumer experience.

[0671] The processing flow will be explained below.

[0672] Step 1:

[0673] The server periodically retrieves the latest price updates from the price update system.

[0674] Specific operation: Refer to the price update database to get new price information. This information includes "product name" and "new price".

[0675] Example: Get the data "Eggs 200 yen → 180 yen".

[0676] Step 2:

[0677] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[0678] Specific operation: The acquired price update information is passed as input data to the AI ​​model, and an appropriate announcement text is automatically generated.

[0679] Example: Generate an announcement saying "Eggs on sale for just 180 yen today only!"

[0680] Step 3:

[0681] The server sends the generated announcement to the terminal (in-store speaker) and provides it as audio.

[0682] Specific operation: The generated announcement text is converted into an audio file and sent to the in-store speaker terminal.

[0683] Example: "We're currently having a special sale on eggs. They're 180 yen per pack. Please come and buy some." is announced over the store's speakers.

[0684] Step 4:

[0685] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[0686] Specific actions: Data from cameras and sensors installed in the store is analyzed to detect when a customer picks up a specific product.

[0687] Example: Recognizing when a customer picks up a carton of eggs from the shelf and reducing stock levels.

[0688] Step 5:

[0689] The server checks the inventory status of the detected product and checks for out-of-stock information.

[0690] What it does: Looks up the updated inventory database and checks if a particular product is below a certain threshold.

[0691] Example: If there are only 2 packs of eggs left in stock, an out-of-stock alert will be issued.

[0692] Step 6:

[0693] If the server detects out-of-stock information, it generates a message to notify staff.

[0694] Specific operation: Based on the out-of-stock information, a message requesting replenishment is generated for staff.

[0695] Example: Generate the message "Low eggs left. Please get more from the warehouse."

[0696] Step 7:

[0697] The server delivers the generated notification message to the staff member via bone conduction earphones.

[0698] Specific operation: Convert notification messages into audio signals and send them to staff through bone conduction earphones.

[0699] Example: A staff member receives a notification that "We are out of eggs. Please restock."

[0700] Step 8:

[0701] The server uses an emotion engine to recognize the user's emotions.

[0702] Specific operation: The emotion engine analyzes the user's facial expressions and behavioral patterns based on data from cameras and sensors installed in the store.

[0703] Example: Detecting emotions such as "interest" or "anxiety" from a customer's facial expression.

[0704] Step 9:

[0705] The server generates an announcement sentence based on the recognized emotion and transmits it to the terminal.

[0706] Specific operation: Based on the detected emotion data, a new announcement is generated and sent to the in-store speaker terminal.

[0707] Example: If a customer is interested in a sale item, add the announcement, "Thank you for your interest in our sale item."

[0708] Step 10:

[0709] The server notifies the staff of the recognized emotion information to assist them in dealing with customers.

[0710] Specific operation: Based on the emotional information detected by the emotion engine, a message is generated for the staff and sent via bone conduction earphones.

[0711] Example: Send a message saying, "Customer A has expressed interest. Please provide additional support."

[0712] Step 11:

[0713] After receiving the notification, the user (staff member) will promptly respond to the customer.

[0714] What happens: Staff receive the notification and provide additional support to the customer, increasing satisfaction.

[0715] Example: A staff member explains the product and makes suggestions to Customer A.

[0716] In this way, a system is realized that streamlines the management of time sales through a multi-stage processing flow and improves the customer experience.

[0717] Example 2

[0718] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0719] While conventional store management systems have basic functions such as obtaining updated price information and managing inventory, they lack the ability to recognize customer emotions and provide personalized responses based on those emotions. This results in a lack of concrete measures to improve customer satisfaction, making efficient store management difficult. Furthermore, even when receiving timely notifications about out-of-stock items or special sales, staff responses are delayed, resulting in reduced productivity.

[0720] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0721] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using artificial intelligence generative technology, means for providing the generated announcements to customers in the store by voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for detecting emotions of customers in the store using an emotion recognition engine, means for generating announcements based on the detected emotions, and means for notifying employees of the detected emotion information. This enables individual responses based on customer emotions, contributing to improved customer satisfaction as well as enabling more efficient store operations and faster staff responses.

[0722] "Price updates" refers to the latest price information for products sold in stores, which is often updated in real time.

[0723] "Generative AI technology" refers to artificial intelligence technology for generating natural language text and speech based on acquired data, and specifically includes generative AI models.

[0724] An "announcement" is a voice message provided to customers in a store, and effectively conveys price information, special sale information, and the like.

[0725] "Object detection technology" is a technology that uses cameras and sensors to monitor and recognize customer behavior and the status of products in a store in real time.

[0726] "Product inventory" refers to the quantity and condition of products sold in a store, and is quantitatively grasped by an inventory management system.

[0727] "Out-of-stock information" is information that notifies you that a particular product is out of stock or in short supply.

[0728] An "emotion recognition engine" is a technology that analyzes emotions from a customer's facial expressions and behavior, and recognizes them as states such as "interest," "satisfaction," or "anxiety."

[0729] "Employees" refers to staff in the store who are responsible for tasks such as product management and customer service.

[0730] "Notification" refers to the transmission of information by the system, including messages to encourage employees to take action.

[0731] The "in-store speaker" and "terminal" refer to audio equipment and its control device for playing the generated announcement text as audio.

[0732] The present invention is a system that aims to improve the efficiency of store operations and customer satisfaction. This system operates mainly around three components: a server, a terminal, and a user. Specific embodiments of this system are described in detail below.

[0733] Get price updates

[0734] The server periodically communicates with the price update system to obtain new price information. This communication is performed via an API, and for example, at 9:00 AM and 3:00 PM, the server downloads the day's special prices and new prices from the database. This data includes the product ID, product name, new price, etc.

[0735] Announcement generation

[0736] The server generates an announcement using generative artificial intelligence technology based on the acquired price update information. A generative AI model (e.g., GPT-3) is used to generate this announcement. For example, the price update information is input as a prompt, and the generated text is "Today's sale information! Fresh eggs are 180 yen." An example of a prompt from a generative AI model is as follows:

[0737] Prompt: "Generate an announcement based on the sale information. Example: Eggs, new price 180 yen."

[0738] Generated announcement: "Today's special offer! Fresh eggs for 180 yen."

[0739] Providing announcement text

[0740] The generated announcement is sent from the server to the terminal. The terminal is connected to the store's speaker system, and the received announcement is provided to customers as audio. For example, at 11:00 a.m. and 4:00 p.m., an announcement such as "Today's special: Fresh eggs for 180 yen. Don't miss out!" is played in the store.

[0741] Object detection surveillance

[0742] The server uses object detection technology to monitor customer behavior and product inventory through cameras and sensors installed in the store. For example, a camera monitors the sale shelves and detects the moment a customer picks up an egg. At that time, inventory information is automatically updated based on the image data.

[0743] Notification of out-of-stock information

[0744] Based on inventory information monitored by object detection technology, the server immediately notifies users (store employees) when an item is about to run out. This notification is sent to bone conduction earphones worn by employees, and for example, a message such as "Eggs are running low. Please order more from the warehouse."

[0745] User Emotion Recognition

[0746] The server uses an emotion recognition engine to collect data on customers' facial expressions and behavior from cameras and sensors installed in the store, and analyzes and recognizes their emotions. For example, a customer's smile when looking at a sale item can be recognized as "interest." This emotion data is updated in real time.

[0747] Emotion-based announcement generation

[0748] The server generates a specific announcement based on the recognized emotion information. For example, if a customer is interested in a special sale item, the server creates an announcement saying, "Thank you for your interest in our special sale items! Please take a look."

[0749] Staff notification of emotional information

[0750] The server notifies the user (store employee) of the recognized emotion information. The employee receives this notification and responds to the customer based on the content, for example, "Customer B is interested in a special sale item. Please provide additional information or support." The staff member approaches the customer and kindly provides product details and other special sale information.

[0751] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0752] Step 1:

[0753] The server retrieves price updates.

[0754] Input: Schedule information, authentication information to make API calls.

[0755] Specific operation: Sends an API request at 9:00 AM and 3:00 PM to retrieve the latest price information (product ID, product name, new price, etc.) from the store's database.

[0756] Output: A list of retrieved price updates.

[0757] Step 2:

[0758] The server generates an announcement based on the acquired price update information.

[0759] Input: Retrieved price update information.

[0760] Specific operation: Create a prompt to input price update information into the generated AI model (e.g., GPT-3) and input it into the AI ​​model. For example, enter the product ID and new price information as a prompt: "Eggs, new price 180 yen."

[0761] Output: The generated announcement (e.g., "Today's special offer! Fresh eggs for 180 yen.").

[0762] Step 3:

[0763] The server sends the generated announcement text to the terminal and provides it as audio from speakers inside the store.

[0764] Input: The generated announcement text.

[0765] Specific operation: The generated announcement text is sent to the terminal in JSON format, and the terminal sends the received data to a speech synthesis engine, which then plays the announcement over the in-store speakers.

[0766] Output: Announcement audio broadcast in the store.

[0767] Step 4:

[0768] The server uses object detection technology to monitor customer behavior and product inventory.

[0769] Input: Video data from cameras and sensors in the store.

[0770] Specific actions: Real-time video data analysis detects the moment a specific product is picked up, for example, when a customer reaches out in front of an egg shelf.

[0771] Output: Updated inventory information.

[0772] Step 5:

[0773] The server notifies staff of out-of-stock information based on the monitored inventory information.

[0774] Input: Updated inventory information.

[0775] Specific operation: When the inventory level falls below a set threshold, a notification method (e.g., bone conduction earphones) is selected and a notification message is generated. For example, a message such as "Egg inventory has fallen below 10 units. Please replenish." is created and sent to staff.

[0776] Output: Staff notification message.

[0777] Step 6:

[0778] The server uses an emotion recognition engine to recognize the emotion of the customer.

[0779] Input: Video data from cameras and sensors in the store.

[0780] Specific behavior: Video data is analyzed in real time to recognize emotions such as "interest," "satisfaction," and "anxiety" from facial expressions and behavior. For example, if a customer smiles when looking at a sale item, this is recognized as "interest."

[0781] Output: Recognized emotion information.

[0782] Step 7:

[0783] The server generates an announcement based on the recognized emotion.

[0784] Input: Recognized emotion information.

[0785] Specific operation: Emotional information is input into the generative AI model to generate appropriate announcements. For example, if a customer expresses interest, the system will generate an announcement such as, "Thank you for your interest in our special sale items! Please take a look."

[0786] Output: The generated announcement.

[0787] Step 8:

[0788] The server notifies the staff of the recognized emotion information.

[0789] Input: Recognized emotion information.

[0790] Specific operation: Generate a notification message to staff based on the emotion information. For example, generate a message saying, "Customer B is interested in a special sale item. Please provide additional guidance or support." and send it to staff.

[0791] Output: Staff notification message.

[0792] (Application example 2)

[0793] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0794] Conventional store management systems lacked the ability to recognize customer emotions and respond appropriately. This made it difficult for employees to properly understand the emotions of individual customers and provide services that match those emotions. Furthermore, while efficient operations are required in terms of inventory management and the provision of special sale information, conventional systems were unable to fully meet these needs.

[0795] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative artificial intelligence technology, means for providing the generated announcements to customers in the store by voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for recognizing user emotions, means for generating and providing announcements based on the recognized emotions, and means for notifying employees of the recognized emotion information. This enables responses and announcements tailored to individual customers' emotions, efficient inventory management, and the provision of special sale information.

[0796] "Price update information" is the latest price change information for products and services, and is data used in store operations.

[0797] "Generative AI technology" is a technology that allows computers to learn independently and imitate creative tasks performed by humans, and is used for text generation and data analysis.

[0798] "Means for generating announcements" refers to a function that automatically generates announcements to provide to customers based on price update information and other data.

[0799] "Means for providing the created announcement text to customers in the store by voice" refers to a function for informing customers of the created announcement text as voice through a speaker or other audio output device.

[0800] "Object detection technology" is a technology that uses cameras and sensors to identify specific objects or human movements and acquire them as data.

[0801] "Means for monitoring customer behavior and product inventory" refers to a function that uses object detection technology to check customer movements within the store and product inventory status, and collects and analyzes the data.

[0802] "Means for notifying employees of out-of-stock information" refers to a function that notifies employees when a product is sold out or when inventory is low.

[0803] "Means for recognizing user emotions" refers to technology that analyzes data such as a customer's facial expressions and tone of voice to identify their emotional state.

[0804] "Means for generating and providing announcements based on recognized emotions" refers to a function that automatically creates appropriate guidance messages for individual customers based on the user's emotional data and provides them in voice or text format.

[0805] The "means of notifying employees of recognized emotional information" is a function that conveys the results of analyzing the user's emotions to employees and encourages them to take appropriate action.

[0806] The present invention relates to the implementation of a system for improving operational efficiency and customer experience in a physical store. Specific embodiments for carrying out the present invention are described below.

[0807] System Overview

[0808] The system includes functions to obtain price update information, generate announcements using generative artificial intelligence technology and provide them in-store, monitor customer behavior and product inventory using object detection technology, notify employees of out-of-stock information, recognize customer emotions and generate announcements based on that information, and notify employees of emotional information.

[0809] Hardware and software used

[0810] Hardware:

[0811] Webcam: Capture your customer's facial expressions in real time.

[0812] Head-mounted display: worn by employees to receive information in real time.

[0813] Speaker: An audio output device for providing announcements within the store.

[0814] software:

[0815] OpenCV: Processes image data obtained from the camera and performs object detection.

[0816] DeepFace: A facial recognition library for analyzing customer emotions.

[0817] Google Cloud Text-to-Speech: Converts the generated announcement text into audio.

[0818] Specific operation of the system

[0819] 1. Get price updates:

[0820] The server periodically retrieves the latest price update information from the price update system. This information is the latest price change information for products and services, and is important data for store operations.

[0821] 2. Announcement generation:

[0822] The server uses generative artificial intelligence technology to generate announcements based on price update information. This technology automatically creates announcements that effectively communicate sales and special offers to customers.

[0823] 3. In-store announcements:

[0824] The server sends the generated announcement to the speakers in the store and provides it by voice, so that all customers in the store can receive the latest sales information in real time.

[0825] 4. Object detection surveillance:

[0826] The server uses OpenCV to monitor customer behavior and product inventory in the store. When a customer picks up a specific product, the behavior is detected and the inventory information is automatically updated.

[0827] 5. Out-of-stock notification:

[0828] When inventory is low, the server notifies employees of the shortage, which is displayed on a head-mounted display worn by the employee, allowing them to immediately take action to replenish the stock.

[0829] 6. Customer Emotion Recognition:

[0830] The server uses DeepFace to analyze the video captured by the webcam and recognize the customer's emotions. The recognized emotions are collected and analyzed as data in real time.

[0831] 7. Emotion-based announcement generation:

[0832] The server generates special announcements based on the recognized emotions and delivers them through speakers and displays in the store, enabling guidance optimized for the customer's current emotions.

[0833] 8. Employee Notification of Emotional Information:

[0834] The server then notifies employees of the recognized emotional information. For example, if a customer is confused, the information is conveyed to employees, who can provide support promptly.

[0835] Examples of specific examples and prompts

[0836] Examples:

[0837] If a customer is smiling while looking at products in the sale section, the server generates and provides a voice announcement saying, "Thank you for your interest in our sale. Please choose your favorite product." If the customer looks confused, the server provides a voice announcement saying, "Is there anything I can help you with? Our staff is here to help you."

[0838] Example prompt sentence:

[0839] "Create a program that analyzes the facial expressions of customers in a store and generates announcements based on their emotions. If the emotion is 'happy', announce, 'Thank you for your interest in our sale. Please choose your favorite product.' If the emotion is 'sad', announce, 'Is there anything I can help you with? Our staff is here to help you.'"

[0840] This invention provides an embodiment of a new system that combines emotion recognition and AI to achieve a high level of automation in store operations and improve customer satisfaction.

[0841] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0842] Step 1:

[0843] The server periodically retrieves the latest price update information from the price update system. The input is data from the price update system, and the output is the latest price update information. This information is the latest price change information for products and services, and is data necessary for store operations.

[0844] Step 2:

[0845] Based on the price update information acquired by the server, announcement text is generated using generative artificial intelligence technology. The input is the price update information, and the output is the generated announcement text. The generated announcement text effectively communicates sales and special offers to customers.

[0846] Step 3:

[0847] The server sends the generated announcement to a terminal in the store and provides it audibly through a speaker. The input is the generated announcement, and the output is a voice announcement played through the in-store speaker. This allows customers in the store to receive special sale information in real time.

[0848] Step 4:

[0849] The server uses OpenCV to detect objects in the image data acquired from the webcam, and monitors customer behavior and product inventory in the store. The input is video data from the webcam, and the output is customer behavior data and inventory information. This allows the detection of actions such as customers picking up sale items.

[0850] Step 5:

[0851] The server uses object detection technology to collect inventory information and notifies employees of out-of-stock information. The input is product inventory information, and the output is a shortage notification sent to bone conduction earphones or a head-mounted display. This allows employees to immediately understand shortages and take action.

[0852] Step 6:

[0853] The server uses DeepFace to analyze video data captured by a webcam and recognize customer emotions. The input is video data from the webcam, and the output is the recognized customer emotion data. This data is obtained from the customer's facial expressions and behavior.

[0854] Step 7:

[0855] The server generates special announcements based on the recognized emotion data and provides them through speakers and displays in the store. The input is the recognized customer emotion data, and the output is the announcement provided through speakers and displays in the store. This makes it possible to provide information that matches the customer's current emotion.

[0856] Step 8:

[0857] The server notifies employees of the recognized emotion information and assists them in handling customers. The input is the recognized emotion data, and the output is a notification message sent to employees. This allows employees to take appropriate action based on the customer's emotion.

[0858] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0859] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0860] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0861] [Third embodiment]

[0862] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0863] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0864] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0865] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0866] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0867] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0868] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0869] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0870] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0871] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0872] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0873] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0874] To implement the present invention, an in-store system is constructed as follows.

[0875] System Overview

[0876] To streamline time sales in stores and improve the consumer experience, the system retrieves price updates, generates and delivers announcements, monitors customer behavior and product inventory using object detection, and notifies staff of out-of-stock information.

[0877] System configuration

[0878] 1. Get price updates

[0879] The server periodically retrieves the latest price updates from the price update system.

[0880] Example: The server collects new price information such as "eggs 200 yen → 180 yen" from a supermarket price update database.

[0881] 2. Announcement Generation

[0882] The server generates announcement text based on the obtained price update information using generative artificial intelligence techniques.

[0883] Example: The server generates an announcement such as "Egg sale, only 180 yen today!"

[0884] 3. Providing announcement text

[0885] The server transmits the generated announcement to a terminal that controls a speaker in the store, and provides it to the customer by voice.

[0886] Example: The terminal announces through the in-store speaker, "We're currently having a special sale on eggs. They're 180 yen per pack. Please buy some."

[0887] 4. Object Detection Monitoring

[0888] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[0889] Example: The server uses data from in-store cameras to capture when a customer picks up an egg and updates the inventory.

[0890] 5. Notification of out-of-stock information

[0891] The server checks the inventory status of the detected product and notifies staff if there is a shortage.

[0892] Example: Generate a notification such as "You're low on eggs. Please get more from the warehouse."

[0893] Users (staff) receive information about out-of-stock items in real time through bone conduction earphones and can quickly replenish products.

[0894] Example: A staff member receives a notification via bone conduction earphones saying, "We are out of eggs. Please replenish them," and restocks the stock from the warehouse.

[0895] A natural language description of the process

[0896] Get price updates

[0897] The server periodically retrieves the latest price update information from the price update system, which is price change information for the product and may fluctuate in real time.

[0898] Announcement generation

[0899] The server uses generative artificial intelligence technology to generate announcements based on the acquired price update information. The generated announcements effectively communicate information about special sales and limited-time sales to customers.

[0900] Providing announcement text

[0901] The server sends the generated announcement text to a terminal in the store, which then provides the content to customers by voice through the in-store speaker, allowing customers to receive special sale information in real time.

[0902] Object detection surveillance

[0903] The server uses object detection technology to monitor customer behavior and product inventory in the store. For example, if a customer picks up a sale item, the server detects that behavior and updates inventory information accordingly.

[0904] Notification of out-of-stock information

[0905] If a shortage occurs based on the monitored inventory information, the server notifies the staff of the shortage via bone conduction earphones, allowing the staff to immediately take action to deal with the shortage.

[0906] This system will streamline the management of time sales, improve the quality of service to customers, and ensure smooth store operations.

[0907] The processing flow will be explained below.

[0908] Step 1:

[0909] The server periodically retrieves the latest price updates from the price update system.

[0910] Specific operation: Refer to the price update database to get new price information. This information includes "product name" and "new price".

[0911] Example: Get the data "Eggs 200 yen → 180 yen".

[0912] Step 2:

[0913] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[0914] Specific operation: The acquired price update information is passed as input data to the AI ​​model, and an appropriate announcement text is automatically generated.

[0915] Example: Generate an announcement for price update information: "Eggs on sale for 180 yen today only!"

[0916] Step 3:

[0917] The server sends the generated announcement to the terminal (in-store speaker) and provides it as audio.

[0918] Specific operation: The generated announcement text is converted into an audio file and sent to the in-store speaker terminal.

[0919] Example: "We're currently having a special sale on eggs. They're 180 yen per pack. Please come and buy some." is announced over the store's speakers.

[0920] Step 4:

[0921] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[0922] Specific actions: Data from cameras and sensors installed in the store is analyzed to detect when a customer picks up a specific product.

[0923] Example: Recognizing when a customer picks up a carton of eggs from the shelf and reducing stock levels.

[0924] Step 5:

[0925] The server checks the inventory status of the detected product and checks for out-of-stock information.

[0926] What it does: Looks up the updated inventory database and checks if a particular product is below a certain threshold.

[0927] Example: If there are only 2 packs of eggs left in stock, an out-of-stock alert will be issued.

[0928] Step 6:

[0929] If the server detects out-of-stock information, it generates a message to notify staff.

[0930] Specific operation: Based on the out-of-stock information, a message requesting replenishment is generated for staff.

[0931] Example: Generate the message "Low eggs left. Please get more from the warehouse."

[0932] Step 7:

[0933] The server delivers the generated notification message to the staff member via bone conduction earphones.

[0934] Specific operation: Convert notification messages into audio signals and send them to staff through bone conduction earphones.

[0935] Example: A staff member receives a notification that "We are out of eggs. Please restock."

[0936] Step 8:

[0937] After receiving the notification, the user (staff member) promptly replenishes the stock.

[0938] Specific actions: Go to the warehouse, retrieve out-of-stock items, and replenish them on the sales floor.

[0939] Example: A staff member takes cartons of eggs from the warehouse and restocks them on the sales shelves.

[0940] Through each step, a system is built that improves the efficiency of time sales and the consumer experience.

[0941] Example 1

[0942] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0943] In traditional store operations, managing special sales and time sales required a lot of manual effort, resulting in inefficiency. Because processes such as providing sales information to consumers, managing inventory, and notifying of out-of-stock items were all done manually, it was difficult to update information in real time, which resulted in reduced customer satisfaction and lost sales opportunities.

[0944] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0945] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative AI technology, means for providing the generated announcements to customers in the store via voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for periodically accessing a price update system to acquire price update information, means for generating announcements by sending prompts to a generative AI model based on the acquired price update information, means for receiving video data from in-store cameras and detecting customer behavior using object detection technology, means for transmitting the generated announcements to in-store terminals, and means for using bone conduction earphones worn by staff to notify out-of-stock information. This enables effective management of limited-time sales, real-time product inventory monitoring, and rapid out-of-stock response.

[0946] "Price update information" is data provided when the price of a product in a store is changed, and includes both the old and new price information.

[0947] "Generative AI technology" is an AI technology for text generation and natural language processing, which generates appropriate output text based on specified input data.

[0948] An "announcement" is a sentence created to inform customers of information such as special sales and limited-time sales, and is a text for providing the content of the announcement by voice.

[0949] "Object detection technology" is a technology for recognizing and detecting specific objects from image data, and is used to monitor customer behavior and merchandise movement within a store.

[0950] "Out-of-stock information" is information that is notified when the product inventory in the store falls below a certain amount, and is information that encourages additional purchases or replenishment.

[0951] A "server" is a computer system that provides a specific service over a network. In this system, it is a device that performs functions such as obtaining price updates, generating announcements, and monitoring using object detection.

[0952] A "terminal" is a device that receives announcements sent from the server and provides them to customers in the store by voice, and is usually connected to a speaker system.

[0953] A "prompt" is an instruction given to a generative AI model, containing hints and instructions for generating a specific output.

[0954] A "generative AI model" is an artificial intelligence model that generates appropriate text based on a given prompt sentence, and in the present invention is primarily used to generate announcement sentences.

[0955] Bone conduction earphones are devices that transmit sound through the bones to the inner ear, allowing employees to receive voice notifications without using their hands.

[0956] The present invention relates to a system for streamlining in-store sales and time-limited sales to improve the consumer experience by obtaining price updates, generating and providing announcements, monitoring customer behavior and product inventory using object detection, and notifying staff of out-of-stock information.

[0957] System configuration

[0958] Get price updates

[0959] The server periodically accesses the price update system to retrieve the latest price updates. This process involves using HTTP requests to retrieve data in JSON format from the API. The retrieved data includes information such as the product name, old price, and new price. For example, the server receives price updates such as "eggs from 200 yen to 180 yen" from a supermarket's price update database.

[0960] Announcement generation

[0961] The server sends a prompt message to the generative AI model based on the obtained price update information. This prompt message contains information about which product's price has changed and how. The generative AI model (e.g., GPT-4) generates an announcement message based on this prompt. For example, a prompt message might be created saying, "The price of the product has changed. Eggs: 200 yen → 180 yen. Please generate a sale announcement message based on this information." The generative AI model would then generate an announcement message saying, "Eggs on sale for 180 yen today only!"

[0962] Providing announcement text

[0963] The server sends the generated announcement to a terminal in the store. The terminal receives the announcement and provides it audibly through the store's speakers. This allows customers to receive sale information in real time. For example, the terminal may broadcast an announcement over the store's speakers saying, "We're currently having a sale on eggs. They're 180 yen a pack. Please buy some."

[0964] Object detection surveillance

[0965] The server uses object detection technology to monitor customer behavior and product inventory in the store. YOLO (You Only Look Once) is one example of the object detection technology. This technology is used to analyze video data from cameras in the store, and if a specific action (such as a customer picking up an egg) is detected, inventory information is updated. For example, the server receives video data from cameras in the store, detects the action of a customer picking up an egg, and updates inventory information accordingly.

[0966] Notification of out-of-stock information

[0967] The server monitors inventory information and generates out-of-stock information when a certain number of items falls below a certain level. This information is notified to staff via bone conduction earphones. For example, the server detects that inventory is running low and generates a notification saying, "We have only a few eggs left. Please replenish them from the warehouse." The staff (users) receive the notification via bone conduction earphones saying, "We are out of eggs. Please replenish them," and quickly replenish the items.

[0968] Hardware and software used

[0969] Price update system: A product management database system used by stores

[0970] Generative AI model: GPT-4 (OpenAI)

[0971] Object detection technology: Image recognition technology such as YOLO (You Only Look Once)

[0972] Notification device: Bone conduction earphones

[0973] Examples and prompts

[0974] Below are some examples of prompt sentences.

[0975] "The price of the product has changed. Eggs: 200 yen → 180 yen. Please generate a sale announcement based on this information."

[0976] "Price update information: Eggs 200 yen → 180 yen. Please use this data to create a sale announcement for your store. For example, please use a format like, 'Egg sale, 180 yen today only!'"

[0977] This system will streamline store operations, improve the quality of service to customers, and provide information in real time.

[0978] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0979] Step 1: Get price updates

[0980] The server periodically accesses the price update system to obtain the latest price update information. At this time, an HTTP request is used as input and JSON format data is received from the API endpoint. Specifically, the server sends a request to the price update system and obtains price update information such as "Eggs 200 yen → 180 yen." The data includes the product name, old price, new price, etc.

[0981] Step 2: Generate announcement text

[0982] The server sends a prompt message to the generative AI model based on the acquired price update information. It uses the price update information as input and obtains an announcement message as output. Specifically, it creates a prompt message saying, "The price of the product has changed. Eggs have gone from 200 yen to 180 yen. Please generate a sale announcement message based on this information." and sends it to the generative AI model (e.g., GPT-4). The generative AI model then generates a response message saying, "Eggs on sale for 180 yen today only!"

[0983] Step 3: Provide the announcement

[0984] The server sends the generated announcement to a terminal in the store. The generated announcement is used as input and is provided to the customer as audio output. The terminal receives this announcement and provides it to the customer through the in-store speaker. For example, the announcement may say, "We're currently having a special sale on eggs. One pack is 180 yen. Please buy some."

[0985] Step 4: Object detection surveillance

[0986] The server uses object detection technology to monitor customer behavior and product inventory in the store. It uses video data from in-store cameras as input and obtains customer behavior and inventory information as output. Specifically, it analyzes the video from the in-store cameras and detects customer behavior using object detection technology (e.g., YOLO). For example, it detects the behavior of a customer picking up an egg and updates inventory information accordingly.

[0987] Step 5: Out-of-stock notification

[0988] The server uses the monitored inventory information to notify staff when a stockout occurs. It uses the updated inventory information as input and generates a stockout notification as output. Specifically, when inventory falls below a certain amount (e.g., there are fewer than 10 eggs left), a notification is generated. A notification stating "Only a few eggs left. Please add more from the warehouse" is created and sent to staff via bone conduction earphones. The user (staff member) receives this notification and replenishes the product from the warehouse.

[0989] (Application example 1)

[0990] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0991] With conventional time sale management systems, it was difficult to obtain real-time in-store price update information and provide it to customers efficiently. Furthermore, monitoring of customer behavior and product inventory management were insufficient, and responding quickly when an item was out of stock was also an issue. Furthermore, the lack of a push notification function to provide customers with timely information limited the ability to maximize sales.

[0992] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0993] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative artificial intelligence technology, means for providing the generated announcements to customers in the store by voice and push notification, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for notifying employees of out-of-stock information using bone conduction earphones, and means for generating prompts for generating announcements based on the price update information using generative artificial intelligence technology. This enables efficient and real-time time sale management, improving customer experience and maximizing sales.

[0994] "Price update information" refers to the latest price information for products in stores, which is periodically obtained from a database or price update system.

[0995] "Generative artificial intelligence technology" is a technology that uses an AI model to generate text, images, etc. based on input data.

[0996] An "announcement" is a sentence generated to effectively convey information about special sales or limited-time sales.

[0997] "Voice and push notifications" refers to providing information through in-store speakers and notifications to customers' smart devices.

[0998] "Object detection technology" is a technology that uses cameras and sensors to monitor customer behavior and product inventory in stores in real time.

[0999] "Product inventory" refers to the current number of products held in the store.

[1000] "Out-of-stock information" is notification information that is generated when the stock of a specific product falls below a certain level.

[1001] "Employees" refers to staff working in the store.

[1002] Bone conduction earphones are special earphones that transmit sound through the bones rather than the ears, and are worn by employees.

[1003] A "generative prompt" is input text for generating an announcement using generative artificial intelligence technology.

[1004] The present invention provides a system for increasing the efficiency of limited-time sales in stores and improving consumer experiences. Specific embodiments for realizing this system will be described below.

[1005] Overall system configuration

[1006] The system is designed to efficiently manage time-limited sales and special offers in stores and provide them to customers in real time. Its main components include a server, smartphones, bone conduction earphones, in-store cameras, object detection technology, generative artificial intelligence technology (generative AI model), and a push notification system.

[1007] Hardware and Software Configuration

[1008] 1. Server

[1009] The server plays the important role of obtaining price update information, generating announcements using generative artificial intelligence technology (generative AI model), and sending them to in-store smartphones and voice systems. Specifically, the server was built using Python and Node.js, and Firebase Firestore was used as the database. The generative AI model used was OpenAI GPT-4.

[1010] 2. Smartphone

[1011] Smartphones are devices that provide information to customers and employees. Developed cross-platform using Flutter, they provide information through push notifications and in-app messages.

[1012] 3. Bone conduction earphones

[1013] The bone conduction earphones worn by employees are devices used to notify them of stock shortages in real time.

[1014] 4. In-store cameras and object detection technology

[1015] In-store cameras monitor customer behavior and product inventory using object detection technology (using OpenCV), which allows for real-time tracking of customer pick-up behavior and shelf inventory.

[1016] Main processes and data flow

[1017] 1. Get price updates

[1018] The server periodically retrieves the latest price information from an external price update system, which is then stored in Firebase Firestore.

[1019] 2. Announcement Generation

[1020] The server generates an announcement using a generative AI model (OpenAI GPT-4) based on the acquired price information. An example of a prompt is, "Please generate special sale information for eggs. Today's special sale price is 180 yen."

[1021] 3. Providing announcement text

[1022] The generated announcement text is sent from the server to the smartphone via push notification and in-store speakers. For example, a notification might be sent to the customer's smartphone saying, "We're currently having a special sale on eggs. One pack costs 180 yen."

[1023] 4. Object Detection Monitoring

[1024] The video data captured by the in-store cameras is sent to a server and analyzed in real time using OpenCV. The analysis results are used to understand what products customers have picked up and changes in inventory levels.

[1025] 5. Notification of out-of-stock information

[1026] Using object detection technology, if inventory falls below a certain threshold, the server will notify employees of the shortage via bone conduction earphones, and will also send a notification to their smartphone, such as "There are only a few eggs left. Please add more from the warehouse."

[1027] This enables efficient and real-time time sale management, improving customer experience and maximizing sales.

[1028] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1029] Step 1:

[1030] The server retrieves the latest price information from an external price update system. To do this, it uses an HTTP request to retrieve price update data, parses the data, and saves it in JSON format. The specific input is the response data from the price update API, and the output is the internal data structure of the parsed price information. Example: Retrieving price information such as "Eggs 200 yen → 180 yen."

[1031] Step 2:

[1032] The server generates an announcement using a generative AI model (OpenAI GPT-4) based on the acquired price information. Here, the input is the acquired price information (e.g., the price of eggs has changed to 180 yen), and the output is the announcement generated by the generative AI model. Specifically, the prompt "Please generate egg sale information. Today's sale price is 180 yen" is input into the generative AI model, which generates the announcement "Egg sale, 180 yen for today only!"

[1033] Step 3:

[1034] The server sends the generated announcement to the smartphone application and the in-store speaker. The input is the generated announcement (e.g., "Eggs on sale for just 180 yen today!"), and the output is a push notification and a voice broadcast. Specifically, it uses Firebase Cloud Messaging (FCM) to send a push notification to the customer's smart device and controls the speaker system to broadcast the announcement throughout the store.

[1035] Step 4:

[1036] The terminal uses in-store cameras to monitor customer behavior and product inventory. The input is video data from the cameras, and the output is customer behavior and inventory information analyzed using an object detection algorithm (using OpenCV). Specifically, it analyzes the video captured by the cameras in real time to capture customer behavior when picking up products and changes in inventory levels.

[1037] Step 5:

[1038] The server notifies employees when an item is out of stock based on inventory information obtained using object detection technology. The input is inventory information obtained and analyzed using object detection technology, and the output is an out-of-stock notification message. Specifically, when the inventory level falls below a certain threshold, the generative AI model generates an out-of-stock notification message and sends the notification to bone conduction earphones and smartphones. For example, a message such as "There are only a few eggs left. Please add more from the warehouse" is generated.

[1039] Step 6:

[1040] The user (employee) receives the out-of-stock notification via bone conduction earphones and a smartphone and responds promptly. The input is the out-of-stock notification message from the server, and the output is the replenishment of stock when the user actually takes action. Specifically, an employee receives a notification saying, "Eggs are out of stock. Please replenish them," and takes action to replenish the product from the warehouse.

[1041] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1042] To put the present invention into practice, an emotion engine is combined with an in-store system to recognize the user's emotions and make announcements or notify staff accordingly.

[1043] System Overview

[1044] In order to improve the efficiency of store operations and the consumer experience, this system acquires price update information, generates and provides announcements, monitors using object detection, notifies of out-of-stock information, and also includes an emotion engine that recognizes user emotions.

[1045] System configuration

[1046] 1. Get price updates

[1047] The server periodically retrieves the latest price updates from the price update system.

[1048] Example: A server collects price change information from a supermarket database.

[1049] 2. Announcement Generation

[1050] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[1051] Example: Generate an announcement such as "Eggs on sale for just 180 yen today only!"

[1052] 3. Providing announcement text

[1053] The server transmits the generated announcement text to the terminal and provides it through a speaker in the store.

[1054] Example: Broadcast an announcement saying, "We're currently having a special sale on eggs. One pack costs 180 yen."

[1055] 4. Object Detection Monitoring

[1056] The server uses object detection technology to monitor customer behavior and product inventory within the store.

[1057] Example: Detecting the behavior of a customer picking up a sale item and updating inventory information.

[1058] 5. Notification of out-of-stock information

[1059] The server checks inventory information and notifies staff when an item is out of stock.

[1060] Example: Generate a notification saying "You're low on eggs. Please add more from the warehouse."

[1061] 6. User Emotion Recognition

[1062] The server uses an emotion engine to detect the emotions of customers in the store and bases further processing on that information.

[1063] Example: An emotion engine recognizes emotions such as "interest" or "anxiety" from a customer's facial expressions and behavior.

[1064] 7. Emotion-based announcement generation

[1065] The server generates an announcement sentence based on the recognized emotion and transmits it to the terminal.

[1066] Example: If a customer expresses interest in a sale, add an announcement saying, "Thank you for your interest in our sale."

[1067] 8. Staff Notification of Emotional Information

[1068] The server uses the recognized emotion information to notify staff and assist them in handling customers.

[1069] Example: Sending a message saying, "Customer A has expressed interest. Please provide additional support."

[1070] The user (staff member) receives the emotional information and responds appropriately to the customer.

[1071] Example: Store staff provides additional support to Customer A to facilitate a purchase.

[1072] A natural language description of the process

[1073] Get price updates

[1074] The server periodically retrieves the latest price updates from the price update system, which are price changes for products and are often updated in real time.

[1075] Announcement generation

[1076] The server uses generative artificial intelligence technology to generate announcements based on the price update information. The generated announcements effectively communicate information about special sales and limited-time sales to customers.

[1077] Providing announcement text

[1078] The server sends the generated announcement text to a terminal in the store, which then provides the content to customers by voice through the in-store speaker, allowing customers to receive special sale information in real time.

[1079] Object detection surveillance

[1080] The server uses object detection technology to monitor customer behavior and product inventory in the store. For example, if a customer picks up a sale item, the server detects that behavior and updates inventory information accordingly.

[1081] Notification of out-of-stock information

[1082] If a shortage occurs based on the monitored inventory information, the server notifies the staff of the shortage via bone conduction earphones, allowing the staff to immediately take action to deal with the shortage.

[1083] User Emotion Recognition

[1084] The server uses an emotion engine to detect the user's emotions from their facial expressions and behavior, enabling more detailed customer service.

[1085] Emotion-based announcement generation

[1086] The server generates special announcements based on the recognized emotions, allowing the information provided to customers to match their current emotions.

[1087] Staff notification of emotional information

[1088] The server notifies the staff of the user's emotional information and assists them in responding to the customer. Based on the information received, the staff responds quickly and appropriately, improving customer satisfaction.

[1089] In this way, through each step, a system is created that streamlines the management of time sales and improves the consumer experience.

[1090] The processing flow will be explained below.

[1091] Step 1:

[1092] The server periodically retrieves the latest price updates from the price update system.

[1093] Specific operation: Refer to the price update database to get new price information. This information includes "product name" and "new price".

[1094] Example: Get the data "Eggs 200 yen → 180 yen".

[1095] Step 2:

[1096] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[1097] Specific operation: The acquired price update information is passed as input data to the AI ​​model, and an appropriate announcement text is automatically generated.

[1098] Example: Generate an announcement saying "Eggs on sale for just 180 yen today only!"

[1099] Step 3:

[1100] The server sends the generated announcement to the terminal (in-store speaker) and provides it as audio.

[1101] Specific operation: The generated announcement text is converted into an audio file and sent to the in-store speaker terminal.

[1102] Example: "We're currently having a special sale on eggs. They're 180 yen per pack. Please come and buy some." is announced over the store's speakers.

[1103] Step 4:

[1104] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[1105] Specific actions: Data from cameras and sensors installed in the store is analyzed to detect when a customer picks up a specific product.

[1106] Example: Recognizing when a customer picks up a carton of eggs from the shelf and reducing stock levels.

[1107] Step 5:

[1108] The server checks the inventory status of the detected product and checks for out-of-stock information.

[1109] What it does: Looks up the updated inventory database and checks if a particular product is below a certain threshold.

[1110] Example: If there are only 2 packs of eggs left in stock, an out-of-stock alert will be issued.

[1111] Step 6:

[1112] If the server detects out-of-stock information, it generates a message to notify staff.

[1113] Specific operation: Based on the out-of-stock information, a message requesting replenishment is generated for staff.

[1114] Example: Generate the message "Low eggs left. Please get more from the warehouse."

[1115] Step 7:

[1116] The server delivers the generated notification message to the staff member via bone conduction earphones.

[1117] Specific operation: Convert notification messages into audio signals and send them to staff through bone conduction earphones.

[1118] Example: A staff member receives a notification that "We are out of eggs. Please restock."

[1119] Step 8:

[1120] The server uses an emotion engine to recognize the user's emotions.

[1121] Specific operation: The emotion engine analyzes the user's facial expressions and behavioral patterns based on data from cameras and sensors installed in the store.

[1122] Example: Detecting emotions such as "interest" or "anxiety" from a customer's facial expression.

[1123] Step 9:

[1124] The server generates an announcement sentence based on the recognized emotion and transmits it to the terminal.

[1125] Specific operation: Based on the detected emotion data, a new announcement is generated and sent to the in-store speaker terminal.

[1126] Example: If a customer is interested in a sale item, add the announcement, "Thank you for your interest in our sale item."

[1127] Step 10:

[1128] The server notifies the staff of the recognized emotion information to assist them in dealing with customers.

[1129] Specific operation: Based on the emotional information detected by the emotion engine, a message is generated for the staff and sent via bone conduction earphones.

[1130] Example: Send a message saying, "Customer A has expressed interest. Please provide additional support."

[1131] Step 11:

[1132] After receiving the notification, the user (staff member) will promptly respond to the customer.

[1133] What happens: Staff receive the notification and provide additional support to the customer, increasing satisfaction.

[1134] Example: A staff member explains the product and makes suggestions to Customer A.

[1135] In this way, a system is realized that streamlines the management of time sales through a multi-stage processing flow and improves the customer experience.

[1136] Example 2

[1137] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1138] While conventional store management systems have basic functions such as obtaining updated price information and managing inventory, they lack the ability to recognize customer emotions and provide personalized responses based on those emotions. This results in a lack of concrete measures to improve customer satisfaction, making efficient store management difficult. Furthermore, even when receiving timely notifications about out-of-stock items or special sales, staff responses are delayed, resulting in reduced productivity.

[1139] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1140] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using artificial intelligence generative technology, means for providing the generated announcements to customers in the store by voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for detecting emotions of customers in the store using an emotion recognition engine, means for generating announcements based on the detected emotions, and means for notifying employees of the detected emotion information. This enables individual responses based on customer emotions, contributing to improved customer satisfaction as well as enabling more efficient store operations and faster staff responses.

[1141] "Price updates" refers to the latest price information for products sold in stores, which is often updated in real time.

[1142] "Generative AI technology" refers to artificial intelligence technology for generating natural language text and speech based on acquired data, and specifically includes generative AI models.

[1143] An "announcement" is a voice message provided to customers in a store, and effectively conveys price information, special sale information, and the like.

[1144] "Object detection technology" is a technology that uses cameras and sensors to monitor and recognize customer behavior and the status of products in a store in real time.

[1145] "Product inventory" refers to the quantity and condition of products sold in a store, and is quantitatively grasped by an inventory management system.

[1146] "Out-of-stock information" is information that notifies you that a particular product is out of stock or in short supply.

[1147] An "emotion recognition engine" is a technology that analyzes emotions from a customer's facial expressions and behavior, and recognizes them as states such as "interest," "satisfaction," or "anxiety."

[1148] "Employees" refers to staff in the store who are responsible for tasks such as product management and customer service.

[1149] "Notification" refers to the transmission of information by the system, including messages to encourage employees to take action.

[1150] The "in-store speaker" and "terminal" refer to audio equipment and its control device for playing the generated announcement text as audio.

[1151] The present invention is a system that aims to improve the efficiency of store operations and customer satisfaction. This system operates mainly around three components: a server, a terminal, and a user. Specific embodiments of this system are described in detail below.

[1152] Get price updates

[1153] The server periodically communicates with the price update system to obtain new price information. This communication is performed via an API, and for example, at 9:00 AM and 3:00 PM, the server downloads the day's special prices and new prices from the database. This data includes the product ID, product name, new price, etc.

[1154] Announcement generation

[1155] The server generates an announcement using generative artificial intelligence technology based on the acquired price update information. A generative AI model (e.g., GPT-3) is used to generate this announcement. For example, the price update information is input as a prompt, and the generated text is "Today's sale information! Fresh eggs are 180 yen." An example of a prompt from a generative AI model is as follows:

[1156] Prompt: "Generate an announcement based on the sale information. Example: Eggs, new price 180 yen."

[1157] Generated announcement: "Today's special offer! Fresh eggs for 180 yen."

[1158] Providing announcement text

[1159] The generated announcement is sent from the server to the terminal. The terminal is connected to the store's speaker system, and the received announcement is provided to customers as audio. For example, at 11:00 a.m. and 4:00 p.m., an announcement such as "Today's special: Fresh eggs for 180 yen. Don't miss out!" is played in the store.

[1160] Object detection surveillance

[1161] The server uses object detection technology to monitor customer behavior and product inventory through cameras and sensors installed in the store. For example, a camera monitors the sale shelves and detects the moment a customer picks up an egg. At that time, inventory information is automatically updated based on the image data.

[1162] Notification of out-of-stock information

[1163] Based on inventory information monitored by object detection technology, the server immediately notifies users (store employees) when an item is about to run out. This notification is sent to bone conduction earphones worn by employees, and for example, a message such as "Eggs are running low. Please order more from the warehouse."

[1164] User Emotion Recognition

[1165] The server uses an emotion recognition engine to collect data on customers' facial expressions and behavior from cameras and sensors installed in the store, and analyzes and recognizes their emotions. For example, a customer's smile when looking at a sale item can be recognized as "interest." This emotion data is updated in real time.

[1166] Emotion-based announcement generation

[1167] The server generates a specific announcement based on the recognized emotion information. For example, if a customer is interested in a special sale item, the server creates an announcement saying, "Thank you for your interest in our special sale items! Please take a look."

[1168] Staff notification of emotional information

[1169] The server notifies the user (store employee) of the recognized emotion information. The employee receives this notification and responds to the customer based on the content, for example, "Customer B is interested in a special sale item. Please provide additional information or support." The staff member approaches the customer and kindly provides product details and other special sale information.

[1170] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1171] Step 1:

[1172] The server retrieves price updates.

[1173] Input: Schedule information, authentication information to make API calls.

[1174] Specific operation: Sends an API request at 9:00 AM and 3:00 PM to retrieve the latest price information (product ID, product name, new price, etc.) from the store's database.

[1175] Output: A list of retrieved price updates.

[1176] Step 2:

[1177] The server generates an announcement based on the acquired price update information.

[1178] Input: Retrieved price update information.

[1179] Specific operation: Create a prompt to input price update information into the generated AI model (e.g., GPT-3) and input it into the AI ​​model. For example, enter the product ID and new price information as a prompt: "Eggs, new price 180 yen."

[1180] Output: The generated announcement (e.g., "Today's special offer! Fresh eggs for 180 yen.").

[1181] Step 3:

[1182] The server sends the generated announcement text to the terminal and provides it as audio from speakers inside the store.

[1183] Input: The generated announcement text.

[1184] Specific operation: The generated announcement text is sent to the terminal in JSON format, and the terminal sends the received data to a speech synthesis engine, which then plays the announcement over the in-store speakers.

[1185] Output: Announcement audio broadcast in the store.

[1186] Step 4:

[1187] The server uses object detection technology to monitor customer behavior and product inventory.

[1188] Input: Video data from cameras and sensors in the store.

[1189] Specific actions: Real-time video data analysis detects the moment a specific product is picked up, for example, when a customer reaches out in front of an egg shelf.

[1190] Output: Updated inventory information.

[1191] Step 5:

[1192] The server notifies staff of out-of-stock information based on the monitored inventory information.

[1193] Input: Updated inventory information.

[1194] Specific operation: When the inventory level falls below a set threshold, a notification method (e.g., bone conduction earphones) is selected and a notification message is generated. For example, a message such as "Egg inventory has fallen below 10 units. Please replenish." is created and sent to staff.

[1195] Output: Staff notification message.

[1196] Step 6:

[1197] The server uses an emotion recognition engine to recognize the emotion of the customer.

[1198] Input: Video data from cameras and sensors in the store.

[1199] Specific behavior: Video data is analyzed in real time to recognize emotions such as "interest," "satisfaction," and "anxiety" from facial expressions and behavior. For example, if a customer smiles when looking at a sale item, this is recognized as "interest."

[1200] Output: Recognized emotion information.

[1201] Step 7:

[1202] The server generates an announcement based on the recognized emotion.

[1203] Input: Recognized emotion information.

[1204] Specific operation: Emotional information is input into the generative AI model to generate appropriate announcements. For example, if a customer expresses interest, the system will generate an announcement such as, "Thank you for your interest in our special sale items! Please take a look."

[1205] Output: The generated announcement.

[1206] Step 8:

[1207] The server notifies the staff of the recognized emotion information.

[1208] Input: Recognized emotion information.

[1209] Specific operation: Generate a notification message to staff based on the emotion information. For example, generate a message saying, "Customer B is interested in a special sale item. Please provide additional guidance or support." and send it to staff.

[1210] Output: Staff notification message.

[1211] (Application example 2)

[1212] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1213] Conventional store management systems lacked the ability to recognize customer emotions and respond appropriately. This made it difficult for employees to properly understand the emotions of individual customers and provide services that match those emotions. Furthermore, while efficient operations are required in terms of inventory management and the provision of special sale information, conventional systems were unable to fully meet these needs.

[1214] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative artificial intelligence technology, means for providing the generated announcements to customers in the store by voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for recognizing user emotions, means for generating and providing announcements based on the recognized emotions, and means for notifying employees of the recognized emotion information. This enables responses and announcements tailored to individual customers' emotions, efficient inventory management, and the provision of special sale information.

[1215] "Price update information" is the latest price change information for products and services, and is data used in store operations.

[1216] "Generative AI technology" is a technology that allows computers to learn independently and imitate creative tasks performed by humans, and is used for text generation and data analysis.

[1217] "Means for generating announcements" refers to a function that automatically generates announcements to provide to customers based on price update information and other data.

[1218] "Means for providing the created announcement text to customers in the store by voice" refers to a function for informing customers of the created announcement text as voice through a speaker or other audio output device.

[1219] "Object detection technology" is a technology that uses cameras and sensors to identify specific objects or human movements and acquire them as data.

[1220] "Means for monitoring customer behavior and product inventory" refers to a function that uses object detection technology to check customer movements within the store and product inventory status, and collects and analyzes the data.

[1221] "Means for notifying employees of out-of-stock information" refers to a function that notifies employees when a product is sold out or when inventory is low.

[1222] "Means for recognizing user emotions" refers to technology that analyzes data such as a customer's facial expressions and tone of voice to identify their emotional state.

[1223] "Means for generating and providing announcements based on recognized emotions" refers to a function that automatically creates appropriate guidance messages for individual customers based on the user's emotional data and provides them in voice or text format.

[1224] The "means of notifying employees of recognized emotional information" is a function that conveys the results of analyzing the user's emotions to employees and encourages them to take appropriate action.

[1225] The present invention relates to the implementation of a system for improving operational efficiency and customer experience in a physical store. Specific embodiments for carrying out the present invention are described below.

[1226] System Overview

[1227] The system includes functions to obtain price update information, generate announcements using generative artificial intelligence technology and provide them in-store, monitor customer behavior and product inventory using object detection technology, notify employees of out-of-stock information, recognize customer emotions and generate announcements based on that information, and notify employees of emotional information.

[1228] Hardware and software used

[1229] Hardware:

[1230] Webcam: Capture your customer's facial expressions in real time.

[1231] Head-mounted display: worn by employees to receive information in real time.

[1232] Speaker: An audio output device for providing announcements within the store.

[1233] software:

[1234] OpenCV: Processes image data obtained from the camera and performs object detection.

[1235] DeepFace: A facial recognition library for analyzing customer emotions.

[1236] Google Cloud Text-to-Speech: Converts the generated announcement text into audio.

[1237] Specific operation of the system

[1238] 1. Get price updates:

[1239] The server periodically retrieves the latest price update information from the price update system. This information is the latest price change information for products and services, and is important data for store operations.

[1240] 2. Announcement generation:

[1241] The server uses generative artificial intelligence technology to generate announcements based on price update information. This technology automatically creates announcements that effectively communicate sales and special offers to customers.

[1242] 3. In-store announcements:

[1243] The server sends the generated announcement to the speakers in the store and provides it by voice, so that all customers in the store can receive the latest sales information in real time.

[1244] 4. Object detection surveillance:

[1245] The server uses OpenCV to monitor customer behavior and product inventory in the store. When a customer picks up a specific product, the behavior is detected and the inventory information is automatically updated.

[1246] 5. Out-of-stock notification:

[1247] When inventory is low, the server notifies employees of the shortage, which is displayed on a head-mounted display worn by the employee, allowing them to immediately take action to replenish the stock.

[1248] 6. Customer Emotion Recognition:

[1249] The server uses DeepFace to analyze the video captured by the webcam and recognize the customer's emotions. The recognized emotions are collected and analyzed as data in real time.

[1250] 7. Emotion-based announcement generation:

[1251] The server generates special announcements based on the recognized emotions and delivers them through speakers and displays in the store, enabling guidance optimized for the customer's current emotions.

[1252] 8. Employee Notification of Emotional Information:

[1253] The server then notifies employees of the recognized emotional information. For example, if a customer is confused, the information is conveyed to employees, who can provide support promptly.

[1254] Examples of specific examples and prompts

[1255] Examples:

[1256] If a customer is smiling while looking at products in the sale section, the server generates and provides a voice announcement saying, "Thank you for your interest in our sale. Please choose your favorite product." If the customer looks confused, the server provides a voice announcement saying, "Is there anything I can help you with? Our staff is here to help you."

[1257] Example prompt sentence:

[1258] "Create a program that analyzes the facial expressions of customers in a store and generates announcements based on their emotions. If the emotion is 'happy', announce, 'Thank you for your interest in our sale. Please choose your favorite product.' If the emotion is 'sad', announce, 'Is there anything I can help you with? Our staff is here to help you.'"

[1259] This invention provides an embodiment of a new system that combines emotion recognition and AI to achieve a high level of automation in store operations and improve customer satisfaction.

[1260] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1261] Step 1:

[1262] The server periodically retrieves the latest price update information from the price update system. The input is data from the price update system, and the output is the latest price update information. This information is the latest price change information for products and services, and is data necessary for store operations.

[1263] Step 2:

[1264] Based on the price update information acquired by the server, announcement text is generated using generative artificial intelligence technology. The input is the price update information, and the output is the generated announcement text. The generated announcement text effectively communicates sales and special offers to customers.

[1265] Step 3:

[1266] The server sends the generated announcement to a terminal in the store and provides it audibly through a speaker. The input is the generated announcement, and the output is a voice announcement played through the in-store speaker. This allows customers in the store to receive special sale information in real time.

[1267] Step 4:

[1268] The server uses OpenCV to detect objects in the image data acquired from the webcam, and monitors customer behavior and product inventory in the store. The input is video data from the webcam, and the output is customer behavior data and inventory information. This allows the detection of actions such as customers picking up sale items.

[1269] Step 5:

[1270] The server uses object detection technology to collect inventory information and notifies employees of out-of-stock information. The input is product inventory information, and the output is a shortage notification sent to bone conduction earphones or a head-mounted display. This allows employees to immediately understand shortages and take action.

[1271] Step 6:

[1272] The server uses DeepFace to analyze video data captured by a webcam and recognize customer emotions. The input is video data from the webcam, and the output is the recognized customer emotion data. This data is obtained from the customer's facial expressions and behavior.

[1273] Step 7:

[1274] The server generates special announcements based on the recognized emotion data and provides them through speakers and displays in the store. The input is the recognized customer emotion data, and the output is the announcement provided through speakers and displays in the store. This makes it possible to provide information that matches the customer's current emotion.

[1275] Step 8:

[1276] The server notifies employees of the recognized emotion information and assists them in handling customers. The input is the recognized emotion data, and the output is a notification message sent to employees. This allows employees to take appropriate action based on the customer's emotion.

[1277] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1278] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1279] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1280] [Fourth embodiment]

[1281] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1282] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1283] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1284] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1285] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1286] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1287] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1288] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1289] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1290] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1291] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1292] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1293] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1294] To implement the present invention, an in-store system is constructed as follows.

[1295] System Overview

[1296] To streamline time sales in stores and improve the consumer experience, the system retrieves price updates, generates and delivers announcements, monitors customer behavior and product inventory using object detection, and notifies staff of out-of-stock information.

[1297] System configuration

[1298] 1. Get price updates

[1299] The server periodically retrieves the latest price updates from the price update system.

[1300] Example: The server collects new price information such as "eggs 200 yen → 180 yen" from a supermarket price update database.

[1301] 2. Announcement Generation

[1302] The server generates announcement text based on the obtained price update information using generative artificial intelligence techniques.

[1303] Example: The server generates an announcement such as "Egg sale, only 180 yen today!"

[1304] 3. Providing announcement text

[1305] The server transmits the generated announcement to a terminal that controls a speaker in the store, and provides it to the customer by voice.

[1306] Example: The terminal announces through the in-store speaker, "We're currently having a special sale on eggs. They're 180 yen per pack. Please buy some."

[1307] 4. Object Detection Monitoring

[1308] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[1309] Example: The server uses data from in-store cameras to capture when a customer picks up an egg and updates the inventory.

[1310] 5. Notification of out-of-stock information

[1311] The server checks the inventory status of the detected product and notifies staff if there is a shortage.

[1312] Example: Generate a notification such as "You're low on eggs. Please get more from the warehouse."

[1313] Users (staff) receive information about out-of-stock items in real time through bone conduction earphones and can quickly replenish products.

[1314] Example: A staff member receives a notification via bone conduction earphones saying, "We are out of eggs. Please replenish them," and restocks the stock from the warehouse.

[1315] A natural language description of the process

[1316] Get price updates

[1317] The server periodically retrieves the latest price update information from the price update system, which is price change information for the product and may fluctuate in real time.

[1318] Announcement generation

[1319] The server uses generative artificial intelligence technology to generate announcements based on the acquired price update information. The generated announcements effectively communicate information about special sales and limited-time sales to customers.

[1320] Providing announcement text

[1321] The server sends the generated announcement text to a terminal in the store, which then provides the content to customers by voice through the in-store speaker, allowing customers to receive special sale information in real time.

[1322] Object detection surveillance

[1323] The server uses object detection technology to monitor customer behavior and product inventory in the store. For example, if a customer picks up a sale item, the server detects that behavior and updates inventory information accordingly.

[1324] Notification of out-of-stock information

[1325] If a shortage occurs based on the monitored inventory information, the server notifies the staff of the shortage via bone conduction earphones, allowing the staff to immediately take action to deal with the shortage.

[1326] This system will streamline the management of time sales, improve the quality of service to customers, and ensure smooth store operations.

[1327] The processing flow will be explained below.

[1328] Step 1:

[1329] The server periodically retrieves the latest price updates from the price update system.

[1330] Specific operation: Refer to the price update database to get new price information. This information includes "product name" and "new price".

[1331] Example: Get the data "Eggs 200 yen → 180 yen".

[1332] Step 2:

[1333] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[1334] Specific operation: The acquired price update information is passed as input data to the AI ​​model, and an appropriate announcement text is automatically generated.

[1335] Example: Generate an announcement for price update information: "Eggs on sale for 180 yen today only!"

[1336] Step 3:

[1337] The server sends the generated announcement to the terminal (in-store speaker) and provides it as audio.

[1338] Specific operation: The generated announcement text is converted into an audio file and sent to the in-store speaker terminal.

[1339] Example: "We're currently having a special sale on eggs. They're 180 yen per pack. Please come and buy some." is announced over the store's speakers.

[1340] Step 4:

[1341] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[1342] Specific actions: Data from cameras and sensors installed in the store is analyzed to detect when a customer picks up a specific product.

[1343] Example: Recognizing when a customer picks up a carton of eggs from the shelf and reducing stock levels.

[1344] Step 5:

[1345] The server checks the inventory status of the detected product and checks for out-of-stock information.

[1346] What it does: Looks up the updated inventory database and checks if a particular product is below a certain threshold.

[1347] Example: If there are only 2 packs of eggs left in stock, an out-of-stock alert will be issued.

[1348] Step 6:

[1349] If the server detects out-of-stock information, it generates a message to notify staff.

[1350] Specific operation: Based on the out-of-stock information, a message requesting replenishment is generated for staff.

[1351] Example: Generate the message "Low eggs left. Please get more from the warehouse."

[1352] Step 7:

[1353] The server delivers the generated notification message to the staff member via bone conduction earphones.

[1354] Specific operation: Convert notification messages into audio signals and send them to staff through bone conduction earphones.

[1355] Example: A staff member receives a notification that "We are out of eggs. Please restock."

[1356] Step 8:

[1357] After receiving the notification, the user (staff member) promptly replenishes the stock.

[1358] Specific actions: Go to the warehouse, retrieve out-of-stock items, and replenish them on the sales floor.

[1359] Example: A staff member takes cartons of eggs from the warehouse and restocks them on the sales shelves.

[1360] Through each step, a system is built that improves the efficiency of time sales and the consumer experience.

[1361] Example 1

[1362] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1363] In traditional store operations, managing special sales and time sales required a lot of manual effort, resulting in inefficiency. Because processes such as providing sales information to consumers, managing inventory, and notifying of out-of-stock items were all done manually, it was difficult to update information in real time, which resulted in reduced customer satisfaction and lost sales opportunities.

[1364] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1365] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative AI technology, means for providing the generated announcements to customers in the store via voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for periodically accessing a price update system to acquire price update information, means for generating announcements by sending prompts to a generative AI model based on the acquired price update information, means for receiving video data from in-store cameras and detecting customer behavior using object detection technology, means for transmitting the generated announcements to in-store terminals, and means for using bone conduction earphones worn by staff to notify out-of-stock information. This enables effective management of limited-time sales, real-time product inventory monitoring, and rapid out-of-stock response.

[1366] "Price update information" is data provided when the price of a product in a store is changed, and includes both the old and new price information.

[1367] "Generative AI technology" is an AI technology for text generation and natural language processing, which generates appropriate output text based on specified input data.

[1368] An "announcement" is a sentence created to inform customers of information such as special sales and limited-time sales, and is a text for providing the content of the announcement by voice.

[1369] "Object detection technology" is a technology for recognizing and detecting specific objects from image data, and is used to monitor customer behavior and merchandise movement within a store.

[1370] "Out-of-stock information" is information that is notified when the product inventory in the store falls below a certain amount, and is information that encourages additional purchases or replenishment.

[1371] A "server" is a computer system that provides a specific service over a network. In this system, it is a device that performs functions such as obtaining price updates, generating announcements, and monitoring using object detection.

[1372] A "terminal" is a device that receives announcements sent from the server and provides them to customers in the store by voice, and is usually connected to a speaker system.

[1373] A "prompt" is an instruction given to a generative AI model, containing hints and instructions for generating a specific output.

[1374] A "generative AI model" is an artificial intelligence model that generates appropriate text based on a given prompt sentence, and in the present invention is primarily used to generate announcement sentences.

[1375] Bone conduction earphones are devices that transmit sound through the bones to the inner ear, allowing employees to receive voice notifications without using their hands.

[1376] The present invention relates to a system for streamlining in-store sales and time-limited sales to improve the consumer experience by obtaining price updates, generating and providing announcements, monitoring customer behavior and product inventory using object detection, and notifying staff of out-of-stock information.

[1377] System configuration

[1378] Get price updates

[1379] The server periodically accesses the price update system to retrieve the latest price updates. This process involves using HTTP requests to retrieve data in JSON format from the API. The retrieved data includes information such as the product name, old price, and new price. For example, the server receives price updates such as "eggs from 200 yen to 180 yen" from a supermarket's price update database.

[1380] Announcement generation

[1381] The server sends a prompt message to the generative AI model based on the obtained price update information. This prompt message contains information about which product's price has changed and how. The generative AI model (e.g., GPT-4) generates an announcement message based on this prompt. For example, a prompt message might be created saying, "The price of the product has changed. Eggs: 200 yen → 180 yen. Please generate a sale announcement message based on this information." The generative AI model would then generate an announcement message saying, "Eggs on sale for 180 yen today only!"

[1382] Providing announcement text

[1383] The server sends the generated announcement to a terminal in the store. The terminal receives the announcement and provides it audibly through the store's speakers. This allows customers to receive sale information in real time. For example, the terminal may broadcast an announcement over the store's speakers saying, "We're currently having a sale on eggs. They're 180 yen a pack. Please buy some."

[1384] Object detection surveillance

[1385] The server uses object detection technology to monitor customer behavior and product inventory in the store. YOLO (You Only Look Once) is one example of the object detection technology. This technology is used to analyze video data from cameras in the store, and if a specific action (such as a customer picking up an egg) is detected, inventory information is updated. For example, the server receives video data from cameras in the store, detects the action of a customer picking up an egg, and updates inventory information accordingly.

[1386] Notification of out-of-stock information

[1387] The server monitors inventory information and generates out-of-stock information when a certain number of items falls below a certain level. This information is notified to staff via bone conduction earphones. For example, the server detects that inventory is running low and generates a notification saying, "We have only a few eggs left. Please replenish them from the warehouse." The staff (users) receive the notification via bone conduction earphones saying, "We are out of eggs. Please replenish them," and quickly replenish the items.

[1388] Hardware and software used

[1389] Price update system: A product management database system used by stores

[1390] Generative AI model: GPT-4 (OpenAI)

[1391] Object detection technology: Image recognition technology such as YOLO (You Only Look Once)

[1392] Notification device: Bone conduction earphones

[1393] Examples and prompts

[1394] Below are some examples of prompt sentences.

[1395] "The price of the product has changed. Eggs: 200 yen → 180 yen. Please generate a sale announcement based on this information."

[1396] "Price update information: Eggs 200 yen → 180 yen. Please use this data to create a sale announcement for your store. For example, please use a format like, 'Egg sale, 180 yen today only!'"

[1397] This system will streamline store operations, improve the quality of service to customers, and provide information in real time.

[1398] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1399] Step 1: Get price updates

[1400] The server periodically accesses the price update system to obtain the latest price update information. At this time, an HTTP request is used as input and JSON format data is received from the API endpoint. Specifically, the server sends a request to the price update system and obtains price update information such as "Eggs 200 yen → 180 yen." The data includes the product name, old price, new price, etc.

[1401] Step 2: Generate announcement text

[1402] The server sends a prompt message to the generative AI model based on the acquired price update information. It uses the price update information as input and obtains an announcement message as output. Specifically, it creates a prompt message saying, "The price of the product has changed. Eggs have gone from 200 yen to 180 yen. Please generate a sale announcement message based on this information." and sends it to the generative AI model (e.g., GPT-4). The generative AI model then generates a response message saying, "Eggs on sale for 180 yen today only!"

[1403] Step 3: Provide the announcement

[1404] The server sends the generated announcement to a terminal in the store. The generated announcement is used as input and is provided to the customer as audio output. The terminal receives this announcement and provides it to the customer through the in-store speaker. For example, the announcement may say, "We're currently having a special sale on eggs. One pack is 180 yen. Please buy some."

[1405] Step 4: Object detection surveillance

[1406] The server uses object detection technology to monitor customer behavior and product inventory in the store. It uses video data from in-store cameras as input and obtains customer behavior and inventory information as output. Specifically, it analyzes the video from the in-store cameras and detects customer behavior using object detection technology (e.g., YOLO). For example, it detects the behavior of a customer picking up an egg and updates inventory information accordingly.

[1407] Step 5: Out-of-stock notification

[1408] The server uses the monitored inventory information to notify staff when a stockout occurs. It uses the updated inventory information as input and generates a stockout notification as output. Specifically, when inventory falls below a certain amount (e.g., there are fewer than 10 eggs left), a notification is generated. A notification stating "Only a few eggs left. Please add more from the warehouse" is created and sent to staff via bone conduction earphones. The user (staff member) receives this notification and replenishes the product from the warehouse.

[1409] (Application example 1)

[1410] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1411] With conventional time sale management systems, it was difficult to obtain real-time in-store price update information and provide it to customers efficiently. Furthermore, monitoring of customer behavior and product inventory management were insufficient, and responding quickly when an item was out of stock was also an issue. Furthermore, the lack of a push notification function to provide customers with timely information limited the ability to maximize sales.

[1412] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1413] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative artificial intelligence technology, means for providing the generated announcements to customers in the store by voice and push notification, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for notifying employees of out-of-stock information using bone conduction earphones, and means for generating prompts for generating announcements based on the price update information using generative artificial intelligence technology. This enables efficient and real-time time sale management, improving customer experience and maximizing sales.

[1414] "Price update information" refers to the latest price information for products in stores, which is periodically obtained from a database or price update system.

[1415] "Generative artificial intelligence technology" is a technology that uses an AI model to generate text, images, etc. based on input data.

[1416] An "announcement" is a sentence generated to effectively convey information about special sales or limited-time sales.

[1417] "Voice and push notifications" refers to providing information through in-store speakers and notifications to customers' smart devices.

[1418] "Object detection technology" is a technology that uses cameras and sensors to monitor customer behavior and product inventory in stores in real time.

[1419] "Product inventory" refers to the current number of products held in the store.

[1420] "Out-of-stock information" is notification information that is generated when the stock of a specific product falls below a certain level.

[1421] "Employees" refers to staff working in the store.

[1422] Bone conduction earphones are special earphones that transmit sound through the bones rather than the ears, and are worn by employees.

[1423] A "generative prompt" is input text for generating an announcement using generative artificial intelligence technology.

[1424] The present invention provides a system for increasing the efficiency of limited-time sales in stores and improving consumer experiences. Specific embodiments for realizing this system will be described below.

[1425] Overall system configuration

[1426] The system is designed to efficiently manage time-limited sales and special offers in stores and provide them to customers in real time. Its main components include a server, smartphones, bone conduction earphones, in-store cameras, object detection technology, generative artificial intelligence technology (generative AI model), and a push notification system.

[1427] Hardware and Software Configuration

[1428] 1. Server

[1429] The server plays the important role of obtaining price update information, generating announcements using generative artificial intelligence technology (generative AI model), and sending them to in-store smartphones and voice systems. Specifically, the server was built using Python and Node.js, and Firebase Firestore was used as the database. The generative AI model used was OpenAI GPT-4.

[1430] 2. Smartphone

[1431] Smartphones are devices that provide information to customers and employees. Developed cross-platform using Flutter, they provide information through push notifications and in-app messages.

[1432] 3. Bone conduction earphones

[1433] The bone conduction earphones worn by employees are devices used to notify them of stock shortages in real time.

[1434] 4. In-store cameras and object detection technology

[1435] In-store cameras monitor customer behavior and product inventory using object detection technology (using OpenCV), which allows for real-time tracking of customer pick-up behavior and shelf inventory.

[1436] Main processes and data flow

[1437] 1. Get price updates

[1438] The server periodically retrieves the latest price information from an external price update system, which is then stored in Firebase Firestore.

[1439] 2. Announcement Generation

[1440] The server generates an announcement using a generative AI model (OpenAI GPT-4) based on the acquired price information. An example of a prompt is, "Please generate special sale information for eggs. Today's special sale price is 180 yen."

[1441] 3. Providing announcement text

[1442] The generated announcement text is sent from the server to the smartphone via push notification and in-store speakers. For example, a notification might be sent to the customer's smartphone saying, "We're currently having a special sale on eggs. One pack costs 180 yen."

[1443] 4. Object Detection Monitoring

[1444] The video data captured by the in-store cameras is sent to a server and analyzed in real time using OpenCV. The analysis results are used to understand what products customers have picked up and changes in inventory levels.

[1445] 5. Notification of out-of-stock information

[1446] Using object detection technology, if inventory falls below a certain threshold, the server will notify employees of the shortage via bone conduction earphones, and will also send a notification to their smartphone, such as "There are only a few eggs left. Please add more from the warehouse."

[1447] This enables efficient and real-time time sale management, improving customer experience and maximizing sales.

[1448] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1449] Step 1:

[1450] The server retrieves the latest price information from an external price update system. To do this, it uses an HTTP request to retrieve price update data, parses the data, and saves it in JSON format. The specific input is the response data from the price update API, and the output is the internal data structure of the parsed price information. Example: Retrieving price information such as "Eggs 200 yen → 180 yen."

[1451] Step 2:

[1452] The server generates an announcement using a generative AI model (OpenAI GPT-4) based on the acquired price information. Here, the input is the acquired price information (e.g., the price of eggs has changed to 180 yen), and the output is the announcement generated by the generative AI model. Specifically, the prompt "Please generate egg sale information. Today's sale price is 180 yen" is input into the generative AI model, which generates the announcement "Egg sale, 180 yen for today only!"

[1453] Step 3:

[1454] The server sends the generated announcement to the smartphone application and the in-store speaker. The input is the generated announcement (e.g., "Eggs on sale for just 180 yen today!"), and the output is a push notification and a voice broadcast. Specifically, it uses Firebase Cloud Messaging (FCM) to send a push notification to the customer's smart device and controls the speaker system to broadcast the announcement throughout the store.

[1455] Step 4:

[1456] The terminal uses in-store cameras to monitor customer behavior and product inventory. The input is video data from the cameras, and the output is customer behavior and inventory information analyzed using an object detection algorithm (using OpenCV). Specifically, it analyzes the video captured by the cameras in real time to capture customer behavior when picking up products and changes in inventory levels.

[1457] Step 5:

[1458] The server notifies employees when an item is out of stock based on inventory information obtained using object detection technology. The input is inventory information obtained and analyzed using object detection technology, and the output is an out-of-stock notification message. Specifically, when the inventory level falls below a certain threshold, the generative AI model generates an out-of-stock notification message and sends the notification to bone conduction earphones and smartphones. For example, a message such as "There are only a few eggs left. Please add more from the warehouse" is generated.

[1459] Step 6:

[1460] The user (employee) receives the out-of-stock notification via bone conduction earphones and a smartphone and responds promptly. The input is the out-of-stock notification message from the server, and the output is the replenishment of stock when the user actually takes action. Specifically, an employee receives a notification saying, "Eggs are out of stock. Please replenish them," and takes action to replenish the product from the warehouse.

[1461] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1462] To put the present invention into practice, an emotion engine is combined with an in-store system to recognize the user's emotions and make announcements or notify staff accordingly.

[1463] System Overview

[1464] In order to improve the efficiency of store operations and the consumer experience, this system acquires price update information, generates and provides announcements, monitors using object detection, notifies of out-of-stock information, and also includes an emotion engine that recognizes user emotions.

[1465] System configuration

[1466] 1. Get price updates

[1467] The server periodically retrieves the latest price updates from the price update system.

[1468] Example: A server collects price change information from a supermarket database.

[1469] 2. Announcement Generation

[1470] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[1471] Example: Generate an announcement such as "Eggs on sale for just 180 yen today only!"

[1472] 3. Providing announcement text

[1473] The server transmits the generated announcement text to the terminal and provides it through a speaker in the store.

[1474] Example: Broadcast an announcement saying, "We're currently having a special sale on eggs. One pack costs 180 yen."

[1475] 4. Object Detection Monitoring

[1476] The server uses object detection technology to monitor customer behavior and product inventory within the store.

[1477] Example: Detecting the behavior of a customer picking up a sale item and updating inventory information.

[1478] 5. Notification of out-of-stock information

[1479] The server checks inventory information and notifies staff when an item is out of stock.

[1480] Example: Generate a notification saying "You're low on eggs. Please add more from the warehouse."

[1481] 6. User Emotion Recognition

[1482] The server uses an emotion engine to detect the emotions of customers in the store and bases further processing on that information.

[1483] Example: An emotion engine recognizes emotions such as "interest" or "anxiety" from a customer's facial expressions and behavior.

[1484] 7. Emotion-based announcement generation

[1485] The server generates an announcement sentence based on the recognized emotion and transmits it to the terminal.

[1486] Example: If a customer expresses interest in a sale, add an announcement saying, "Thank you for your interest in our sale."

[1487] 8. Staff Notification of Emotional Information

[1488] The server uses the recognized emotion information to notify staff and assist them in handling customers.

[1489] Example: Sending a message saying, "Customer A has expressed interest. Please provide additional support."

[1490] The user (staff member) receives the emotional information and responds appropriately to the customer.

[1491] Example: Store staff provides additional support to Customer A to facilitate a purchase.

[1492] A natural language description of the process

[1493] Get price updates

[1494] The server periodically retrieves the latest price updates from the price update system, which are price changes for products and are often updated in real time.

[1495] Announcement generation

[1496] The server uses generative artificial intelligence technology to generate announcements based on the price update information. The generated announcements effectively communicate information about special sales and limited-time sales to customers.

[1497] Providing announcement text

[1498] The server sends the generated announcement text to a terminal in the store, which then provides the content to customers by voice through the in-store speaker, allowing customers to receive special sale information in real time.

[1499] Object detection surveillance

[1500] The server uses object detection technology to monitor customer behavior and product inventory in the store. For example, if a customer picks up a sale item, the server detects that behavior and updates inventory information accordingly.

[1501] Notification of out-of-stock information

[1502] If a shortage occurs based on the monitored inventory information, the server notifies the staff of the shortage via bone conduction earphones, allowing the staff to immediately take action to deal with the shortage.

[1503] User Emotion Recognition

[1504] The server uses an emotion engine to detect the user's emotions from their facial expressions and behavior, enabling more detailed customer service.

[1505] Emotion-based announcement generation

[1506] The server generates special announcements based on the recognized emotions, allowing the information provided to customers to match their current emotions.

[1507] Staff notification of emotional information

[1508] The server notifies the staff of the user's emotional information and assists them in responding to the customer. Based on the information received, the staff responds quickly and appropriately, improving customer satisfaction.

[1509] In this way, through each step, a system is created that streamlines the management of time sales and improves the consumer experience.

[1510] The processing flow will be explained below.

[1511] Step 1:

[1512] The server periodically retrieves the latest price updates from the price update system.

[1513] Specific operation: Refer to the price update database to get new price information. This information includes "product name" and "new price".

[1514] Example: Get the data "Eggs 200 yen → 180 yen".

[1515] Step 2:

[1516] The server uses generative artificial intelligence techniques to generate announcements based on the obtained price update information.

[1517] Specific operation: The acquired price update information is passed as input data to the AI ​​model, and an appropriate announcement text is automatically generated.

[1518] Example: Generate an announcement saying "Eggs on sale for just 180 yen today only!"

[1519] Step 3:

[1520] The server sends the generated announcement to the terminal (in-store speaker) and provides it as audio.

[1521] Specific operation: The generated announcement text is converted into an audio file and sent to the in-store speaker terminal.

[1522] Example: "We're currently having a special sale on eggs. They're 180 yen per pack. Please come and buy some." is announced over the store's speakers.

[1523] Step 4:

[1524] The server uses object detection technology to monitor customer behavior and product inventory in the store in real time.

[1525] Specific actions: Data from cameras and sensors installed in the store is analyzed to detect when a customer picks up a specific product.

[1526] Example: Recognizing when a customer picks up a carton of eggs from the shelf and reducing stock levels.

[1527] Step 5:

[1528] The server checks the inventory status of the detected product and checks for out-of-stock information.

[1529] What it does: Looks up the updated inventory database and checks if a particular product is below a certain threshold.

[1530] Example: If there are only 2 packs of eggs left in stock, an out-of-stock alert will be issued.

[1531] Step 6:

[1532] If the server detects out-of-stock information, it generates a message to notify staff.

[1533] Specific operation: Based on the out-of-stock information, a message requesting replenishment is generated for staff.

[1534] Example: Generate the message "Low eggs left. Please get more from the warehouse."

[1535] Step 7:

[1536] The server delivers the generated notification message to the staff member via bone conduction earphones.

[1537] Specific operation: Convert notification messages into audio signals and send them to staff through bone conduction earphones.

[1538] Example: A staff member receives a notification that "We are out of eggs. Please restock."

[1539] Step 8:

[1540] The server uses an emotion engine to recognize the user's emotions.

[1541] Specific operation: The emotion engine analyzes the user's facial expressions and behavioral patterns based on data from cameras and sensors installed in the store.

[1542] Example: Detecting emotions such as "interest" or "anxiety" from a customer's facial expression.

[1543] Step 9:

[1544] The server generates an announcement sentence based on the recognized emotion and transmits it to the terminal.

[1545] Specific operation: Based on the detected emotion data, a new announcement is generated and sent to the in-store speaker terminal.

[1546] Example: If a customer is interested in a sale item, add the announcement, "Thank you for your interest in our sale item."

[1547] Step 10:

[1548] The server notifies the staff of the recognized emotion information to assist them in dealing with customers.

[1549] Specific operation: Based on the emotional information detected by the emotion engine, a message is generated for the staff and sent via bone conduction earphones.

[1550] Example: Send a message saying, "Customer A has expressed interest. Please provide additional support."

[1551] Step 11:

[1552] After receiving the notification, the user (staff member) will promptly respond to the customer.

[1553] What happens: Staff receive the notification and provide additional support to the customer, increasing satisfaction.

[1554] Example: A staff member explains the product and makes suggestions to Customer A.

[1555] In this way, a system is realized that streamlines the management of time sales through a multi-stage processing flow and improves the customer experience.

[1556] Example 2

[1557] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1558] While conventional store management systems have basic functions such as obtaining updated price information and managing inventory, they lack the ability to recognize customer emotions and provide personalized responses based on those emotions. This results in a lack of concrete measures to improve customer satisfaction, making efficient store management difficult. Furthermore, even when receiving timely notifications about out-of-stock items or special sales, staff responses are delayed, resulting in reduced productivity.

[1559] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1560] In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using artificial intelligence generative technology, means for providing the generated announcements to customers in the store by voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for detecting emotions of customers in the store using an emotion recognition engine, means for generating announcements based on the detected emotions, and means for notifying employees of the detected emotion information. This enables individual responses based on customer emotions, contributing to improved customer satisfaction as well as enabling more efficient store operations and faster staff responses.

[1561] "Price updates" refers to the latest price information for products sold in stores, which is often updated in real time.

[1562] "Generative AI technology" refers to artificial intelligence technology for generating natural language text and speech based on acquired data, and specifically includes generative AI models.

[1563] An "announcement" is a voice message provided to customers in a store, and effectively conveys price information, special sale information, and the like.

[1564] "Object detection technology" is a technology that uses cameras and sensors to monitor and recognize customer behavior and the status of products in a store in real time.

[1565] "Product inventory" refers to the quantity and condition of products sold in a store, and is quantitatively grasped by an inventory management system.

[1566] "Out-of-stock information" is information that notifies you that a particular product is out of stock or in short supply.

[1567] An "emotion recognition engine" is a technology that analyzes emotions from a customer's facial expressions and behavior, and recognizes them as states such as "interest," "satisfaction," or "anxiety."

[1568] "Employees" refers to staff in the store who are responsible for tasks such as product management and customer service.

[1569] "Notification" refers to the transmission of information by the system, including messages to encourage employees to take action.

[1570] The "in-store speaker" and "terminal" refer to audio equipment and its control device for playing the generated announcement text as audio.

[1571] The present invention is a system that aims to improve the efficiency of store operations and customer satisfaction. This system operates mainly around three components: a server, a terminal, and a user. Specific embodiments of this system are described in detail below.

[1572] Get price updates

[1573] The server periodically communicates with the price update system to obtain new price information. This communication is performed via an API, and for example, at 9:00 AM and 3:00 PM, the server downloads the day's special prices and new prices from the database. This data includes the product ID, product name, new price, etc.

[1574] Announcement generation

[1575] The server generates an announcement using generative artificial intelligence technology based on the acquired price update information. A generative AI model (e.g., GPT-3) is used to generate this announcement. For example, the price update information is input as a prompt, and the generated text is "Today's sale information! Fresh eggs are 180 yen." An example of a prompt from a generative AI model is as follows:

[1576] Prompt: "Generate an announcement based on the sale information. Example: Eggs, new price 180 yen."

[1577] Generated announcement: "Today's special offer! Fresh eggs for 180 yen."

[1578] Providing announcement text

[1579] The generated announcement is sent from the server to the terminal. The terminal is connected to the store's speaker system, and the received announcement is provided to customers as audio. For example, at 11:00 a.m. and 4:00 p.m., an announcement such as "Today's special: Fresh eggs for 180 yen. Don't miss out!" is played in the store.

[1580] Object detection surveillance

[1581] The server uses object detection technology to monitor customer behavior and product inventory through cameras and sensors installed in the store. For example, a camera monitors the sale shelves and detects the moment a customer picks up an egg. At that time, inventory information is automatically updated based on the image data.

[1582] Notification of out-of-stock information

[1583] Based on inventory information monitored by object detection technology, the server immediately notifies users (store employees) when an item is about to run out. This notification is sent to bone conduction earphones worn by employees, and for example, a message such as "Eggs are running low. Please order more from the warehouse."

[1584] User Emotion Recognition

[1585] The server uses an emotion recognition engine to collect data on customers' facial expressions and behavior from cameras and sensors installed in the store, and analyzes and recognizes their emotions. For example, a customer's smile when looking at a sale item can be recognized as "interest." This emotion data is updated in real time.

[1586] Emotion-based announcement generation

[1587] The server generates a specific announcement based on the recognized emotion information. For example, if a customer is interested in a special sale item, the server creates an announcement saying, "Thank you for your interest in our special sale items! Please take a look."

[1588] Staff notification of emotional information

[1589] The server notifies the user (store employee) of the recognized emotion information. The employee receives this notification and responds to the customer based on the content, for example, "Customer B is interested in a special sale item. Please provide additional information or support." The staff member approaches the customer and kindly provides product details and other special sale information.

[1590] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1591] Step 1:

[1592] The server retrieves price updates.

[1593] Input: Schedule information, authentication information to make API calls.

[1594] Specific operation: Sends an API request at 9:00 AM and 3:00 PM to retrieve the latest price information (product ID, product name, new price, etc.) from the store's database.

[1595] Output: A list of retrieved price updates.

[1596] Step 2:

[1597] The server generates an announcement based on the acquired price update information.

[1598] Input: Retrieved price update information.

[1599] Specific operation: Create a prompt to input price update information into the generated AI model (e.g., GPT-3) and input it into the AI ​​model. For example, enter the product ID and new price information as a prompt: "Eggs, new price 180 yen."

[1600] Output: The generated announcement (e.g., "Today's special offer! Fresh eggs for 180 yen.").

[1601] Step 3:

[1602] The server sends the generated announcement text to the terminal and provides it as audio from speakers inside the store.

[1603] Input: The generated announcement text.

[1604] Specific operation: The generated announcement text is sent to the terminal in JSON format, and the terminal sends the received data to a speech synthesis engine, which then plays the announcement over the in-store speakers.

[1605] Output: Announcement audio broadcast in the store.

[1606] Step 4:

[1607] The server uses object detection technology to monitor customer behavior and product inventory.

[1608] Input: Video data from cameras and sensors in the store.

[1609] Specific actions: Real-time video data analysis detects the moment a specific product is picked up, for example, when a customer reaches out in front of an egg shelf.

[1610] Output: Updated inventory information.

[1611] Step 5:

[1612] The server notifies staff of out-of-stock information based on the monitored inventory information.

[1613] Input: Updated inventory information.

[1614] Specific operation: When the inventory level falls below a set threshold, a notification method (e.g., bone conduction earphones) is selected and a notification message is generated. For example, a message such as "Egg inventory has fallen below 10 units. Please replenish." is created and sent to staff.

[1615] Output: Staff notification message.

[1616] Step 6:

[1617] The server uses an emotion recognition engine to recognize the emotion of the customer.

[1618] Input: Video data from cameras and sensors in the store.

[1619] Specific behavior: Video data is analyzed in real time to recognize emotions such as "interest," "satisfaction," and "anxiety" from facial expressions and behavior. For example, if a customer smiles when looking at a sale item, this is recognized as "interest."

[1620] Output: Recognized emotion information.

[1621] Step 7:

[1622] The server generates an announcement based on the recognized emotion.

[1623] Input: Recognized emotion information.

[1624] Specific operation: Emotional information is input into the generative AI model to generate appropriate announcements. For example, if a customer expresses interest, the system will generate an announcement such as, "Thank you for your interest in our special sale items! Please take a look."

[1625] Output: The generated announcement.

[1626] Step 8:

[1627] The server notifies the staff of the recognized emotion information.

[1628] Input: Recognized emotion information.

[1629] Specific operation: Generate a notification message to staff based on the emotion information. For example, generate a message saying, "Customer B is interested in a special sale item. Please provide additional guidance or support." and send it to staff.

[1630] Output: Staff notification message.

[1631] (Application example 2)

[1632] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1633] Conventional store management systems lacked the ability to recognize customer emotions and respond appropriately. This made it difficult for employees to properly understand the emotions of individual customers and provide services that match those emotions. Furthermore, while efficient operations are required in terms of inventory management and the provision of special sale information, conventional systems were unable to fully meet these needs.

[1634] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring price update information, means for generating announcements based on the acquired price update information using generative artificial intelligence technology, means for providing the generated announcements to customers in the store by voice, means for monitoring customer behavior and product inventory in the store using object detection technology, means for notifying employees of out-of-stock information based on the monitored product inventory information, means for recognizing user emotions, means for generating and providing announcements based on the recognized emotions, and means for notifying employees of the recognized emotion information. This enables responses and announcements tailored to individual customers' emotions, efficient inventory management, and the provision of special sale information.

[1635] "Price update information" is the latest price change information for products and services, and is data used in store operations.

[1636] "Generative AI technology" is a technology that allows computers to learn independently and imitate creative tasks performed by humans, and is used for text generation and data analysis.

[1637] "Means for generating announcements" refers to a function that automatically generates announcements to provide to customers based on price update information and other data.

[1638] "Means for providing the created announcement text to customers in the store by voice" refers to a function for informing customers of the created announcement text as voice through a speaker or other audio output device.

[1639] "Object detection technology" is a technology that uses cameras and sensors to identify specific objects or human movements and acquire them as data.

[1640] "Means for monitoring customer behavior and product inventory" refers to a function that uses object detection technology to check customer movements within the store and product inventory status, and collects and analyzes the data.

[1641] "Means for notifying employees of out-of-stock information" refers to a function that notifies employees when a product is sold out or when inventory is low.

[1642] "Means for recognizing user emotions" refers to technology that analyzes data such as a customer's facial expressions and tone of voice to identify their emotional state.

[1643] "Means for generating and providing announcements based on recognized emotions" refers to a function that automatically creates appropriate guidance messages for individual customers based on the user's emotional data and provides them in voice or text format.

[1644] The "means of notifying employees of recognized emotional information" is a function that conveys the results of analyzing the user's emotions to employees and encourages them to take appropriate action.

[1645] The present invention relates to the implementation of a system for improving operational efficiency and customer experience in a physical store. Specific embodiments for carrying out the present invention are described below.

[1646] System Overview

[1647] The system includes functions to obtain price update information, generate announcements using generative artificial intelligence technology and provide them in-store, monitor customer behavior and product inventory using object detection technology, notify employees of out-of-stock information, recognize customer emotions and generate announcements based on that information, and notify employees of emotional information.

[1648] Hardware and software used

[1649] Hardware:

[1650] Webcam: Capture your customer's facial expressions in real time.

[1651] Head-mounted display: worn by employees to receive information in real time.

[1652] Speaker: An audio output device for providing announcements within the store.

[1653] software:

[1654] OpenCV: Processes image data obtained from the camera and performs object detection.

[1655] DeepFace: A facial recognition library for analyzing customer emotions.

[1656] Google Cloud Text-to-Speech: Converts the generated announcement text into audio.

[1657] Specific operation of the system

[1658] 1. Get price updates:

[1659] The server periodically retrieves the latest price update information from the price update system. This information is the latest price change information for products and services, and is important data for store operations.

[1660] 2. Announcement generation:

[1661] The server uses generative artificial intelligence technology to generate announcements based on price update information. This technology automatically creates announcements that effectively communicate sales and special offers to customers.

[1662] 3. In-store announcements:

[1663] The server sends the generated announcement to the speakers in the store and provides it by voice, so that all customers in the store can receive the latest sales information in real time.

[1664] 4. Object detection surveillance:

[1665] The server uses OpenCV to monitor customer behavior and product inventory in the store. When a customer picks up a specific product, the behavior is detected and the inventory information is automatically updated.

[1666] 5. Out-of-stock notification:

[1667] When inventory is low, the server notifies employees of the shortage, which is displayed on a head-mounted display worn by the employee, allowing them to immediately take action to replenish the stock.

[1668] 6. Customer Emotion Recognition:

[1669] The server uses DeepFace to analyze the video captured by the webcam and recognize the customer's emotions. The recognized emotions are collected and analyzed as data in real time.

[1670] 7. Emotion-based announcement generation:

[1671] The server generates special announcements based on the recognized emotions and delivers them through speakers and displays in the store, enabling guidance optimized for the customer's current emotions.

[1672] 8. Employee Notification of Emotional Information:

[1673] The server then notifies employees of the recognized emotional information. For example, if a customer is confused, the information is conveyed to employees, who can provide support promptly.

[1674] Examples of specific examples and prompts

[1675] Examples:

[1676] If a customer is smiling while looking at products in the sale section, the server generates and provides a voice announcement saying, "Thank you for your interest in our sale. Please choose your favorite product." If the customer looks confused, the server provides a voice announcement saying, "Is there anything I can help you with? Our staff is here to help you."

[1677] Example prompt sentence:

[1678] "Create a program that analyzes the facial expressions of customers in a store and generates announcements based on their emotions. If the emotion is 'happy', announce, 'Thank you for your interest in our sale. Please choose your favorite product.' If the emotion is 'sad', announce, 'Is there anything I can help you with? Our staff is here to help you.'"

[1679] This invention provides an embodiment of a new system that combines emotion recognition and AI to achieve a high level of automation in store operations and improve customer satisfaction.

[1680] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1681] Step 1:

[1682] The server periodically retrieves the latest price update information from the price update system. The input is data from the price update system, and the output is the latest price update information. This information is the latest price change information for products and services, and is data necessary for store operations.

[1683] Step 2:

[1684] Based on the price update information acquired by the server, announcement text is generated using generative artificial intelligence technology. The input is the price update information, and the output is the generated announcement text. The generated announcement text effectively communicates sales and special offers to customers.

[1685] Step 3:

[1686] The server sends the generated announcement to a terminal in the store and provides it audibly through a speaker. The input is the generated announcement, and the output is a voice announcement played through the in-store speaker. This allows customers in the store to receive special sale information in real time.

[1687] Step 4:

[1688] The server uses OpenCV to detect objects in the image data acquired from the webcam, and monitors customer behavior and product inventory in the store. The input is video data from the webcam, and the output is customer behavior data and inventory information. This allows the detection of actions such as customers picking up sale items.

[1689] Step 5:

[1690] The server uses object detection technology to collect inventory information and notifies employees of out-of-stock information. The input is product inventory information, and the output is a shortage notification sent to bone conduction earphones or a head-mounted display. This allows employees to immediately understand shortages and take action.

[1691] Step 6:

[1692] The server uses DeepFace to analyze video data captured by a webcam and recognize customer emotions. The input is video data from the webcam, and the output is the recognized customer emotion data. This data is obtained from the customer's facial expressions and behavior.

[1693] Step 7:

[1694] The server generates special announcements based on the recognized emotion data and provides them through speakers and displays in the store. The input is the recognized customer emotion data, and the output is the announcement provided through speakers and displays in the store. This makes it possible to provide information that matches the customer's current emotion.

[1695] Step 8:

[1696] The server notifies employees of the recognized emotion information and assists them in handling customers. The input is the recognized emotion data, and the output is a notification message sent to employees. This allows employees to take appropriate action based on the customer's emotion.

[1697] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1698] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1699] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1700] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1701] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1702] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1703] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1704] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1705] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1706] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1707] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1708] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1709] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1710] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1711] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1712] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1713] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1714] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1715] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1716] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1717] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1718] The following is further disclosed regarding the above embodiment.

[1719] (Claim 1)

[1720] a means for obtaining price updates;

[1721] means for generating announcement text based on the obtained price update information using generative artificial intelligence technology;

[1722] a means for providing the generated announcement text by voice to customers in the store;

[1723] means for monitoring customer activity and product inventory within the store using object detection technology;

[1724] a means for notifying employees of out-of-stock information based on the monitored product inventory information;

[1725] A system including:

[1726] (Claim 2)

[1727] 10. The system of claim 1, further comprising means for periodically broadcasting announcements generated using generative artificial intelligence technology through speakers within the store.

[1728] (Claim 3)

[1729] 10. The system of claim 1, further comprising means for notifying employees of out-of-stock information using bone conduction earphones worn by the employees.

[1730]

[1731] "Example 1"

[1732] (Claim 1)

[1733] a means for obtaining price updates;

[1734] means for generating announcement text based on the obtained price update information using generative artificial intelligence technology;

[1735] a means for providing the generated announcement text by voice to customers in the store;

[1736] means for monitoring customer activity and product inventory within the store using object detection technology;

[1737] a means for notifying employees of out-of-stock information based on the monitored product inventory information;

[1738] a means of periodically accessing a price update system to obtain price updates;

[1739] A means for generating an announcement text by sending a prompt text to a generation AI model based on the acquired price update information;

[1740] A means for receiving video data from a camera in the store and detecting customer behavior using object detection technology;

[1741] means for transmitting the generated announcement text to a terminal in the store;

[1742] A method using bone conduction earphones worn by staff to notify them of out-of-stock information;

[1743] A system including:

[1744] (Claim 2)

[1745] 10. The system of claim 1, further comprising means for providing the generated announcement by voice through a plurality of speakers in the store.

[1746] (Claim 3)

[1747] 10. The system of claim 1, further comprising means for notifying employees of out-of-stock information using bone conduction earphones worn by the employees.

[1748] "Application Example 1"

[1749] (Claim 1)

[1750] a means for obtaining price updates;

[1751] means for generating announcement text based on the obtained price update information using generative artificial intelligence technology;

[1752] a means for providing the generated announcement text to customers in the store by voice and push notification;

[1753] means for monitoring customer activity and product inventory within the store using object detection technology;

[1754] a means for notifying employees of out-of-stock information based on the monitored product inventory information;

[1755] A method for notifying employees of out-of-stock information using bone conduction earphones;

[1756] means for generating a prompt for generating an announcement based on the price update information using generative artificial intelligence techniques;

[1757] A system including:

[1758] (Claim 2)

[1759] The system of claim 1 further comprising means for periodically broadcasting announcements generated using generative artificial intelligence technology through speakers in the store and means for providing the announcements to customers' smart devices via push notifications.

[1760] (Claim 3)

[1761] The system of claim 1 further comprising: means for generating stock-out information to be notified to employees based on data acquired using object detection technology; and means for notifying the generated stock-out information to the employee's smart device.

[1762] "Example 2: Combining Emotion Engines"

[1763] (Claim 1)

[1764] a means for obtaining price updates;

[1765] means for generating announcement text based on the obtained price update information using generative artificial intelligence technology;

[1766] a means for providing the generated announcement text by voice to customers in the store;

[1767] means for monitoring customer activity and product inventory within the store using object detection technology;

[1768] a means for notifying employees of out-of-stock information based on the monitored product inventory information;

[1769] means for detecting emotions of customers in the store using an emotion recognition engine;

[1770] means for generating an announcement sentence based on the detected emotion;

[1771] a means for notifying employees of the detected emotional information;

[1772] A system including:

[1773] (Claim 2)

[1774] The system according to claim 1, wherein announcements generated using generative artificial intelligence technology are periodically broadcast through speakers within the store.

[1775] (Claim 3)

[1776] The system according to claim 1, wherein bone conduction earphones worn by employees are used to notify employees of out-of-stock information.

[1777] "Application example 2 when combining emotion engines"

[1778] (Claim 1)

[1779] a means for obtaining price updates;

[1780] means for generating announcement text based on the obtained price update information using generative artificial intelligence technology;

[1781] a means for providing the generated announcement text by voice to customers in the store;

[1782] means for monitoring customer activity and product inventory within the store using object detection technology;

[1783] a means for notifying employees of out-of-stock information based on the monitored product inventory information;

[1784] means for recognizing a user's emotion;

[1785] means for generating and providing an announcement based on the recognized emotion;

[1786] a means of informing employees of the perceived emotional information;

[1787] A system including:

[1788] (Claim 2)

[1789] 10. The system of claim 1, further comprising means for periodically broadcasting announcements generated using generative artificial intelligence technology through speakers within the store.

[1790] (Claim 3)

[1791] 10. The system of claim 1, further comprising means for notifying employees of out-of-stock information using bone conduction earphones worn by the employees. [Explanation of symbols]

[1792] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for obtaining price updates; means for generating announcement text based on the obtained price update information using generative artificial intelligence technology; a means for providing the generated announcement text by voice to customers in the store; means for monitoring customer activity and product inventory within the store using object detection technology; a means for notifying employees of out-of-stock information based on the monitored product inventory information; A system including:

2. 10. The system of claim 1, further comprising means for periodically broadcasting announcements generated using generative artificial intelligence technology through speakers within the store.

3. 10. The system of claim 1, further comprising means for notifying employees of out-of-stock information using bone conduction earphones worn by the employees.

Images