Product information presentation system

Abstract

To provide a system.SOLUTION: A product information presentation system is provided which includes: means for a server to receive bar-code information and acquire an online price, evaluation, and past transaction history of a product corresponding to a bar-code from a database; means for the server to provide information on the product in natural language; means for the server to use a generative AI model utilizing a natural language processing technique to generate information for supporting purchase decisions; means for the server to recognize a user's emotion; and means for the server to customize the generated information on the basis of the user's emotion.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a product information presentation system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Previously, when comparing products while shopping or on the go, it was necessary to manually search for product information, making it difficult to make efficient price comparisons or make purchasing decisions. [Means for solving the problem]

[0005] The present invention aims to provide a product information presentation system that enables users to easily check online product prices, reviews, and past transaction history while shopping or on the go, thereby supporting purchasing decisions.

[0006] Specifically, when a user scans a product's barcode, the server receives the barcode information and retrieves the online price, reviews, and past transaction history of the product corresponding to the barcode in a database. The server then provides product information in natural language to the user's device. The server also uses generative AI to generate information about the product that will support the user's purchasing decision.

[0007] In this way, the present invention eliminates the need for manual product comparison and provides users with a means to make efficient price comparisons and assist them in making purchasing decisions.

[0008] The "product information presentation system" is a system that allows users to instantly check online prices, reviews, and past transaction history by scanning the barcode of a product in a physical store.

[0009] "Barcode information" refers to information related to the barcode used to identify a product, and includes the numbers and patterns of the barcode.

[0010] "Online prices, reviews, and past transaction history" refers to information such as the product's online selling price, user reviews, and past transaction information.

[0011] "Providing product information in natural language" means providing information such as product prices, ratings, and transaction history to users in natural language (human language).

[0012] "Generative AI" is an AI model that uses natural language processing technology to

[0013] Generates information to support purchasing decisions based on on-line prices, ratings, and past transaction history. [Brief description of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] 4 is a sequence diagram showing a process flow of the data processing system according to the first embodiment. FIG. [Figure 12] 11 is a sequence diagram showing a process flow of the data processing system in application example 1. FIG. [Figure 13] FIG. 11 is a sequence diagram showing the flow of processing of the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 11 is a sequence diagram showing the flow of processing in the data processing system in application example 2 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0018] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] An embodiment for implementing the present invention includes the following elements.

[0036] 1. Terminal: A device that allows users to scan product barcodes in physical stores.

[0037] 2. Server: A system that receives barcode information, obtains the product's online price, rating, and past transaction history, and provides them to the user's terminal.

[0038] 3. Database: A database for storing product information and managing online prices, ratings, and past transaction history for products corresponding to barcodes.

[0039] 4. Natural language processing model: A model that uses natural language processing technologies such as generative AI to provide product information in natural language and generate information to support purchasing decisions.

[0040] As a specific embodiment, the invention is carried out in the following procedure.

[0041] 1. The user scans the product's barcode in a physical store using the device's camera.

[0042] 2. The terminal sends the barcode information to the server.

[0043] 3. The server searches the database for the product that corresponds to the barcode and retrieves online prices, reviews, and past transaction history.

[0044] 4. The server uses a natural language processing model to generate product information in natural language and provide it to the user's device.

[0045] 5. The user makes a purchasing decision based on the information provided.

[0046] In this way, by implementing the present invention, a user can simply scan a product's barcode while shopping in a physical store or on the go, and instantly check the product's online price, reviews, and past transaction history, providing information to support their purchasing decision.

[0047] The process flow will be explained below.

[0048] Step 1: The user scans the product barcode with the device's camera at the physical store.

[0049] -The user scans the product's barcode with the device's camera in a physical store.

[0050] Step 2: The terminal sends the barcode information to the server

[0051] -The terminal transmits the barcode information to the server.

[0052] Step 3: The server receives the barcode information and searches it in the database

[0053] -The server receives the barcode information and searches the database for the product that corresponds to the barcode.

[0054] Step 4: The server retrieves online prices, ratings, and past transaction history.

[0055] -The server retrieves the online price, reviews, and past transaction history of the product corresponding to the barcode from a database.

[0056] Step 5: The server generates information using a natural language processing model

[0057] -The server uses a natural language processing model (such as generative AI) to generate product information in natural language.

[0058] Step 6: The server provides the generated information to the user's device.

[0059] The server provides the generated product information to the user's terminal.

[0060] For example, when a user scans a smartphone barcode in a brick-and-mortar store, the server verifies that the smartphone that corresponds to that barcode has an online price of 100,000 yen and has a high rating in past transaction history. Furthermore, the natural language processing model suggests other smartphones in the same price range, and the server displays information about the suggested smartphones.

[0061] For example, the server may present information to the user's terminal such as, "The online price of this product is 100,000 yen, and there have been many transactions in the past. It has high ratings and is popular." The server may also use a natural language processing model to generate information that supports purchasing decisions regarding the product. For example, if a user is unsure whether to purchase a particular product, the server may present information such as, "The online price of this product is 100,000 yen, and past transaction history indicates that there is high demand for it. In addition, compared to similar products, this product has some unique features. Therefore, why not consider purchasing it?"

[0062] Users can refer to this information when making a decision about purchasing a smartphone. Above is an explanation of the program's processing steps and specific operations.

[0063] Example 1

[0064] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0065] When purchasing products in physical stores, modern consumers want to easily obtain detailed information about the product, such as its price, reviews, and past transaction history. However, with current systems, it is difficult to obtain this information instantly, and consumers lack the information they need to make appropriate purchasing decisions. There is a need to solve this problem and improve consumers' purchasing experience in physical stores.

[0066] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0067] In this invention, the server includes a means for a user to scan a barcode of a product using a terminal, a means for the terminal to transmit the scanned barcode information to the server, a means for the server to search a database for the received barcode information and obtain the online price, evaluation, and past transaction history of the product corresponding to the barcode, a means for the server to use a generative AI model to convert the obtained product information into natural language, a means for the server to generate information supporting a purchase decision using the generative AI model and provide the information to the terminal, and a means for the terminal to display the provided information to the user. This allows consumers to instantly obtain detailed information about products in a physical store and make appropriate purchasing decisions.

[0068] "User" refers to a consumer who operates the system and scans product barcodes to obtain information.

[0069] "Terminal" refers to a device used by a user that has the function of scanning product barcodes and transmitting the barcode information to a server. Examples include smartphones and tablets.

[0070] A "server" refers to a computer system that has the function of receiving barcode information sent from a terminal, querying a database to obtain product information, and further generating information using natural language processing technology.

[0071] A "barcode" is an identifier consisting of a series of lines and numbers attached to a product, and refers to a code used to identify and obtain information about the product.

[0072] "Database" means an electronic storage device for storing and managing product information, including online product prices, ratings, and past transaction history.

[0073] "Online Price" means the selling price for a Product available on the Internet.

[0074] "Ratings" refers to ratings and reviews of products by consumers and experts who have previously purchased the product.

[0075] "Transaction History" refers to the record of past purchases and sales of a particular commodity.

[0076] "Generative AI model" refers to an artificial intelligence model that uses natural language processing technology to convert acquired product information into text that is easy for humans to understand. Specific examples include ChatGPT (registered trademark).

[0077] "Natural language processing technology" is a technology that enables the understanding and generation of human language, and refers to the technology used to analyze and generate text data.

[0078] "Generating information" refers to converting the data acquired using a generative AI model into a natural language text format and providing it to users in a form that is easy for them to understand.

[0079] This invention provides a product information presentation system that allows users to instantly obtain detailed product information when purchasing a product in a brick-and-mortar store. This system is composed of the following elements: a terminal, a server, a database, and a generative AI model.

[0080] Using the terminal

[0081] A user uses a device such as a smartphone to launch a barcode scanning app and scan the barcode of a product. The device then reads the barcode with a camera, encodes the information as a string, and sends it to the server.

[0082] Examples:

[0083] "When a user is in a supermarket, they use their smartphone camera to scan the barcode of an item they are considering purchasing."

[0084] Server Processing

[0085] The server receives the barcode information sent from the terminal and retrieves the corresponding product information by referencing the information in a database, which stores the product's online price, reviews, and past transaction history. The server uses SQL queries to search the database and retrieve the required information.

[0086] Specific software and hardware used:

[0087] Server: Apache (registered trademark), MySQL (registered trademark)

[0088] Using generative AI models

[0089] The server uses a generative AI model (e.g. ChatGPT) to convert the acquired product information into natural language. The generative AI model analyzes information including the product's online price, reviews, and past transaction history, and generates natural language text in a form that is easy for the user to understand.

[0090] Examples:

[0091] "The server searches the database for the barcode '1234567890123' and retrieves the product's price, reviews, and past transaction history."

[0092] Providing and displaying information

[0093] The generated text information is transmitted from the server to the terminal, which displays the received text information on a user interface so that the user can visually confirm the information.

[0094] Examples:

[0095] "The server sends the generated natural language explanation to the terminal, and the terminal displays the explanation."

[0096] Examples of prompt statements

[0097] Examples of prompts that users might input to a generative AI model include:

[0098] "How much does this item cost online? The barcode is 1234567890123."

[0099] "What is the rating for this item? Barcode is 1234567890123."

[0100] This allows users to instantly obtain detailed information about products in physical stores, providing information to support their purchasing decisions.

[0101] The system allows users to easily access detailed product information while shopping in a physical store, allowing them to make more informed purchasing decisions.

[0102] Recheck the hardware and software you will be using

[0103] Smartphones (e.g. iPhone (registered trademark) and ANDROID (registered trademark) devices)

[0104] Barcode Scanning App

[0105] Server (Apache (registered trademark), MySQL (registered trademark))

[0106] Natural language processing models (e.g. ChatGPT)

[0107] In the above manner, the present invention can be carried out.

[0108] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0109] Step 1:

[0110] User Actions

[0111] A user launches a barcode scanning app on their smartphone at a physical store and scans the barcode of a product. The user holds the camera over the product, frames the barcode, and presses the scan button. The input data is the image of the barcode, and the output data is the encoded barcode information.

[0112] Specific behavior:

[0113] "Users use their smartphone camera to scan the barcode of a product they are considering purchasing at a supermarket."

[0114] Step 2:

[0115] Terminal operation

[0116] The terminal encodes the scanned barcode information and sends it to the server as an HTTP request. The input data is the encoded barcode information, and the output data is the HTTP request to the server.

[0117] Specific behavior:

[0118] "The terminal encodes the barcode as a string of characters and sends it over the internet to a server."

[0119] Step 3:

[0120] Server Processing

[0121] The server decodes the received barcode information and searches the database for the corresponding product. The input data is the barcode information, and the output data is detailed information about the retrieved product (price, reviews, past transaction history).

[0122] Specific behavior:

[0123] "The server decodes the barcode number and runs a SQL query based on that number to retrieve product information from a database."

[0124] Step 4:

[0125] Server Processing

[0126] The server inputs the obtained product details into a generative AI model (e.g. ChatGPT) to convert them into natural language. The generative AI model analyzes information including online prices, reviews, past transaction history, etc., and generates natural language text. The input data is the product details, and the output data is the generated natural language text.

[0127] Specific behavior:

[0128] "The server inputs product details into ChatGPT and generates natural language text to support the purchasing decision."

[0129] Step 5:

[0130] Server Operation

[0131] The server transmits the generated natural language text to the user's terminal. The input data is the generated natural language text, and the output data is the text transmission to the user's terminal.

[0132] Specific behavior:

[0133] "The server sends the generated text to the user's device as an HTTP response."

[0134] Step 6:

[0135] Terminal operation

[0136] The terminal displays the received natural language text on a user interface, allowing the user to visually confirm the information. Input data is the natural language text received from the server, and output data is the information displayed on the user interface.

[0137] Specific behavior:

[0138] "The terminal displays the received text on the screen, allowing the user to visually confirm the product details."

[0139] These processing steps allow users to easily obtain product information in physical stores and make appropriate purchasing decisions.

[0140] (Application example 1)

[0141] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0142] When purchasing goods at a brick-and-mortar store, modern consumers find it difficult to instantly check online prices, ratings, and past transaction history to make optimal purchasing decisions. In addition, there is a lack of means to quickly provide such information, and it is difficult to receive advice and offers individually. As a result, consumers are at risk of purchasing at inappropriate prices, and sellers are also faced with the challenge of being unable to implement appropriate marketing measures.

[0143] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0144] In this invention, the server includes means for receiving barcode information and obtaining online prices, ratings, and past transaction history of an item corresponding to the barcode in a database, means for providing information about the item in natural language, means for generating information to support a purchasing decision using a generative AI model that utilizes natural language processing technology, means for generating prompt sentences for the generative AI model, means for providing advice and offers to the user based on the generated prompt sentences, and means for enabling a user to scan an item in a physical store and refer to online prices, ratings, and past transaction history.

[0145] This allows users to check online prices and ratings in real time when purchasing goods in physical stores, and to refer to past transaction history to make optimal purchasing decisions. In addition, they can receive advice and offers from generative AI models, which increases convenience for consumers.

[0146] A "server" is a computer system that provides data and services to other computers and devices over a computer network.

[0147] "Barcode information" refers to information that represents data corresponding to a particular item by reading a series of black and white lines affixed to the item.

[0148] A "database" is a system for efficiently storing, managing, and searching large amounts of digital data.

[0149] "Online price of the Goods" means the price at which the Goods are offered on an internet sales platform.

[0150] A "rating" is a score or review based on a user's impressions or opinions about an item.

[0151] "Transaction history" refers to historical information about past sales of the item.

[0152] "Natural language" is language used daily by humans and is exploited by technologies that enable computers to understand and generate it.

[0153] "Natural language processing technology" refers to technology that enables computers to understand, generate, and analyze human natural language.

[0154] A "generative AI model" is an artificial intelligence model that automatically generates new text or data based on given input data.

[0155] A "prompt statement" is an instruction or input statement given to a generative AI model, which serves as a guideline for obtaining the desired output result.

[0156] A "user" is someone who uses a system or service.

[0157] A "terminal" refers to a hardware device that a user can directly operate, including a smartphone or tablet.

[0158] The embodiment of the present invention is specifically realized by a system including the following procedures.

[0159] First, when a user is considering purchasing an item in a physical store, he or she uses a device such as a smartphone or tablet to scan the barcode of the target item with a camera. The scanned barcode information is sent to the server in real time.

[0160] The server searches the database for the corresponding item based on the received barcode information, which includes the item's online price, reviews, and past transaction history, to obtain detailed information about the item.

[0161] Next, the server uses natural language processing technology to pass the acquired data to a generative AI model. The generative AI model generates specific instructions, or prompts, and generates information to provide to the user based on the prompts. For example, the generated prompts are as follows:

[0162] example:

[0163] Please provide the product information corresponding to barcode "1234567890123".

[0164] Product name: Microwave

[0165] Online Price: 12000 yen

[0166] Rating: 4.5

[0167] Past transaction history: Sold 50 times in the past year. The lowest price was 10,000 yen.

[0168] The generated information is presented to the user via a terminal, allowing the user to instantly check online price comparisons, reviews, and past sales history of the item, providing information to support purchasing decisions.

[0169] In addition, the server generates advice and offers for the user based on prompts sent to the generative AI model, and these are also provided through the terminal, allowing the user to receive advantageous information and additional suggestions when purchasing goods.

[0170] Specific hardware used to realize this system includes terminals such as smartphones and tablets, and server computers. Software used includes a database management system and a generative AI model that utilizes natural language processing technology (such as ChatGPT).

[0171] In this way, the present invention provides a smarter and more efficient way for users to shop in physical stores.

[0172] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0173] Step 1:

[0174] The user scans the barcode of an item they are considering purchasing in a physical store using the camera on their smartphone, tablet or other device.

[0175] Input: Item barcode

[0176] Output: Scanned barcode information (numeric data)

[0177] Specific behavior: Read the barcode using the device's camera application and obtain its information.

[0178] Step 2:

[0179] The terminal transmits the scanned barcode information to the server.

[0180] Input: Scanned barcode information

[0181] Output: A message confirming the request was sent to the server.

[0182] Specific operation: The terminal application sends the barcode information to the server as an HTTP POST request.

[0183] Step 3:

[0184] Based on the barcode information received by the server, the database is searched for corresponding item information.

[0185] Input: Barcode information

[0186] Output: Item information retrieved from the database (price, rating, transaction history, etc.)

[0187] Specific operation: The search algorithm in the server searches the corresponding items in the database and retrieves the item information.

[0188] Step 4:

[0189] The server uses natural language processing technology to generate a prompt sentence to pass the product information it has acquired to the generative AI model.

[0190] Input: Product information (price, rating, transaction history, etc.)

[0191] Output: The generated prompt statement

[0192] Specific operation: The server constructs a text prompt based on the product information and prepares it for input into the generative AI model.

[0193] Step 5:

[0194] The generative AI model generates information to provide to the user based on the prompt sentence.

[0195] Input: prompt statement

[0196] Output: Information for the user in natural language (advice, offers, etc.)

[0197] What it does: A generative AI model parses the prompt and generates appropriate natural language text.

[0198] Step 6:

[0199] The server transmits the generated information to the terminal.

[0200] Input: User-facing information in natural language

[0201] Output: Response message to terminal

[0202] Specific operation: The server sends the generated information to the terminal using an HTTP response.

[0203] Step 7:

[0204] The terminal displays the received information to the user.

[0205] Input: User-request information received from the server

[0206] Output: Displays the price, rating, transaction history, additional advice and offers for the item.

[0207] Specific operation: The terminal application analyzes the received text information and displays it in the user interface.

[0208] Example prompt:

[0209] "Please provide details of the product that corresponds to barcode '1234567890123'. Product name: Microwave oven. Online price: 12,000 yen. Rating: 4.5. Past transaction history: Sold 50 times in the past year. The lowest price was 10,000 yen."

[0210] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0211] An embodiment for implementing the present invention includes the following elements.

[0212] 1. Terminal: A device that allows users to scan product barcodes in physical stores.

[0213] 2. Server: A system that receives barcode information, obtains the product's online price, rating, and past transaction history, and provides them to the user's terminal.

[0214] 3. Database: A database for storing product information and managing online prices, ratings, and past transaction history for products corresponding to barcodes.

[0215] 4. Natural language processing model: A model that uses natural language processing technologies such as generative AI to generate product information in natural language and provide it to the user's device.

[0216] 5. Emotion Engine: An engine for recognizing user emotions.

[0217] As a specific embodiment, the invention is carried out in the following procedure.

[0218] 1. The user scans the barcode of a product in a physical store using the device's camera.

[0219] 2. The terminal sends the barcode information to the server.

[0220] 3. The server searches the database for the product that corresponds to the barcode and retrieves online prices, reviews, and past transaction history.

[0221] 4. The server uses a natural language processing model to generate product information in natural language and provide it to the user's device.

[0222] 5. The server uses the emotion engine to recognize the user's emotions.

[0223] 6. The server appropriately customizes the generated product information based on the user's emotions and provides it to the user.

[0224] In this way, by implementing the present invention, a user can simply scan the barcode of a product while shopping at a brick-and-mortar store or on the go, and instantly check the product's online price, reviews, and past transaction history, and obtain information to support purchasing decisions. Furthermore, by combining it with an emotion engine, it is possible to recognize the user's emotions and customize product information accordingly.

[0225] The process flow will be explained below.

[0226] Step 1: The user scans the product barcode with the device's camera at the physical store.

[0227] -The user scans the product's barcode with the device's camera in a physical store.

[0228] Step 2: The terminal sends the barcode information to the server

[0229] -The terminal transmits the barcode information to the server.

[0230] Step 3: The server receives the barcode information and searches it in the database

[0231] -The server receives the barcode information and searches the database for the product that corresponds to the barcode.

[0232] Step 4: The server retrieves online prices, ratings, and past transaction history.

[0233] -The server retrieves the online price, reviews, and past transaction history of the product corresponding to the barcode from a database.

[0234] Step 5: The server generates information using a natural language processing model

[0235] -The server uses a natural language processing model (such as generative AI) to generate product information in natural language.

[0236] Step 6: The server uses the emotion engine to recognize the user's emotion.

[0237] -The server uses an emotion engine to recognize the user's emotions.

[0238] Step 7: Customize server-generated information based on user emotions

[0239] The server appropriately customizes the generated product information based on the user's emotions.

[0240] Step 8: The server provides customized information to the user's device.

[0241] The server provides customized product information to the user's terminal.

[0242] The above is an explanation of the program processing steps and specific operations.

[0243] Example 2

[0244] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0245] Conventional information provision systems have difficulty in fully stimulating users' purchasing motivation because the information that can be obtained by scanning product barcodes is limited.In addition, it is not possible to provide information that takes into account the user's emotions and preferences, making it impossible to meet individual needs.

[0246] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving barcode information and acquiring the online price, evaluation, and past transaction history of the product corresponding to the barcode in the database, a means for providing product information in natural language, a means for generating information supporting purchase decisions using a generative model that utilizes natural language processing technology, a means for recognizing the user's emotions, and a means for customizing the generated information based on the user's emotions. This allows the user to diversify the information obtained from the scanned barcode and receive information provided that is customized according to emotions.

[0247] A "server" is a central processing unit that receives requests from users and processes data and provides information.

[0248] "Barcode information" is data contained in a barcode for identifying a product.

[0249] A "database" is a data management system that stores information in an organized format and makes it possible to search and retrieve it.

[0250] "Natural language processing technology" is a technology that enables computers to understand and generate human language.

[0251] A "generative model" is an artificial intelligence model that generates new text or content based on input data.

[0252] "Means for recognizing emotions" refers to technology that identifies a user's emotional state by analyzing facial expressions and voice data.

[0253] "Means for customizing information" refers to technology that adjusts the content of generated information based on the user's emotions and preferences, and provides it in accordance with individual needs.

[0254] "Online price" means the selling price of the product published on the Internet.

[0255] A "rating" is an opinion or rating given by a user or expert on the quality or performance of a product.

[0256] "Past transaction history" is data indicating the past buying and selling history and usage history of a product.

[0257] The product information presentation system according to the present invention functions by combining multiple hardware and software elements. To implement the system, a user terminal, a server, a database, a natural language processing model, and an emotion recognition engine are all required.

[0258] First, a user scans the barcode of a product at a physical store using a smartphone or a dedicated barcode scanner. The terminal receives the barcode information and sends it to the server. Specifically, a smartphone with a barcode reader application installed reads the barcode using its built-in camera and sends an HTTP POST request to the server.

[0259] The server searches a database based on the received barcode information to obtain the online price, reviews, and past transaction history of the corresponding product. The database management system used by the server includes MySQL and PostgreSQL (registered trademark). For example, the server issues the SQL query "SELECT FROM products WHERE barcode = '9781234567897'" to obtain the corresponding data.

[0260] Next, the server uses a generative AI model to generate information using natural language processing technology based on the acquired product information. This generative AI model is composed of an advanced generative AI such as ChatGPT, which receives the required prompt as input and generates natural language text. As a specific example, the server generates a prompt such as "This book has high reviews and the latest price is $20. It is also popular based on past transaction history," and passes it to the AI ​​model, generating natural sentences.

[0261] In addition, the server uses an emotion recognition engine to recognize the user's emotions. It analyzes facial images and voice data sent from the user's device to identify the user's emotional state. For example, if the user has a surprised expression, it will be recognized as "surprise." The server then customizes the generated product information based on the user's emotions, changing the expression to reflect the user's emotions, such as "This price is an amazing deal!"

[0262] Finally, the server sends the customized product information to the user's device and displays it. Users can instantly obtain product information in an easy-to-view format by simply scanning the barcode in a physical store. In addition, the customized information based on emotions can support more effective purchasing decisions.

[0263] Specific examples of prompt sentences include the following:

[0264] "This book has great reviews, a current price of $20, and past transaction history shows it's popular."

[0265] In this way, by implementing the present invention, it becomes possible to provide highly convenient information in a brick-and-mortar store or on the go, thereby improving the user's purchasing experience.

[0266] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0267] Step 1:

[0268] A user scans the product's barcode.

[0269] Specific operations: The user launches a barcode reader app on their smartphone and uses the camera to scan the barcode of the product.

[0270] Input: Item barcode.

[0271] Output: Barcode data (e.g. barcode number "9781234567897").

[0272] Step 2:

[0273] The terminal transmits the barcode information to the server.

[0274] Specific behavior: The barcode reader app reads the barcode and sends the barcode number to the server in an HTTP POST request.

[0275] Input: Barcode data.

[0276] Output: HTTP request containing the barcode data.

[0277] Step 3:

[0278] The server searches the database.

[0279] Specific operation: The server receives the barcode number and issues an SQL query to the database management system (e.g. MySQL, PostgreSQL). It executes the query "SELECT FROM products WHERE barcode = '9781234567897'".

[0280] Input: Barcode data.

[0281] Output: SQL query result (product information).

[0282] Step 4:

[0283] The server obtains the product information.

[0284] Specific operation: The server stores the product information (price, rating, past transaction history) obtained from the database in memory.

[0285] Input: Product information resulting from a SQL query.

[0286] Output: Product information held in memory (e.g. price "$20", rating "Highly Rated", transaction history "Popular").

[0287] Step 5:

[0288] The server generates the description using a natural language processing model.

[0289] Specific operation: The server passes the product information as a prompt to a generative AI model (e.g. ChatGPT) and generates natural language text, such as "This book has high reviews, the latest price is $20, and it is also popular based on past transaction history."

[0290] Input: Product information.

[0291] Output: The generated description (natural language text).

[0292] Step 6:

[0293] The server recognizes the user's emotions using an emotion engine.

[0294] Specific operation: Facial expression images and voice data sent from the user's device are passed to an emotion recognition engine on the server, which analyzes the user's emotional state.

[0295] Input: facial expression images and audio data.

[0296] Output: The perceived emotion (e.g. "surprise").

[0297] Step 7:

[0298] The server customizes the generated product information.

[0299] What happens: The server customizes the generated description based on the recognized emotional state, such as "This price is an amazing deal!"

[0300] Input: Generated description, recognized sentiment.

[0301] Output: A customized description.

[0302] Step 8:

[0303] The server transmits the customized product information to the user's terminal.

[0304] Specific operation: The server sends the customized explanation to the user's terminal as an HTTP response, and the terminal displays it.

[0305] Input: Customized description.

[0306] Output: Product information displayed on the user's device.

[0307] Through these steps, users can simply scan the barcode of a product to instantly check online prices, ratings, past transaction history, and get personalized information based on their sentiment.

[0308] (Application example 2)

[0309] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0310] When shopping in a brick-and-mortar store, it is difficult to instantly grasp the price, reviews, and past transaction history of a product. Also, when referring to information on the web, it takes a lot of time and effort to find information that is appropriate for oneself. In particular, when a user is in an emotional state, their judgment is easily influenced, which can lead to inefficient purchasing behavior.

[0311] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for receiving barcode information and acquiring the online price, evaluation, and past transaction history of the product corresponding to the barcode in the database, a means for providing the product information in natural language, a means for generating information supporting a purchase decision using an AI model using natural language processing technology, and a means for recognizing the user's emotion using emotion recognition technology and customizing the generated information based on the recognition result. This enables a user to obtain customized information corresponding to their individual emotional state in real time by simply scanning a product in a physical store.

[0312] "Barcode information" refers to data of an identification code attached to a product, and is information that can identify the product by scanning it.

[0313] "Online price" means the selling price of a Product offered on the Internet.

[0314] "Rating" refers to reviews and scores of products by users and experts, and is information that reflects opinions regarding the quality and usability of the product.

[0315] "Past transaction history" refers to information regarding past purchases and sales of the product.

[0316] A "natural language" is a language that is used daily by humans and that can be analyzed and generated by machines.

[0317] "Natural language processing technology" is a technology that enables computers to understand, interpret, and manipulate text data.

[0318] An "AI model" is a mathematical model that utilizes artificial intelligence algorithms to perform specific tasks.

[0319] "Emotion recognition technology" is a technology that determines a user's emotional state by analyzing their facial expressions and tone of voice.

[0320] "Customize" means adapting or modifying content or information to fit a particular need or situation.

[0321] A "server" is a computer system that receives and processes requests from clients over a network and provides information.

[0322] "Terminal" refers to a device that is directly operated by a user, and includes, for example, a smartphone or smart glasses.

[0323] The system for implementing the present invention includes a server, a terminal, and a database. The purpose is to enable users to scan products while shopping in a brick-and-mortar store and instantly provide online information and customized advice. The specific configuration and operation are shown below.

[0324] System Configuration

[0325] 1. Server:

[0326] How to receive barcode information:

[0327] The server receives the barcode information sent by the terminal. This is done using an HTTP request.

[0328] Ways to retrieve information from the database:

[0329] The server searches a database based on the received barcode information to obtain the online price, reviews, and past transaction history of the corresponding product.

[0330] Means of generating information in natural language:

[0331] The server uses natural language processing technology (e.g., the ChatGPT model from the Transformers library) to generate the acquired product information in natural language.

[0332] Ways to use emotion recognition technology to personalize communications:

[0333] Use an emotion recognition engine (e.g., the EmotionRecognition library) to recognize the user's emotions and customize the generated information based on the results.

[0334] 2. Terminal:

[0335] How users can scan items:

[0336] Using the camera on the device (a smartphone or smart glasses), the user scans the barcode of the product.

[0337] Methods for sending information to the server:

[0338] The scanned barcode information is sent from the terminal to the server.

[0339] To view the information obtained:

[0340] The product information and customization information generated in natural language and obtained from the server are displayed on the user's terminal.

[0341] Hardware and Software

[0342] Hardware: Smartphones, smart glasses, servers

[0343] Software: OpenCV (barcode scanning), Requests (API communication), Transformers (natural language generation), EmotionRecognition (emotion recognition)

[0344] Examples

[0345] Assume the user is in a brick-and-mortar store and scans the barcode on their laptop. The server receives the barcode information and inputs the following prompt sentence into the natural language generation model:

[0346] Example prompt:

[0347] "Product description: This amazing laptop is packed with the latest features, priced at $798 and has a user rating of 4.5."

[0348] A natural language generation model generates a detailed product description based on the prompt. An emotion recognition engine then analyzes the user's facial expressions and tone of voice to determine their emotions. For example, if the system detects that the user is anxious, the product description will include additional information to reassure the user.

[0349] This allows users to simply scan a product in a physical store and obtain real-time information on not only the price and reviews but also customized information based on their own emotions, making purchasing decisions easier.

[0350] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0351] Step 1:

[0352] The user scans the product's barcode using the device's camera.

[0353] Input: Product barcode captured by the device camera

[0354] Output: Barcode data

[0355] Specific operation: Activate the device's camera and use the OpenCV library to read the barcode and extract the barcode data. Once the barcode data is obtained, proceed to the next step.

[0356] Step 2:

[0357] The terminal transmits the acquired barcode data to the server.

[0358] Input: Barcode data

[0359] Output: HTTP request to the server

[0360] Specific operation: The terminal sends the acquired barcode data to the server as an HTTP request. The request includes the barcode data.

[0361] Step 3:

[0362] The server receives the barcode information and searches a database to obtain the corresponding product information.

[0363] Input: Barcode data

[0364] Output: Product information (price, rating, past transaction history)

[0365] Specific operation: The server searches the database based on the received barcode data. If the appropriate product information is found, it acquires the information and proceeds to the next step.

[0366] Step 4:

[0367] Based on the product information acquired by the server, a product description is generated in natural language using a generative AI model.

[0368] Input: Product information

[0369] Output: A product description generated in natural language

[0370] Specific operation: The server inputs product information as a prompt sentence into a generative AI model (e.g., Transformers' ChatGPT model). The model generates a product description in natural language based on the information and outputs the result.

[0371] Step 5:

[0372] In order for the server to recognize the user's emotions, it uses emotion recognition technology to analyze facial and voice data.

[0373] Input: User facial and voice data

[0374] Output: The user's emotional state.

[0375] How it works: The user provides facial expressions and voice using a dedicated camera or microphone. The server analyzes this data using the EmotionRecognition library and recognizes the user's emotional state.

[0376] Step 6:

[0377] The server generates customized product information based on the generated product description and the user's emotional state.

[0378] Input: Natural language generated product description, user's emotional state

[0379] Output:Customized product information

[0380] How it works: The server adds or adjusts the generated product description depending on the user's recognized emotional state. For example, if the user is anxious, the server adds information that emphasizes the reliability and guarantees of the product.

[0381] Step 7:

[0382] The device displays the customized product information.

[0383] Input:Customized product information

[0384] Output: Product information displayed to the user

[0385] Specific operation: The user's device (smartphone or smart glasses) receives the customized product information sent from the server and displays it on the screen. This allows the user to obtain information that suits their individual needs and emotional state.

[0386] These steps allow users to obtain real-time relevant information about the products they scan in physical stores and receive customized purchasing support information based on their emotions.

[0387] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0388] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0389] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0390] [Second embodiment]

[0391] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0392] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0393] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0394] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0395] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0396] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0397] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0398] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0399] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0400] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0401] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0402] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0403] An embodiment for implementing the present invention includes the following elements.

[0404] 1. Terminal: A device that allows users to scan product barcodes in physical stores.

[0405] 2. Server: A system that receives barcode information, obtains the product's online price, rating, and past transaction history, and provides them to the user's terminal.

[0406] 3. Database: A database for storing product information and managing online prices, ratings, and past transaction history for products corresponding to barcodes.

[0407] 4. Natural language processing model: A model that uses natural language processing technologies such as generative AI to provide product information in natural language and generate information to support purchasing decisions.

[0408] As a specific embodiment, the invention is carried out in the following procedure.

[0409] 1. The user scans the product's barcode in a physical store using the device's camera.

[0410] 2. The terminal sends the barcode information to the server.

[0411] 3. The server searches the database for the product that corresponds to the barcode and retrieves online prices, reviews, and past transaction history.

[0412] 4. The server uses a natural language processing model to generate product information in natural language and provide it to the user's device.

[0413] 5. The user makes a purchasing decision based on the information provided.

[0414] In this way, by implementing the present invention, a user can simply scan a product's barcode while shopping in a physical store or on the go, and instantly check the product's online price, reviews, and past transaction history, providing information to support their purchasing decision.

[0415] The process flow will be explained below.

[0416] Step 1: The user scans the product barcode with the device's camera at the physical store.

[0417] -The user scans the product's barcode with the device's camera in a physical store.

[0418] Step 2: The terminal sends the barcode information to the server

[0419] -The terminal transmits the barcode information to the server.

[0420] Step 3: The server receives the barcode information and searches it in the database

[0421] -The server receives the barcode information and searches the database for the product that corresponds to the barcode.

[0422] Step 4: The server retrieves online prices, ratings, and past transaction history.

[0423] -The server retrieves the online price, reviews, and past transaction history of the product corresponding to the barcode from a database.

[0424] Step 5: The server generates information using a natural language processing model

[0425] -The server uses a natural language processing model (such as generative AI) to generate product information in natural language.

[0426] Step 6: The server provides the generated information to the user's device.

[0427] The server provides the generated product information to the user's terminal.

[0428] The above is an explanation of the program processing steps and specific operations.

[0429] Example 1

[0430] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0431] When purchasing products in physical stores, modern consumers want to easily obtain detailed information about the product, such as its price, reviews, and past transaction history. However, with current systems, it is difficult to obtain this information instantly, and consumers lack the information they need to make appropriate purchasing decisions. There is a need to solve this problem and improve consumers' purchasing experience in physical stores.

[0432] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0433] In this invention, the server includes a means for a user to scan a barcode of a product using a terminal, a means for the terminal to transmit the scanned barcode information to the server, a means for the server to search a database for the received barcode information and obtain the online price, evaluation, and past transaction history of the product corresponding to the barcode, a means for the server to use a generative AI model to convert the obtained product information into natural language, a means for the server to generate information supporting a purchase decision using the generative AI model and provide the information to the terminal, and a means for the terminal to display the provided information to the user. This allows consumers to instantly obtain detailed information about products in a physical store and make appropriate purchasing decisions.

[0434] "User" refers to a consumer who operates the system and scans product barcodes to obtain information.

[0435] "Terminal" refers to a device used by a user that has the function of scanning product barcodes and transmitting the barcode information to a server. Examples include smartphones and tablets.

[0436] A "server" refers to a computer system that has the function of receiving barcode information sent from a terminal, querying a database to obtain product information, and further generating information using natural language processing technology.

[0437] A "barcode" is an identifier consisting of a series of lines and numbers attached to a product, and refers to a code used to identify and obtain information about the product.

[0438] "Database" means an electronic storage device for storing and managing product information, including online product prices, ratings, and past transaction history.

[0439] "Online Price" means the selling price for a Product available on the Internet.

[0440] "Ratings" refers to ratings and reviews of products by consumers and experts who have previously purchased the product.

[0441] "Transaction History" refers to the record of past purchases and sales of a particular commodity.

[0442] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to convert acquired product information into text that is easy for humans to understand. Specific examples include ChatGPT.

[0443] "Natural language processing technology" is a technology that enables the understanding and generation of human language, and refers to the technology used to analyze and generate text data.

[0444] "Generating information" refers to converting the data acquired using a generative AI model into a natural language text format and providing it to users in a form that is easy for them to understand.

[0445] This invention provides a product information presentation system that allows users to instantly obtain detailed product information when purchasing a product in a brick-and-mortar store. This system is composed of the following elements: a terminal, a server, a database, and a generative AI model.

[0446] Using the terminal

[0447] A user uses a device such as a smartphone to launch a barcode scanning app and scan the barcode of a product. The device then reads the barcode with a camera, encodes the information as a string, and sends it to the server.

[0448] Examples:

[0449] "When a user is in a supermarket, they use their smartphone camera to scan the barcode of an item they are considering purchasing."

[0450] Server Processing

[0451] The server receives the barcode information sent from the terminal and retrieves the corresponding product information by referencing the information in a database, which stores the product's online price, reviews, and past transaction history. The server uses SQL queries to search the database and retrieve the required information.

[0452] Specific software and hardware used:

[0453] Server: Apache, MySQL

[0454] Using generative AI models

[0455] The server uses a generative AI model (e.g. ChatGPT) to convert the acquired product information into natural language. The generative AI model analyzes information including the product's online price, reviews, and past transaction history, and generates natural language text in a form that is easy for the user to understand.

[0456] Examples:

[0457] "The server searches the database for the barcode '1234567890123' and retrieves the product's price, reviews, and past transaction history."

[0458] Providing and displaying information

[0459] The generated text information is transmitted from the server to the terminal, which displays the received text information on a user interface so that the user can visually confirm the information.

[0460] Examples:

[0461] "The server sends the generated natural language explanation to the terminal, and the terminal displays the explanation."

[0462] Examples of prompt statements

[0463] Examples of prompts that users might input to a generative AI model include:

[0464] "How much does this item cost online? The barcode is 1234567890123."

[0465] "What is the rating for this item? Barcode is 1234567890123."

[0466] This allows users to instantly obtain detailed information about products in physical stores, providing information to support their purchasing decisions.

[0467] The system allows users to easily access detailed product information while shopping in a physical store, allowing them to make more informed purchasing decisions.

[0468] Recheck the hardware and software you will be using

[0469] Smartphone (e.g. iPhone or Android device)

[0470] Barcode Scanning App

[0471] Server (Apache, MySQL)

[0472] Natural language processing models (e.g. ChatGPT)

[0473] In the above manner, the present invention can be carried out.

[0474] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0475] Step 1:

[0476] User Actions

[0477] A user launches a barcode scanning app on their smartphone at a physical store and scans the barcode of a product. The user holds the camera over the product, frames the barcode, and presses the scan button. The input data is the image of the barcode, and the output data is the encoded barcode information.

[0478] Specific behavior:

[0479] "Users use their smartphone camera to scan the barcode of a product they are considering purchasing at a supermarket."

[0480] Step 2:

[0481] Terminal operation

[0482] The terminal encodes the scanned barcode information and sends it to the server as an HTTP request. The input data is the encoded barcode information, and the output data is the HTTP request to the server.

[0483] Specific behavior:

[0484] "The terminal encodes the barcode as a string of characters and sends it over the internet to a server."

[0485] Step 3:

[0486] Server Processing

[0487] The server decodes the received barcode information and searches the database for the corresponding product. The input data is the barcode information, and the output data is detailed information about the retrieved product (price, reviews, past transaction history).

[0488] Specific behavior:

[0489] "The server decodes the barcode number and runs a SQL query based on that number to retrieve product information from a database."

[0490] Step 4:

[0491] Server Processing

[0492] The server inputs the obtained product details into a generative AI model (e.g. ChatGPT) to convert them into natural language. The generative AI model analyzes information including online prices, reviews, past transaction history, etc., and generates natural language text. The input data is the product details, and the output data is the generated natural language text.

[0493] Specific behavior:

[0494] "The server inputs product details into ChatGPT and generates natural language text to support the purchasing decision."

[0495] Step 5:

[0496] Server Operation

[0497] The server transmits the generated natural language text to the user's terminal. The input data is the generated natural language text, and the output data is the text transmission to the user's terminal.

[0498] Specific behavior:

[0499] "The server sends the generated text to the user's device as an HTTP response."

[0500] Step 6:

[0501] Terminal operation

[0502] The terminal displays the received natural language text on a user interface, allowing the user to visually confirm the information. Input data is the natural language text received from the server, and output data is the information displayed on the user interface.

[0503] Specific behavior:

[0504] "The terminal displays the received text on the screen, allowing the user to visually confirm the product details."

[0505] These processing steps allow users to easily obtain product information in physical stores and make appropriate purchasing decisions.

[0506] (Application example 1)

[0507] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0508] When purchasing goods at a brick-and-mortar store, modern consumers find it difficult to instantly check online prices, ratings, and past transaction history to make optimal purchasing decisions. In addition, there is a lack of means to quickly provide such information, and it is difficult to receive advice and offers individually. As a result, consumers are at risk of purchasing at inappropriate prices, and sellers are also faced with the challenge of being unable to implement appropriate marketing measures.

[0509] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0510] In this invention, the server includes means for receiving barcode information and obtaining online prices, ratings, and past transaction history of an item corresponding to the barcode in a database, means for providing information about the item in natural language, means for generating information to support a purchasing decision using a generative AI model that utilizes natural language processing technology, means for generating prompt sentences for the generative AI model, means for providing advice and offers to the user based on the generated prompt sentences, and means for enabling a user to scan an item in a physical store and refer to online prices, ratings, and past transaction history.

[0511] This allows users to check online prices and ratings in real time when purchasing goods in physical stores, and to refer to past transaction history to make optimal purchasing decisions. In addition, they can receive advice and offers from generative AI models, which increases convenience for consumers.

[0512] A "server" is a computer system that provides data and services to other computers and devices over a computer network.

[0513] "Barcode information" refers to information that represents data corresponding to a particular item by reading a series of black and white lines affixed to the item.

[0514] A "database" is a system for efficiently storing, managing, and searching large amounts of digital data.

[0515] "Online price of the Goods" means the price at which the Goods are offered on an internet sales platform.

[0516] A "rating" is a score or review based on a user's impressions or opinions about an item.

[0517] "Transaction history" refers to historical information about past sales of the item.

[0518] "Natural language" is language used daily by humans and is exploited by technologies that enable computers to understand and generate it.

[0519] "Natural language processing technology" refers to technology that enables computers to understand, generate, and analyze human natural language.

[0520] A "generative AI model" is an artificial intelligence model that automatically generates new text or data based on given input data.

[0521] A "prompt statement" is an instruction or input statement given to a generative AI model, which serves as a guideline for obtaining the desired output result.

[0522] A "user" is someone who uses a system or service.

[0523] A "terminal" refers to a hardware device that a user can directly operate, including a smartphone or tablet.

[0524] The embodiment of the present invention is specifically realized by a system including the following procedures.

[0525] First, when a user is considering purchasing an item in a physical store, he or she uses a device such as a smartphone or tablet to scan the barcode of the target item with a camera. The scanned barcode information is sent to the server in real time.

[0526] The server searches the database for the corresponding item based on the received barcode information, which includes the item's online price, reviews, and past transaction history, to obtain detailed information about the item.

[0527] Next, the server uses natural language processing technology to pass the acquired data to a generative AI model. The generative AI model generates specific instructions, or prompts, and generates information to provide to the user based on the prompts. For example, the generated prompts are as follows:

[0528] example:

[0529] Please provide the product information corresponding to barcode "1234567890123".

[0530] Product name: Microwave

[0531] Online Price: 12000 yen

[0532] Rating: 4.5

[0533] Past transaction history: Sold 50 times in the past year. The lowest price was 10,000 yen.

[0534] The generated information is presented to the user via a terminal, allowing the user to instantly check online price comparisons, reviews, and past sales history of the item, providing information to support purchasing decisions.

[0535] In addition, the server generates advice and offers for the user based on prompts sent to the generative AI model, and these are also provided through the terminal, allowing the user to receive advantageous information and additional suggestions when purchasing goods.

[0536] Specific hardware used to realize this system includes terminals such as smartphones and tablets, and server computers. Software used includes a database management system and a generative AI model that utilizes natural language processing technology (such as ChatGPT).

[0537] In this way, the present invention provides a smarter and more efficient way for users to shop in physical stores.

[0538] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0539] Step 1:

[0540] The user scans the barcode of an item they are considering purchasing in a physical store using the camera on their smartphone, tablet or other device.

[0541] Input: Item barcode

[0542] Output: Scanned barcode information (numeric data)

[0543] Specific behavior: Read the barcode using the device's camera application and obtain its information.

[0544] Step 2:

[0545] The terminal transmits the scanned barcode information to the server.

[0546] Input: Scanned barcode information

[0547] Output: A message confirming the request was sent to the server.

[0548] Specific operation: The terminal application sends the barcode information to the server as an HTTP POST request.

[0549] Step 3:

[0550] Based on the barcode information received by the server, the database is searched for corresponding item information.

[0551] Input: Barcode information

[0552] Output: Item information retrieved from the database (price, rating, transaction history, etc.)

[0553] Specific operation: The search algorithm in the server searches the corresponding items in the database and retrieves the item information.

[0554] Step 4:

[0555] The server uses natural language processing technology to generate a prompt sentence to pass the product information it has acquired to the generative AI model.

[0556] Input: Product information (price, rating, transaction history, etc.)

[0557] Output: The generated prompt statement

[0558] Specific operation: The server constructs a text prompt based on the product information and prepares it for input into the generative AI model.

[0559] Step 5:

[0560] The generative AI model generates information to provide to the user based on the prompt sentence.

[0561] Input: prompt statement

[0562] Output: Information for the user in natural language (advice, offers, etc.)

[0563] What it does: A generative AI model parses the prompt and generates appropriate natural language text.

[0564] Step 6:

[0565] The server transmits the generated information to the terminal.

[0566] Input: User-facing information in natural language

[0567] Output: Response message to terminal

[0568] Specific operation: The server sends the generated information to the terminal using an HTTP response.

[0569] Step 7:

[0570] The terminal displays the received information to the user.

[0571] Input: User-request information received from the server

[0572] Output: Displays the price, rating, transaction history, additional advice and offers for the item.

[0573] Specific operation: The terminal application analyzes the received text information and displays it in the user interface.

[0574] Example prompt:

[0575] "Please provide details of the product that corresponds to barcode '1234567890123'. Product name: Microwave oven. Online price: 12,000 yen. Rating: 4.5. Past transaction history: Sold 50 times in the past year. The lowest price was 10,000 yen."

[0576] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0577] An embodiment for implementing the present invention includes the following elements.

[0578] 1. Terminal: A device that allows users to scan product barcodes in physical stores.

[0579] 2. Server: A system that receives barcode information, obtains the product's online price, rating, and past transaction history, and provides them to the user's terminal.

[0580] 3. Database: A database for storing product information and managing online prices, ratings, and past transaction history for products corresponding to barcodes.

[0581] 4. Natural language processing model: A model that uses natural language processing technologies such as generative AI to generate product information in natural language and provide it to the user's device.

[0582] 5. Emotion Engine: An engine for recognizing user emotions.

[0583] As a specific embodiment, the invention is carried out in the following procedure.

[0584] 1. The user scans the barcode of a product in a physical store using the device's camera.

[0585] 2. The terminal sends the barcode information to the server.

[0586] 3. The server searches the database for the product that corresponds to the barcode and retrieves online prices, reviews, and past transaction history.

[0587] 4. The server uses a natural language processing model to generate product information in natural language and provide it to the user's device.

[0588] 5. The server uses the emotion engine to recognize the user's emotions.

[0589] 6. The server appropriately customizes the generated product information based on the user's emotions and provides it to the user.

[0590] In this way, by implementing the present invention, a user can simply scan the barcode of a product while shopping at a brick-and-mortar store or on the go, and instantly check the product's online price, reviews, and past transaction history, and obtain information to support purchasing decisions. Furthermore, by combining it with an emotion engine, it is possible to recognize the user's emotions and customize product information accordingly.

[0591] The process flow will be explained below.

[0592] Step 1: The user scans the product barcode with the device's camera at the physical store.

[0593] -The user scans the product's barcode with the device's camera in a physical store.

[0594] Step 2: The terminal sends the barcode information to the server

[0595] -The terminal transmits the barcode information to the server.

[0596] Step 3: The server receives the barcode information and searches it in the database

[0597] -The server receives the barcode information and searches the database for the product that corresponds to the barcode.

[0598] Step 4: The server retrieves online prices, ratings, and past transaction history.

[0599] -The server retrieves the online price, reviews, and past transaction history of the product corresponding to the barcode from a database.

[0600] Step 5: The server generates information using a natural language processing model

[0601] -The server uses a natural language processing model (such as generative AI) to generate product information in natural language.

[0602] Step 6: The server uses the emotion engine to recognize the user's emotion.

[0603] -The server uses an emotion engine to recognize the user's emotions.

[0604] Step 7: Customize server-generated information based on user emotions

[0605] The server appropriately customizes the generated product information based on the user's emotions.

[0606] Step 8: The server provides customized information to the user's device.

[0607] The server provides customized product information to the user's terminal.

[0608] This concludes the explanation of the program's processing steps and specific operations.

[0609] Example 2

[0610] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0611] Conventional information provision systems have difficulty in fully stimulating users' purchasing motivation because the information that can be obtained by scanning product barcodes is limited.In addition, it is not possible to provide information that takes into account the user's emotions and preferences, making it impossible to meet individual needs.

[0612] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving barcode information and acquiring the online price, evaluation, and past transaction history of the product corresponding to the barcode in the database, a means for providing product information in natural language, a means for generating information supporting purchase decisions using a generative model that utilizes natural language processing technology, a means for recognizing the user's emotions, and a means for customizing the generated information based on the user's emotions. This allows the user to diversify the information obtained from the scanned barcode and receive information provided that is customized according to emotions.

[0613] A "server" is a central processing unit that receives requests from users and processes data and provides information.

[0614] "Barcode information" is data contained in a barcode for identifying a product.

[0615] A "database" is a data management system that stores information in an organized format and makes it possible to search and retrieve it.

[0616] "Natural language processing technology" is a technology that enables computers to understand and generate human language.

[0617] A "generative model" is an artificial intelligence model that generates new text or content based on input data.

[0618] "Means for recognizing emotions" refers to technology that identifies a user's emotional state by analyzing facial expressions and voice data.

[0619] "Means for customizing information" refers to technology that adjusts the content of generated information based on the user's emotions and preferences, and provides it in accordance with individual needs.

[0620] "Online price" means the selling price of the product published on the Internet.

[0621] A "rating" is an opinion or rating given by a user or expert on the quality or performance of a product.

[0622] "Past transaction history" is data indicating the past buying and selling history and usage history of a product.

[0623] The product information presentation system according to the present invention functions by combining multiple hardware and software elements. To implement the system, a user terminal, a server, a database, a natural language processing model, and an emotion recognition engine are all required.

[0624] First, a user scans the barcode of a product at a physical store using a smartphone or a dedicated barcode scanner. The terminal receives the barcode information and sends it to the server. Specifically, a smartphone with a barcode reader application installed reads the barcode using its built-in camera and sends an HTTP POST request to the server.

[0625] The server searches the database based on the received barcode information to obtain the online price, reviews, and past transaction history of the corresponding product. The database management system used by the server includes MySQL and PostgreSQL. For example, the server issues the SQL query "SELECT FROM products WHERE barcode = '9781234567897'" to obtain the corresponding data.

[0626] Next, the server uses a generative AI model to generate information using natural language processing technology based on the acquired product information. This generative AI model is composed of an advanced generative AI such as ChatGPT, which receives the required prompt as input and generates natural language text. As a specific example, the server generates a prompt such as "This book has high reviews and the latest price is $20. It is also popular based on past transaction history," and passes it to the AI ​​model, generating natural sentences.

[0627] In addition, the server uses an emotion recognition engine to recognize the user's emotions. It analyzes facial images and voice data sent from the user's device to identify the user's emotional state. For example, if the user has a surprised expression, it will be recognized as "surprise." The server then customizes the generated product information based on the user's emotions, changing the expression to reflect the user's emotions, such as "This price is an amazing deal!"

[0628] Finally, the server sends the customized product information to the user's device and displays it. Users can instantly obtain product information in an easy-to-view format by simply scanning the barcode in a physical store. In addition, the customized information based on emotions can support more effective purchasing decisions.

[0629] Specific examples of prompt sentences include the following:

[0630] "This book has great reviews, a current price of $20, and past transaction history shows it's popular."

[0631] In this way, by implementing the present invention, it becomes possible to provide highly convenient information in a brick-and-mortar store or on the go, thereby improving the user's purchasing experience.

[0632] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0633] Step 1:

[0634] A user scans the product's barcode.

[0635] Specific operations: The user launches a barcode reader app on their smartphone and uses the camera to scan the barcode of the product.

[0636] Input: Item barcode.

[0637] Output: Barcode data (e.g. barcode number "9781234567897").

[0638] Step 2:

[0639] The terminal transmits the barcode information to the server.

[0640] Specific behavior: The barcode reader app reads the barcode and sends the barcode number to the server in an HTTP POST request.

[0641] Input: Barcode data.

[0642] Output: HTTP request containing the barcode data.

[0643] Step 3:

[0644] The server searches the database.

[0645] Specific operation: The server receives the barcode number and issues an SQL query to the database management system (e.g. MySQL, PostgreSQL). It executes the query "SELECT FROM products WHERE barcode = '9781234567897'".

[0646] Input: Barcode data.

[0647] Output: SQL query result (product information).

[0648] Step 4:

[0649] The server obtains the product information.

[0650] Specific operation: The server stores the product information (price, rating, past transaction history) obtained from the database in memory.

[0651] Input: Product information resulting from a SQL query.

[0652] Output: Product information held in memory (e.g. price "$20", rating "Highly Rated", transaction history "Popular").

[0653] Step 5:

[0654] The server generates the description using a natural language processing model.

[0655] Specific operation: The server passes the product information as a prompt to a generative AI model (e.g. ChatGPT) and generates natural language text, such as "This book has high reviews, the latest price is $20, and it is also popular based on past transaction history."

[0656] Input: Product information.

[0657] Output: The generated description (natural language text).

[0658] Step 6:

[0659] The server recognizes the user's emotions using an emotion engine.

[0660] Specific operation: Facial expression images and voice data sent from the user's device are passed to an emotion recognition engine on the server, which analyzes the user's emotional state.

[0661] Input: facial expression images and audio data.

[0662] Output: The perceived emotion (e.g. "surprise").

[0663] Step 7:

[0664] The server customizes the generated product information.

[0665] What happens: The server customizes the generated description based on the recognized emotional state, adjusting it to something like "This price is an amazing deal!"

[0666] Input: Generated description, recognized sentiment.

[0667] Output: A customized description.

[0668] Step 8:

[0669] The server transmits the customized product information to the user's terminal.

[0670] Specific operation: The server sends the customized explanation to the user's terminal as an HTTP response, and the terminal displays it.

[0671] Input: Customized description.

[0672] Output: Product information displayed on the user's device.

[0673] Through these steps, users can simply scan the barcode of a product to instantly check online prices, ratings, past transaction history, and get personalized information based on their sentiment.

[0674] (Application example 2)

[0675] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0676] When shopping in a brick-and-mortar store, it is difficult to instantly grasp the price, reviews, and past transaction history of a product. Also, when referring to information on the web, it takes a lot of time and effort to find information that is appropriate for oneself. In particular, when a user is in an emotional state, their judgment is easily influenced, which can lead to inefficient purchasing behavior.

[0677] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for receiving barcode information and acquiring the online price, evaluation, and past transaction history of the product corresponding to the barcode in the database, a means for providing the product information in natural language, a means for generating information supporting a purchase decision using an AI model using natural language processing technology, and a means for recognizing the user's emotion using emotion recognition technology and customizing the generated information based on the recognition result. This enables a user to obtain customized information corresponding to their individual emotional state in real time by simply scanning a product in a physical store.

[0678] "Barcode information" refers to data of an identification code attached to a product, and is information that can identify the product by scanning it.

[0679] "Online price" means the selling price of a Product offered on the Internet.

[0680] "Rating" refers to reviews and scores of products by users and experts, and is information that reflects opinions about the quality and usability of the product.

[0681] "Past transaction history" refers to information regarding past purchases and sales of the product.

[0682] A "natural language" is a language that is used daily by humans and that can be analyzed and generated by machines.

[0683] "Natural language processing technology" is a technology that enables computers to understand, interpret, and manipulate text data.

[0684] An "AI model" is a mathematical model that utilizes artificial intelligence algorithms to perform specific tasks.

[0685] "Emotion recognition technology" is a technology that determines a user's emotional state by analyzing their facial expressions and tone of voice.

[0686] "Customization" means adapting or modifying content or information to fit a particular need or situation.

[0687] A "server" is a computer system that receives and processes requests from clients over a network and provides information.

[0688] "Terminal" refers to a device that is directly operated by a user, and includes, for example, a smartphone or smart glasses.

[0689] The system for implementing the present invention includes a server, a terminal, and a database. The purpose is to enable users to scan products while shopping in a brick-and-mortar store and instantly provide online information and customized advice. The specific configuration and operation are shown below.

[0690] System Configuration

[0691] 1. Server:

[0692] How to receive barcode information:

[0693] The server receives the barcode information sent by the terminal. This is done using an HTTP request.

[0694] Ways to retrieve information from the database:

[0695] The server searches a database based on the received barcode information to obtain the online price, reviews, and past transaction history of the corresponding product.

[0696] Means of generating information in natural language:

[0697] The server uses natural language processing technology (e.g., the ChatGPT model from the Transformers library) to generate the acquired product information in natural language.

[0698] Ways to use emotion recognition technology to personalize communications:

[0699] Use an emotion recognition engine (e.g., the EmotionRecognition library) to recognize the user's emotions and customize the generated information based on the results.

[0700] 2. Terminal:

[0701] How users can scan items:

[0702] Using the camera on the device (a smartphone or smart glasses), the user scans the barcode of the product.

[0703] Methods for sending information to the server:

[0704] The scanned barcode information is sent from the terminal to the server.

[0705] To view the information obtained:

[0706] The product information and customization information generated in natural language and obtained from the server are displayed on the user's terminal.

[0707] Hardware and Software

[0708] Hardware: Smartphones, smart glasses, servers

[0709] Software: OpenCV (barcode scanning), Requests (API communication), Transformers (natural language generation), EmotionRecognition (emotion recognition)

[0710] Examples

[0711] Assume the user is in a brick-and-mortar store and scans the barcode on their laptop. The server receives the barcode information and inputs the following prompt sentence into the natural language generation model:

[0712] Example prompt:

[0713] "Product description: This amazing laptop is packed with the latest features, priced at $798 and has a user rating of 4.5."

[0714] A natural language generation model generates a detailed product description based on the prompt. An emotion recognition engine then analyzes the user's facial expressions and tone of voice to determine their emotions. For example, if the system detects that the user is anxious, the product description will include additional information to reassure the user.

[0715] This allows users to simply scan a product in a physical store and obtain real-time information on not only the price and reviews but also customized information based on their own emotions, making purchasing decisions easier.

[0716] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0717] Step 1:

[0718] The user scans the product's barcode using the device's camera.

[0719] Input: Product barcode captured by the device camera

[0720] Output: Barcode data

[0721] Specific operation: Activate the device's camera and use the OpenCV library to read the barcode and extract the barcode data. Once the barcode data is obtained, proceed to the next step.

[0722] Step 2:

[0723] The terminal transmits the acquired barcode data to the server.

[0724] Input: Barcode data

[0725] Output: HTTP request to the server

[0726] Specific operation: The terminal sends the acquired barcode data to the server as an HTTP request. The request includes the barcode data.

[0727] Step 3:

[0728] The server receives the barcode information and searches a database to obtain the corresponding product information.

[0729] Input: Barcode data

[0730] Output: Product information (price, rating, past transaction history)

[0731] Specific operation: The server searches the database based on the received barcode data. If the appropriate product information is found, it retrieves the information and proceeds to the next step.

[0732] Step 4:

[0733] Based on the product information acquired by the server, a product description is generated in natural language using a generative AI model.

[0734] Input: Product information

[0735] Output: A product description generated in natural language

[0736] Specific operation: The server inputs product information as a prompt sentence into a generative AI model (e.g., Transformers' ChatGPT model). The model generates a product description in natural language based on the information and outputs the result.

[0737] Step 5:

[0738] In order for the server to recognize the user's emotions, it uses emotion recognition technology to analyze facial and voice data.

[0739] Input: User facial and voice data

[0740] Output: The user's emotional state.

[0741] How it works: The user provides facial expressions and voice using a dedicated camera or microphone. The server analyzes this data using the EmotionRecognition library and recognizes the user's emotional state.

[0742] Step 6:

[0743] The server generates customized product information based on the generated product description and the user's emotional state.

[0744] Input: Natural language generated product description, user's emotional state

[0745] Output:Customized product information

[0746] How it works: The server adds or adjusts the generated product description depending on the user's recognized emotional state. For example, if the user is in an anxious state, the server adds information that emphasizes the reliability and guarantees of the product.

[0747] Step 7:

[0748] The device displays the customized product information.

[0749] Input:Customized product information

[0750] Output: Product information displayed to the user

[0751] Specific operation: The user's device (smartphone or smart glasses) receives the customized product information sent from the server and displays it on the screen. This allows the user to obtain information that suits their individual needs and emotional state.

[0752] These steps allow users to obtain real-time relevant information about the products they scan in physical stores and receive customized purchasing support information based on their emotions.

[0753] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0754] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0755] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0756] [Third embodiment]

[0757] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0758] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0759] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0760] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0761] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0762] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0763] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0764] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0765] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0766] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0767] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0768] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server", and the headset type terminal 314 will be referred to as the "terminal".

[0769] An embodiment for implementing the present invention includes the following elements.

[0770] 1. Terminal: A device that allows users to scan product barcodes in physical stores.

[0771] 2. Server: A system that receives barcode information, obtains the product's online price, rating, and past transaction history, and provides them to the user's terminal.

[0772] 3. Database: A database for storing product information and managing online prices, ratings, and past transaction history for products corresponding to barcodes.

[0773] 4. Natural language processing model: A model that uses natural language processing technologies such as generative AI to provide product information in natural language and generate information to support purchasing decisions.

[0774] As a specific embodiment, the invention is carried out in the following procedure.

[0775] 1. The user scans the barcode of a product in a physical store using the device's camera.

[0776] 2. The terminal sends the barcode information to the server.

[0777] 3. The server searches the database for the item that corresponds to the barcode and displays the online price

[0778] Get stock quotes, ratings, and past transaction history.

[0779] 4. The server uses a natural language processing model to generate product information in natural language and provide it to the user's device.

[0780] 5. The user makes a purchasing decision based on the information provided.

[0781] In this way, by implementing the present invention, a user can simply scan a product's barcode while shopping in a physical store or on the go, and instantly check the product's online price, reviews, and past transaction history, providing information to support their purchasing decision.

[0782] The process flow will be explained below.

[0783] Step 1: The user scans the product barcode with the device's camera at the physical store.

[0784] -The user scans the product's barcode with the device's camera in a physical store.

[0785] Step 2: The terminal sends the barcode information to the server

[0786] -The terminal transmits the barcode information to the server.

[0787] Step 3: The server receives the barcode information and searches it in the database

[0788] -The server receives the barcode information and searches the database for the product that corresponds to the barcode.

[0789] Step 4: The server retrieves online prices, ratings, and past transaction history.

[0790] -The server retrieves the online price, reviews, and past transaction history of the product corresponding to the barcode from a database.

[0791] Step 5: The server generates information using a natural language processing model

[0792] -The server uses a natural language processing model (such as generative AI) to generate product information in natural language.

[0793] Step 6: The server provides the generated information to the user's device.

[0794] The server provides the generated product information to the user's terminal.

[0795] The above is an explanation of the program processing steps and specific operations.

[0796] Example 1

[0797] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0798] When purchasing products in physical stores, modern consumers want to easily obtain detailed information about the product, such as its price, reviews, and past transaction history. However, with current systems, it is difficult to obtain this information instantly, and consumers lack the information they need to make appropriate purchasing decisions. There is a need to solve this problem and improve consumers' purchasing experience in physical stores.

[0799] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0800] In this invention, the server includes a means for a user to scan a barcode of a product using a terminal, a means for the terminal to transmit the scanned barcode information to the server, a means for the server to search a database for the received barcode information and obtain the online price, evaluation, and past transaction history of the product corresponding to the barcode, a means for the server to use a generative AI model to convert the obtained product information into natural language, a means for the server to generate information supporting a purchase decision using the generative AI model and provide the information to the terminal, and a means for the terminal to display the provided information to the user. This allows consumers to instantly obtain detailed information about products in a physical store and make appropriate purchasing decisions.

[0801] "User" refers to a consumer who operates the system and scans product barcodes to obtain information.

[0802] "Terminal" refers to a device used by a user that has the function of scanning product barcodes and transmitting the barcode information to a server. Examples include smartphones and tablets.

[0803] A "server" refers to a computer system that has the function of receiving barcode information sent from a terminal, querying a database to obtain product information, and further generating information using natural language processing technology.

[0804] A "barcode" is an identifier consisting of a series of lines and numbers attached to a product, and refers to a code used to identify and obtain information about the product.

[0805] "Database" means an electronic storage device for storing and managing product information, including online product prices, ratings, and past transaction history.

[0806] "Online Price" means the selling price for a Product available on the Internet.

[0807] "Ratings" refers to ratings and reviews of products by consumers and experts who have previously purchased the product.

[0808] "Transaction History" refers to the record of past purchases and sales of a particular commodity.

[0809] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to convert acquired product information into text that is easy for humans to understand. Specific examples include ChatGPT.

[0810] "Natural language processing technology" is a technology that enables the understanding and generation of human language, and refers to the technology used to analyze and generate text data.

[0811] "Generating information" refers to converting the data acquired using a generative AI model into a natural language text format and providing it to users in a form that is easy for them to understand.

[0812] This invention provides a product information presentation system that allows users to instantly obtain detailed product information when purchasing a product in a brick-and-mortar store. This system is composed of the following elements: a terminal, a server, a database, and a generative AI model.

[0813] Using the terminal

[0814] A user uses a device such as a smartphone to launch a barcode scanning app and scan the barcode of a product. The device then reads the barcode with a camera, encodes the information as a string, and sends it to the server.

[0815] Examples:

[0816] "When a user is in a supermarket, they use their smartphone camera to scan the barcode of an item they are considering purchasing."

[0817] Server Processing

[0818] The server receives the barcode information sent from the terminal and retrieves the corresponding product information by referencing the information in a database, which stores the product's online price, reviews, and past transaction history. The server uses SQL queries to search the database and retrieve the required information.

[0819] Specific software and hardware used:

[0820] Server: Apache, MySQL

[0821] Using generative AI models

[0822] The server uses a generative AI model (e.g. ChatGPT) to convert the acquired product information into natural language. The generative AI model analyzes information including the product's online price, reviews, and past transaction history, and generates natural language text in a form that is easy for the user to understand.

[0823] Examples:

[0824] "The server searches the database for the barcode '1234567890123' and retrieves the product's price, reviews, and past transaction history."

[0825] Providing and displaying information

[0826] The generated text information is transmitted from the server to the terminal, which displays the received text information on a user interface so that the user can visually confirm the information.

[0827] Examples:

[0828] "The server sends the generated natural language explanation to the terminal, and the terminal displays the explanation."

[0829] Examples of prompt statements

[0830] Examples of prompts that users might input to a generative AI model include:

[0831] "How much does this item cost online? The barcode is 1234567890123."

[0832] "What is the rating for this item? Barcode is 1234567890123."

[0833] This allows users to instantly obtain detailed information about products in physical stores, providing information to support their purchasing decisions.

[0834] The system allows users to easily access detailed product information while shopping in a physical store, allowing them to make more informed purchasing decisions.

[0835] Recheck the hardware and software you will be using

[0836] Smartphone (e.g. iPhone or Android device)

[0837] Barcode Scanning App

[0838] Server (Apache, MySQL)

[0839] Natural language processing models (e.g. ChatGPT)

[0840] In the above manner, the present invention can be carried out.

[0841] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0842] Step 1:

[0843] User Actions

[0844] A user launches a barcode scanning app on their smartphone at a physical store and scans the barcode of a product. The user holds the camera over the product, frames the barcode, and presses the scan button. The input data is the image of the barcode, and the output data is the encoded barcode information.

[0845] Specific behavior:

[0846] "Users use their smartphone camera to scan the barcode of a product they are considering purchasing at a supermarket."

[0847] Step 2:

[0848] Terminal operation

[0849] The terminal encodes the scanned barcode information and sends it to the server as an HTTP request. The input data is the encoded barcode information, and the output data is the HTTP request to the server.

[0850] Specific behavior:

[0851] "The terminal encodes the barcode as a string of characters and sends it over the internet to a server."

[0852] Step 3:

[0853] Server Processing

[0854] The server decodes the received barcode information and searches the database for the corresponding product. The input data is the barcode information, and the output data is detailed information about the retrieved product (price, reviews, past transaction history).

[0855] Specific behavior:

[0856] "The server decodes the barcode number and runs a SQL query based on that number to retrieve product information from a database."

[0857] Step 4:

[0858] Server Processing

[0859] The server inputs the obtained product details into a generative AI model (e.g. ChatGPT) to convert them into natural language. The generative AI model analyzes information including online prices, reviews, past transaction history, etc., and generates natural language text. The input data is the product details, and the output data is the generated natural language text.

[0860] Specific behavior:

[0861] "The server inputs product details into ChatGPT and generates natural language text to support the purchasing decision."

[0862] Step 5:

[0863] Server Operation

[0864] The server transmits the generated natural language text to the user's terminal. The input data is the generated natural language text, and the output data is the text transmission to the user's terminal.

[0865] Specific behavior:

[0866] "The server sends the generated text to the user's device as an HTTP response."

[0867] Step 6:

[0868] Terminal operation

[0869] The terminal displays the received natural language text on a user interface, allowing the user to visually confirm the information. Input data is the natural language text received from the server, and output data is the information displayed on the user interface.

[0870] Specific behavior:

[0871] "The terminal displays the received text on the screen, allowing the user to visually confirm the product details."

[0872] These processing steps allow users to easily obtain product information in physical stores and make appropriate purchasing decisions.

[0873] (Application example 1)

[0874] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0875] When purchasing goods at a brick-and-mortar store, modern consumers find it difficult to instantly check online prices, ratings, and past transaction history to make optimal purchasing decisions. In addition, there is a lack of means to quickly provide such information, and it is difficult to receive advice and offers individually. As a result, consumers are at risk of purchasing at inappropriate prices, and sellers are also faced with the challenge of being unable to implement appropriate marketing measures.

[0876] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0877] In this invention, the server includes means for receiving barcode information and obtaining online prices, ratings, and past transaction history of an item corresponding to the barcode in a database, means for providing information about the item in natural language, means for generating information to support a purchasing decision using a generative AI model that utilizes natural language processing technology, means for generating prompt sentences for the generative AI model, means for providing advice and offers to the user based on the generated prompt sentences, and means for enabling a user to scan an item in a physical store and refer to online prices, ratings, and past transaction history.

[0878] This allows users to check online prices and ratings in real time when purchasing goods in physical stores, and to refer to past transaction history to make optimal purchasing decisions. In addition, they can receive advice and offers from generative AI models, which increases convenience for consumers.

[0879] A "server" is a computer system that provides data and services to other computers and devices over a computer network.

[0880] "Barcode information" refers to information that represents data corresponding to a particular item by reading a series of black and white lines affixed to the item.

[0881] A "database" is a system for efficiently storing, managing, and searching large amounts of digital data.

[0882] "Online price of the Goods" means the price at which the Goods are offered on an internet sales platform.

[0883] A "rating" is a score or review based on a user's impressions or opinions about an item.

[0884] "Transaction history" refers to historical information about past sales of the item.

[0885] "Natural language" is language used daily by humans and is exploited by technologies that enable computers to understand and generate it.

[0886] "Natural language processing technology" refers to technology that enables computers to understand, generate, and analyze human natural language.

[0887] A "generative AI model" is an artificial intelligence model that automatically generates new text or data based on given input data.

[0888] A "prompt statement" is an instruction or input statement given to a generative AI model, which serves as a guideline for obtaining the desired output result.

[0889] A "user" is someone who uses a system or service.

[0890] A "terminal" refers to a hardware device that a user can directly operate, including a smartphone or tablet.

[0891] The embodiment of the present invention is specifically realized by a system including the following procedures.

[0892] First, when a user is considering purchasing an item in a physical store, he or she uses a device such as a smartphone or tablet to scan the barcode of the target item with a camera. The scanned barcode information is sent to the server in real time.

[0893] The server searches the database for the corresponding item based on the received barcode information, which includes the item's online price, reviews, and past transaction history, to obtain detailed information about the item.

[0894] Next, the server uses natural language processing technology to pass the acquired data to a generative AI model. The generative AI model generates specific instructions, or prompts, and generates information to provide to the user based on the prompts. For example, the generated prompts are as follows:

[0895] example:

[0896] Please provide the product information corresponding to barcode "1234567890123".

[0897] Product name: Microwave

[0898] Online Price: 12000 yen

[0899] Rating: 4.5

[0900] Past transaction history: Sold 50 times in the past year. The lowest price was 10,000 yen.

[0901] The generated information is presented to the user via a terminal, allowing the user to instantly check online price comparisons, reviews, and past sales history of the item, providing information to support purchasing decisions.

[0902] In addition, the server generates advice and offers for the user based on prompts sent to the generative AI model, and these are also provided through the terminal, allowing the user to receive advantageous information and additional suggestions when purchasing goods.

[0903] Specific hardware used to realize this system includes terminals such as smartphones and tablets, and server computers. Software used includes a database management system and a generative AI model that utilizes natural language processing technology (such as ChatGPT).

[0904] In this way, the present invention provides a smarter and more efficient way for users to shop in physical stores.

[0905] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0906] Step 1:

[0907] The user scans the barcode of an item they are considering purchasing in a physical store using the camera on their smartphone, tablet or other device.

[0908] Input: Item barcode

[0909] Output: Scanned barcode information (numeric data)

[0910] Specific behavior: Read the barcode using the device's camera application and obtain its information.

[0911] Step 2:

[0912] The terminal transmits the scanned barcode information to the server.

[0913] Input: Scanned barcode information

[0914] Output: A message confirming the request was sent to the server.

[0915] Specific operation: The terminal application sends the barcode information to the server as an HTTP POST request.

[0916] Step 3:

[0917] Based on the barcode information received by the server, the database is searched for corresponding item information.

[0918] Input: Barcode information

[0919] Output: Item information retrieved from the database (price, rating, transaction history, etc.)

[0920] Specific operation: The search algorithm in the server searches the corresponding items in the database and retrieves the item information.

[0921] Step 4:

[0922] The server uses natural language processing technology to generate a prompt sentence to pass the product information it has acquired to the generative AI model.

[0923] Input: Product information (price, rating, transaction history, etc.)

[0924] Output: The generated prompt statement

[0925] Specific operation: The server constructs a text prompt based on the product information and prepares it for input into the generative AI model.

[0926] Step 5:

[0927] The generative AI model generates information to provide to the user based on the prompt sentence.

[0928] Input: prompt statement

[0929] Output: Information for the user in natural language (advice, offers, etc.)

[0930] What it does: A generative AI model parses the prompt and generates appropriate natural language text.

[0931] Step 6:

[0932] The server transmits the generated information to the terminal.

[0933] Input: User-facing information in natural language

[0934] Output: Response message to terminal

[0935] Specific operation: The server sends the generated information to the terminal using an HTTP response.

[0936] Step 7:

[0937] The terminal displays the received information to the user.

[0938] Input: User-request information received from the server

[0939] Output: Displays the price, rating, transaction history, additional advice and offers for the item.

[0940] Specific operation: The terminal application analyzes the received text information and displays it in the user interface.

[0941] Example prompt:

[0942] "Please provide details of the product that corresponds to barcode '1234567890123'. Product name: Microwave oven. Online price: 12,000 yen. Rating: 4.5. Past transaction history: Sold 50 times in the past year. The lowest price was 10,000 yen."

[0943] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0944] An embodiment for implementing the present invention includes the following elements.

[0945] 1. Terminal: A device that allows users to scan product barcodes in physical stores.

[0946] 2. Server: A system that receives barcode information, obtains the product's online price, rating, and past transaction history, and provides them to the user's terminal.

[0947] 3. Database: A database for storing product information and managing online prices, ratings, and past transaction history for products corresponding to barcodes.

[0948] 4. Natural language processing model: A model that uses natural language processing technologies such as generative AI to generate product information in natural language and provide it to the user's device.

[0949] 5. Emotion Engine: An engine for recognizing user emotions.

[0950] As a specific embodiment, the invention is carried out in the following procedure.

[0951] 1. The user scans the barcode of a product in a physical store using the device's camera.

[0952] 2. The terminal sends the barcode information to the server.

[0953] 3. The server searches the database for the product that corresponds to the barcode and retrieves online prices, reviews, and past transaction history.

[0954] 4. The server uses a natural language processing model to generate product information in natural language and provide it to the user's device.

[0955] 5. The server uses the emotion engine to recognize the user's emotions.

[0956] 6. The server appropriately customizes the generated product information based on the user's emotions and provides it to the user.

[0957] In this way, by implementing the present invention, a user can simply scan the barcode of a product while shopping at a brick-and-mortar store or on the go, and instantly check the product's online price, reviews, and past transaction history, and obtain information to support purchasing decisions. Furthermore, by combining it with an emotion engine, it is possible to recognize the user's emotions and customize product information accordingly.

[0958] The process flow will be explained below.

[0959] Step 1: The user scans the product barcode with the device's camera at the physical store.

[0960] -The user scans the product's barcode with the device's camera in a physical store.

[0961] Step 2: The terminal sends the barcode information to the server

[0962] -The terminal transmits the barcode information to the server.

[0963] Step 3: The server receives the barcode information and searches it in the database

[0964] -The server receives the barcode information and searches the database for the product that corresponds to the barcode.

[0965] Step 4: The server retrieves online prices, ratings, and past transaction history.

[0966] -The server retrieves the online price, reviews, and past transaction history of the product corresponding to the barcode from a database.

[0967] Step 5: The server generates information using a natural language processing model

[0968] -The server uses a natural language processing model (such as generative AI) to generate product information in natural language.

[0969] Step 6: The server uses the emotion engine to recognize the user's emotion.

[0970] -The server uses an emotion engine to recognize the user's emotions.

[0971] Step 7: Customize server-generated information based on user emotions

[0972] The server appropriately customizes the generated product information based on the user's emotions.

[0973] Step 8: The server provides customized information to the user's device.

[0974] The server provides customized product information to the user's terminal.

[0975] The above is an explanation of the program processing steps and specific operations.

[0976] Example 2

[0977] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0978] Conventional information provision systems have difficulty in fully stimulating users' purchasing motivation because the information that can be obtained by scanning product barcodes is limited.In addition, it is not possible to provide information that takes into account the user's emotions and preferences, making it impossible to meet individual needs.

[0979] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving barcode information and acquiring the online price, evaluation, and past transaction history of the product corresponding to the barcode in the database, a means for providing product information in natural language, a means for generating information supporting purchase decisions using a generative model that utilizes natural language processing technology, a means for recognizing the user's emotions, and a means for customizing the generated information based on the user's emotions. This allows the user to diversify the information obtained from the scanned barcode and receive information provided that is customized according to emotions.

[0980] A "server" is a central processing unit that receives requests from users and processes data and provides information.

[0981] "Barcode information" is data contained in a barcode for identifying a product.

[0982] A "database" is a data management system that stores information in an organized format and makes it possible to search and retrieve it.

[0983] "Natural language processing technology" is a technology that enables computers to understand and generate human language.

[0984] A "generative model" is an artificial intelligence model that generates new text or content based on input data.

[0985] "Means for recognizing emotions" refers to technology that identifies a user's emotional state by analyzing facial expressions and voice data.

[0986] "Means for customizing information" refers to technology that adjusts the content of generated information based on the user's emotions and preferences, and provides it in accordance with individual needs.

[0987] "Online price" means the selling price of the product published on the Internet.

[0988] A "rating" is an opinion or rating given by a user or expert on the quality or performance of a product.

[0989] "Past transaction history" is data indicating the past buying and selling history and usage history of a product.

[0990] The product information presentation system according to the present invention functions by combining multiple hardware and software elements. To implement the system, a user terminal, a server, a database, a natural language processing model, and an emotion recognition engine are all required.

[0991] First, a user scans the barcode of a product at a physical store using a smartphone or a dedicated barcode scanner. The terminal receives the barcode information and sends it to the server. Specifically, a smartphone with a barcode reader application installed reads the barcode using its built-in camera and sends an HTTP POST request to the server.

[0992] The server searches the database based on the received barcode information to obtain the online price, reviews, and past transaction history of the corresponding product. The database management system used by the server includes MySQL and PostgreSQL. For example, the server issues the SQL query "SELECT FROM products WHERE barcode = '9781234567897'" to obtain the corresponding data.

[0993] Next, the server uses a generative AI model to generate information using natural language processing technology based on the acquired product information. This generative AI model is composed of an advanced generative AI such as ChatGPT, which receives the required prompt as input and generates natural language text. As a specific example, the server generates a prompt such as "This book has high reviews and the latest price is $20. It is also popular based on past transaction history," and passes it to the AI ​​model, generating natural sentences.

[0994] In addition, the server uses an emotion recognition engine to recognize the user's emotions. It analyzes facial images and voice data sent from the user's device to identify the user's emotional state. For example, if the user has a surprised expression, it will be recognized as "surprise." The server then customizes the generated product information based on the user's emotions, changing the expression to reflect the user's emotions, such as "This price is an amazing deal!"

[0995] Finally, the server sends the customized product information to the user's device and displays it. Users can instantly obtain product information in an easy-to-view format by simply scanning the barcode in a physical store. In addition, the customized information based on emotions can support more effective purchasing decisions.

[0996] Specific examples of prompt sentences include the following:

[0997] "This book has great reviews, a current price of $20, and past transaction history shows it's popular."

[0998] In this way, by implementing the present invention, it becomes possible to provide highly convenient information in a brick-and-mortar store or on the go, thereby improving the user's purchasing experience.

[0999] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1000] Step 1:

[1001] A user scans the product's barcode.

[1002] Specific operations: The user launches a barcode reader app on their smartphone and uses the camera to scan the barcode of the product.

[1003] Input: Item barcode.

[1004] Output: Barcode data (e.g. barcode number "9781234567897").

[1005] Step 2:

[1006] The terminal transmits the barcode information to the server.

[1007] Specific behavior: The barcode reader app reads the barcode and sends the barcode number to the server in an HTTP POST request.

[1008] Input: Barcode data.

[1009] Output: HTTP request containing the barcode data.

[1010] Step 3:

[1011] The server searches the database.

[1012] Specific operation: The server receives the barcode number and issues an SQL query to the database management system (e.g. MySQL, PostgreSQL). It executes the query "SELECT FROM products WHERE barcode = '9781234567897'".

[1013] Input: Barcode data.

[1014] Output: SQL query result (product information).

[1015] Step 4:

[1016] The server obtains the product information.

[1017] Specific operation: The server stores the product information (price, rating, past transaction history) obtained from the database in memory.

[1018] Input: Product information resulting from a SQL query.

[1019] Output: Product information held in memory (e.g. price "$20", rating "Highly Rated", transaction history "Popular").

[1020] Step 5:

[1021] The server generates the description using a natural language processing model.

[1022] Specific operation: The server passes the product information as a prompt to a generative AI model (e.g. ChatGPT) and generates natural language text, such as "This book has high reviews, the latest price is $20, and it is also popular based on past transaction history."

[1023] Input: Product information.

[1024] Output: The generated description (natural language text).

[1025] Step 6:

[1026] The server recognizes the user's emotions using an emotion engine.

[1027] Specific operation: Facial expression images and voice data sent from the user's device are passed to an emotion recognition engine on the server, which analyzes the user's emotional state.

[1028] Input: facial expression images and audio data.

[1029] Output: The perceived emotion (e.g. "surprise").

[1030] Step 7:

[1031] The server customizes the generated product information.

[1032] What happens: The server customizes the generated description based on the recognized emotional state, adjusting it to something like "This price is an amazing deal!"

[1033] Input: Generated description, recognized sentiment.

[1034] Output: A customized description.

[1035] Step 8:

[1036] The server transmits the customized product information to the user's terminal.

[1037] Specific operation: The server sends the customized explanation to the user's terminal as an HTTP response, and the terminal displays it.

[1038] Input: Customized description.

[1039] Output: Product information displayed on the user's device.

[1040] Through these steps, users can simply scan the barcode of a product to instantly check online prices, ratings, past transaction history, and get personalized information based on their sentiment.

[1041] (Application example 2)

[1042] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1043] When shopping in a brick-and-mortar store, it is difficult to instantly grasp the price, reviews, and past transaction history of a product. Also, when referring to information on the web, it takes a lot of time and effort to find information that is appropriate for oneself. In particular, when a user is in an emotional state, their judgment is easily influenced, which can lead to inefficient purchasing behavior.

[1044] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for receiving barcode information and acquiring the online price, evaluation, and past transaction history of the product corresponding to the barcode in the database, a means for providing the product information in natural language, a means for generating information supporting a purchase decision using an AI model using natural language processing technology, and a means for recognizing the user's emotion using emotion recognition technology and customizing the generated information based on the recognition result. This enables a user to obtain customized information corresponding to their individual emotional state in real time by simply scanning a product in a physical store.

[1045] "Barcode information" refers to data of an identification code attached to a product, and is information that can identify the product by scanning it.

[1046] "Online price" means the selling price of a Product offered on the Internet.

[1047] "Rating" refers to reviews and scores of products by users and experts, and is information that reflects opinions about the quality and usability of the product.

[1048] "Past transaction history" refers to information regarding past purchases and sales of the product.

[1049] A "natural language" is a language that is used daily by humans and that can be analyzed and generated by machines.

[1050] "Natural language processing technology" is a technology that enables computers to understand, interpret, and manipulate text data.

[1051] An "AI model" is a mathematical model that utilizes artificial intelligence algorithms to perform specific tasks.

[1052] "Emotion recognition technology" is a technology that determines a user's emotional state by analyzing their facial expressions and tone of voice.

[1053] "Customization" means adapting or modifying content or information to fit a particular need or situation.

[1054] A "server" is a computer system that receives and processes requests from clients over a network and provides information.

[1055] "Terminal" refers to a device that is directly operated by a user, and includes, for example, a smartphone or smart glasses.

[1056] The system for implementing the present invention includes a server, a terminal, and a database. The purpose is to enable users to scan products while shopping in a brick-and-mortar store and instantly provide online information and customized advice. The specific configuration and operation are shown below.

[1057] System Configuration

[1058] 1. Server:

[1059] How to receive barcode information:

[1060] The server receives the barcode information sent by the terminal. This is done using an HTTP request.

[1061] Ways to retrieve information from the database:

[1062] The server searches a database based on the received barcode information to obtain the online price, reviews, and past transaction history of the corresponding product.

[1063] Means of generating information in natural language:

[1064] The server uses natural language processing technology (e.g., the ChatGPT model from the Transformers library) to generate the acquired product information in natural language.

[1065] Ways to use emotion recognition technology to personalize communications:

[1066] Use an emotion recognition engine (e.g., the EmotionRecognition library) to recognize the user's emotions and customize the generated information based on the results.

[1067] 2. Terminal:

[1068] How users can scan items:

[1069] Using the camera on the device (a smartphone or smart glasses), the user scans the barcode of the product.

[1070] Methods for sending information to the server:

[1071] The scanned barcode information is sent from the terminal to the server.

[1072] To view the information obtained:

[1073] The product information and customization information generated in natural language and obtained from the server are displayed on the user's terminal.

[1074] Hardware and Software

[1075] Hardware: Smartphones, smart glasses, servers

[1076] Software: OpenCV (barcode scanning), Requests (API communication), Transformers (natural language generation), EmotionRecognition (emotion recognition)

[1077] Examples

[1078] Assume the user is in a brick-and-mortar store and scans the barcode on their laptop. The server receives the barcode information and inputs the following prompt sentence into the natural language generation model:

[1079] Example prompt:

[1080] "Product description: This amazing laptop is packed with the latest features, priced at $798 and has a user rating of 4.5."

[1081] A natural language generation model generates a detailed product description based on the prompt. An emotion recognition engine then analyzes the user's facial expressions and tone of voice to determine their emotions. For example, if the system detects that the user is anxious, the product description will include additional information to reassure the user.

[1082] This allows users to simply scan a product in a physical store and obtain real-time information on not only the price and reviews but also customized information based on their own emotions, making purchasing decisions easier.

[1083] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1084] Step 1:

[1085] The user scans the product's barcode using the device's camera.

[1086] Input: Product barcode captured by the device camera

[1087] Output: Barcode data

[1088] Specific operation: Activate the device's camera and use the OpenCV library to read the barcode and extract the barcode data. Once the barcode data is obtained, proceed to the next step.

[1089] Step 2:

[1090] The terminal transmits the acquired barcode data to the server.

[1091] Input: Barcode data

[1092] Output: HTTP request to the server

[1093] Specific operation: The terminal sends the acquired barcode data to the server as an HTTP request. The request includes the barcode data.

[1094] Step 3:

[1095] The server receives the barcode information and searches a database to obtain the corresponding product information.

[1096] Input: Barcode data

[1097] Output: Product information (price, rating, past transaction history)

[1098] Specific operation: The server searches the database based on the received barcode data. If the appropriate product information is found, it retrieves the information and proceeds to the next step.

[1099] Step 4:

[1100] Based on the product information acquired by the server, a product description is generated in natural language using a generative AI model.

[1101] Input: Product information

[1102] Output: A product description generated in natural language

[1103] Specific operation: The server inputs product information as a prompt sentence into a generative AI model (e.g., Transformers' ChatGPT model). The model generates a product description in natural language based on the information and outputs the result.

[1104] Step 5:

[1105] In order for the server to recognize the user's emotions, it uses emotion recognition technology to analyze facial and voice data.

[1106] Input: User facial and voice data

[1107] Output: The user's emotional state.

[1108] How it works: The user provides facial expressions and voice using a dedicated camera or microphone. The server analyzes this data using the EmotionRecognition library and recognizes the user's emotional state.

[1109] Step 6:

[1110] The server generates customized product information based on the generated product description and the user's emotional state.

[1111] Input: Natural language generated product description, user's emotional state

[1112] Output:Customized product information

[1113] How it works: The server adds or adjusts the generated product description depending on the user's recognized emotional state. For example, if the user is in an anxious state, the server adds information that emphasizes the reliability and guarantees of the product.

[1114] Step 7:

[1115] The device displays the customized product information.

[1116] Input:Customized product information

[1117] Output: Product information displayed to the user

[1118] Specific operation: The user's device (smartphone or smart glasses) receives the customized product information sent from the server and displays it on the screen. This allows the user to obtain information that suits their individual needs and emotional state.

[1119] These steps allow users to obtain real-time relevant information about the products they scan in physical stores and receive customized purchasing support information based on their emotions.

[1120] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1121] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1122] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1123] [Fourth embodiment]

[1124] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1125] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1126] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1127] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1128] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1129] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1130] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1131] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1132] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1133] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1134] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1135] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1136] Next, a description will be given of the specific processing by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1137] An embodiment for implementing the present invention includes the following elements.

[1138] 1. Terminal: A device that allows users to scan product barcodes in physical stores.

[1139] 2. Server: A system that receives barcode information, obtains the product's online price, rating, and past transaction history, and provides them to the user's terminal.

[1140] 3. Database: A database for storing product information and managing online prices, ratings, and past transaction history for products corresponding to barcodes.

[1141] 4. Natural language processing model: A model that uses natural language processing technologies such as generative AI to provide product information in natural language and generate information to support purchasing decisions.

[1142] As a specific embodiment, the invention is carried out in the following procedure.

[1143] 1. The user scans the barcode of a product in a physical store using the device's camera.

[1144] 2. The terminal sends the barcode information to the server.

[1145] 3. The server searches the database for the product that corresponds to the barcode and retrieves online prices, reviews, and past transaction history.

[1146] 4. The server uses a natural language processing model to generate product information in natural language and provide it to the user's device.

[1147] 5. The user makes a purchasing decision based on the information provided.

[1148] In this way, by implementing the present invention, a user can simply scan a product's barcode while shopping in a physical store or on the go, and instantly check the product's online price, reviews, and past transaction history, providing information to support their purchasing decision.

[1149] The process flow will be explained below.

[1150] Step 1: The user scans the product barcode with the device's camera at the physical store.

[1151] -The user scans the product's barcode with the device's camera in a physical store.

[1152] Step 2: The terminal sends the barcode information to the server

[1153] -The terminal transmits the barcode information to the server.

[1154] Step 3: The server receives the barcode information and searches it in the database

[1155] -The server receives the barcode information and searches the database for the product that corresponds to the barcode.

[1156] Step 4: The server retrieves online prices, ratings, and past transaction history.

[1157] -The server retrieves the online price, reviews, and past transaction history of the product corresponding to the barcode from a database.

[1158] Step 5: The server generates information using a natural language processing model

[1159] -The server uses a natural language processing model (such as generative AI) to generate product information in natural language.

[1160] Step 6: The server provides the generated information to the user's device.

[1161] The server provides the generated product information to the user's terminal.

[1162] The above is an explanation of the program processing steps and specific operations.

[1163] Example 1

[1164] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the robot 414 is referred to as a "terminal."

[1165] When purchasing products in physical stores, modern consumers want to easily obtain detailed information about the product, such as its price, reviews, and past transaction history. However, with current systems, it is difficult to obtain this information instantly, and consumers lack the information they need to make appropriate purchasing decisions. There is a need to solve this problem and improve consumers' purchasing experience in physical stores.

[1166] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1167] In this invention, the server includes a means for a user to scan a barcode of a product using a terminal, a means for the terminal to transmit the scanned barcode information to the server, a means for the server to search a database for the received barcode information and obtain the online price, evaluation, and past transaction history of the product corresponding to the barcode, a means for the server to use a generative AI model to convert the obtained product information into natural language, a means for the server to generate information supporting a purchase decision using the generative AI model and provide the information to the terminal, and a means for the terminal to display the provided information to the user. This allows consumers to instantly obtain detailed information about products in a physical store and make appropriate purchasing decisions.

[1168] "User" refers to a consumer who operates the system and scans product barcodes to obtain information.

[1169] "Terminal" refers to a device used by a user that has the function of scanning product barcodes and transmitting the barcode information to a server. Examples include smartphones and tablets.

[1170] A "server" refers to a computer system that has the function of receiving barcode information sent from a terminal, querying a database to obtain product information, and further generating information using natural language processing technology.

[1171] A "barcode" is an identifier consisting of a series of lines and numbers attached to a product, and refers to a code used to identify and obtain information about the product.

[1172] "Database" means an electronic storage device for storing and managing product information, including online product prices, ratings, and past transaction history.

[1173] "Online Price" means the selling price for a Product available on the Internet.

[1174] "Ratings" refers to ratings and reviews of products by consumers and experts who have previously purchased the product.

[1175] "Transaction History" refers to the record of past purchases and sales of a particular commodity.

[1176] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to convert acquired product information into text that is easy for humans to understand. Specific examples include ChatGPT.

[1177] "Natural language processing technology" is a technology that enables the understanding and generation of human language, and refers to the technology used to analyze and generate text data.

[1178] "Generating information" refers to converting the data acquired using a generative AI model into a natural language text format and providing it to users in a form that is easy for them to understand.

[1179] This invention provides a product information presentation system that allows users to instantly obtain detailed product information when purchasing a product in a brick-and-mortar store. This system is composed of the following elements: a terminal, a server, a database, and a generative AI model.

[1180] Using the terminal

[1181] A user uses a device such as a smartphone to launch a barcode scanning app and scan the barcode of a product. The device then reads the barcode with a camera, encodes the information as a string, and sends it to the server.

[1182] Examples:

[1183] "When a user is in a supermarket, they use their smartphone camera to scan the barcode of an item they are considering purchasing."

[1184] Server Processing

[1185] The server receives the barcode information sent from the terminal and retrieves the corresponding product information by referencing the information in a database, which stores the product's online price, reviews, and past transaction history. The server uses SQL queries to search the database and retrieve the required information.

[1186] Specific software and hardware used:

[1187] Server: Apache, MySQL

[1188] Using generative AI models

[1189] The server uses a generative AI model (e.g. ChatGPT) to convert the acquired product information into natural language. The generative AI model analyzes information including the product's online price, reviews, and past transaction history, and generates natural language text in a form that is easy for the user to understand.

[1190] Examples:

[1191] "The server searches the database for the barcode '1234567890123' and retrieves the product's price, reviews, and past transaction history."

[1192] Providing and displaying information

[1193] The generated text information is transmitted from the server to the terminal, which displays the received text information on a user interface so that the user can visually confirm the information.

[1194] Examples:

[1195] "The server sends the generated natural language explanation to the terminal, and the terminal displays the explanation."

[1196] Examples of prompt statements

[1197] Examples of prompts that users might input to a generative AI model include:

[1198] "How much does this item cost online? The barcode is 1234567890123."

[1199] "What is the rating for this item? Barcode is 1234567890123."

[1200] This allows users to instantly obtain detailed information about products in physical stores, providing information to support their purchasing decisions.

[1201] The system allows users to easily access detailed product information while shopping in a physical store, allowing them to make more informed purchasing decisions.

[1202] Recheck the hardware and software you will be using

[1203] Smartphone (e.g. iPhone or Android device)

[1204] Barcode Scanning App

[1205] Server (Apache, MySQL)

[1206] Natural language processing models (e.g. ChatGPT)

[1207] In the above manner, the present invention can be carried out.

[1208] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1209] Step 1:

[1210] User Actions

[1211] A user launches a barcode scanning app on their smartphone at a physical store and scans the barcode of a product. The user holds the camera over the product, frames the barcode, and presses the scan button. The input data is the image of the barcode, and the output data is the encoded barcode information.

[1212] Specific behavior:

[1213] "Users use their smartphone camera to scan the barcode of a product they are considering purchasing at a supermarket."

[1214] Step 2:

[1215] Terminal operation

[1216] The terminal encodes the scanned barcode information and sends it to the server as an HTTP request. The input data is the encoded barcode information, and the output data is the HTTP request to the server.

[1217] Specific behavior:

[1218] "The terminal encodes the barcode as a string of characters and sends it over the internet to a server."

[1219] Step 3:

[1220] Server Processing

[1221] The server decodes the received barcode information and searches the database for the corresponding product. The input data is the barcode information, and the output data is detailed information about the retrieved product (price, reviews, past transaction history).

[1222] Specific behavior:

[1223] "The server decodes the barcode number and runs a SQL query based on that number to retrieve product information from a database."

[1224] Step 4:

[1225] Server Processing

[1226] The server inputs the obtained product details into a generative AI model (e.g. ChatGPT) to convert them into natural language. The generative AI model analyzes information including online prices, reviews, past transaction history, etc., and generates natural language text. The input data is the product details, and the output data is the generated natural language text.

[1227] Specific behavior:

[1228] "The server inputs product details into ChatGPT and generates natural language text to support the purchasing decision."

[1229] Step 5:

[1230] Server Operation

[1231] The server transmits the generated natural language text to the user's terminal. The input data is the generated natural language text, and the output data is the text transmission to the user's terminal.

[1232] Specific behavior:

[1233] "The server sends the generated text to the user's device as an HTTP response."

[1234] Step 6:

[1235] Terminal operation

[1236] The terminal displays the received natural language text on a user interface, allowing the user to visually confirm the information. Input data is the natural language text received from the server, and output data is the information displayed on the user interface.

[1237] Specific behavior:

[1238] "The terminal displays the received text on the screen, allowing the user to visually confirm the product details."

[1239] These processing steps allow users to easily obtain product information in physical stores and make appropriate purchasing decisions.

[1240] (Application example 1)

[1241] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1242] When purchasing goods at a brick-and-mortar store, modern consumers find it difficult to instantly check online prices, ratings, and past transaction history to make optimal purchasing decisions. In addition, there is a lack of means to quickly provide such information, and it is difficult to receive advice and offers individually. As a result, consumers are at risk of purchasing at inappropriate prices, and sellers are also faced with the challenge of being unable to implement appropriate marketing measures.

[1243] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1244] In this invention, the server includes means for receiving barcode information and obtaining online prices, ratings, and past transaction history of an item corresponding to the barcode in a database, means for providing information about the item in natural language, means for generating information to support a purchasing decision using a generative AI model that utilizes natural language processing technology, means for generating prompt sentences for the generative AI model, means for providing advice and offers to the user based on the generated prompt sentences, and means for enabling a user to scan an item in a physical store and refer to online prices, ratings, and past transaction history.

[1245] This allows users to check online prices and ratings in real time when purchasing goods in physical stores, and to refer to past transaction history to make optimal purchasing decisions. In addition, they can receive advice and offers from generative AI models, which increases convenience for consumers.

[1246] A "server" is a computer system that provides data and services to other computers and devices over a computer network.

[1247] "Barcode information" refers to information that represents data corresponding to a particular item by reading a series of black and white lines affixed to the item.

[1248] A "database" is a system for efficiently storing, managing, and searching large amounts of digital data.

[1249] "Online price of the Goods" means the price at which the Goods are offered on an internet sales platform.

[1250] A "rating" is a score or review based on a user's impressions or opinions about an item.

[1251] "Transaction history" refers to historical information about past sales of the item.

[1252] "Natural language" is language used daily by humans and is exploited by technologies that enable computers to understand and generate it.

[1253] "Natural language processing technology" refers to technology that enables computers to understand, generate, and analyze human natural language.

[1254] A "generative AI model" is an artificial intelligence model that automatically generates new text or data based on given input data.

[1255] A "prompt statement" is an instruction or input statement given to a generative AI model, which serves as a guideline for obtaining the desired output result.

[1256] A "user" is someone who uses a system or service.

[1257] A "terminal" refers to a hardware device that a user can directly operate, including a smartphone or tablet.

[1258] The embodiment of the present invention is specifically realized by a system including the following procedures.

[1259] First, when a user is considering purchasing an item in a physical store, he or she uses a device such as a smartphone or tablet to scan the barcode of the target item with a camera. The scanned barcode information is sent to the server in real time.

[1260] The server searches the database for the corresponding item based on the received barcode information, which includes the item's online price, reviews, and past transaction history, to obtain detailed information about the item.

[1261] Next, the server uses natural language processing technology to pass the acquired data to a generative AI model. The generative AI model generates specific instructions, or prompts, and generates information to provide to the user based on the prompts. For example, the generated prompts are as follows:

[1262] example:

[1263] Please provide the product information corresponding to barcode "1234567890123".

[1264] Product name: Microwave

[1265] Online Price: 12000 yen

[1266] Rating: 4.5

[1267] Past transaction history: Sold 50 times in the past year. The lowest price was 10,000 yen.

[1268] The generated information is presented to the user via a terminal, allowing the user to instantly check online price comparisons, reviews, and past sales history of the item, providing information to support purchasing decisions.

[1269] In addition, the server generates advice and offers for the user based on prompts sent to the generative AI model, and these are also provided through the terminal, allowing the user to receive advantageous information and additional suggestions when purchasing goods.

[1270] Specific hardware used to realize this system includes terminals such as smartphones and tablets, and server computers. Software used includes a database management system and a generative AI model that utilizes natural language processing technology (such as ChatGPT).

[1271] In this way, the present invention provides a smarter and more efficient way for users to shop in physical stores.

[1272] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1273] Step 1:

[1274] The user scans the barcode of an item they are considering purchasing in a physical store using the camera on their smartphone, tablet or other device.

[1275] Input: Item barcode

[1276] Output: Scanned barcode information (numeric data)

[1277] Specific behavior: Read the barcode using the device's camera application and obtain its information.

[1278] Step 2:

[1279] The terminal transmits the scanned barcode information to the server.

[1280] Input: Scanned barcode information

[1281] Output: A message confirming the request was sent to the server.

[1282] Specific operation: The terminal application sends the barcode information to the server as an HTTP POST request.

[1283] Step 3:

[1284] Based on the barcode information received by the server, the database is searched for corresponding item information.

[1285] Input: Barcode information

[1286] Output: Item information retrieved from the database (price, rating, transaction history, etc.)

[1287] Specific operation: The search algorithm in the server searches the corresponding items in the database and retrieves the item information.

[1288] Step 4:

[1289] The server uses natural language processing technology to generate a prompt sentence to pass the product information it has acquired to the generative AI model.

[1290] Input: Product information (price, rating, transaction history, etc.)

[1291] Output: The generated prompt statement

[1292] Specific operation: The server constructs a text prompt based on the product information and prepares it for input into the generative AI model.

[1293] Step 5:

[1294] The generative AI model generates information to provide to the user based on the prompt sentence.

[1295] Input: prompt statement

[1296] Output: Information for the user in natural language (advice, offers, etc.)

[1297] What it does: A generative AI model parses the prompt and generates appropriate natural language text.

[1298] Step 6:

[1299] The server transmits the generated information to the terminal.

[1300] Input: User-facing information in natural language

[1301] Output: Response message to terminal

[1302] Specific operation: The server sends the generated information to the terminal using an HTTP response.

[1303] Step 7:

[1304] The terminal displays the received information to the user.

[1305] Input: User-request information received from the server

[1306] Output: Displays the price, rating, transaction history, additional advice and offers for the item.

[1307] Specific operation: The terminal application analyzes the received text information and displays it in the user interface.

[1308] Example prompt:

[1309] "Please provide details of the product that corresponds to barcode '1234567890123'. Product name: Microwave oven. Online price: 12,000 yen. Rating: 4.5. Past transaction history: Sold 50 times in the past year. The lowest price was 10,000 yen."

[1310] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1311] An embodiment for implementing the present invention includes the following elements.

[1312] 1. Terminal: A device that allows users to scan product barcodes in physical stores.

[1313] 2. Server: A system that receives barcode information, obtains the product's online price, rating, and past transaction history, and provides them to the user's terminal.

[1314] 3. Database: A database for storing product information and managing online prices, ratings, and past transaction history for products corresponding to barcodes.

[1315] 4. Natural language processing model: A model that uses natural language processing technologies such as generative AI to generate product information in natural language and provide it to the user's device.

[1316] 5. Emotion Engine: An engine for recognizing user emotions.

[1317] As a specific embodiment, the invention is carried out in the following procedure.

[1318] 1. The user scans the product's barcode in a physical store using the device's camera.

[1319] 2. The terminal sends the barcode information to the server.

[1320] 3. The server searches the database for the product that corresponds to the barcode and retrieves online prices, reviews, and past transaction history.

[1321] 4. The server uses a natural language processing model to generate product information in natural language and provide it to the user's device.

[1322] 5. The server uses the emotion engine to recognize the user's emotions.

[1323] 6. The server appropriately customizes the generated product information based on the user's emotions and provides it to the user.

[1324] In this way, by implementing the present invention, a user can simply scan the barcode of a product while shopping at a brick-and-mortar store or on the go, and instantly check the product's online price, reviews, and past transaction history, and obtain information to support purchasing decisions. Furthermore, by combining it with an emotion engine, it is possible to recognize the user's emotions and customize product information accordingly.

[1325] The process flow will be explained below.

[1326] Step 1: The user scans the product barcode with the device's camera at the physical store.

[1327] -The user scans the product's barcode with the device's camera in a physical store.

[1328] Step 2: The terminal sends the barcode information to the server

[1329] -The terminal transmits the barcode information to the server.

[1330] Step 3: The server receives the barcode information and searches it in the database

[1331] -The server receives the barcode information and searches the database for the product that corresponds to the barcode.

[1332] Step 4: The server retrieves online prices, ratings, and past transaction history.

[1333] -The server retrieves the online price, reviews, and past transaction history of the product corresponding to the barcode from a database.

[1334] Step 5: The server generates information using a natural language processing model

[1335] -The server uses a natural language processing model (such as generative AI) to generate product information in natural language.

[1336] Step 6: The server uses the emotion engine to recognize the user's emotion.

[1337] -The server uses an emotion engine to recognize the user's emotions.

[1338] Step 7: Customize server-generated information based on user emotions

[1339] The server appropriately customizes the generated product information based on the user's emotions.

[1340] Step 8: The server provides customized information to the user's device.

[1341] The server provides customized product information to the user's terminal.

[1342] The above is an explanation of the program processing steps and specific operations.

[1343] Example 2

[1344] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the robot 414 is referred to as a "terminal."

[1345] Conventional information provision systems have difficulty in fully stimulating users' purchasing motivation because the information that can be obtained by scanning product barcodes is limited.In addition, it is not possible to provide information that takes into account the user's emotions and preferences, making it impossible to meet individual needs.

[1346] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving barcode information and acquiring the online price, evaluation, and past transaction history of the product corresponding to the barcode in the database, a means for providing product information in natural language, a means for generating information supporting purchase decisions using a generative model that utilizes natural language processing technology, a means for recognizing the user's emotions, and a means for customizing the generated information based on the user's emotions. This allows the user to diversify the information obtained from the scanned barcode and receive information provided that is customized according to emotions.

[1347] A "server" is a central processing unit that receives requests from users and processes data and provides information.

[1348] "Barcode information" is data contained in a barcode for identifying a product.

[1349] A "database" is a data management system that stores information in an organized format and makes it possible to search and retrieve it.

[1350] "Natural language processing technology" is a technology that enables computers to understand and generate human language.

[1351] A "generative model" is an artificial intelligence model that generates new text or content based on input data.

[1352] "Means for recognizing emotions" refers to technology that identifies a user's emotional state by analyzing facial expressions and voice data.

[1353] "Means for customizing information" refers to technology that adjusts the content of generated information based on the user's emotions and preferences, and provides it in accordance with individual needs.

[1354] "Online price" means the selling price of the product published on the Internet.

[1355] A "rating" is an opinion or rating given by a user or expert on the quality or performance of a product.

[1356] "Past transaction history" is data indicating the past buying and selling history and usage history of a product.

[1357] The product information presentation system according to the present invention functions by combining multiple hardware and software elements. To implement the system, a user terminal, a server, a database, a natural language processing model, and an emotion recognition engine are all required.

[1358] First, a user scans the barcode of a product at a physical store using a smartphone or a dedicated barcode scanner. The terminal receives the barcode information and sends it to the server. Specifically, a smartphone with a barcode reader application installed reads the barcode using its built-in camera and sends an HTTP POST request to the server.

[1359] The server searches the database based on the received barcode information to obtain the online price, reviews, and past transaction history of the corresponding product. The database management system used by the server includes MySQL and PostgreSQL. For example, the server issues the SQL query "SELECT FROM products WHERE barcode = '9781234567897'" to obtain the corresponding data.

[1360] Next, the server uses a generative AI model to generate information using natural language processing technology based on the acquired product information. This generative AI model is composed of an advanced generative AI such as ChatGPT, which receives the required prompt as input and generates natural language text. As a specific example, the server generates a prompt such as "This book has high reviews and the latest price is $20. It is also popular based on past transaction history," and passes it to the AI ​​model, generating natural sentences.

[1361] In addition, the server uses an emotion recognition engine to recognize the user's emotions. It analyzes facial images and voice data sent from the user's device to identify the user's emotional state. For example, if the user has a surprised expression, it will be recognized as "surprise." The server then customizes the generated product information based on the user's emotions, changing the expression to reflect the user's emotions, such as "This price is an amazing deal!"

[1362] Finally, the server sends the customized product information to the user's device and displays it. Users can instantly obtain product information in an easy-to-view format by simply scanning the barcode in a physical store. In addition, the customized information based on emotions can support more effective purchasing decisions.

[1363] Specific examples of prompt sentences include the following:

[1364] "This book has great reviews, a current price of $20, and past transaction history shows it's popular."

[1365] In this way, by implementing the present invention, it becomes possible to provide highly convenient information in a brick-and-mortar store or on the go, thereby improving the user's purchasing experience.

[1366] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1367] Step 1:

[1368] A user scans the product's barcode.

[1369] Specific operations: The user launches a barcode reader app on their smartphone and uses the camera to scan the barcode of the product.

[1370] Input: Item barcode.

[1371] Output: Barcode data (e.g. barcode number "9781234567897").

[1372] Step 2:

[1373] The terminal transmits the barcode information to the server.

[1374] Specific behavior: The barcode reader app reads the barcode and sends the barcode number to the server in an HTTP POST request.

[1375] Input: Barcode data.

[1376] Output: HTTP request containing the barcode data.

[1377] Step 3:

[1378] The server searches the database.

[1379] Specific operation: The server receives the barcode number and issues an SQL query to the database management system (e.g. MySQL, PostgreSQL). It executes the query "SELECT FROM products WHERE barcode = '9781234567897'".

[1380] Input: Barcode data.

[1381] Output: SQL query result (product information).

[1382] Step 4:

[1383] The server obtains the product information.

[1384] Specific operation: The server stores the product information (price, rating, past transaction history) obtained from the database in memory.

[1385] Input: Product information resulting from a SQL query.

[1386] Output: Product information held in memory (e.g. price "$20", rating "Highly Rated", transaction history "Popular").

[1387] Step 5:

[1388] The server generates the description using a natural language processing model.

[1389] Specific operation: The server passes the product information as a prompt to a generative AI model (e.g. ChatGPT) and generates natural language text, such as "This book has high reviews, the latest price is $20, and it is also popular based on past transaction history."

[1390] Input: Product information.

[1391] Output: The generated description (natural language text).

[1392] Step 6:

[1393] The server recognizes the user's emotions using an emotion engine.

[1394] Specific operation: Facial expression images and voice data sent from the user's device are passed to an emotion recognition engine on the server, which analyzes the user's emotional state.

[1395] Input: facial expression images and audio data.

[1396] Output: The perceived emotion (e.g. "surprise").

[1397] Step 7:

[1398] The server customizes the generated product information.

[1399] What happens: The server customizes the generated description based on the recognized emotional state, such as "This price is an amazing deal!"

[1400] Input: Generated description, recognized sentiment.

[1401] Output: A customized description.

[1402] Step 8:

[1403] The server transmits the customized product information to the user's terminal.

[1404] Specific operation: The server sends the customized explanation to the user's terminal as an HTTP response, and the terminal displays it.

[1405] Input: Customized description.

[1406] Output: Product information displayed on the user's device.

[1407] Through these steps, users can simply scan the barcode of a product to instantly check online prices, ratings, past transaction history, and get personalized information based on their sentiment.

[1408] (Application example 2)

[1409] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1410] When shopping in a brick-and-mortar store, it is difficult to instantly grasp the price, reviews, and past transaction history of a product. Also, when referring to information on the web, it takes a lot of time and effort to find information that is appropriate for oneself. In particular, when a user is in an emotional state, their judgment is easily influenced, which can lead to inefficient purchasing behavior.

[1411] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for receiving barcode information and acquiring the online price, evaluation, and past transaction history of the product corresponding to the barcode in the database, a means for providing the product information in natural language, a means for generating information supporting a purchase decision using an AI model using natural language processing technology, and a means for recognizing the user's emotion using emotion recognition technology and customizing the generated information based on the recognition result. This enables a user to obtain customized information corresponding to their individual emotional state in real time by simply scanning a product in a physical store.

[1412] "Barcode information" refers to data of an identification code attached to a product, and is information that can identify the product by scanning it.

[1413] "Online price" means the selling price of a Product offered on the Internet.

[1414] "Rating" refers to reviews and scores of products by users and experts, and is information that reflects opinions about the quality and usability of the product.

[1415] "Past transaction history" refers to information regarding past purchases and sales of the product.

[1416] A "natural language" is a language that is used daily by humans and that can be analyzed and generated by machines.

[1417] "Natural language processing technology" is a technology that enables computers to understand, interpret, and manipulate text data.

[1418] An "AI model" is a mathematical model that utilizes artificial intelligence algorithms to perform specific tasks.

[1419] "Emotion recognition technology" is a technology that determines a user's emotional state by analyzing their facial expressions and tone of voice.

[1420] "Customization" means adapting or modifying content or information to fit a particular need or situation.

[1421] A "server" is a computer system that receives and processes requests from clients over a network and provides information.

[1422] "Terminal" refers to a device that is directly operated by a user, and includes, for example, a smartphone or smart glasses.

[1423] The system for implementing the present invention includes a server, a terminal, and a database. The purpose is to enable users to scan products while shopping in a brick-and-mortar store and instantly provide online information and customized advice. The specific configuration and operation are shown below.

[1424] System Configuration

[1425] 1. Server:

[1426] How to receive barcode information:

[1427] The server receives the barcode information sent by the terminal. This is done using an HTTP request.

[1428] Ways to retrieve information from the database:

[1429] The server searches a database based on the received barcode information to obtain the online price, reviews, and past transaction history of the corresponding product.

[1430] Means of generating information in natural language:

[1431] The server uses natural language processing technology (e.g., the ChatGPT model from the Transformers library) to generate the acquired product information in natural language.

[1432] Ways to use emotion recognition technology to personalize communications:

[1433] Use an emotion recognition engine (e.g., the EmotionRecognition library) to recognize the user's emotions and customize the generated information based on the results.

[1434] 2. Terminal:

[1435] How users can scan items:

[1436] Using the camera on the device (a smartphone or smart glasses), the user scans the barcode of the product.

[1437] Methods for sending information to the server:

[1438] The scanned barcode information is sent from the terminal to the server.

[1439] To view the information obtained:

[1440] The product information and customization information generated in natural language and obtained from the server are displayed on the user's terminal.

[1441] Hardware and Software

[1442] Hardware: Smartphones, smart glasses, servers

[1443] Software: OpenCV (barcode scanning), Requests (API communication), Transformers (natural language generation), EmotionRecognition (emotion recognition)

[1444] Examples

[1445] Assume the user is in a brick-and-mortar store and scans the barcode on their laptop. The server receives the barcode information and inputs the following prompt sentence into the natural language generation model:

[1446] Example prompt:

[1447] "Product description: This amazing laptop is packed with the latest features, priced at $798 and has a user rating of 4.5."

[1448] A natural language generation model generates a detailed product description based on the prompt. An emotion recognition engine then analyzes the user's facial expressions and tone of voice to determine their emotions. For example, if the system detects that the user is anxious, the product description will include additional information to reassure the user.

[1449] This allows users to simply scan a product in a physical store and obtain real-time information on not only the price and reviews but also customized information based on their own emotions, making purchasing decisions easier.

[1450] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1451] Step 1:

[1452] The user scans the product's barcode using the device's camera.

[1453] Input: Product barcode captured by the device camera

[1454] Output: Barcode data

[1455] Specific operation: Activate the device's camera and use the OpenCV library to read the barcode and extract the barcode data. Once the barcode data is obtained, proceed to the next step.

[1456] Step 2:

[1457] The terminal transmits the acquired barcode data to the server.

[1458] Input: Barcode data

[1459] Output: HTTP request to the server

[1460] Specific operation: The terminal sends the acquired barcode data to the server as an HTTP request. The request includes the barcode data.

[1461] Step 3:

[1462] The server receives the barcode information and searches a database to obtain the corresponding product information.

[1463] Input: Barcode data

[1464] Output: Product information (price, rating, past transaction history)

[1465] Specific operation: The server searches the database based on the received barcode data. If the appropriate product information is found, it retrieves the information and proceeds to the next step.

[1466] Step 4:

[1467] Based on the product information acquired by the server, a product description is generated in natural language using a generative AI model.

[1468] Input: Product information

[1469] Output: A product description generated in natural language

[1470] Specific operation: The server inputs product information as a prompt sentence into a generative AI model (e.g., Transformers' ChatGPT model). The model generates a product description in natural language based on the information and outputs the result.

[1471] Step 5:

[1472] In order for the server to recognize the user's emotions, it uses emotion recognition technology to analyze facial and voice data.

[1473] Input: User facial and voice data

[1474] Output: The user's emotional state.

[1475] How it works: The user provides facial expressions and voice using a dedicated camera or microphone. The server analyzes this data using the EmotionRecognition library and recognizes the user's emotional state.

[1476] Step 6:

[1477] The server generates customized product information based on the generated product description and the user's emotional state.

[1478] Input: Natural language generated product description, user's emotional state

[1479] Output:Customized product information

[1480] How it works: The server adds or adjusts the generated product description depending on the user's recognized emotional state. For example, if the user is in an anxious state, the server adds information that emphasizes the reliability and guarantees of the product.

[1481] Step 7:

[1482] The device displays the customized product information.

[1483] Input:Customized product information

[1484] Output: Product information displayed to the user

[1485] Specific operation: The user's device (smartphone or smart glasses) receives the customized product information sent from the server and displays it on the screen. This allows the user to obtain information that suits their individual needs and emotional state.

[1486] These steps allow users to obtain real-time relevant information about the products they scan in physical stores and receive customized purchasing support information based on their emotions.

[1487] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1488] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1489] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the robot 414.

[1490] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[1491] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1492] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1493] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1494] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[1495] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[1496] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[1497] Although the system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, the system according to the present disclosure is not necessarily implemented in a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program that runs on a personal computer, or an application that runs on a smartphone or the like. The method according to the present disclosure may be provided to a user in the form of SaaS (Software as a Service).

[1498] In the above embodiment, an example is given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to input data.

[1499] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1500] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[1501] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1502] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[1503] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[1504] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1505] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[1506] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[1507] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[1508] The following is further disclosed regarding the above embodiment.

[1509] (Claim 1) a server means for receiving the barcode information and retrieving online prices, ratings, and past transaction history of the product corresponding to the barcode in a database; A means for providing information about the product in a natural language by the server; A means for generating information supporting purchase decisions by using an AI model that utilizes natural language processing technology in the server; A product information presentation system comprising:

[1510] (Claim 2) A means for scanning the barcode of the product on a user's terminal; means for transmitting information of the barcode to the server; 2. The product information presentation system according to claim 1,

[1511] (Claim 3) A means for the server to search a database for a product corresponding to the barcode; 3. The product information presentation system according to claim 1 or 2, comprising:

[1512] (Claim 4) 2. The product information presentation system according to claim 1, further comprising an emotion engine for recognizing user emotions.

[1513] (Claim 5) 3. The product information presentation system according to claim 2, further comprising an emotion engine for recognizing user emotions.

[1514] (Claim 6) 4. The product information presentation system according to claim 3, further comprising an emotion engine for recognizing user emotions.

[1515] "Example 1" (Claim 1) A means for a user to scan a barcode of an item using a terminal; A means for the terminal to transmit scanned barcode information to a server; A means for the server to search a database for the received barcode information and obtain online prices, ratings, and past transaction history of the product corresponding to the barcode; A means for the server to use a generative AI model to convert the acquired product information into natural language; A means for the server to generate information supporting a purchase decision using the generative AI model and provide the information to the terminal; means for the terminal to display the provided information to a user; A system including:

[1516] (Claim 2) A means for a user's terminal to scan a barcode of a product and transmit the barcode information to a server; A means for the server to generate information on the acquired products using natural language processing technology and provide the information to the terminal; 2. The system of claim 1, comprising:

[1517] (Claim 3) A means for the server to search for products corresponding to the barcode in the database and input the acquired information into a generative AI model to perform natural language processing; 2. The system of claim 1, comprising:

[1518] "Application example 1" (Claim 1) a server means for receiving the barcode information and retrieving online prices, ratings, and past transaction history for the item corresponding to the barcode in a database; A means for providing information about the item in a natural language by the server; The server generates information to support purchase decisions using a generative AI model that uses natural language processing technology; The server has a means for generating a prompt sentence for a generative AI model; means for the server to provide advice and offers to the user based on the generated prompt sentence; means for enabling a user to scan an item in a physical store and access online prices, ratings, and past transaction history; A system including:

[1519] (Claim 2) A means for a user's terminal to scan a barcode of the item and transmit the barcode information to the server; means for returning information about the item in natural language; 2. The system of claim 1, comprising:

[1520] (Claim 3) A means for the server to search a database for an item corresponding to the barcode and obtain product information; A means for providing a prompt sentence to a generative AI model based on the acquired product information; 3. The system of claim 1 or claim 2, comprising:

[1521] "Example 2 of combining emotion engines" (Claim 1) a server means for receiving the barcode information and retrieving online prices, ratings, and past transaction history of the product corresponding to the barcode in a database; A means for providing information about the product in a natural language by the server; A means for generating information supporting purchase decisions by using a generative model that utilizes natural language processing technology in the server; The server has a means for recognizing a user's emotion; A means for customizing the generated information based on the user's emotions, A system including:

[1522] (Claim 2) A means for scanning the barcode of the product on a user's terminal; means for transmitting information of the barcode to the server; 2. The system of claim 1.

[1523] (Claim 3) A means for the server to search a database for a product corresponding to the barcode; 2. The system of claim 1.

[1524] "Application example 2 when combining emotion engines" (Claim 1) a server means for receiving the barcode information and retrieving online prices, ratings, and past transaction history of the product corresponding to the barcode in a database; A means for providing information about the product in a natural language by the server; A means for generating information supporting purchase decisions by using an AI model that utilizes natural language processing technology in the server; A means for the server to recognize the user's emotion using emotion recognition technology and customize the generated information based on the recognition result; A system including:

[1525] (Claim 2) A means for scanning the barcode of the product on a user's terminal; means for transmitting information of the barcode to the server; A means for displaying the information obtained from the server on a user's smartphone or smart glasses; A means for providing customized product information based on the user's emotion in the display; 2. The system of claim 1, comprising:

[1526] (Claim 3) A means for the server to search a database for a product corresponding to the barcode; 3. The system of claim 1 or claim 2, comprising: [Explanation of symbols]

[1527] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a server means for receiving the barcode information and retrieving online prices, ratings, and past transaction history of the product corresponding to the barcode in a database; A means for providing information about the product in a natural language by the server; The server generates information to support purchase decisions using a generative AI model that utilizes natural language processing technology; The server has a means for recognizing a user's emotion; A means for customizing the generated information based on the user's emotions, A product information presentation system comprising:

2. A means for scanning the barcode of the product on a user's terminal; means for transmitting information of the barcode to the server; The product information presentation system according to claim 1 ,

3. A means for the server to search a database for a product corresponding to the barcode; The product information presentation system according to claim 1 or 2, further comprising:

Images