System

Abstract

A system is provided.SOLUTION: A system for providing information corresponding to a store, comprising: means for registering specific information of the store in a database; means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model; means for AR-displaying the generated virtual character; means for allowing the virtual character to respond to a user's inquiry in an interactive manner; and means for giving points corresponding to use to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional information provision systems using QR codes (registered trademark) have limitations in how easily users can obtain and understand store information. Specifically, static text and images alone cannot adequately convey the depth and appeal of the information, resulting in a poor user experience. Furthermore, interaction between stores and users is limited, making it difficult to improve user satisfaction and repeat visits. This calls for a method to stimulate interaction between stores and users and provide new value. [Means for solving the problem]

[0005] To address the above-mentioned issues, the present invention provides a system that generates a virtual character corresponding to a store when the QR code of the store is scanned with a smartphone. The system includes a means for registering unique store information in a database, a means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, and a means for displaying the generated virtual character in AR. The system also includes a means for the virtual character to respond to user inquiries in an interactive format and a means for awarding points to the user based on usage. This allows users to obtain information in a visually easy-to-understand format and gain a deeper understanding through the interaction. Furthermore, awarding points encourages users to continue using the system, and by offering new services and benefits when certain conditions are met, user satisfaction can be increased.

[0006] "Store-specific information" refers to unique information about products, services, features, etc. related to a particular store.

[0007] "Database" refers to an information system that systematically stores store-specific information and allows access and retrieval as needed.

[0008] A "large-scale language model" is an artificial intelligence model that is trained based on massive amounts of data, and refers to technology that can generate text and perform natural language processing.

[0009] A "virtual character" is a digitally generated character that has specific information and roles and that can interact and provide guidance.

[0010] "AR display" refers to a technology that uses augmented reality technology to superimpose virtual objects onto the actual environment.

[0011] "Dialogue" refers to a two-way communication between a user and a system, involving a question-and-answer exchange.

[0012] "Points" are units of reward awarded based on user behavior and usage, and can be exchanged for special offers or new services by meeting certain conditions.

[0013] "New services" refer to services that are provided in addition to existing services or products and that are novel and valuable to users. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The system of the present invention works by scanning a store's QR code with a smartphone, generating a virtual character that corresponds to the store and provides information about products and services to the user. This system is realized through the interaction of three parties: a server, a terminal, and a user.

[0036] Server processing

[0037] 1. Generate a QR code and register store information

[0038] The server generates a unique QR code for each store.

[0039] The unique information of the store accessed through this QR code (product information, service information, store features, etc.) is registered in a database.

[0040] 2. AI character generation

[0041] When a user scans a QR code, the server retrieves the store's information from the database.

[0042] The server uses a large-scale language model to generate a virtual character from the acquired store information.

[0043] Processing by device (smartphone)

[0044] 1. Scan the QR code

[0045] The user scans the store's QR code with the device's camera.

[0046] 2. Displaying an AI character and starting a conversation

[0047] Information about the AI ​​character generated from the server is received and displayed in AR on the device.

[0048] The characters provide information interactively in response to the user's questions and requests.

[0049] User Action

[0050] 1. Scan the QR code

[0051] The user scans the store's QR code with a smartphone app.

[0052] After scanning, the store information is sent to a server, which generates a virtual character and sends that information back to the device.

[0053] 2. Dialogue and Questions

[0054] Users ask the AI ​​character questions about products and services.

[0055] For example, if you ask, "What drink do you recommend?" the character will respond with something like, "I recommend the lavender latte."

[0056] Specific examples

[0057] In the case of "Cafe La Pearl"

[0058] 1. Scan the QR code

[0059] When a user scans the Cafe La Pearl QR code with their smartphone, the device sends the information to the server.

[0060] The server obtains information specific to Cafe La Pearl (e.g., products include lavender latte, espresso, and matcha latte) and generates a virtual character.

[0061] 2. Character Display

[0062] The generated virtual character is displayed on the user's smartphone.

[0063] When the user asks, "What drink would you recommend?" the character responds, "I recommend the elegant lavender latte. It has a calming scent."

[0064] Earn points and get new services

[0065] 1. Earn points when you use the service

[0066] By using this system, users can earn points, which are awarded based on their actions and usage and are managed on their devices.

[0067] For example, users can earn points by interacting with characters or spending a lot of time in the store.

[0068] 2. Offer of Benefits

[0069] Depending on the points accumulated, new services will be offered when certain conditions are met. For example, if you reach 50 points, you will receive a free drink on your next visit.

[0070] This will increase the repeat rate of users and enhance customer satisfaction.

[0071] This invention promotes interactive exchanges between stores and users, improving the user experience. In particular, it achieves a high level of interactivity by combining the generation of virtual characters using a large-scale language model, display using AR technology, and a point system. This system also provides great benefits to stores, as it allows users to gain a deeper understanding of store information and encourages continued use.

[0072] The processing flow will be explained below.

[0073] Step 1:

[0074] The server generates a unique QR code for each store, which contains a specific store ID and allows access to the store's specific information. The QR code is then stored in a database.

[0075] Step 2:

[0076] The server registers the product information, service information, and store features provided by the store in a database, allowing the information to be quickly retrieved when the QR code is scanned later.

[0077] Step 3:

[0078] A user scans a store's QR code using the camera on their smartphone. The QR code contains the store's ID, which the user's smartphone reads.

[0079] Step 4:

[0080] The user's device sends a request to the server containing the read store ID, which may also contain the user's identification information.

[0081] Step 5:

[0082] The server retrieves the store's unique information from the database based on the received store ID, including the store name, product list, and service details.

[0083] Step 6:

[0084] The server uses a large-scale language model (LLM) to generate virtual characters based on the acquired store information. The characters reflect the store's atmosphere and characteristics.

[0085] Step 7:

[0086] The generated virtual character data is sent to the user's device, including the character's visual data and dialogue script.

[0087] Step 8:

[0088] The user's device displays the received virtual character in AR, allowing the user to see the virtual character appear in real space through their smartphone.

[0089] Step 9:

[0090] Users can ask the virtual character questions about products and services, such as "What drink do you recommend?"

[0091] Step 10:

[0092] The device processes the user's question using voice recognition or text input, sends the question to the server, or may respond locally if it can be processed locally.

[0093] Step 11:

[0094] The server analyzes the questions sent by the user and generates appropriate answers using pre-defined dialogue scripts and large-scale language models, which are then returned to the terminal.

[0095] Step 12:

[0096] The user's device displays the answer received from the server through a virtual character. For example, the character may respond by voice or text, saying, "I recommend a lavender latte."

[0097] Step 13:

[0098] Users can enjoy an interactive experience by interacting with the virtual character and asking additional questions or requests as needed.

[0099] Step 14:

[0100] The server awards users points based on their usage of the system. The points are recorded in the user's account, and new services and benefits are offered when certain conditions are met.

[0101] Step 15:

[0102] Users can use the points they have accumulated to receive new services and benefits, such as a free drink on their next visit when they collect 50 points.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] Conventional store information systems provided information to users in a one-way manner, making it difficult to flexibly respond to the needs of individual users. Furthermore, when store information was updated, there was a lack of a way to notify users in a timely manner. Furthermore, incentives to increase user frequency of use were not effectively provided, making it difficult to increase repeat visits.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in augmented reality, means for a user to read a store's QR code using a mobile device, means for transmitting the read QR code information to the server, means for the virtual character to respond to the user's inquiries in an interactive format, and means for awarding points to the user based on usage. This allows users to obtain information tailored to their needs in real time, improving the convenience of store use through interactive exchanges. Furthermore, utilizing a point system can increase the rate of user repeat visits.

[0108] A "server" is a computer system or device that provides services to clients over a network.

[0109] A "database" is a system or software that stores and manages data in an organized manner, and later extracts and uses it efficiently.

[0110] A "large-scale language model" is an artificial intelligence model that is trained based on large amounts of text data and has the ability to automatically generate text and understand meaning.

[0111] A "virtual character" is a computer-generated, computer-generated or text-based character used to interact with a user.

[0112] "Augmented reality display" is a technology that displays digital information overlaid on the real environment, making it possible to merge the real world with the virtual world.

[0113] A "mobile terminal" is a small, portable electronic device, including a smartphone, tablet, or mobile phone.

[0114] A "QR code" is a type of two-dimensional barcode that displays information in a matrix format, allowing for high-speed reading.

[0115] "Points" are units of reward awarded based on user behavior and frequency of use, and have value that can be exchanged for specific benefits or privileges.

[0116] MODE FOR CARRYING OUT THE INVENTION

[0117] The system of the present invention scans a store's QR code with a mobile device, and generates a virtual character corresponding to the store, which then guides the user through its products and services. The system is realized through the interaction of three parties: a server, a terminal, and a user.

[0118] Server processing

[0119] The server first uses Python's qrcode library to generate a unique QR code for each store. This QR code is generated by embedding the store's unique ID as data. The generated QR code is saved as an image file. Next, the server registers the store's unique information in a database such as MySQL (registered trademark). The registered information includes the store name, product list, and service details.

[0120] When a user scans a QR code, the server retrieves the store information corresponding to that QR code from a database. Based on the retrieved information, a virtual character is generated using a large-scale language model (e.g., OpenAI's GPT-3 (registered trademark)). Specifically, a prompt sentence is created and sent to the AI ​​to generate the virtual character.

[0121] Processing by device (smartphone)

[0122] Users can use the react-native-camera library in React Native to scan a store's QR code using their mobile device's camera. The scanned QR code information is then sent from the device to the server. The generated virtual character information received from the server is then displayed in augmented reality on the device. This is achieved by combining software such as Unity and ARFoundation.

[0123] User Action

[0124] Users scan the QR code using an app on their mobile device. After scanning, store information is sent to the server. The information about the virtual character returned by the server is displayed on the device. Users can ask the virtual character questions about products and services. For example, if they ask, "What drink do you recommend?", the virtual character will respond, "I recommend the lavender latte."

[0125] Earn points and receive rewards

[0126] Each time a user uses the system, they are awarded points based on their behavior and frequency of use. Points are managed on the device and added up based on the user's behavior. When a certain number of points are accumulated, they are given special benefits that meet certain conditions, such as a free drink on their next visit.

[0127] For example, the prompt is:

[0128] You are a virtual character at Cafe La Pearl. What drink would you recommend to our customers?

[0129] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0130] Step 1: Generate a QR code

[0131] The server uses Python's qrcode library to generate a unique QR code for each store. The input is the store's unique ID, and the output is an image file of the QR code. Specifically, the server embeds the store ID as QR code data and saves the QR code as an image file.

[0132] Step 2: Register your store information

[0133] The server registers store-specific information in a database such as MySQL. The input is store information (product list, service details, store features, etc.), and the output is the information stored in the database. Specifically, the server executes SQL queries to insert information such as products, services, and features into the database.

[0134] Step 3: Scan the QR code

[0135] A user scans a store's QR code using the camera on their smartphone. The input is the QR code image captured by the camera, and the output is the QR code data. Specifically, the user opens the smartphone app, points the camera at the QR code, and scans it to obtain the data.

[0136] Step 4: Send QR code data to the server

[0137] The device sends the data read from the scanned QR code to the server. The input is the QR code data, and the output is the QR code information sent to the server. Specifically, the device sends the read QR code data to the server as an HTTP request.

[0138] Step 5: Get store information

[0139] The server retrieves store information from a database based on the received QR code data. The input is the QR code data (store ID), and the output is the store information retrieved from the database. Specifically, the server executes a database query using the store ID to retrieve information about the corresponding store.

[0140] Step 6: Generate a virtual character

[0141] The server uses a large-scale language model (e.g., OpenAI's GPT-3) to generate a virtual character from the acquired store information. The input is the store information and a prompt sentence for the generating AI model, and the output is the response of the generated virtual character. Specifically, the server creates a prompt sentence based on the store information and sends a request to the AI.

[0142] Step 7: Sending information about your virtual character

[0143] The server sends information about the generated virtual character to the terminal. The input is the response of the generated virtual character, and the output is the character information sent to the terminal. Specifically, the server sends the response of the virtual character to the terminal as an HTTP response.

[0144] Step 8: AR display of virtual characters

[0145] The device displays the received virtual character in augmented reality. The input is information about the virtual character, and the output is the virtual character displayed on the device. Specifically, Unity and ARFoundation are used to display the virtual character on the screen.

[0146] Step 9: Interact with the user

[0147] The user asks a question to a virtual character displayed in augmented reality. The input is the user's question, and the output is the response from the virtual character. For example, if the user asks, "What drink do you recommend?", the virtual character will respond, "I recommend a lavender latte."

[0148] Step 10: Points awarded

[0149] The server awards points based on the user's usage. The input is the user's behavioral data (frequency and content of interactions), and the output is the awarded points. Specifically, the server records the user's usage, calculates points based on that, and stores them in a points database.

[0150] Step 11: Offering rewards under certain conditions

[0151] The server provides new services and benefits when the awarded points meet certain conditions. The input is the user's point data, and the output is the benefits to be provided. Specifically, the server checks the user's points, and if the conditions are met, generates a reward coupon or service and notifies the user.

[0152] (Application example 1)

[0153] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0154] Currently, physical stores have limited methods for providing guidance and services to users, and there is a lack of effective means for communicating in-store information and product services. Furthermore, when users visit a store to obtain information, it is difficult to provide interactive and personalized support, making it difficult to improve customer satisfaction. Furthermore, there is a lack of methods for providing incentives based on user usage, making it difficult to increase repeat visit rates.

[0155] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0156] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in AR, means for the virtual character to respond to user inquiries in an interactive format, means for awarding points to the user according to usage, and means for accessing the server and generating a virtual character by reading a QR code in the store. This enables interactive and personalized information provision and service guidance in physical stores, thereby improving customer satisfaction and repeat business rates.

[0157] A "system for providing store-specific information" is a system that effectively guides users to information and services provided by stores.

[0158] "Store-specific information" refers to data such as product information, service information, and store characteristics that are unique to each store.

[0159] A "database" is a system or structure for storing and managing information in an organized manner.

[0160] A "large-scale language model" is an advanced AI model based on machine learning that has the ability to understand and generate human language.

[0161] A "virtual character" is a digitally generated character that can be interacted with.

[0162] "AR display" refers to the use of augmented reality technology to overlay digital information and objects onto the real world.

[0163] "Means of responding to user inquiries" refers to a function that enables a virtual character to provide appropriate responses to questions or requests from users.

[0164] "Means of awarding points to users according to usage" refers to a function that accumulates points each time a user uses the system, and manages and awards those points.

[0165] "Means for reading QR codes in stores" refers to the function that allows users to use a device such as a smartphone to scan QR codes installed in stores.

[0166] A "server" is a centralized computer system that processes and provides information over a network.

[0167] The system of the present invention functions through the mutual cooperation of three parties: a server, a terminal (smartphone), and a user. This system uses the following hardware and software.

[0168] Server processing

[0169] The server has the function of collecting store-specific information and registering it in a database. For example, the database stores product information, service information, and distinctive features of the store. MySQL is a suitable database to use. The Python library qrcode is used to generate QR codes, which are then distributed to each store.

[0170] When a user scans a QR code inside a store, the server uses a large-scale language model (e.g., OpenAI GPT-4 (registered trademark)) to generate a virtual character corresponding to the store. This character guides the user about products and services based on the store's unique information.

[0171] Processing by device (smartphone)

[0172] When a user scans a QR code in a store using their smartphone camera, the information in the QR code is read by the device. Using an AR library such as Vuforia, the device displays a virtual character overlaid on the real world. Unity can be used to display and interact with the AR character.

[0173] When interacting with a virtual character, the server generates a response based on the user's input and sends it to the device. For example, when a user asks, "What products do you recommend?", the smartphone sends the request to the server, which then queries the generative AI model based on the received information and provides an appropriate response to the device.

[0174] User Action

[0175] Users use a smartphone application to scan a QR code in a store, which sends store information to a server and generates a virtual character.

[0176] Users can then interact with the virtual character. For example, if they ask, "What drink do you recommend?" the character will respond in real time with, "I recommend the elegant lavender latte. It has a calming aroma." By continuing to interact, users can also accumulate points, which can be used to receive special benefits.

[0177] Specific examples

[0178] As a concrete example, consider the case where a user scans a QR code for a particular cafe. When the QR code is read, the server generates a virtual character based on the cafe's menu and features, and the character appears on the smartphone. When the user asks, "What's the recommendation this week?", the character responds, "This week's recommendation is our special chocolate cake."

[0179] Example prompts to input to a generative AI model:

[0180] Please generate lines for your virtual character based on the following information.

[0181] Store name: A specific cafe

[0182] Offerings: Chocolate Cake, Lavender Latte, Matcha Latte

[0183] Character's role: Store guide

[0184] User asks: "What's your pick this week?"

[0185] Character Quotes: <Generated quote goes here>

[0186] Hardware and Software Examples

[0187] Server: The database uses MySQL, and the Python qrcode library is used to generate QR codes.

[0188] Device: Scan the QR code with your smartphone camera. AR display will be performed using Unity and Vuforia.

[0189] Generative AI model: Dialogue content is generated using OpenAI GPT-4.

[0190] With the above configuration, the present invention makes it possible to provide interactive and personalized information and service guidance in brick-and-mortar stores, thereby improving customer satisfaction and repeat visit rates.

[0191] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0192] Step 1:

[0193] The server registers the store's unique information in a database. In this process, product information, service information, and characteristic features provided by the store are stored in a MySQL database. The specific input is the store's unique information, and the output is a database record in which this information is registered.

[0194] Step 2:

[0195] The server generates a unique QR code for each store. This QR code is generated using the Python library qrcode. The input is the store ID and URL, and the output is the generated QR code image file. This QR code is then distributed to each store.

[0196] Step 3:

[0197] A user uses the camera on their device (smartphone) to read a QR code in a store. The input is the QR code, and the output is a URL or identifier extracted from the QR code.

[0198] Step 4:

[0199] The device sends the information of the scanned QR code to the server. The input is the URL or identifier obtained from the QR code, and the output is that information is sent to the server.

[0200] Step 5:

[0201] The server retrieves the unique information of the corresponding store from the database based on the information in the received QR code. The input is the QR code identifier, and the output is the information of the corresponding store.

[0202] Step 6:

[0203] The server generates a virtual character using a large-scale language model (e.g., OpenAI GPT-4). In this step, a prompt is generated based on the acquired store information and sent to the AI ​​model. The input is the store's unique information and the generated prompt, and the output is the lines and movement information of the generated virtual character.

[0204] Step 7:

[0205] The server transmits information about the generated virtual character to the terminal. The input is the information about the generated virtual character, and the output is the transmission of this information to the terminal.

[0206] Step 8:

[0207] The device uses Unity and Vuforia to display AR based on the received virtual character information. The input is virtual character information, and the output is an AR character superimposed on the real world.

[0208] Step 9:

[0209] The user asks a question or makes a request to the virtual character, for example, "What products do you recommend?" The input is the user's question or request, and the output is the text data.

[0210] Step 10:

[0211] The terminal sends the user's question to the server. The input is the user's question text, and the output is that information is sent to the server.

[0212] Step 11:

[0213] The server receives the user's question and generates an appropriate response based on the generative AI model. The input is the user's question text, and the output is the AI-generated response text.

[0214] Step 12:

[0215] The server sends the generated response text to the terminal. The input is the generated response text, and the output is that information is sent to the terminal.

[0216] Step 13:

[0217] The terminal conveys the received response text to the user through the virtual character. The input is the response text, and the output is the voice of the virtual character or a text display.

[0218] Step 14:

[0219] The server awards points according to the user's usage. These points are updated in the database and managed for the user's use. The input is the user's usage data, and the output is the updated point data.

[0220] The above are the specific processing steps of the system for implementing the present invention.

[0221] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0222] This invention is a system in which a virtual character corresponding to a store is generated by scanning the store's QR code with a smartphone, and the virtual character guides the user to the store's products and services. The invention also combines an emotion engine that recognizes the user's emotions. This system is realized through interaction between the server, the terminal, and the user.

[0223] Server processing

[0224] 1. Generate a QR code and register store information

[0225] The server generates a unique QR code for each store, which contains a specific store ID and can be used to access store-specific information.

[0226] The server registers the product information, service information, and store features provided by the store in a database.

[0227] 2. AI character generation

[0228] When a user scans a QR code, the server retrieves the store's information from the database.

[0229] The server uses a large-scale language model to generate a virtual character that corresponds to the store from the acquired store information.

[0230] 3. Equipped with an emotional engine

[0231] The virtual character is equipped with an emotion engine that recognizes the user's emotions by analyzing the user's facial expressions, tone of voice, and choice of words.

[0232] Processing by device (smartphone)

[0233] 1. Scan the QR code

[0234] The user scans the store's QR code with the device's camera.

[0235] 2. Displaying an AI character and starting a conversation

[0236] The AI ​​character information sent from the server is received and displayed in AR on the device, allowing users to see the virtual character appear in real space through their smartphone.

[0237] The characters provide information interactively in response to the user's questions and requests.

[0238] 3. Emotion recognition and response changes

[0239] The emotion engine analyzes the user's emotional state, and the virtual character's responses change based on the analysis results.

[0240] For example, if the user looks tired, the character will make special suggestions such as, "Why don't you take a short break?"

[0241] User Action

[0242] 1. Scan the QR code

[0243] Users scan the store's QR code with their smartphone, after which the store's information is sent to a server and a virtual character is generated.

[0244] 2. Dialogue and Questions

[0245] Users can ask the virtual character questions about products and services, such as, "What drink do you recommend?"

[0246] The virtual character provides the store with the most relevant information and responds appropriately to questions.

[0247] 3. Specialized responses based on emotions

[0248] Users can receive special suggestions and responses based on their emotional state. For example, if a user is happy, they may receive suggestions such as, "Would you like to try our special dessert?"

[0249] Specific examples

[0250] In the case of "Cafe La Pearl"

[0251] 1. Scan the QR code

[0252] The user scans the "Cafe La Pearl" QR code with their smartphone, and the device sends the information to the server.

[0253] The server obtains information specific to Cafe La Pearl (for example, product information such as lavender latte, espresso, and matcha latte) and generates a virtual character based on that information.

[0254] 2. Character Display

[0255] The generated virtual character is displayed in AR on the user's smartphone. When the user asks, "What drink do you recommend?", the character responds, "I recommend the elegant lavender latte."

[0256] 3. Emotion recognition and response

[0257] The character analyzes the user's facial expressions and tone of voice, and if it determines that the user is tired, it will suggest, "Would you like me to show you a sofa seat where you can relax?"

[0258] Earn points and get new services

[0259] 1. Earn points when you use the service

[0260] By using this system, users earn points, which are recorded in their account and, when certain conditions are met, can be used to receive new services and benefits.

[0261] 2. Offer of Benefits

[0262] New services and benefits will be offered depending on the points accumulated. For example, 50 points will give you a free drink on your next visit.

[0263] This invention promotes interactive exchanges between stores and users, improving the user experience. In particular, by using virtual characters equipped with an emotion engine, it is possible to provide optimal services according to the user's emotional state. This system also provides great benefits to stores, as it allows users to gain a deeper understanding of store information and encourages continued use.

[0264] The processing flow will be explained below.

[0265] Step 1:

[0266] The server generates a unique QR code for each store. The generated QR code contains a specific store ID, which allows access to the store's unique information. After generation, this QR code and basic store information are stored in a database.

[0267] Step 2:

[0268] The server registers the product information, service information, and detailed store features provided by the store in a database, allowing the necessary information to be quickly retrieved when the QR code is scanned later.

[0269] Step 3:

[0270] The user uses the smartphone camera to scan the store's QR code, which then reads the store ID contained in the QR code and stores it on the user's device.

[0271] Step 4:

[0272] The user's device sends a request to the server containing the scanned store ID, which may also contain the user's identification information (e.g., user ID).

[0273] Step 5:

[0274] The server retrieves the store's unique information from the database based on the store ID received from the terminal, including the store name, product list, and service details.

[0275] Step 6:

[0276] The server uses a large-scale language model (LLM) to generate a virtual character based on the acquired store information, which reflects the store's atmosphere and characteristics.

[0277] Step 7:

[0278] Once the virtual character data is generated, the server connects the data to the emotion engine, which contains the algorithms and necessary data for recognizing the user's emotions.

[0279] Step 8:

[0280] The generated virtual character and emotion engine data are sent to the user's device, including the character's visual data, dialogue scripts, and emotion recognition algorithms.

[0281] Step 9:

[0282] The user's device displays the received virtual character in AR, allowing the user to see the virtual character in real space through the smartphone screen.

[0283] Step 10:

[0284] The user asks the virtual character a question about a product or service. For example, the user might ask, "What drink do you recommend?"

[0285] Step 11:

[0286] The user's device analyzes the user's question using voice recognition or text input, and sends the question content to the server. The emotion engine also analyzes the user's facial expressions and tone of voice, and sends this data to the server as appropriate.

[0287] Step 12:

[0288] The server receives and analyzes the user's question and transmitted emotion data, and uses a large-scale language model and emotion engine to generate appropriate answers and emotion-based responses to the user's question.

[0289] Step 13:

[0290] The server then sends the generated answers and responses to the user's device, including text and voice responses to questions, as well as special emotional responses.

[0291] Step 14:

[0292] The user's device displays the received answer through a virtual character. For example, the character may respond, "I recommend a lavender latte," and even make special suggestions, such as, "You look tired. A lavender latte will help you relax."

[0293] Step 15:

[0294] The server awards users points based on their usage of the system, which are recorded in their accounts and can be used to receive new services or rewards when certain conditions are met.

[0295] Step 16:

[0296] Users can use their accumulated points to receive new services and benefits. For example, 50 points will give you a free drink on your next visit.

[0297] Example 2

[0298] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0299] Conventional store guide systems lacked interactive methods for users to gain a deeper understanding of store information, resulting in a limited user experience. Furthermore, services were not provided that took user emotions into consideration, making it difficult to meet individual needs. Furthermore, point systems and rewards based on usage were insufficient, and there was a lack of methods to encourage continued use.

[0300] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0301] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in augmented reality, means for a user to read a QR code using a camera on a terminal and transmit the information to the server, means for the virtual character to recognize the user's emotions and adjust responses based on the recognition results, and means for awarding points to the user based on usage. This allows users to interactively acquire store information and receive optimal services based on their emotions. Furthermore, the point system encourages continued usage and improves the user experience.

[0302] "Store-specific information" refers to individual information such as product information, service information, and store features related to a specific store.

[0303] A "database" refers to a system for efficiently storing, retrieving, and managing information.

[0304] A "large-scale language model" is an artificial intelligence model trained on a huge amount of text data, and is capable of performing natural language processing.

[0305] "Virtual character" refers to a computer-generated digital character that has the ability to interact and respond.

[0306] "Augmented reality display" refers to a technology that overlays digital information and objects onto the real world.

[0307] "Terminal" refers to a device such as a computer or smartphone that is directly operated by a user.

[0308] A "QR code" is a code that encodes information in two dimensions and can be read by a camera or other device.

[0309] "Emotion recognition" refers to technology that analyzes a user's facial expressions, tone of voice, word choice, etc. to identify their emotional state.

[0310] "Adjusting the response" refers to dynamically changing the information or suggestions provided by the virtual character based on the results of emotion recognition.

[0311] "Giving points" refers to giving points that are managed within the system to a user based on the user's usage record.

[0312] "Benefits" refers to additional services or benefits provided to users based on conditions such as points.

[0313] "Interactive" refers to the ability to communicate and operate two-way.

[0314] This invention is a system in which a user reads a QR code installed in a store with the camera of their device, and a virtual character corresponding to that store is generated, and the virtual character guides the user about products and services. This system is realized through the interaction of three parties: a server, a device, and a user.

[0315] The server first registers unique information corresponding to each store in a database. This information includes the store name, product information, service information, and store features. A relational database such as MySQL or PostgreSQL is used for the database.

[0316] The server then generates a unique QR code for each store. The QR code contains a specific store ID, which allows the server to access the store's unique information. The QR code is generated using the Python qrcode library.

[0317] When a user scans a store's QR code using the device's camera, the device sends the information to the server. This communication is securely carried out using the HTTPS protocol.

[0318] The server retrieves the store information from the database based on the QR code, and generates a virtual character based on that information using a large-scale language model (e.g., GPT-4). An example of a prompt is "Please generate a virtual character for the store based on the following information: {Store information}."

[0319] The generated virtual character is sent to the device and displayed in augmented reality (AR) on the device. This AR display uses Unity or Unreal Engine. The user can see the virtual character appear in real space through their smartphone.

[0320] The interaction between the user and the virtual character takes place in the form of the user asking questions about products or services, and the virtual character responding to them. The virtual character is equipped with an emotion recognition function that recognizes emotions by analyzing the user's facial expressions, tone of voice, and choice of words. This function uses Google® Cloud's Vision API and Microsoft® Azure® emotion analysis API.

[0321] The virtual character's responses are adjusted based on the emotion recognition technology used. For example, if the virtual character recognizes that the user is tired, it will suggest, "Why don't you take a break?"

[0322] Additionally, by using the system, users can earn points, which are recorded in their account and, when certain conditions are met, can unlock new services and rewards (such as a free drink on their next visit).

[0323] This allows users to interactively obtain store information and receive optimal service according to their emotions. Furthermore, the point system encourages continued use and improves the user experience.

[0324] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0325] Step 1:

[0326] Registering store-specific information

[0327] Subject: Server

[0328] Description: The server registers product information, service information, and store features provided by stores in a database.

[0329] Input: Product information, service information, and store features offered by the store

[0330] Data processing: This information is stored in a database in an appropriate format, for example, by inserting it into a relational database table.

[0331] Output: Store information saved in the database

[0332] Specific operation: The server registers the information received through the administrator interface in a database using an SQL query.

[0333] Step 2:

[0334] Generate a QR code

[0335] Subject: Server

[0336] Description: The server generates a unique QR code for each store. The QR code contains a specific store ID, which can be used to access store-specific information.

[0337] Input: Store ID

[0338] Data processing: Encode the store ID and generate a QR code.

[0339] Output: Generated QR code

[0340] Specific operation: Generate a QR code using Python's qrcode library and save it as an image file.

[0341] Step 3:

[0342] Scanning QR codes

[0343] Subject: User

[0344] Description: The user scans the QR code displayed in the store with the device's camera.

[0345] Input: QR code

[0346] Data processing: Decode the contents of the QR code to obtain the store ID.

[0347] Output: Store ID

[0348] What happens: A user scans a QR code using their smartphone's camera app and decodes the information.

[0349] Step 4:

[0350] Sending information

[0351] Subject: Terminal

[0352] Description: The device sends the information from the scanned QR code to the server.

[0353] Input: QR code content (store ID)

[0354] Data processing: Format the QR code content into an HTTP POST request format.

[0355] Output: Store ID sent to the server

[0356] Specific operation: The device sends data to the server using the HTTPS protocol.

[0357] Step 5:

[0358] Obtaining store information

[0359] Subject: Server

[0360] Description: The server retrieves the store information from the database.

[0361] Input: Store ID

[0362] Data processing: Retrieve store information from the database using an SQL query.

[0363] Output: Retrieved store information

[0364] Specific operation: Based on the store ID received by the server, a query is sent to the database to obtain information about the corresponding store.

[0365] Step 6:

[0366] Virtual character generation

[0367] Subject: Server

[0368] Description: The server uses a large-scale language model to generate a virtual character that corresponds to the store from the acquired store information.

[0369] Input: Store information

[0370] Data processing: Create prompt sentences for the generative AI model and input them into the AI ​​model.

[0371] Output: Data of the generated virtual character

[0372] Specific operation: Based on the store information, the server sends a prompt message to the AI ​​model saying, "Please generate a virtual character for the store based on the following information: {Store information}" to generate a virtual character.

[0373] Step 7:

[0374] Virtual character display

[0375] Subject: Terminal

[0376] Description: The device displays the virtual character information sent from the server in the AR application.

[0377] Input: Virtual character data

[0378] Data processing: Convert into a format suitable for AR display.

[0379] Output: Virtual character displayed on the device

[0380] Specific operation: The device uses Unity or Unreal Engine to display a virtual character overlaid on real space.

[0381] Step 8:

[0382] Starting a conversation

[0383] Subject: User

[0384] Description: The user asks the virtual character questions about products and services.

[0385] Input: User question

[0386] Data processing: The user's question is converted into text format and sent to the server.

[0387] Output: The question sent to the server

[0388] Specific operation: The user inputs a question using the input interface on the terminal and sends it to the server.

[0389] Step 9:

[0390] emotion recognition

[0391] Subject: Terminal

[0392] Description: The emotion engine recognizes emotions by analyzing the user's facial expressions, tone of voice, and word choice.

[0393] Input: User audio and video data

[0394] Data processing: Analyzing audio and video data using emotion recognition algorithms.

[0395] Output: Perceived emotional state

[0396] What it does: Evaluates the user's emotional state using Google Cloud's Vision API and Microsoft Azure's Sentiment Analysis API.

[0397] Step 10:

[0398] Response adjustment

[0399] Subject: Server

[0400] Description: Adjust the virtual character's response based on the analysis results.

[0401] Input: perceived emotional state, user question

[0402] Data processing: Send prompts to the response generation AI model to get the appropriate response.

[0403] Output: Adjusted response

[0404] Specific operation: Based on the emotion recognition results and the question content, the server sends a prompt to the AI ​​model saying, "If the user feels tired, please make suggestions to help them relax," and generates a response.

[0405] Step 11:

[0406] Points awarded

[0407] Subject: Server

[0408] Description: The server will give points to users for using this system.

[0409] Input: User action log

[0410] Data processing: Calculate points based on the algorithm of the points system and add them to the user's account.

[0411] Output: Updated points

[0412] Specific operation: The server continuously monitors the user's usage history and calculates the points for each specific action, such as "scanning a QR code gives 10 points," and assigns them to the user's account.

[0413] (Application example 2)

[0414] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0415] The user experience in traditional brick-and-mortar stores has been limited in the information provided, making it difficult to fully utilize the unique characteristics and services of each store.In addition, there is a lack of guidance that takes into account the user's emotional state, making it difficult to provide highly satisfying services that meet individual needs.

[0416] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a generative AI model, means for displaying the generated virtual character in AR, means for the virtual character to respond to user inquiries in an interactive format, means for recognizing the user's emotions and changing the virtual character's response based on the user's emotional state, and means for awarding points to the user based on usage. This makes it possible to provide appropriate product information and service suggestions based on the user's emotional state, significantly improving the user experience in physical stores.

[0417] "Store-specific information" refers to information about a specific store, such as its location, product lineup, special services, and business hours.

[0418] "Means of registering in a database" refers to a method of storing and managing a store's unique information in digital format using a server or cloud computing.

[0419] A "generative AI model" is an artificial intelligence system that uses natural language processing and machine learning techniques to generate virtual characters based on input data.

[0420] "Virtual Character" means a digital character created by a generative AI model and displayed through a user interface.

[0421] "AR display" is a method of displaying digital content overlaid on the real environment using augmented reality technology.

[0422] "Interactive response means" refers to a method in which a virtual character uses voice or text to communicate two-way in response to a user's questions or requests.

[0423] "Means for recognizing emotions and varying response content based on emotional state" refers to a method for determining a user's emotions by analyzing the user's facial expressions, tone of voice, choice of words, etc., and adjusting the virtual character's response content based on the results.

[0424] "Means for awarding points to users" refers to a means for adding digital points to users based on their system usage or specific actions, and allowing those points to be used for future benefits or services.

[0425] The present invention is a system that allows users to receive product information tailored to their emotions in a physical store. This system is realized by the interaction of three main entities: a server, a terminal (smartphone), and the user.

[0426] Server processing

[0427] 1. Register store information

[0428] The server registers the unique information provided by each store in the database. This information includes location, product lineup, special services, business hours, etc. For example, product information such as "sneakers, business shoes, sandals" is registered as information for the shoe zone.

[0429] 2. Virtual Character Generation

[0430] When a user scans a QR code in a store with their device, the server retrieves the corresponding store information from the database and generates a virtual character corresponding to the store using a generative AI model (e.g., GPT-4). An example of a prompt sentence is as follows:

[0431] "You are a virtual character from the Shoe Zone. Please introduce the following products to users: sneakers, business shoes, and sandals."

[0432] 3. Emotion recognition

[0433] The virtual character is equipped with an emotion engine that recognizes the user's emotions. The server is equipped with algorithms (e.g., the FER library) to analyze the user's facial expressions, tone of voice, and word choice, etc. This allows it to diagnose the user's emotional state in real time and send that information back to the device.

[0434] Processing by device (smartphone)

[0435] 1. Scan the QR code

[0436] When a user scans a QR code located in each zone of the store (for example, the shoe department or the clothing department) with their smartphone, the corresponding store ID is sent to the server.

[0437] 2. Displaying and interacting with virtual characters

[0438] The virtual character sent from the server is displayed on the smartphone in AR. The user can directly interact with the virtual character displayed overlaid on the real environment through the smartphone screen. For example, if the user asks, "What shoes do you recommend?" the virtual character will respond, "I recommend sneakers."

[0439] 3. Emotion recognition and response changes

[0440] The device adjusts the virtual character's response based on the emotion diagnosis results received from the server. For example, if the device detects that the user is tired, the virtual character will suggest, "Would you like to try some relaxing shoes?"

[0441] User Action

[0442] 1. Scan the QR code

[0443] Users scan the QR code in the store with their smartphone, and this information is sent to the server, which retrieves information about participating stores.

[0444] 2. Dialogue and Questions

[0445] Users can ask the virtual character questions about products and services. For example, if the user asks, "What are the current sales information?", the virtual character will respond with, "Sneakers are 20% off today."

[0446] 3. Earning points and providing new services

[0447] Points are awarded each time the system is used, and new services and benefits are offered depending on the points accumulated. For example, "50 points will get you a free drink on your next visit."

[0448] In this way, it will be possible to provide appropriate product information and service suggestions based on the user's emotional state, significantly improving the user experience in physical stores.

[0449] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0450] Step 1:

[0451] The server registers the unique information provided by each store (location, product lineup, special services, business hours, etc.) in a database. Specifically, store staff enter the information through the management screen, and this is automatically saved in a digital format in the database. The input is in the form of text or image data, and is indexed as soon as it is registered in the database, allowing for quick subsequent searches and processing.

[0452] Step 2:

[0453] When a user scans a QR code in a physical store, the device sends the scanned QR code information (store ID) to the server. The device's camera function is used to read the QR code, and the scanned data is parsed to extract the store ID, which is then sent to the server as an API request. The input at this time is the QR code image data, and the output is the store ID.

[0454] Step 3:

[0455] When the server receives the store ID, it retrieves the corresponding store's unique information from the database and uses a generative AI model (e.g., GPT-4) to create a virtual character generation prompt. The server uses this prompt to drive a generative AI model based on the store information. An example of a virtual character generation prompt is: "You are a virtual character from the Shoe Zone. Please introduce the following products to the user: sneakers, business shoes, sandals." The input is the store ID and unique information, and the output is a dialogue script for the generated virtual character.

[0456] Step 4:

[0457] The server sends the generated dialogue script for the virtual character to the device. The device displays the virtual character in AR based on the received information and begins dialogue with the user. AR technology (such as ARCore or ARKit) is used to overlay the virtual character on the real world via the smartphone screen. The input is the dialogue script for the virtual character, and the output is the AR character displayed on the user's device screen.

[0458] Step 5:

[0459] When a user interacts with a virtual character and asks a question or makes a request, the device captures the voice and text data. The device then sends the captured data to a server, which analyzes it and generates an appropriate response. Natural language processing technology is used for the analysis, and text tokenization and semantic analysis are performed to understand the user's intent and the content of the question. The input is the user's voice and text data, and the output is the analyzed question.

[0460] Step 6:

[0461] The server uses an emotion engine to analyze the user's emotional state. Based on the facial expression information obtained from the voice data and camera footage, the server uses an emotion recognition algorithm (e.g., the FER library) to identify the user's emotional state. The input is the user's facial expression and voice data, and the output is the recognized emotional state.

[0462] Step 7:

[0463] The server determines the virtual character's response based on the recognized emotional state and the user's question, and sends it back to the device. For example, if the server recognizes that the user is tired, it generates a response such as "Would you like to try some relaxing shoes?" The input in this case is the emotional state and the question, and the output is a customized response.

[0464] Step 8:

[0465] When a user uses the information provided, tries on products, or makes a purchase, points are awarded according to usage. The device sends this information to the server, which adds points to the user's account based on the points system. The input is usage data, and the output is updated points information.

[0466] This series of processing steps allows users to significantly improve their shopping experience in physical stores.

[0467] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0468] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0469] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0470] [Second embodiment]

[0471] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0472] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0473] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0474] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0475] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0476] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0477] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0478] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0479] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0480] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0481] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0482] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0483] The system of the present invention works by scanning a store's QR code with a smartphone, generating a virtual character that corresponds to the store and provides information about products and services to the user. This system is realized through the interaction of three parties: a server, a terminal, and a user.

[0484] Server processing

[0485] 1. Generate a QR code and register store information

[0486] The server generates a unique QR code for each store.

[0487] The unique information of the store accessed through this QR code (product information, service information, store features, etc.) is registered in a database.

[0488] 2. AI character generation

[0489] When a user scans a QR code, the server retrieves the store's information from the database.

[0490] The server uses a large-scale language model to generate a virtual character from the acquired store information.

[0491] Processing by device (smartphone)

[0492] 1. Scan the QR code

[0493] The user scans the store's QR code with the device's camera.

[0494] 2. Displaying an AI character and starting a conversation

[0495] Information about the AI ​​character generated from the server is received and displayed in AR on the device.

[0496] The characters provide information interactively in response to the user's questions and requests.

[0497] User Action

[0498] 1. Scan the QR code

[0499] The user scans the store's QR code with a smartphone app.

[0500] After scanning, the store information is sent to a server, which generates a virtual character and sends that information back to the device.

[0501] 2. Dialogue and Questions

[0502] Users ask the AI ​​character questions about products and services.

[0503] For example, if you ask, "What drink do you recommend?" the character will respond with something like, "I recommend the lavender latte."

[0504] Specific examples

[0505] In the case of "Cafe La Pearl"

[0506] 1. Scan the QR code

[0507] When a user scans the Cafe La Pearl QR code with their smartphone, the device sends the information to the server.

[0508] The server obtains information specific to Cafe La Pearl (e.g., products include lavender latte, espresso, and matcha latte) and generates a virtual character.

[0509] 2. Character Display

[0510] The generated virtual character is displayed on the user's smartphone.

[0511] When the user asks, "What drink would you recommend?" the character responds, "I recommend the elegant lavender latte. It has a calming scent."

[0512] Earn points and get new services

[0513] 1. Earn points when you use the service

[0514] By using this system, users can earn points, which are awarded based on their actions and usage and are managed on their devices.

[0515] For example, users can earn points by interacting with characters or spending a lot of time in the store.

[0516] 2. Offer of Benefits

[0517] Depending on the points accumulated, new services will be offered when certain conditions are met. For example, if you reach 50 points, you will receive a free drink on your next visit.

[0518] This will increase the repeat rate of users and enhance customer satisfaction.

[0519] This invention promotes interactive exchanges between stores and users, improving the user experience. In particular, it achieves a high level of interactivity by combining the generation of virtual characters using a large-scale language model, display using AR technology, and a point system. This system also provides great benefits to stores, as it allows users to gain a deeper understanding of store information and encourages continued use.

[0520] The processing flow will be explained below.

[0521] Step 1:

[0522] The server generates a unique QR code for each store, which contains a specific store ID and allows access to the store's specific information. The QR code is then stored in a database.

[0523] Step 2:

[0524] The server registers the product information, service information, and store features provided by the store in a database, allowing the information to be quickly retrieved when the QR code is scanned later.

[0525] Step 3:

[0526] A user scans a store's QR code using the camera on their smartphone. The QR code contains the store's ID, which the user's smartphone reads.

[0527] Step 4:

[0528] The user's device sends a request to the server containing the read store ID, which may also contain the user's identification information.

[0529] Step 5:

[0530] The server retrieves the store's unique information from the database based on the received store ID, including the store name, product list, and service details.

[0531] Step 6:

[0532] The server uses a large-scale language model (LLM) to generate virtual characters based on the acquired store information. The characters reflect the store's atmosphere and characteristics.

[0533] Step 7:

[0534] The generated virtual character data is sent to the user's device, including the character's visual data and dialogue script.

[0535] Step 8:

[0536] The user's device displays the received virtual character in AR, allowing the user to see the virtual character appear in real space through their smartphone.

[0537] Step 9:

[0538] Users can ask the virtual character questions about products and services, such as "What drink do you recommend?"

[0539] Step 10:

[0540] The device processes the user's question using voice recognition or text input, sends the question to the server, or may respond locally if it can be processed locally.

[0541] Step 11:

[0542] The server analyzes the questions sent by the user and generates appropriate answers using pre-defined dialogue scripts and large-scale language models, which are then returned to the terminal.

[0543] Step 12:

[0544] The user's device displays the answer received from the server through a virtual character. For example, the character may respond by voice or text, saying, "I recommend a lavender latte."

[0545] Step 13:

[0546] Users can enjoy an interactive experience by interacting with the virtual character and asking additional questions or requests as needed.

[0547] Step 14:

[0548] The server awards users points based on their usage of the system. The points are recorded in the user's account, and new services and benefits are offered when certain conditions are met.

[0549] Step 15:

[0550] Users can use the points they have accumulated to receive new services and benefits, such as a free drink on their next visit when they collect 50 points.

[0551] Example 1

[0552] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0553] Conventional store information systems provided information to users in a one-way manner, making it difficult to flexibly respond to the needs of individual users. Furthermore, when store information was updated, there was a lack of a way to notify users in a timely manner. Furthermore, incentives to increase user frequency of use were not effectively provided, making it difficult to increase repeat visits.

[0554] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0555] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in augmented reality, means for a user to read a store's QR code using a mobile device, means for transmitting the read QR code information to the server, means for the virtual character to respond to the user's inquiries in an interactive format, and means for awarding points to the user based on usage. This allows users to obtain information tailored to their needs in real time, improving the convenience of store use through interactive exchanges. Furthermore, utilizing a point system can increase the rate of user repeat visits.

[0556] A "server" is a computer system or device that provides services to clients over a network.

[0557] A "database" is a system or software that stores and manages data in an organized manner, and later extracts and uses it efficiently.

[0558] A "large-scale language model" is an artificial intelligence model that is trained based on large amounts of text data and has the ability to automatically generate text and understand meaning.

[0559] A "virtual character" is a computer-generated, computer-generated or text-based character used to interact with a user.

[0560] "Augmented reality display" is a technology that displays digital information overlaid on the real environment, making it possible to merge the real world with the virtual world.

[0561] A "mobile terminal" is a small, portable electronic device, including a smartphone, tablet, or mobile phone.

[0562] A "QR code" is a type of two-dimensional barcode that displays information in a matrix format, allowing for high-speed reading.

[0563] "Points" are units of reward awarded based on user behavior and frequency of use, and have value that can be exchanged for specific benefits or privileges.

[0564] MODE FOR CARRYING OUT THE INVENTION

[0565] The system of the present invention scans a store's QR code with a mobile device, and generates a virtual character corresponding to the store, which then guides the user through its products and services. The system is realized through the interaction of three parties: a server, a terminal, and a user.

[0566] Server processing

[0567] The server first uses Python's qrcode library to generate a unique QR code for each store. This QR code is generated by embedding the store's unique ID as data. The generated QR code is saved as an image file. Next, the server registers the store's unique information in a database such as MySQL. The registered information includes the store name, product list, service details, etc.

[0568] When a user scans a QR code, the server retrieves the store information corresponding to that QR code from a database. Based on the retrieved information, a virtual character is generated using a large-scale language model (for example, OpenAI's GPT-3). Specifically, a prompt sentence is created and sent to the AI ​​to generate the virtual character.

[0569] Processing by device (smartphone)

[0570] Users can use the react-native-camera library in React Native to scan a store's QR code using their mobile device's camera. The scanned QR code information is then sent from the device to the server. The generated virtual character information received from the server is then displayed in augmented reality on the device. This is achieved by combining software such as Unity and ARFoundation.

[0571] User Action

[0572] Users scan the QR code using an app on their mobile device. After scanning, store information is sent to the server. The information about the virtual character returned by the server is displayed on the device. Users can ask the virtual character questions about products and services. For example, if they ask, "What drink do you recommend?", the virtual character will respond, "I recommend the lavender latte."

[0573] Earn points and receive rewards

[0574] Each time a user uses the system, they are awarded points based on their behavior and frequency of use. Points are managed on the device and added up based on the user's behavior. When a certain number of points are accumulated, they are given special benefits that meet certain conditions, such as a free drink on their next visit.

[0575] For example, the prompt is:

[0576] You are a virtual character at Cafe La Pearl. What drink would you recommend to our customers?

[0577] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0578] Step 1: Generate a QR code

[0579] The server uses Python's qrcode library to generate a unique QR code for each store. The input is the store's unique ID, and the output is an image file of the QR code. Specifically, the server embeds the store ID as QR code data and saves the QR code as an image file.

[0580] Step 2: Register your store information

[0581] The server registers store-specific information in a database such as MySQL. The input is store information (product list, service details, store features, etc.), and the output is the information stored in the database. Specifically, the server executes SQL queries to insert information such as products, services, and features into the database.

[0582] Step 3: Scan the QR code

[0583] A user scans a store's QR code using the camera on their smartphone. The input is the QR code image captured by the camera, and the output is the QR code data. Specifically, the user opens the smartphone app, points the camera at the QR code, and scans it to obtain the data.

[0584] Step 4: Send QR code data to the server

[0585] The device sends the data read from the scanned QR code to the server. The input is the QR code data, and the output is the QR code information sent to the server. Specifically, the device sends the read QR code data to the server as an HTTP request.

[0586] Step 5: Get store information

[0587] The server retrieves store information from a database based on the received QR code data. The input is the QR code data (store ID), and the output is the store information retrieved from the database. Specifically, the server executes a database query using the store ID to retrieve information about the corresponding store.

[0588] Step 6: Generate a virtual character

[0589] The server uses a large-scale language model (e.g., OpenAI's GPT-3) to generate a virtual character from the acquired store information. The input is the store information and a prompt sentence for the generating AI model, and the output is the response of the generated virtual character. Specifically, the server creates a prompt sentence based on the store information and sends a request to the AI.

[0590] Step 7: Sending information about your virtual character

[0591] The server sends information about the generated virtual character to the terminal. The input is the response of the generated virtual character, and the output is the character information sent to the terminal. Specifically, the server sends the response of the virtual character to the terminal as an HTTP response.

[0592] Step 8: AR display of virtual characters

[0593] The device displays the received virtual character in augmented reality. The input is information about the virtual character, and the output is the virtual character displayed on the device. Specifically, Unity and ARFoundation are used to display the virtual character on the screen.

[0594] Step 9: Interact with the user

[0595] The user asks a question to a virtual character displayed in augmented reality. The input is the user's question, and the output is the response from the virtual character. For example, if the user asks, "What drink do you recommend?", the virtual character will respond, "I recommend a lavender latte."

[0596] Step 10: Points awarded

[0597] The server awards points based on the user's usage. The input is the user's behavioral data (frequency and content of interactions), and the output is the awarded points. Specifically, the server records the user's usage, calculates points based on that, and stores them in a points database.

[0598] Step 11: Offering rewards under certain conditions

[0599] The server provides new services and benefits when the awarded points meet certain conditions. The input is the user's point data, and the output is the benefits to be provided. Specifically, the server checks the user's points, and if the conditions are met, generates a reward coupon or service and notifies the user.

[0600] (Application example 1)

[0601] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0602] Currently, physical stores have limited methods for providing guidance and services to users, and there is a lack of effective means for communicating in-store information and product services. Furthermore, when users visit a store to obtain information, it is difficult to provide interactive and personalized support, making it difficult to improve customer satisfaction. Furthermore, there is a lack of methods for providing incentives based on user usage, making it difficult to increase repeat visit rates.

[0603] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0604] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in AR, means for the virtual character to respond to user inquiries in an interactive format, means for awarding points to the user according to usage, and means for accessing the server and generating a virtual character by reading a QR code in the store. This enables interactive and personalized information provision and service guidance in physical stores, thereby improving customer satisfaction and repeat business rates.

[0605] A "system for providing store-specific information" is a system that effectively guides users to information and services provided by stores.

[0606] "Store-specific information" refers to data such as product information, service information, and store characteristics that are unique to each store.

[0607] A "database" is a system or structure for storing and managing information in an organized manner.

[0608] A "large-scale language model" is an advanced AI model based on machine learning that has the ability to understand and generate human language.

[0609] A "virtual character" is a digitally generated character that can be interacted with.

[0610] "AR display" refers to the use of augmented reality technology to overlay digital information and objects onto the real world.

[0611] "Means of responding to user inquiries" refers to a function that enables a virtual character to provide appropriate responses to questions or requests from users.

[0612] "Means of awarding points to users according to usage" refers to a function that accumulates points each time a user uses the system, and manages and awards those points.

[0613] "Means for reading QR codes in stores" refers to the function that allows users to use a device such as a smartphone to scan QR codes installed in stores.

[0614] A "server" is a centralized computer system that processes and provides information over a network.

[0615] The system of the present invention functions through the mutual cooperation of three parties: a server, a terminal (smartphone), and a user. This system uses the following hardware and software.

[0616] Server processing

[0617] The server has the function of collecting store-specific information and registering it in a database. For example, the database stores product information, service information, and distinctive features of the store. MySQL is a suitable database to use. The Python library qrcode is used to generate QR codes, which are then distributed to each store.

[0618] When a user scans a QR code inside a store, the server uses a large-scale language model (e.g., OpenAI GPT-4) to generate a virtual character that corresponds to the store. This character then guides the user about products and services based on the store's unique information.

[0619] Processing by device (smartphone)

[0620] When a user scans a QR code in a store using their smartphone camera, the information in the QR code is read by the device. Using an AR library such as Vuforia, the device displays a virtual character overlaid on the real world. Unity can be used to display and interact with the AR character.

[0621] When interacting with a virtual character, the server generates a response based on the user's input and sends it to the device. For example, when a user asks, "What products do you recommend?", the smartphone sends the request to the server, which then queries the generative AI model based on the received information and provides an appropriate response to the device.

[0622] User Action

[0623] Users use a smartphone application to scan a QR code in a store, which sends store information to a server and generates a virtual character.

[0624] Users can then interact with the virtual character. For example, if they ask, "What drink do you recommend?" the character will respond in real time with, "I recommend the elegant lavender latte. It has a calming aroma." By continuing to interact, users can also accumulate points, which can be used to receive special benefits.

[0625] Specific examples

[0626] As a concrete example, consider the case where a user scans a QR code for a particular cafe. When the QR code is read, the server generates a virtual character based on the cafe's menu and features, and the character appears on the smartphone. When the user asks, "What's the recommendation this week?", the character responds, "This week's recommendation is our special chocolate cake."

[0627] Example prompts to input to a generative AI model:

[0628] Please generate lines for your virtual character based on the following information.

[0629] Store name: A specific cafe

[0630] Offerings: Chocolate Cake, Lavender Latte, Matcha Latte

[0631] Character's role: Store guide

[0632] User asks: "What's your pick this week?"

[0633] Character Quotes: <Generated quote goes here>

[0634] Hardware and Software Examples

[0635] Server: The database uses MySQL, and the Python qrcode library is used to generate QR codes.

[0636] Device: Scan the QR code with your smartphone camera. AR display will be performed using Unity and Vuforia.

[0637] Generative AI model: Dialogue content is generated using OpenAI GPT-4.

[0638] With the above configuration, the present invention makes it possible to provide interactive and personalized information and service guidance in brick-and-mortar stores, thereby improving customer satisfaction and repeat visit rates.

[0639] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0640] Step 1:

[0641] The server registers the store's unique information in a database. In this process, product information, service information, and characteristic features provided by the store are stored in a MySQL database. The specific input is the store's unique information, and the output is a database record in which this information is registered.

[0642] Step 2:

[0643] The server generates a unique QR code for each store. This QR code is generated using the Python library qrcode. The input is the store ID and URL, and the output is the generated QR code image file. This QR code is then distributed to each store.

[0644] Step 3:

[0645] A user uses the camera on their device (smartphone) to read a QR code in a store. The input is the QR code, and the output is a URL or identifier extracted from the QR code.

[0646] Step 4:

[0647] The device sends the information of the scanned QR code to the server. The input is the URL or identifier obtained from the QR code, and the output is that information is sent to the server.

[0648] Step 5:

[0649] The server retrieves the unique information of the corresponding store from the database based on the information in the received QR code. The input is the QR code identifier, and the output is the information of the corresponding store.

[0650] Step 6:

[0651] The server generates a virtual character using a large-scale language model (e.g., OpenAI GPT-4). In this step, a prompt is generated based on the acquired store information and sent to the AI ​​model. The input is the store's unique information and the generated prompt, and the output is the lines and movement information of the generated virtual character.

[0652] Step 7:

[0653] The server transmits information about the generated virtual character to the terminal. The input is the information about the generated virtual character, and the output is the transmission of this information to the terminal.

[0654] Step 8:

[0655] The device uses Unity and Vuforia to display AR based on the received virtual character information. The input is virtual character information, and the output is an AR character superimposed on the real world.

[0656] Step 9:

[0657] The user asks a question or makes a request to the virtual character, for example, "What products do you recommend?" The input is the user's question or request, and the output is the text data.

[0658] Step 10:

[0659] The terminal sends the user's question to the server. The input is the user's question text, and the output is that information is sent to the server.

[0660] Step 11:

[0661] The server receives the user's question and generates an appropriate response based on the generative AI model. The input is the user's question text, and the output is the AI-generated response text.

[0662] Step 12:

[0663] The server sends the generated response text to the terminal. The input is the generated response text, and the output is that information is sent to the terminal.

[0664] Step 13:

[0665] The terminal conveys the received response text to the user through the virtual character. The input is the response text, and the output is the voice of the virtual character or a text display.

[0666] Step 14:

[0667] The server awards points according to the user's usage. These points are updated in the database and managed for the user's use. The input is the user's usage data, and the output is the updated point data.

[0668] The above are the specific processing steps of the system for implementing the present invention.

[0669] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0670] This invention is a system in which a virtual character corresponding to a store is generated by scanning the store's QR code with a smartphone, and the virtual character guides the user to the store's products and services. The invention also combines an emotion engine that recognizes the user's emotions. This system is realized through interaction between the server, the terminal, and the user.

[0671] Server processing

[0672] 1. Generate a QR code and register store information

[0673] The server generates a unique QR code for each store, which contains a specific store ID and can be used to access store-specific information.

[0674] The server registers the product information, service information, and store features provided by the store in a database.

[0675] 2. AI character generation

[0676] When a user scans a QR code, the server retrieves the store's information from the database.

[0677] The server uses a large-scale language model to generate a virtual character that corresponds to the store from the acquired store information.

[0678] 3. Equipped with an emotional engine

[0679] The virtual character is equipped with an emotion engine that recognizes the user's emotions by analyzing the user's facial expressions, tone of voice, and choice of words.

[0680] Processing by device (smartphone)

[0681] 1. Scan the QR code

[0682] The user scans the store's QR code with the device's camera.

[0683] 2. Displaying an AI character and starting a conversation

[0684] The AI ​​character information sent from the server is received and displayed in AR on the device, allowing users to see the virtual character appear in real space through their smartphone.

[0685] The characters provide information interactively in response to the user's questions and requests.

[0686] 3. Emotion recognition and response changes

[0687] The emotion engine analyzes the user's emotional state, and the virtual character's responses change based on the analysis results.

[0688] For example, if the user looks tired, the character will make special suggestions such as, "Why don't you take a short break?"

[0689] User Action

[0690] 1. Scan the QR code

[0691] Users scan the store's QR code with their smartphone, after which the store's information is sent to a server and a virtual character is generated.

[0692] 2. Dialogue and Questions

[0693] Users can ask the virtual character questions about products and services, such as, "What drink do you recommend?"

[0694] The virtual character provides the store with the most relevant information and responds appropriately to questions.

[0695] 3. Specialized responses based on emotions

[0696] Users can receive special suggestions and responses based on their emotional state. For example, if a user is happy, they may receive suggestions such as, "Would you like to try our special dessert?"

[0697] Specific examples

[0698] In the case of "Cafe La Pearl"

[0699] 1. Scan the QR code

[0700] The user scans the "Cafe La Pearl" QR code with their smartphone, and the device sends the information to the server.

[0701] The server obtains information specific to Cafe La Pearl (for example, product information such as lavender latte, espresso, and matcha latte) and generates a virtual character based on that information.

[0702] 2. Character Display

[0703] The generated virtual character is displayed in AR on the user's smartphone. When the user asks, "What drink do you recommend?", the character responds, "I recommend the elegant lavender latte."

[0704] 3. Emotion recognition and response

[0705] The character analyzes the user's facial expressions and tone of voice, and if it determines that the user is tired, it will suggest, "Would you like me to show you a sofa seat where you can relax?"

[0706] Earn points and get new services

[0707] 1. Earn points when you use the service

[0708] By using this system, users earn points, which are recorded in their account and, when certain conditions are met, can be used to receive new services and benefits.

[0709] 2. Offer of Benefits

[0710] New services and benefits will be offered depending on the points accumulated. For example, 50 points will give you a free drink on your next visit.

[0711] This invention promotes interactive exchanges between stores and users, improving the user experience. In particular, by using virtual characters equipped with an emotion engine, it is possible to provide optimal services according to the user's emotional state. This system also provides great benefits to stores, as it allows users to gain a deeper understanding of store information and encourages continued use.

[0712] The processing flow will be explained below.

[0713] Step 1:

[0714] The server generates a unique QR code for each store. The generated QR code contains a specific store ID, which allows access to the store's unique information. After generation, this QR code and basic store information are stored in a database.

[0715] Step 2:

[0716] The server registers the product information, service information, and detailed store features provided by the store in a database, allowing the necessary information to be quickly retrieved when the QR code is scanned later.

[0717] Step 3:

[0718] The user uses the smartphone camera to scan the store's QR code, which then reads the store ID contained in the QR code and stores it on the user's device.

[0719] Step 4:

[0720] The user's device sends a request to the server containing the scanned store ID, which may also contain the user's identification information (e.g., user ID).

[0721] Step 5:

[0722] The server retrieves the store's unique information from the database based on the store ID received from the terminal, including the store name, product list, and service details.

[0723] Step 6:

[0724] The server uses a large-scale language model (LLM) to generate a virtual character based on the acquired store information, which reflects the store's atmosphere and characteristics.

[0725] Step 7:

[0726] Once the virtual character data is generated, the server connects the data to the emotion engine, which contains the algorithms and necessary data for recognizing the user's emotions.

[0727] Step 8:

[0728] The generated virtual character and emotion engine data are sent to the user's device, including the character's visual data, dialogue scripts, and emotion recognition algorithms.

[0729] Step 9:

[0730] The user's device displays the received virtual character in AR, allowing the user to see the virtual character in real space through the smartphone screen.

[0731] Step 10:

[0732] The user asks the virtual character a question about a product or service. For example, the user might ask, "What drink do you recommend?"

[0733] Step 11:

[0734] The user's device analyzes the user's question using voice recognition or text input, and sends the question content to the server. The emotion engine also analyzes the user's facial expressions and tone of voice, and sends this data to the server as appropriate.

[0735] Step 12:

[0736] The server receives and analyzes the user's question and transmitted emotion data, and uses a large-scale language model and emotion engine to generate appropriate answers and emotion-based responses to the user's question.

[0737] Step 13:

[0738] The server then sends the generated answers and responses to the user's device, including text and voice responses to questions, as well as special emotional responses.

[0739] Step 14:

[0740] The user's device displays the received answer through a virtual character. For example, the character may respond, "I recommend a lavender latte," and even make special suggestions, such as, "You look tired. A lavender latte will help you relax."

[0741] Step 15:

[0742] The server awards users points based on their usage of the system, which are recorded in their accounts and can be used to receive new services or rewards when certain conditions are met.

[0743] Step 16:

[0744] Users can use their accumulated points to receive new services and benefits. For example, 50 points will give you a free drink on your next visit.

[0745] Example 2

[0746] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0747] Conventional store guide systems lacked interactive methods for users to gain a deeper understanding of store information, resulting in a limited user experience. Furthermore, services were not provided that took user emotions into consideration, making it difficult to meet individual needs. Furthermore, point systems and rewards based on usage were insufficient, and there was a lack of methods to encourage continued use.

[0748] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0749] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in augmented reality, means for a user to read a QR code using a camera on a terminal and transmit the information to the server, means for the virtual character to recognize the user's emotions and adjust responses based on the recognition results, and means for awarding points to the user based on usage. This allows users to interactively acquire store information and receive optimal services based on their emotions. Furthermore, the point system encourages continued usage and improves the user experience.

[0750] "Store-specific information" refers to individual information such as product information, service information, and store features related to a specific store.

[0751] A "database" refers to a system for efficiently storing, retrieving, and managing information.

[0752] A "large-scale language model" is an artificial intelligence model trained on a huge amount of text data, and is capable of performing natural language processing.

[0753] "Virtual character" refers to a computer-generated digital character that has the ability to interact and respond.

[0754] "Augmented reality display" refers to a technology that overlays digital information and objects onto the real world.

[0755] "Terminal" refers to a device such as a computer or smartphone that is directly operated by a user.

[0756] A "QR code" is a code that encodes information in two dimensions and can be read by a camera or other device.

[0757] "Emotion recognition" refers to technology that analyzes a user's facial expressions, tone of voice, word choice, etc. to identify their emotional state.

[0758] "Adjusting the response" refers to dynamically changing the information or suggestions provided by the virtual character based on the results of emotion recognition.

[0759] "Giving points" refers to giving points that are managed within the system to a user based on the user's usage record.

[0760] "Benefits" refers to additional services or benefits provided to users based on conditions such as points.

[0761] "Interactive" refers to the ability to communicate and operate two-way.

[0762] This invention is a system in which a user reads a QR code installed in a store with the camera of their device, and a virtual character corresponding to that store is generated, and the virtual character guides the user about products and services. This system is realized through the interaction of three parties: a server, a device, and a user.

[0763] The server first registers unique information corresponding to each store in a database. This information includes the store name, product information, service information, and store features. A relational database such as MySQL or PostgreSQL is used for the database.

[0764] The server then generates a unique QR code for each store. The QR code contains a specific store ID, which allows the server to access the store's unique information. The QR code is generated using the Python qrcode library.

[0765] When a user scans a store's QR code using the device's camera, the device sends the information to the server. This communication is securely carried out using the HTTPS protocol.

[0766] The server retrieves the store information from the database based on the QR code, and generates a virtual character based on that information using a large-scale language model (e.g., GPT-4). An example of a prompt is "Please generate a virtual character for the store based on the following information: {Store information}."

[0767] The generated virtual character is sent to the device and displayed in augmented reality (AR) on the device. This AR display uses Unity or Unreal Engine. The user can see the virtual character appear in real space through their smartphone.

[0768] The interaction between the user and the virtual character is carried out by the user asking questions about products and services, and the virtual character responding to them. The virtual character is equipped with an emotion recognition function that recognizes emotions by analyzing the user's facial expressions, tone of voice, and choice of words. This function uses Google Cloud's Vision API and Microsoft Azure's emotion analysis API.

[0769] The virtual character's responses are adjusted based on the emotion recognition technology used. For example, if the virtual character recognizes that the user is tired, it will suggest, "Why don't you take a break?"

[0770] Additionally, by using the system, users can earn points, which are recorded in their account and, when certain conditions are met, can unlock new services and rewards (such as a free drink on their next visit).

[0771] This allows users to interactively obtain store information and receive optimal service according to their emotions. Furthermore, the point system encourages continued use and improves the user experience.

[0772] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0773] Step 1:

[0774] Registering store-specific information

[0775] Subject: Server

[0776] Description: The server registers product information, service information, and store features provided by stores in a database.

[0777] Input: Product information, service information, and store features offered by the store

[0778] Data processing: This information is stored in a database in an appropriate format, for example, by inserting it into a relational database table.

[0779] Output: Store information saved in the database

[0780] Specific operation: The server registers the information received through the administrator interface in a database using an SQL query.

[0781] Step 2:

[0782] Generate a QR code

[0783] Subject: Server

[0784] Description: The server generates a unique QR code for each store. The QR code contains a specific store ID, which can be used to access store-specific information.

[0785] Input: Store ID

[0786] Data processing: Encode the store ID and generate a QR code.

[0787] Output: Generated QR code

[0788] Specific operation: Generate a QR code using Python's qrcode library and save it as an image file.

[0789] Step 3:

[0790] Scanning QR codes

[0791] Subject: User

[0792] Description: The user scans the QR code displayed in the store with the device's camera.

[0793] Input: QR code

[0794] Data processing: Decode the contents of the QR code to obtain the store ID.

[0795] Output: Store ID

[0796] What happens: A user scans a QR code using their smartphone's camera app and decodes the information.

[0797] Step 4:

[0798] Sending information

[0799] Subject: Terminal

[0800] Description: The device sends the information from the scanned QR code to the server.

[0801] Input: QR code content (store ID)

[0802] Data processing: Format the QR code content into an HTTP POST request format.

[0803] Output: Store ID sent to the server

[0804] Specific operation: The device sends data to the server using the HTTPS protocol.

[0805] Step 5:

[0806] Obtaining store information

[0807] Subject: Server

[0808] Description: The server retrieves the store information from the database.

[0809] Input: Store ID

[0810] Data processing: Retrieve store information from the database using an SQL query.

[0811] Output: Retrieved store information

[0812] Specific operation: Based on the store ID received by the server, a query is sent to the database to obtain information about the corresponding store.

[0813] Step 6:

[0814] Virtual character generation

[0815] Subject: Server

[0816] Description: The server uses a large-scale language model to generate a virtual character that corresponds to the store from the acquired store information.

[0817] Input: Store information

[0818] Data processing: Create prompt sentences for the generative AI model and input them into the AI ​​model.

[0819] Output: Data of the generated virtual character

[0820] Specific operation: Based on the store information, the server sends a prompt message to the AI ​​model saying, "Please generate a virtual character for the store based on the following information: {Store information}" to generate a virtual character.

[0821] Step 7:

[0822] Virtual character display

[0823] Subject: Terminal

[0824] Description: The device displays the virtual character information sent from the server in the AR application.

[0825] Input: Virtual character data

[0826] Data processing: Convert into a format suitable for AR display.

[0827] Output: Virtual character displayed on the device

[0828] Specific operation: The device uses Unity or Unreal Engine to display a virtual character overlaid on real space.

[0829] Step 8:

[0830] Starting a conversation

[0831] Subject: User

[0832] Description: The user asks the virtual character questions about products and services.

[0833] Input: User question

[0834] Data processing: The user's question is converted into text format and sent to the server.

[0835] Output: The question sent to the server

[0836] Specific operation: The user inputs a question using the input interface on the terminal and sends it to the server.

[0837] Step 9:

[0838] emotion recognition

[0839] Subject: Terminal

[0840] Description: The emotion engine recognizes emotions by analyzing the user's facial expressions, tone of voice, and word choice.

[0841] Input: User audio and video data

[0842] Data processing: Analyzing audio and video data using emotion recognition algorithms.

[0843] Output: Perceived emotional state

[0844] What it does: Evaluates the user's emotional state using Google Cloud's Vision API and Microsoft Azure's Sentiment Analysis API.

[0845] Step 10:

[0846] Response adjustment

[0847] Subject: Server

[0848] Description: Adjust the virtual character's response based on the analysis results.

[0849] Input: perceived emotional state, user question

[0850] Data processing: Send prompts to the response generation AI model to get the appropriate response.

[0851] Output: Adjusted response

[0852] Specific operation: Based on the emotion recognition results and the question content, the server sends a prompt to the AI ​​model saying, "If the user feels tired, please make suggestions to help them relax," and generates a response.

[0853] Step 11:

[0854] Points awarded

[0855] Subject: Server

[0856] Description: The server will give points to users for using this system.

[0857] Input: User action log

[0858] Data processing: Calculate points based on the algorithm of the points system and add them to the user's account.

[0859] Output: Updated points

[0860] Specific operation: The server continuously monitors the user's usage history and calculates the points for each specific action, such as "scanning a QR code gives 10 points," and assigns them to the user's account.

[0861] (Application example 2)

[0862] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0863] The user experience in traditional brick-and-mortar stores has been limited in the information provided, making it difficult to fully utilize the unique characteristics and services of each store.In addition, there is a lack of guidance that takes into account the user's emotional state, making it difficult to provide highly satisfying services that meet individual needs.

[0864] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a generative AI model, means for displaying the generated virtual character in AR, means for the virtual character to respond to user inquiries in an interactive format, means for recognizing the user's emotions and changing the virtual character's response based on the user's emotional state, and means for awarding points to the user based on usage. This makes it possible to provide appropriate product information and service suggestions based on the user's emotional state, significantly improving the user experience in physical stores.

[0865] "Store-specific information" refers to information about a specific store, such as its location, product lineup, special services, and business hours.

[0866] "Means of registering in a database" refers to a method of storing and managing a store's unique information in digital format using a server or cloud computing.

[0867] A "generative AI model" is an artificial intelligence system that uses natural language processing and machine learning techniques to generate virtual characters based on input data.

[0868] "Virtual Character" means a digital character created by a generative AI model and displayed through a user interface.

[0869] "AR display" is a method of displaying digital content overlaid on the real environment using augmented reality technology.

[0870] "Interactive response means" refers to a method in which a virtual character uses voice or text to communicate two-way in response to a user's questions or requests.

[0871] "Means for recognizing emotions and varying response content based on emotional state" refers to a method for determining a user's emotions by analyzing the user's facial expressions, tone of voice, choice of words, etc., and adjusting the virtual character's response content based on the results.

[0872] "Means for awarding points to users" refers to a means for adding digital points to users based on their system usage or specific actions, and allowing those points to be used for future benefits or services.

[0873] The present invention is a system that allows users to receive product information tailored to their emotions in a physical store. This system is realized by the interaction of three main entities: a server, a terminal (smartphone), and the user.

[0874] Server processing

[0875] 1. Register store information

[0876] The server registers the unique information provided by each store in the database. This information includes location, product lineup, special services, business hours, etc. For example, product information such as "sneakers, business shoes, sandals" is registered as information for the shoe zone.

[0877] 2. Virtual Character Generation

[0878] When a user scans a QR code in a store with their device, the server retrieves the corresponding store information from the database and generates a virtual character corresponding to the store using a generative AI model (e.g., GPT-4). An example of a prompt sentence is as follows:

[0879] "You are a virtual character from the Shoe Zone. Please introduce the following products to users: sneakers, business shoes, and sandals."

[0880] 3. Emotion recognition

[0881] The virtual character is equipped with an emotion engine that recognizes the user's emotions. The server is equipped with algorithms (e.g., the FER library) to analyze the user's facial expressions, tone of voice, and word choice, etc. This allows it to diagnose the user's emotional state in real time and send that information back to the device.

[0882] Processing by device (smartphone)

[0883] 1. Scan the QR code

[0884] When a user scans a QR code located in each zone of the store (for example, the shoe department or the clothing department) with their smartphone, the corresponding store ID is sent to the server.

[0885] 2. Displaying and interacting with virtual characters

[0886] The virtual character sent from the server is displayed on the smartphone in AR. The user can directly interact with the virtual character displayed overlaid on the real environment through the smartphone screen. For example, if the user asks, "What shoes do you recommend?" the virtual character will respond, "I recommend sneakers."

[0887] 3. Emotion recognition and response changes

[0888] The device adjusts the virtual character's response based on the emotion diagnosis results received from the server. For example, if the device detects that the user is tired, the virtual character will suggest, "Would you like to try some relaxing shoes?"

[0889] User Action

[0890] 1. Scan the QR code

[0891] Users scan the QR code in the store with their smartphone, and this information is sent to the server, which retrieves information about participating stores.

[0892] 2. Dialogue and Questions

[0893] Users can ask the virtual character questions about products and services. For example, if the user asks, "What are the current sales information?", the virtual character will respond with, "Sneakers are 20% off today."

[0894] 3. Earning points and providing new services

[0895] Points are awarded each time the system is used, and new services and benefits are offered depending on the points accumulated. For example, "50 points will get you a free drink on your next visit."

[0896] In this way, it will be possible to provide appropriate product information and service suggestions based on the user's emotional state, significantly improving the user experience in physical stores.

[0897] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0898] Step 1:

[0899] The server registers the unique information provided by each store (location, product lineup, special services, business hours, etc.) in a database. Specifically, store staff enter the information through the management screen, and this is automatically saved in a digital format in the database. The input is in the form of text or image data, and is indexed as soon as it is registered in the database, allowing for quick subsequent searches and processing.

[0900] Step 2:

[0901] When a user scans a QR code in a physical store, the device sends the scanned QR code information (store ID) to the server. The device's camera function is used to read the QR code, and the scanned data is parsed to extract the store ID, which is then sent to the server as an API request. The input at this time is the QR code image data, and the output is the store ID.

[0902] Step 3:

[0903] When the server receives the store ID, it retrieves the corresponding store's unique information from the database and uses a generative AI model (e.g., GPT-4) to create a virtual character generation prompt. The server uses this prompt to drive a generative AI model based on the store information. An example of a virtual character generation prompt is: "You are a virtual character from the Shoe Zone. Please introduce the following products to the user: sneakers, business shoes, sandals." The input is the store ID and unique information, and the output is a dialogue script for the generated virtual character.

[0904] Step 4:

[0905] The server sends the generated dialogue script for the virtual character to the device. The device displays the virtual character in AR based on the received information and begins dialogue with the user. AR technology (such as ARCore or ARKit) is used to overlay the virtual character on the real world via the smartphone screen. The input is the dialogue script for the virtual character, and the output is the AR character displayed on the user's device screen.

[0906] Step 5:

[0907] When a user interacts with a virtual character and asks a question or makes a request, the device captures the voice and text data. The device then sends the captured data to a server, which analyzes it and generates an appropriate response. Natural language processing technology is used for the analysis, and text tokenization and semantic analysis are performed to understand the user's intent and the content of the question. The input is the user's voice and text data, and the output is the analyzed question.

[0908] Step 6:

[0909] The server uses an emotion engine to analyze the user's emotional state. Based on the facial expression information obtained from the voice data and camera footage, the server uses an emotion recognition algorithm (e.g., the FER library) to identify the user's emotional state. The input is the user's facial expression and voice data, and the output is the recognized emotional state.

[0910] Step 7:

[0911] The server determines the virtual character's response based on the recognized emotional state and the user's question, and sends it back to the device. For example, if the server recognizes that the user is tired, it generates a response such as "Would you like to try some relaxing shoes?" The input in this case is the emotional state and the question, and the output is a customized response.

[0912] Step 8:

[0913] When a user uses the information provided, tries on products, or makes a purchase, points are awarded according to usage. The device sends this information to the server, which adds points to the user's account based on the points system. The input is usage data, and the output is updated points information.

[0914] This series of processing steps allows users to significantly improve their shopping experience in physical stores.

[0915] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0916] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0917] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0918] [Third embodiment]

[0919] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0920] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0921] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0922] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0923] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0924] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0925] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0926] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0927] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0928] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0929] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0930] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0931] The system of the present invention works by scanning a store's QR code with a smartphone, generating a virtual character that corresponds to the store and provides information about products and services to the user. This system is realized through the interaction of three parties: a server, a terminal, and a user.

[0932] Server processing

[0933] 1. Generate a QR code and register store information

[0934] The server generates a unique QR code for each store.

[0935] The unique information of the store accessed through this QR code (product information, service information, store features, etc.) is registered in a database.

[0936] 2. AI character generation

[0937] When a user scans a QR code, the server retrieves the store's information from the database.

[0938] The server uses a large-scale language model to generate a virtual character from the acquired store information.

[0939] Processing by device (smartphone)

[0940] 1. Scan the QR code

[0941] The user scans the store's QR code with the device's camera.

[0942] 2. Displaying an AI character and starting a conversation

[0943] Information about the AI ​​character generated from the server is received and displayed in AR on the device.

[0944] The characters provide information interactively in response to the user's questions and requests.

[0945] User Action

[0946] 1. Scan the QR code

[0947] The user scans the store's QR code with a smartphone app.

[0948] After scanning, the store information is sent to a server, which generates a virtual character and sends that information back to the device.

[0949] 2. Dialogue and Questions

[0950] Users ask the AI ​​character questions about products and services.

[0951] For example, if you ask, "What drink do you recommend?" the character will respond with something like, "I recommend the lavender latte."

[0952] Specific examples

[0953] In the case of "Cafe La Pearl"

[0954] 1. Scan the QR code

[0955] When a user scans the Cafe La Pearl QR code with their smartphone, the device sends the information to the server.

[0956] The server obtains information specific to Cafe La Pearl (e.g., products include lavender latte, espresso, and matcha latte) and generates a virtual character.

[0957] 2. Character Display

[0958] The generated virtual character is displayed on the user's smartphone.

[0959] When the user asks, "What drink would you recommend?" the character responds, "I recommend the elegant lavender latte. It has a calming scent."

[0960] Earn points and get new services

[0961] 1. Earn points when you use the service

[0962] By using this system, users can earn points, which are awarded based on their actions and usage and are managed on their devices.

[0963] For example, users can earn points by interacting with characters or spending a lot of time in the store.

[0964] 2. Offer of Benefits

[0965] Depending on the points accumulated, new services will be offered when certain conditions are met. For example, if you reach 50 points, you will receive a free drink on your next visit.

[0966] This will increase the repeat rate of users and enhance customer satisfaction.

[0967] This invention promotes interactive exchanges between stores and users, improving the user experience. In particular, it achieves a high level of interactivity by combining the generation of virtual characters using a large-scale language model, display using AR technology, and a point system. This system also provides great benefits to stores, as it allows users to gain a deeper understanding of store information and encourages continued use.

[0968] The processing flow will be explained below.

[0969] Step 1:

[0970] The server generates a unique QR code for each store, which contains a specific store ID and allows access to the store's specific information. The QR code is then stored in a database.

[0971] Step 2:

[0972] The server registers the product information, service information, and store features provided by the store in a database, allowing the information to be quickly retrieved when the QR code is scanned later.

[0973] Step 3:

[0974] A user scans a store's QR code using the camera on their smartphone. The QR code contains the store's ID, which the user's smartphone reads.

[0975] Step 4:

[0976] The user's device sends a request to the server containing the read store ID, which may also contain the user's identification information.

[0977] Step 5:

[0978] The server retrieves the store's unique information from the database based on the received store ID, including the store name, product list, and service details.

[0979] Step 6:

[0980] The server uses a large-scale language model (LLM) to generate virtual characters based on the acquired store information. The characters reflect the store's atmosphere and characteristics.

[0981] Step 7:

[0982] The generated virtual character data is sent to the user's device, including the character's visual data and dialogue script.

[0983] Step 8:

[0984] The user's device displays the received virtual character in AR, allowing the user to see the virtual character appear in real space through their smartphone.

[0985] Step 9:

[0986] Users can ask the virtual character questions about products and services, such as "What drink do you recommend?"

[0987] Step 10:

[0988] The device processes the user's question using voice recognition or text input, sends the question to the server, or may respond locally if it can be processed locally.

[0989] Step 11:

[0990] The server analyzes the questions sent by the user and generates appropriate answers using pre-defined dialogue scripts and large-scale language models, which are then returned to the terminal.

[0991] Step 12:

[0992] The user's device displays the answer received from the server through a virtual character. For example, the character may respond by voice or text, saying, "I recommend a lavender latte."

[0993] Step 13:

[0994] Users can enjoy an interactive experience by interacting with the virtual character and asking additional questions or requests as needed.

[0995] Step 14:

[0996] The server awards users points based on their usage of the system. The points are recorded in the user's account, and new services and benefits are offered when certain conditions are met.

[0997] Step 15:

[0998] Users can use the points they have accumulated to receive new services and benefits, such as a free drink on their next visit when they collect 50 points.

[0999] Example 1

[1000] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1001] Conventional store information systems provided information to users in a one-way manner, making it difficult to flexibly respond to the needs of individual users. Furthermore, when store information was updated, there was a lack of a way to notify users in a timely manner. Furthermore, incentives to increase user frequency of use were not effectively provided, making it difficult to increase repeat visits.

[1002] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1003] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in augmented reality, means for a user to read a store's QR code using a mobile device, means for transmitting the read QR code information to the server, means for the virtual character to respond to the user's inquiries in an interactive format, and means for awarding points to the user based on usage. This allows users to obtain information tailored to their needs in real time, improving the convenience of store use through interactive exchanges. Furthermore, utilizing a point system can increase the rate of user repeat visits.

[1004] A "server" is a computer system or device that provides services to clients over a network.

[1005] A "database" is a system or software that stores and manages data in an organized manner, and later extracts and uses it efficiently.

[1006] A "large-scale language model" is an artificial intelligence model that is trained based on large amounts of text data and has the ability to automatically generate text and understand meaning.

[1007] A "virtual character" is a computer-generated, computer-generated or text-based character used to interact with a user.

[1008] "Augmented reality display" is a technology that displays digital information overlaid on the real environment, making it possible to merge the real world with the virtual world.

[1009] A "mobile terminal" is a small, portable electronic device, including a smartphone, tablet, or mobile phone.

[1010] A "QR code" is a type of two-dimensional barcode that displays information in a matrix format, allowing for high-speed reading.

[1011] "Points" are units of reward awarded based on user behavior and frequency of use, and have value that can be exchanged for specific benefits or privileges.

[1012] MODE FOR CARRYING OUT THE INVENTION

[1013] The system of the present invention scans a store's QR code with a mobile device, and generates a virtual character corresponding to the store, which then guides the user through its products and services. The system is realized through the interaction of three parties: a server, a terminal, and a user.

[1014] Server processing

[1015] The server first uses Python's qrcode library to generate a unique QR code for each store. This QR code is generated by embedding the store's unique ID as data. The generated QR code is saved as an image file. Next, the server registers the store's unique information in a database such as MySQL. The registered information includes the store name, product list, service details, etc.

[1016] When a user scans a QR code, the server retrieves the store information corresponding to that QR code from a database. Based on the retrieved information, a virtual character is generated using a large-scale language model (for example, OpenAI's GPT-3). Specifically, a prompt sentence is created and sent to the AI ​​to generate the virtual character.

[1017] Processing by device (smartphone)

[1018] Users can use the react-native-camera library in React Native to scan a store's QR code using their mobile device's camera. The scanned QR code information is then sent from the device to the server. The generated virtual character information received from the server is then displayed in augmented reality on the device. This is achieved by combining software such as Unity and ARFoundation.

[1019] User Action

[1020] Users scan the QR code using an app on their mobile device. After scanning, store information is sent to the server. The information about the virtual character returned by the server is displayed on the device. Users can ask the virtual character questions about products and services. For example, if they ask, "What drink do you recommend?", the virtual character will respond, "I recommend the lavender latte."

[1021] Earn points and receive rewards

[1022] Each time a user uses the system, they are awarded points based on their behavior and frequency of use. Points are managed on the device and added up based on the user's behavior. When a certain number of points are accumulated, they are given special benefits that meet certain conditions, such as a free drink on their next visit.

[1023] For example, the prompt is:

[1024] You are a virtual character at Cafe La Pearl. What drink would you recommend to our customers?

[1025] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1026] Step 1: Generate a QR code

[1027] The server uses Python's qrcode library to generate a unique QR code for each store. The input is the store's unique ID, and the output is an image file of the QR code. Specifically, the server embeds the store ID as QR code data and saves the QR code as an image file.

[1028] Step 2: Register your store information

[1029] The server registers store-specific information in a database such as MySQL. The input is store information (product list, service details, store features, etc.), and the output is the information stored in the database. Specifically, the server executes SQL queries to insert information such as products, services, and features into the database.

[1030] Step 3: Scan the QR code

[1031] A user scans a store's QR code using the camera on their smartphone. The input is the QR code image captured by the camera, and the output is the QR code data. Specifically, the user opens the smartphone app, points the camera at the QR code, and scans it to obtain the data.

[1032] Step 4: Send QR code data to the server

[1033] The device sends the data read from the scanned QR code to the server. The input is the QR code data, and the output is the QR code information sent to the server. Specifically, the device sends the read QR code data to the server as an HTTP request.

[1034] Step 5: Get store information

[1035] The server retrieves store information from a database based on the received QR code data. The input is the QR code data (store ID), and the output is the store information retrieved from the database. Specifically, the server executes a database query using the store ID to retrieve information about the corresponding store.

[1036] Step 6: Generate a virtual character

[1037] The server uses a large-scale language model (e.g., OpenAI's GPT-3) to generate a virtual character from the acquired store information. The input is the store information and a prompt sentence for the generating AI model, and the output is the response of the generated virtual character. Specifically, the server creates a prompt sentence based on the store information and sends a request to the AI.

[1038] Step 7: Sending information about your virtual character

[1039] The server sends information about the generated virtual character to the terminal. The input is the response of the generated virtual character, and the output is the character information sent to the terminal. Specifically, the server sends the response of the virtual character to the terminal as an HTTP response.

[1040] Step 8: AR display of virtual characters

[1041] The device displays the received virtual character in augmented reality. The input is information about the virtual character, and the output is the virtual character displayed on the device. Specifically, Unity and ARFoundation are used to display the virtual character on the screen.

[1042] Step 9: Interact with the user

[1043] The user asks a question to a virtual character displayed in augmented reality. The input is the user's question, and the output is the response from the virtual character. For example, if the user asks, "What drink do you recommend?", the virtual character will respond, "I recommend a lavender latte."

[1044] Step 10: Points awarded

[1045] The server awards points based on the user's usage. The input is the user's behavioral data (frequency and content of interactions), and the output is the awarded points. Specifically, the server records the user's usage, calculates points based on that, and stores them in a points database.

[1046] Step 11: Offering rewards under certain conditions

[1047] The server provides new services and benefits when the awarded points meet certain conditions. The input is the user's point data, and the output is the benefits to be provided. Specifically, the server checks the user's points, and if the conditions are met, generates a reward coupon or service and notifies the user.

[1048] (Application example 1)

[1049] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1050] Currently, physical stores have limited methods for providing guidance and services to users, and there is a lack of effective means for communicating in-store information and product services. Furthermore, when users visit a store to obtain information, it is difficult to provide interactive and personalized support, making it difficult to improve customer satisfaction. Furthermore, there is a lack of methods for providing incentives based on user usage, making it difficult to increase repeat visit rates.

[1051] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1052] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in AR, means for the virtual character to respond to user inquiries in an interactive format, means for awarding points to the user according to usage, and means for accessing the server and generating a virtual character by reading a QR code in the store. This enables interactive and personalized information provision and service guidance in physical stores, thereby improving customer satisfaction and repeat business rates.

[1053] A "system for providing store-specific information" is a system that effectively guides users to information and services provided by stores.

[1054] "Store-specific information" refers to data such as product information, service information, and store characteristics that are unique to each store.

[1055] A "database" is a system or structure for storing and managing information in an organized manner.

[1056] A "large-scale language model" is an advanced AI model based on machine learning that has the ability to understand and generate human language.

[1057] A "virtual character" is a digitally generated character that can be interacted with.

[1058] "AR display" refers to the use of augmented reality technology to overlay digital information and objects onto the real world.

[1059] "Means of responding to user inquiries" refers to a function that enables a virtual character to provide appropriate responses to questions or requests from users.

[1060] "Means of awarding points to users according to usage" refers to a function that accumulates points each time a user uses the system, and manages and awards those points.

[1061] "Means for reading QR codes in stores" refers to the function that allows users to use a device such as a smartphone to scan QR codes installed in stores.

[1062] A "server" is a centralized computer system that processes and provides information over a network.

[1063] The system of the present invention functions through the mutual cooperation of three parties: a server, a terminal (smartphone), and a user. This system uses the following hardware and software.

[1064] Server processing

[1065] The server has the function of collecting store-specific information and registering it in a database. For example, the database stores product information, service information, and distinctive features of the store. MySQL is a suitable database to use. The Python library qrcode is used to generate QR codes, which are then distributed to each store.

[1066] When a user scans a QR code inside a store, the server uses a large-scale language model (e.g., OpenAI GPT-4) to generate a virtual character that corresponds to the store. This character then guides the user about products and services based on the store's unique information.

[1067] Processing by device (smartphone)

[1068] When a user scans a QR code in a store using their smartphone camera, the information in the QR code is read by the device. Using an AR library such as Vuforia, the device displays a virtual character overlaid on the real world. Unity can be used to display and interact with the AR character.

[1069] When interacting with a virtual character, the server generates a response based on the user's input and sends it to the device. For example, when a user asks, "What products do you recommend?", the smartphone sends the request to the server, which then queries the generative AI model based on the received information and provides an appropriate response to the device.

[1070] User Action

[1071] Users use a smartphone application to scan a QR code in a store, which sends store information to a server and generates a virtual character.

[1072] Users can then interact with the virtual character. For example, if they ask, "What drink do you recommend?" the character will respond in real time with, "I recommend the elegant lavender latte. It has a calming aroma." By continuing to interact, users can also accumulate points, which can be used to receive special benefits.

[1073] Specific examples

[1074] As a concrete example, consider the case where a user scans a QR code for a particular cafe. When the QR code is read, the server generates a virtual character based on the cafe's menu and features, and the character appears on the smartphone. When the user asks, "What's the recommendation this week?", the character responds, "This week's recommendation is our special chocolate cake."

[1075] Example prompts to input to a generative AI model:

[1076] Please generate lines for your virtual character based on the following information.

[1077] Store name: A specific cafe

[1078] Offerings: Chocolate Cake, Lavender Latte, Matcha Latte

[1079] Character's role: Store guide

[1080] User asks: "What's your pick this week?"

[1081] Character Quotes: <Generated quote goes here>

[1082] Hardware and Software Examples

[1083] Server: The database uses MySQL, and the Python qrcode library is used to generate QR codes.

[1084] Device: Scan the QR code with your smartphone camera. AR display will be performed using Unity and Vuforia.

[1085] Generative AI model: Dialogue content is generated using OpenAI GPT-4.

[1086] With the above configuration, the present invention makes it possible to provide interactive and personalized information and service guidance in brick-and-mortar stores, thereby improving customer satisfaction and repeat visit rates.

[1087] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1088] Step 1:

[1089] The server registers the store's unique information in a database. In this process, product information, service information, and characteristic features provided by the store are stored in a MySQL database. The specific input is the store's unique information, and the output is a database record in which this information is registered.

[1090] Step 2:

[1091] The server generates a unique QR code for each store. This QR code is generated using the Python library qrcode. The input is the store ID and URL, and the output is the generated QR code image file. This QR code is then distributed to each store.

[1092] Step 3:

[1093] A user uses the camera on their device (smartphone) to read a QR code in a store. The input is the QR code, and the output is a URL or identifier extracted from the QR code.

[1094] Step 4:

[1095] The device sends the information of the scanned QR code to the server. The input is the URL or identifier obtained from the QR code, and the output is that information is sent to the server.

[1096] Step 5:

[1097] The server retrieves the unique information of the corresponding store from the database based on the information in the received QR code. The input is the QR code identifier, and the output is the information of the corresponding store.

[1098] Step 6:

[1099] The server generates a virtual character using a large-scale language model (e.g., OpenAI GPT-4). In this step, a prompt is generated based on the acquired store information and sent to the AI ​​model. The input is the store's unique information and the generated prompt, and the output is the lines and movement information of the generated virtual character.

[1100] Step 7:

[1101] The server transmits information about the generated virtual character to the terminal. The input is the information about the generated virtual character, and the output is the transmission of this information to the terminal.

[1102] Step 8:

[1103] The device uses Unity and Vuforia to display AR based on the received virtual character information. The input is virtual character information, and the output is an AR character superimposed on the real world.

[1104] Step 9:

[1105] The user asks a question or makes a request to the virtual character, for example, "What products do you recommend?" The input is the user's question or request, and the output is the text data.

[1106] Step 10:

[1107] The terminal sends the user's question to the server. The input is the user's question text, and the output is that information is sent to the server.

[1108] Step 11:

[1109] The server receives the user's question and generates an appropriate response based on the generative AI model. The input is the user's question text, and the output is the AI-generated response text.

[1110] Step 12:

[1111] The server sends the generated response text to the terminal. The input is the generated response text, and the output is that information is sent to the terminal.

[1112] Step 13:

[1113] The terminal conveys the received response text to the user through the virtual character. The input is the response text, and the output is the voice of the virtual character or a text display.

[1114] Step 14:

[1115] The server awards points according to the user's usage. These points are updated in the database and managed for the user's use. The input is the user's usage data, and the output is the updated point data.

[1116] The above are the specific processing steps of the system for implementing the present invention.

[1117] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1118] This invention is a system in which a virtual character corresponding to a store is generated by scanning the store's QR code with a smartphone, and the virtual character guides the user to the store's products and services. The invention also combines an emotion engine that recognizes the user's emotions. This system is realized through interaction between the server, the terminal, and the user.

[1119] Server processing

[1120] 1. Generate a QR code and register store information

[1121] The server generates a unique QR code for each store, which contains a specific store ID and can be used to access store-specific information.

[1122] The server registers the product information, service information, and store features provided by the store in a database.

[1123] 2. AI character generation

[1124] When a user scans a QR code, the server retrieves the store's information from the database.

[1125] The server uses a large-scale language model to generate a virtual character that corresponds to the store from the acquired store information.

[1126] 3. Equipped with an emotional engine

[1127] The virtual character is equipped with an emotion engine that recognizes the user's emotions by analyzing the user's facial expressions, tone of voice, and choice of words.

[1128] Processing by device (smartphone)

[1129] 1. Scan the QR code

[1130] The user scans the store's QR code with the device's camera.

[1131] 2. Displaying an AI character and starting a conversation

[1132] The AI ​​character information sent from the server is received and displayed in AR on the device, allowing users to see the virtual character appear in real space through their smartphone.

[1133] The characters provide information interactively in response to the user's questions and requests.

[1134] 3. Emotion recognition and response changes

[1135] The emotion engine analyzes the user's emotional state, and the virtual character's responses change based on the analysis results.

[1136] For example, if the user looks tired, the character will make special suggestions such as, "Why don't you take a short break?"

[1137] User Action

[1138] 1. Scan the QR code

[1139] Users scan the store's QR code with their smartphone, after which the store's information is sent to a server and a virtual character is generated.

[1140] 2. Dialogue and Questions

[1141] Users can ask the virtual character questions about products and services, such as, "What drink do you recommend?"

[1142] The virtual character provides the store with the most relevant information and responds appropriately to questions.

[1143] 3. Specialized responses based on emotions

[1144] Users can receive special suggestions and responses based on their emotional state. For example, if a user is happy, they may receive suggestions such as, "Would you like to try our special dessert?"

[1145] Specific examples

[1146] In the case of "Cafe La Pearl"

[1147] 1. Scan the QR code

[1148] The user scans the "Cafe La Pearl" QR code with their smartphone, and the device sends the information to the server.

[1149] The server obtains information specific to Cafe La Pearl (for example, product information such as lavender latte, espresso, and matcha latte) and generates a virtual character based on that information.

[1150] 2. Character Display

[1151] The generated virtual character is displayed in AR on the user's smartphone. When the user asks, "What drink do you recommend?", the character responds, "I recommend the elegant lavender latte."

[1152] 3. Emotion recognition and response

[1153] The character analyzes the user's facial expressions and tone of voice, and if it determines that the user is tired, it will suggest, "Would you like me to show you a sofa seat where you can relax?"

[1154] Earn points and get new services

[1155] 1. Earn points when you use the service

[1156] By using this system, users earn points, which are recorded in their account and, when certain conditions are met, can be used to receive new services and benefits.

[1157] 2. Offer of Benefits

[1158] New services and benefits will be offered depending on the points accumulated. For example, 50 points will give you a free drink on your next visit.

[1159] This invention promotes interactive exchanges between stores and users, improving the user experience. In particular, by using virtual characters equipped with an emotion engine, it is possible to provide optimal services according to the user's emotional state. This system also provides great benefits to stores, as it allows users to gain a deeper understanding of store information and encourages continued use.

[1160] The processing flow will be explained below.

[1161] Step 1:

[1162] The server generates a unique QR code for each store. The generated QR code contains a specific store ID, which allows access to the store's unique information. After generation, this QR code and basic store information are stored in a database.

[1163] Step 2:

[1164] The server registers the product information, service information, and detailed store features provided by the store in a database, allowing the necessary information to be quickly retrieved when the QR code is scanned later.

[1165] Step 3:

[1166] The user uses the smartphone camera to scan the store's QR code, which then reads the store ID contained in the QR code and stores it on the user's device.

[1167] Step 4:

[1168] The user's device sends a request to the server containing the scanned store ID, which may also contain the user's identification information (e.g., user ID).

[1169] Step 5:

[1170] The server retrieves the store's unique information from the database based on the store ID received from the terminal, including the store name, product list, and service details.

[1171] Step 6:

[1172] The server uses a large-scale language model (LLM) to generate a virtual character based on the acquired store information, which reflects the store's atmosphere and characteristics.

[1173] Step 7:

[1174] Once the virtual character data is generated, the server connects the data to the emotion engine, which contains the algorithms and necessary data for recognizing the user's emotions.

[1175] Step 8:

[1176] The generated virtual character and emotion engine data are sent to the user's device, including the character's visual data, dialogue scripts, and emotion recognition algorithms.

[1177] Step 9:

[1178] The user's device displays the received virtual character in AR, allowing the user to see the virtual character in real space through the smartphone screen.

[1179] Step 10:

[1180] The user asks the virtual character a question about a product or service. For example, the user might ask, "What drink do you recommend?"

[1181] Step 11:

[1182] The user's device analyzes the user's question using voice recognition or text input, and sends the question content to the server. The emotion engine also analyzes the user's facial expressions and tone of voice, and sends this data to the server as appropriate.

[1183] Step 12:

[1184] The server receives and analyzes the user's question and transmitted emotion data, and uses a large-scale language model and emotion engine to generate appropriate answers and emotion-based responses to the user's question.

[1185] Step 13:

[1186] The server then sends the generated answers and responses to the user's device, including text and voice responses to questions, as well as special emotional responses.

[1187] Step 14:

[1188] The user's device displays the received answer through a virtual character. For example, the character may respond, "I recommend a lavender latte," and even make special suggestions, such as, "You look tired. A lavender latte will help you relax."

[1189] Step 15:

[1190] The server awards users points based on their usage of the system, which are recorded in their accounts and can be used to receive new services or rewards when certain conditions are met.

[1191] Step 16:

[1192] Users can use their accumulated points to receive new services and benefits. For example, 50 points will give you a free drink on your next visit.

[1193] Example 2

[1194] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1195] Conventional store guide systems lacked interactive methods for users to gain a deeper understanding of store information, resulting in a limited user experience. Furthermore, services were not provided that took user emotions into consideration, making it difficult to meet individual needs. Furthermore, point systems and rewards based on usage were insufficient, and there was a lack of methods to encourage continued use.

[1196] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1197] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in augmented reality, means for a user to read a QR code using a camera on a terminal and transmit the information to the server, means for the virtual character to recognize the user's emotions and adjust responses based on the recognition results, and means for awarding points to the user based on usage. This allows users to interactively acquire store information and receive optimal services based on their emotions. Furthermore, the point system encourages continued usage and improves the user experience.

[1198] "Store-specific information" refers to individual information such as product information, service information, and store features related to a specific store.

[1199] A "database" refers to a system for efficiently storing, retrieving, and managing information.

[1200] A "large-scale language model" is an artificial intelligence model trained on a huge amount of text data, and is capable of performing natural language processing.

[1201] "Virtual character" refers to a computer-generated digital character that has the ability to interact and respond.

[1202] "Augmented reality display" refers to a technology that overlays digital information and objects onto the real world.

[1203] "Terminal" refers to a device such as a computer or smartphone that is directly operated by a user.

[1204] A "QR code" is a code that encodes information in two dimensions and can be read by a camera or other device.

[1205] "Emotion recognition" refers to technology that analyzes a user's facial expressions, tone of voice, word choice, etc. to identify their emotional state.

[1206] "Adjusting the response" refers to dynamically changing the information or suggestions provided by the virtual character based on the results of emotion recognition.

[1207] "Giving points" refers to giving points that are managed within the system to a user based on the user's usage record.

[1208] "Benefits" refers to additional services or benefits provided to users based on conditions such as points.

[1209] "Interactive" refers to the ability to communicate and operate two-way.

[1210] This invention is a system in which a user reads a QR code installed in a store with the camera of their device, and a virtual character corresponding to that store is generated, and the virtual character guides the user about products and services. This system is realized through the interaction of three parties: a server, a device, and a user.

[1211] The server first registers unique information corresponding to each store in a database. This information includes the store name, product information, service information, and store features. A relational database such as MySQL or PostgreSQL is used for the database.

[1212] The server then generates a unique QR code for each store. The QR code contains a specific store ID, which allows the server to access the store's unique information. The QR code is generated using the Python qrcode library.

[1213] When a user scans a store's QR code using the device's camera, the device sends the information to the server. This communication is securely carried out using the HTTPS protocol.

[1214] The server retrieves the store information from the database based on the QR code, and generates a virtual character based on that information using a large-scale language model (e.g., GPT-4). An example of a prompt is "Please generate a virtual character for the store based on the following information: {Store information}."

[1215] The generated virtual character is sent to the device and displayed in augmented reality (AR) on the device. This AR display uses Unity or Unreal Engine. The user can see the virtual character appear in real space through their smartphone.

[1216] The interaction between the user and the virtual character is carried out by the user asking questions about products and services, and the virtual character responding to them. The virtual character is equipped with an emotion recognition function that recognizes emotions by analyzing the user's facial expressions, tone of voice, and choice of words. This function uses Google Cloud's Vision API and Microsoft Azure's emotion analysis API.

[1217] The virtual character's responses are adjusted based on the emotion recognition technology used. For example, if the virtual character recognizes that the user is tired, it will suggest, "Why don't you take a break?"

[1218] Additionally, by using the system, users can earn points, which are recorded in their account and, when certain conditions are met, can unlock new services and rewards (such as a free drink on their next visit).

[1219] This allows users to interactively obtain store information and receive optimal service according to their emotions. Furthermore, the point system encourages continued use and improves the user experience.

[1220] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1221] Step 1:

[1222] Registering store-specific information

[1223] Subject: Server

[1224] Description: The server registers product information, service information, and store features provided by stores in a database.

[1225] Input: Product information, service information, and store features offered by the store

[1226] Data processing: This information is stored in a database in an appropriate format, for example, by inserting it into a relational database table.

[1227] Output: Store information saved in the database

[1228] Specific operation: The server registers the information received through the administrator interface in a database using an SQL query.

[1229] Step 2:

[1230] Generate a QR code

[1231] Subject: Server

[1232] Description: The server generates a unique QR code for each store. The QR code contains a specific store ID, which can be used to access store-specific information.

[1233] Input: Store ID

[1234] Data processing: Encode the store ID and generate a QR code.

[1235] Output: Generated QR code

[1236] Specific operation: Generate a QR code using Python's qrcode library and save it as an image file.

[1237] Step 3:

[1238] Scanning QR codes

[1239] Subject: User

[1240] Description: The user scans the QR code displayed in the store with the device's camera.

[1241] Input: QR code

[1242] Data processing: Decode the contents of the QR code to obtain the store ID.

[1243] Output: Store ID

[1244] What happens: A user scans a QR code using their smartphone's camera app and decodes the information.

[1245] Step 4:

[1246] Sending information

[1247] Subject: Terminal

[1248] Description: The device sends the information from the scanned QR code to the server.

[1249] Input: QR code content (store ID)

[1250] Data processing: Format the QR code content into an HTTP POST request format.

[1251] Output: Store ID sent to the server

[1252] Specific operation: The device sends data to the server using the HTTPS protocol.

[1253] Step 5:

[1254] Obtaining store information

[1255] Subject: Server

[1256] Description: The server retrieves the store information from the database.

[1257] Input: Store ID

[1258] Data processing: Retrieve store information from the database using an SQL query.

[1259] Output: Retrieved store information

[1260] Specific operation: Based on the store ID received by the server, a query is sent to the database to obtain information about the corresponding store.

[1261] Step 6:

[1262] Virtual character generation

[1263] Subject: Server

[1264] Description: The server uses a large-scale language model to generate a virtual character that corresponds to the store from the acquired store information.

[1265] Input: Store information

[1266] Data processing: Create prompt sentences for the generative AI model and input them into the AI ​​model.

[1267] Output: Data of the generated virtual character

[1268] Specific operation: Based on the store information, the server sends a prompt message to the AI ​​model saying, "Please generate a virtual character for the store based on the following information: {Store information}" to generate a virtual character.

[1269] Step 7:

[1270] Virtual character display

[1271] Subject: Terminal

[1272] Description: The device displays the virtual character information sent from the server in the AR application.

[1273] Input: Virtual character data

[1274] Data processing: Convert into a format suitable for AR display.

[1275] Output: Virtual character displayed on the device

[1276] Specific operation: The device uses Unity or Unreal Engine to display a virtual character overlaid on real space.

[1277] Step 8:

[1278] Starting a conversation

[1279] Subject: User

[1280] Description: The user asks the virtual character questions about products and services.

[1281] Input: User question

[1282] Data processing: The user's question is converted into text format and sent to the server.

[1283] Output: The question sent to the server

[1284] Specific operation: The user inputs a question using the input interface on the terminal and sends it to the server.

[1285] Step 9:

[1286] emotion recognition

[1287] Subject: Terminal

[1288] Description: The emotion engine recognizes emotions by analyzing the user's facial expressions, tone of voice, and word choice.

[1289] Input: User audio and video data

[1290] Data processing: Analyzing audio and video data using emotion recognition algorithms.

[1291] Output: Perceived emotional state

[1292] What it does: Evaluates the user's emotional state using Google Cloud's Vision API and Microsoft Azure's Sentiment Analysis API.

[1293] Step 10:

[1294] Response adjustment

[1295] Subject: Server

[1296] Description: Adjust the virtual character's response based on the analysis results.

[1297] Input: perceived emotional state, user question

[1298] Data processing: Send prompts to the response generation AI model to get the appropriate response.

[1299] Output: Adjusted response

[1300] Specific operation: Based on the emotion recognition results and the question content, the server sends a prompt to the AI ​​model saying, "If the user feels tired, please make suggestions to help them relax," and generates a response.

[1301] Step 11:

[1302] Points awarded

[1303] Subject: Server

[1304] Description: The server will give points to users for using this system.

[1305] Input: User action log

[1306] Data processing: Calculate points based on the algorithm of the points system and add them to the user's account.

[1307] Output: Updated points

[1308] Specific operation: The server continuously monitors the user's usage history and calculates the points for each specific action, such as "scanning a QR code gives 10 points," and assigns them to the user's account.

[1309] (Application example 2)

[1310] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1311] The user experience in traditional brick-and-mortar stores has been limited in the information provided, making it difficult to fully utilize the unique characteristics and services of each store.In addition, there is a lack of guidance that takes into account the user's emotional state, making it difficult to provide highly satisfying services that meet individual needs.

[1312] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a generative AI model, means for displaying the generated virtual character in AR, means for the virtual character to respond to user inquiries in an interactive format, means for recognizing the user's emotions and changing the virtual character's response based on the user's emotional state, and means for awarding points to the user based on usage. This makes it possible to provide appropriate product information and service suggestions based on the user's emotional state, significantly improving the user experience in physical stores.

[1313] "Store-specific information" refers to information about a specific store, such as its location, product lineup, special services, and business hours.

[1314] "Means of registering in a database" refers to a method of storing and managing a store's unique information in digital format using a server or cloud computing.

[1315] A "generative AI model" is an artificial intelligence system that uses natural language processing and machine learning techniques to generate virtual characters based on input data.

[1316] "Virtual Character" means a digital character created by a generative AI model and displayed through a user interface.

[1317] "AR display" is a method of displaying digital content overlaid on the real environment using augmented reality technology.

[1318] "Interactive response means" refers to a method in which a virtual character uses voice or text to communicate two-way in response to a user's questions or requests.

[1319] "Means for recognizing emotions and varying response content based on emotional state" refers to a method for determining a user's emotions by analyzing the user's facial expressions, tone of voice, choice of words, etc., and adjusting the virtual character's response content based on the results.

[1320] "Means for awarding points to users" refers to a means for adding digital points to users based on their system usage or specific actions, and allowing those points to be used for future benefits or services.

[1321] The present invention is a system that allows users to receive product information tailored to their emotions in a physical store. This system is realized by the interaction of three main entities: a server, a terminal (smartphone), and the user.

[1322] Server processing

[1323] 1. Register store information

[1324] The server registers the unique information provided by each store in the database. This information includes location, product lineup, special services, business hours, etc. For example, product information such as "sneakers, business shoes, sandals" is registered as information for the shoe zone.

[1325] 2. Virtual Character Generation

[1326] When a user scans a QR code in a store with their device, the server retrieves the corresponding store information from the database and generates a virtual character corresponding to the store using a generative AI model (e.g., GPT-4). An example of a prompt sentence is as follows:

[1327] "You are a virtual character from the Shoe Zone. Please introduce the following products to users: sneakers, business shoes, and sandals."

[1328] 3. Emotion recognition

[1329] The virtual character is equipped with an emotion engine that recognizes the user's emotions. The server is equipped with algorithms (e.g., the FER library) to analyze the user's facial expressions, tone of voice, and word choice, etc. This allows it to diagnose the user's emotional state in real time and send that information back to the device.

[1330] Processing by device (smartphone)

[1331] 1. Scan the QR code

[1332] When a user scans a QR code located in each zone of the store (for example, the shoe department or the clothing department) with their smartphone, the corresponding store ID is sent to the server.

[1333] 2. Displaying and interacting with virtual characters

[1334] The virtual character sent from the server is displayed on the smartphone in AR. The user can directly interact with the virtual character displayed overlaid on the real environment through the smartphone screen. For example, if the user asks, "What shoes do you recommend?" the virtual character will respond, "I recommend sneakers."

[1335] 3. Emotion recognition and response changes

[1336] The device adjusts the virtual character's response based on the emotion diagnosis results received from the server. For example, if the device detects that the user is tired, the virtual character will suggest, "Would you like to try some relaxing shoes?"

[1337] User Action

[1338] 1. Scan the QR code

[1339] Users scan the QR code in the store with their smartphone, and this information is sent to the server, which retrieves information about participating stores.

[1340] 2. Dialogue and Questions

[1341] Users can ask the virtual character questions about products and services. For example, if the user asks, "What are the current sales information?", the virtual character will respond with, "Sneakers are 20% off today."

[1342] 3. Earning points and providing new services

[1343] Points are awarded each time the system is used, and new services and benefits are offered depending on the points accumulated. For example, "50 points will get you a free drink on your next visit."

[1344] In this way, it will be possible to provide appropriate product information and service suggestions based on the user's emotional state, significantly improving the user experience in physical stores.

[1345] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1346] Step 1:

[1347] The server registers the unique information provided by each store (location, product lineup, special services, business hours, etc.) in a database. Specifically, store staff enter the information through the management screen, and this is automatically saved in a digital format in the database. The input is in the form of text or image data, and is indexed as soon as it is registered in the database, allowing for quick subsequent searches and processing.

[1348] Step 2:

[1349] When a user scans a QR code in a physical store, the device sends the scanned QR code information (store ID) to the server. The device's camera function is used to read the QR code, and the scanned data is parsed to extract the store ID, which is then sent to the server as an API request. The input at this time is the QR code image data, and the output is the store ID.

[1350] Step 3:

[1351] When the server receives the store ID, it retrieves the corresponding store's unique information from the database and uses a generative AI model (e.g., GPT-4) to create a virtual character generation prompt. The server uses this prompt to drive a generative AI model based on the store information. An example of a virtual character generation prompt is: "You are a virtual character from the Shoe Zone. Please introduce the following products to the user: sneakers, business shoes, sandals." The input is the store ID and unique information, and the output is a dialogue script for the generated virtual character.

[1352] Step 4:

[1353] The server sends the generated dialogue script for the virtual character to the device. The device displays the virtual character in AR based on the received information and begins dialogue with the user. AR technology (such as ARCore or ARKit) is used to overlay the virtual character on the real world via the smartphone screen. The input is the dialogue script for the virtual character, and the output is the AR character displayed on the user's device screen.

[1354] Step 5:

[1355] When a user interacts with a virtual character and asks a question or makes a request, the device captures the voice and text data. The device then sends the captured data to a server, which analyzes it and generates an appropriate response. Natural language processing technology is used for the analysis, and text tokenization and semantic analysis are performed to understand the user's intent and the content of the question. The input is the user's voice and text data, and the output is the analyzed question.

[1356] Step 6:

[1357] The server uses an emotion engine to analyze the user's emotional state. Based on the facial expression information obtained from the voice data and camera footage, the server uses an emotion recognition algorithm (e.g., the FER library) to identify the user's emotional state. The input is the user's facial expression and voice data, and the output is the recognized emotional state.

[1358] Step 7:

[1359] The server determines the virtual character's response based on the recognized emotional state and the user's question, and sends it back to the device. For example, if the server recognizes that the user is tired, it generates a response such as "Would you like to try some relaxing shoes?" The input in this case is the emotional state and the question, and the output is a customized response.

[1360] Step 8:

[1361] When a user uses the information provided, tries on products, or makes a purchase, points are awarded according to usage. The device sends this information to the server, which adds points to the user's account based on the points system. The input is usage data, and the output is updated points information.

[1362] This series of processing steps allows users to significantly improve their shopping experience in physical stores.

[1363] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1364] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1365] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1366] [Fourth embodiment]

[1367] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1368] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1369] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1370] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1371] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1372] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1373] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1374] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1375] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1376] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1377] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1378] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1379] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1380] The system of the present invention works by scanning a store's QR code with a smartphone, generating a virtual character that corresponds to the store and provides information about products and services to the user. This system is realized through the interaction of three parties: a server, a terminal, and a user.

[1381] Server processing

[1382] 1. Generate a QR code and register store information

[1383] The server generates a unique QR code for each store.

[1384] The unique information of the store accessed through this QR code (product information, service information, store features, etc.) is registered in a database.

[1385] 2. AI character generation

[1386] When a user scans a QR code, the server retrieves the store's information from the database.

[1387] The server uses a large-scale language model to generate a virtual character from the acquired store information.

[1388] Processing by device (smartphone)

[1389] 1. Scan the QR code

[1390] The user scans the store's QR code with the device's camera.

[1391] 2. Displaying an AI character and starting a conversation

[1392] Information about the AI ​​character generated from the server is received and displayed in AR on the device.

[1393] The characters provide information interactively in response to the user's questions and requests.

[1394] User Action

[1395] 1. Scan the QR code

[1396] The user scans the store's QR code with a smartphone app.

[1397] After scanning, the store information is sent to a server, which generates a virtual character and sends that information back to the device.

[1398] 2. Dialogue and Questions

[1399] Users ask the AI ​​character questions about products and services.

[1400] For example, if you ask, "What drink do you recommend?" the character will respond with something like, "I recommend the lavender latte."

[1401] Specific examples

[1402] In the case of "Cafe La Pearl"

[1403] 1. Scan the QR code

[1404] When a user scans the Cafe La Pearl QR code with their smartphone, the device sends the information to the server.

[1405] The server obtains information specific to Cafe La Pearl (e.g., products include lavender latte, espresso, and matcha latte) and generates a virtual character.

[1406] 2. Character Display

[1407] The generated virtual character is displayed on the user's smartphone.

[1408] When the user asks, "What drink would you recommend?" the character responds, "I recommend the elegant lavender latte. It has a calming scent."

[1409] Earn points and get new services

[1410] 1. Earn points when you use the service

[1411] By using this system, users can earn points, which are awarded based on their actions and usage and are managed on their devices.

[1412] For example, users can earn points by interacting with characters or spending a lot of time in the store.

[1413] 2. Offer of Benefits

[1414] Depending on the points accumulated, new services will be offered when certain conditions are met. For example, if you reach 50 points, you will receive a free drink on your next visit.

[1415] This will increase the repeat rate of users and enhance customer satisfaction.

[1416] This invention promotes interactive exchanges between stores and users, improving the user experience. In particular, it achieves a high level of interactivity by combining the generation of virtual characters using a large-scale language model, display using AR technology, and a point system. This system also provides great benefits to stores, as it allows users to gain a deeper understanding of store information and encourages continued use.

[1417] The processing flow will be explained below.

[1418] Step 1:

[1419] The server generates a unique QR code for each store, which contains a specific store ID and allows access to the store's specific information. The QR code is then stored in a database.

[1420] Step 2:

[1421] The server registers the product information, service information, and store features provided by the store in a database, allowing the information to be quickly retrieved when the QR code is scanned later.

[1422] Step 3:

[1423] A user scans a store's QR code using the camera on their smartphone. The QR code contains the store's ID, which the user's smartphone reads.

[1424] Step 4:

[1425] The user's device sends a request to the server containing the read store ID, which may also contain the user's identification information.

[1426] Step 5:

[1427] The server retrieves the store's unique information from the database based on the received store ID, including the store name, product list, and service details.

[1428] Step 6:

[1429] The server uses a large-scale language model (LLM) to generate virtual characters based on the acquired store information. The characters reflect the store's atmosphere and characteristics.

[1430] Step 7:

[1431] The generated virtual character data is sent to the user's device, including the character's visual data and dialogue script.

[1432] Step 8:

[1433] The user's device displays the received virtual character in AR, allowing the user to see the virtual character appear in real space through their smartphone.

[1434] Step 9:

[1435] Users can ask the virtual character questions about products and services, such as "What drink do you recommend?"

[1436] Step 10:

[1437] The device processes the user's question using voice recognition or text input, sends the question to the server, or may respond locally if it can be processed locally.

[1438] Step 11:

[1439] The server analyzes the questions sent by the user and generates appropriate answers using pre-defined dialogue scripts and large-scale language models, which are then returned to the terminal.

[1440] Step 12:

[1441] The user's device displays the answer received from the server through a virtual character. For example, the character may respond by voice or text, saying, "I recommend a lavender latte."

[1442] Step 13:

[1443] Users can enjoy an interactive experience by interacting with the virtual character and asking additional questions or requests as needed.

[1444] Step 14:

[1445] The server awards users points based on their usage of the system. The points are recorded in the user's account, and new services and benefits are offered when certain conditions are met.

[1446] Step 15:

[1447] Users can use the points they have accumulated to receive new services and benefits, such as a free drink on their next visit when they collect 50 points.

[1448] Example 1

[1449] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1450] Conventional store information systems provided information to users in a one-way manner, making it difficult to flexibly respond to the needs of individual users. Furthermore, when store information was updated, there was a lack of a way to notify users in a timely manner. Furthermore, incentives to increase user frequency of use were not effectively provided, making it difficult to increase repeat visits.

[1451] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1452] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in augmented reality, means for a user to read a store's QR code using a mobile device, means for transmitting the read QR code information to the server, means for the virtual character to respond to the user's inquiries in an interactive format, and means for awarding points to the user based on usage. This allows users to obtain information tailored to their needs in real time, improving the convenience of store use through interactive exchanges. Furthermore, utilizing a point system can increase the rate of user repeat visits.

[1453] A "server" is a computer system or device that provides services to clients over a network.

[1454] A "database" is a system or software that stores and manages data in an organized manner, and later extracts and uses it efficiently.

[1455] A "large-scale language model" is an artificial intelligence model that is trained based on large amounts of text data and has the ability to automatically generate text and understand meaning.

[1456] A "virtual character" is a computer-generated, computer-generated or text-based character used to interact with a user.

[1457] "Augmented reality display" is a technology that displays digital information overlaid on the real environment, making it possible to merge the real world with the virtual world.

[1458] A "mobile terminal" is a small, portable electronic device, including a smartphone, tablet, or mobile phone.

[1459] A "QR code" is a type of two-dimensional barcode that displays information in a matrix format, allowing for high-speed reading.

[1460] "Points" are units of reward awarded based on user behavior and frequency of use, and have value that can be exchanged for specific benefits or privileges.

[1461] MODE FOR CARRYING OUT THE INVENTION

[1462] The system of the present invention scans a store's QR code with a mobile device, and generates a virtual character corresponding to the store, which then guides the user through its products and services. The system is realized through the interaction of three parties: a server, a terminal, and a user.

[1463] Server processing

[1464] The server first uses Python's qrcode library to generate a unique QR code for each store. This QR code is generated by embedding the store's unique ID as data. The generated QR code is saved as an image file. Next, the server registers the store's unique information in a database such as MySQL. The registered information includes the store name, product list, service details, etc.

[1465] When a user scans a QR code, the server retrieves the store information corresponding to that QR code from a database. Based on the retrieved information, a virtual character is generated using a large-scale language model (for example, OpenAI's GPT-3). Specifically, a prompt sentence is created and sent to the AI ​​to generate the virtual character.

[1466] Processing by device (smartphone)

[1467] Users can use the react-native-camera library in React Native to scan a store's QR code using their mobile device's camera. The scanned QR code information is then sent from the device to the server. The generated virtual character information received from the server is then displayed in augmented reality on the device. This is achieved by combining software such as Unity and ARFoundation.

[1468] User Action

[1469] Users scan the QR code using an app on their mobile device. After scanning, store information is sent to the server. The information about the virtual character returned by the server is displayed on the device. Users can ask the virtual character questions about products and services. For example, if they ask, "What drink do you recommend?", the virtual character will respond, "I recommend the lavender latte."

[1470] Earn points and receive rewards

[1471] Each time a user uses the system, they are awarded points based on their behavior and frequency of use. Points are managed on the device and added up based on the user's behavior. When a certain number of points are accumulated, they are given special benefits that meet certain conditions, such as a free drink on their next visit.

[1472] For example, the prompt is:

[1473] You are a virtual character at Cafe La Pearl. What drink would you recommend to our customers?

[1474] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1475] Step 1: Generate a QR code

[1476] The server uses Python's qrcode library to generate a unique QR code for each store. The input is the store's unique ID, and the output is an image file of the QR code. Specifically, the server embeds the store ID as QR code data and saves the QR code as an image file.

[1477] Step 2: Register your store information

[1478] The server registers store-specific information in a database such as MySQL. The input is store information (product list, service details, store features, etc.), and the output is the information stored in the database. Specifically, the server executes SQL queries to insert information such as products, services, and features into the database.

[1479] Step 3: Scan the QR code

[1480] A user scans a store's QR code using the camera on their smartphone. The input is the QR code image captured by the camera, and the output is the QR code data. Specifically, the user opens the smartphone app, points the camera at the QR code, and scans it to obtain the data.

[1481] Step 4: Send QR code data to the server

[1482] The device sends the data read from the scanned QR code to the server. The input is the QR code data, and the output is the QR code information sent to the server. Specifically, the device sends the read QR code data to the server as an HTTP request.

[1483] Step 5: Get store information

[1484] The server retrieves store information from a database based on the received QR code data. The input is the QR code data (store ID), and the output is the store information retrieved from the database. Specifically, the server executes a database query using the store ID to retrieve information about the corresponding store.

[1485] Step 6: Generate a virtual character

[1486] The server uses a large-scale language model (e.g., OpenAI's GPT-3) to generate a virtual character from the acquired store information. The input is the store information and a prompt sentence for the generating AI model, and the output is the response of the generated virtual character. Specifically, the server creates a prompt sentence based on the store information and sends a request to the AI.

[1487] Step 7: Sending information about your virtual character

[1488] The server sends information about the generated virtual character to the terminal. The input is the response of the generated virtual character, and the output is the character information sent to the terminal. Specifically, the server sends the response of the virtual character to the terminal as an HTTP response.

[1489] Step 8: AR display of virtual characters

[1490] The device displays the received virtual character in augmented reality. The input is information about the virtual character, and the output is the virtual character displayed on the device. Specifically, Unity and ARFoundation are used to display the virtual character on the screen.

[1491] Step 9: Interact with the user

[1492] The user asks a question to a virtual character displayed in augmented reality. The input is the user's question, and the output is the response from the virtual character. For example, if the user asks, "What drink do you recommend?", the virtual character will respond, "I recommend a lavender latte."

[1493] Step 10: Points awarded

[1494] The server awards points based on the user's usage. The input is the user's behavioral data (frequency and content of interactions), and the output is the awarded points. Specifically, the server records the user's usage, calculates points based on that, and stores them in a points database.

[1495] Step 11: Offering rewards under certain conditions

[1496] The server provides new services and benefits when the awarded points meet certain conditions. The input is the user's point data, and the output is the benefits to be provided. Specifically, the server checks the user's points, and if the conditions are met, generates a reward coupon or service and notifies the user.

[1497] (Application example 1)

[1498] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1499] Currently, physical stores have limited methods for providing guidance and services to users, and there is a lack of effective means for communicating in-store information and product services. Furthermore, when users visit a store to obtain information, it is difficult to provide interactive and personalized support, making it difficult to improve customer satisfaction. Furthermore, there is a lack of methods for providing incentives based on user usage, making it difficult to increase repeat visit rates.

[1500] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1501] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in AR, means for the virtual character to respond to user inquiries in an interactive format, means for awarding points to the user according to usage, and means for accessing the server and generating a virtual character by reading a QR code in the store. This enables interactive and personalized information provision and service guidance in physical stores, thereby improving customer satisfaction and repeat business rates.

[1502] A "system for providing store-specific information" is a system that effectively guides users to information and services provided by stores.

[1503] "Store-specific information" refers to data such as product information, service information, and store characteristics that are unique to each store.

[1504] A "database" is a system or structure for storing and managing information in an organized manner.

[1505] A "large-scale language model" is an advanced AI model based on machine learning that has the ability to understand and generate human language.

[1506] A "virtual character" is a digitally generated character that can be interacted with.

[1507] "AR display" refers to the use of augmented reality technology to overlay digital information and objects onto the real world.

[1508] "Means of responding to user inquiries" refers to a function that enables a virtual character to provide appropriate responses to questions or requests from users.

[1509] "Means of awarding points to users according to usage" refers to a function that accumulates points each time a user uses the system, and manages and awards those points.

[1510] "Means for reading QR codes in stores" refers to the function that allows users to use a device such as a smartphone to scan QR codes installed in stores.

[1511] A "server" is a centralized computer system that processes and provides information over a network.

[1512] The system of the present invention functions through the mutual cooperation of three parties: a server, a terminal (smartphone), and a user. This system uses the following hardware and software.

[1513] Server processing

[1514] The server has the function of collecting store-specific information and registering it in a database. For example, the database stores product information, service information, and distinctive features of the store. MySQL is a suitable database to use. The Python library qrcode is used to generate QR codes, which are then distributed to each store.

[1515] When a user scans a QR code inside a store, the server uses a large-scale language model (e.g., OpenAI GPT-4) to generate a virtual character that corresponds to the store. This character then guides the user about products and services based on the store's unique information.

[1516] Processing by device (smartphone)

[1517] When a user scans a QR code in a store using their smartphone camera, the information in the QR code is read by the device. Using an AR library such as Vuforia, the device displays a virtual character overlaid on the real world. Unity can be used to display and interact with the AR character.

[1518] When interacting with a virtual character, the server generates a response based on the user's input and sends it to the device. For example, when a user asks, "What products do you recommend?", the smartphone sends the request to the server, which then queries the generative AI model based on the received information and provides an appropriate response to the device.

[1519] User Action

[1520] Users use a smartphone application to scan a QR code in a store, which sends store information to a server and generates a virtual character.

[1521] Users can then interact with the virtual character. For example, if they ask, "What drink do you recommend?" the character will respond in real time with, "I recommend the elegant lavender latte. It has a calming aroma." By continuing to interact, users can also accumulate points, which can be used to receive special benefits.

[1522] Specific examples

[1523] As a concrete example, consider the case where a user scans a QR code for a particular cafe. When the QR code is read, the server generates a virtual character based on the cafe's menu and features, and the character appears on the smartphone. When the user asks, "What's the recommendation this week?", the character responds, "This week's recommendation is our special chocolate cake."

[1524] Example prompts to input to a generative AI model:

[1525] Please generate lines for your virtual character based on the following information.

[1526] Store name: A specific cafe

[1527] Offerings: Chocolate Cake, Lavender Latte, Matcha Latte

[1528] Character's role: Store guide

[1529] User asks: "What's your pick this week?"

[1530] Character Quotes: <Generated quote goes here>

[1531] Hardware and Software Examples

[1532] Server: The database uses MySQL, and the Python qrcode library is used to generate QR codes.

[1533] Device: Scan the QR code with your smartphone camera. AR display will be performed using Unity and Vuforia.

[1534] Generative AI model: Dialogue content is generated using OpenAI GPT-4.

[1535] With the above configuration, the present invention makes it possible to provide interactive and personalized information and service guidance in brick-and-mortar stores, thereby improving customer satisfaction and repeat visit rates.

[1536] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1537] Step 1:

[1538] The server registers the store's unique information in a database. In this process, product information, service information, and characteristic features provided by the store are stored in a MySQL database. The specific input is the store's unique information, and the output is a database record in which this information is registered.

[1539] Step 2:

[1540] The server generates a unique QR code for each store. This QR code is generated using the Python library qrcode. The input is the store ID and URL, and the output is the generated QR code image file. This QR code is then distributed to each store.

[1541] Step 3:

[1542] A user uses the camera on their device (smartphone) to read a QR code in a store. The input is the QR code, and the output is a URL or identifier extracted from the QR code.

[1543] Step 4:

[1544] The device sends the information of the scanned QR code to the server. The input is the URL or identifier obtained from the QR code, and the output is that information is sent to the server.

[1545] Step 5:

[1546] The server retrieves the unique information of the corresponding store from the database based on the information in the received QR code. The input is the QR code identifier, and the output is the information of the corresponding store.

[1547] Step 6:

[1548] The server generates a virtual character using a large-scale language model (e.g., OpenAI GPT-4). In this step, a prompt is generated based on the acquired store information and sent to the AI ​​model. The input is the store's unique information and the generated prompt, and the output is the lines and movement information of the generated virtual character.

[1549] Step 7:

[1550] The server transmits information about the generated virtual character to the terminal. The input is the information about the generated virtual character, and the output is the transmission of this information to the terminal.

[1551] Step 8:

[1552] The device uses Unity and Vuforia to display AR based on the received virtual character information. The input is virtual character information, and the output is an AR character superimposed on the real world.

[1553] Step 9:

[1554] The user asks a question or makes a request to the virtual character, for example, "What products do you recommend?" The input is the user's question or request, and the output is the text data.

[1555] Step 10:

[1556] The terminal sends the user's question to the server. The input is the user's question text, and the output is that information is sent to the server.

[1557] Step 11:

[1558] The server receives the user's question and generates an appropriate response based on the generative AI model. The input is the user's question text, and the output is the AI-generated response text.

[1559] Step 12:

[1560] The server sends the generated response text to the terminal. The input is the generated response text, and the output is that information is sent to the terminal.

[1561] Step 13:

[1562] The terminal conveys the received response text to the user through the virtual character. The input is the response text, and the output is the voice of the virtual character or a text display.

[1563] Step 14:

[1564] The server awards points according to the user's usage. These points are updated in the database and managed for the user's use. The input is the user's usage data, and the output is the updated point data.

[1565] The above are the specific processing steps of the system for implementing the present invention.

[1566] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1567] This invention is a system in which a virtual character corresponding to a store is generated by scanning the store's QR code with a smartphone, and the virtual character guides the user to the store's products and services. The invention also combines an emotion engine that recognizes the user's emotions. This system is realized through interaction between the server, the terminal, and the user.

[1568] Server processing

[1569] 1. Generate a QR code and register store information

[1570] The server generates a unique QR code for each store, which contains a specific store ID and can be used to access store-specific information.

[1571] The server registers the product information, service information, and store features provided by the store in a database.

[1572] 2. AI character generation

[1573] When a user scans a QR code, the server retrieves the store's information from the database.

[1574] The server uses a large-scale language model to generate a virtual character that corresponds to the store from the acquired store information.

[1575] 3. Equipped with an emotional engine

[1576] The virtual character is equipped with an emotion engine that recognizes the user's emotions by analyzing the user's facial expressions, tone of voice, and choice of words.

[1577] Processing by device (smartphone)

[1578] 1. Scan the QR code

[1579] The user scans the store's QR code with the device's camera.

[1580] 2. Displaying an AI character and starting a conversation

[1581] The AI ​​character information sent from the server is received and displayed in AR on the device, allowing users to see the virtual character appear in real space through their smartphone.

[1582] The characters provide information interactively in response to the user's questions and requests.

[1583] 3. Emotion recognition and response changes

[1584] The emotion engine analyzes the user's emotional state, and the virtual character's responses change based on the analysis results.

[1585] For example, if the user looks tired, the character will make special suggestions such as, "Why don't you take a short break?"

[1586] User Action

[1587] 1. Scan the QR code

[1588] Users scan the store's QR code with their smartphone, after which the store's information is sent to a server and a virtual character is generated.

[1589] 2. Dialogue and Questions

[1590] Users can ask the virtual character questions about products and services, such as, "What drink do you recommend?"

[1591] The virtual character provides the store with the most relevant information and responds appropriately to questions.

[1592] 3. Specialized responses based on emotions

[1593] Users can receive special suggestions and responses based on their emotional state. For example, if a user is happy, they may receive suggestions such as, "Would you like to try our special dessert?"

[1594] Specific examples

[1595] In the case of "Cafe La Pearl"

[1596] 1. Scan the QR code

[1597] The user scans the "Cafe La Pearl" QR code with their smartphone, and the device sends the information to the server.

[1598] The server obtains information specific to Cafe La Pearl (for example, product information such as lavender latte, espresso, and matcha latte) and generates a virtual character based on that information.

[1599] 2. Character Display

[1600] The generated virtual character is displayed in AR on the user's smartphone. When the user asks, "What drink do you recommend?", the character responds, "I recommend the elegant lavender latte."

[1601] 3. Emotion recognition and response

[1602] The character analyzes the user's facial expressions and tone of voice, and if it determines that the user is tired, it will suggest, "Would you like me to show you a sofa seat where you can relax?"

[1603] Earn points and get new services

[1604] 1. Earn points when you use the service

[1605] By using this system, users earn points, which are recorded in their account and, when certain conditions are met, can be used to receive new services and benefits.

[1606] 2. Offer of Benefits

[1607] New services and benefits will be offered depending on the points accumulated. For example, 50 points will give you a free drink on your next visit.

[1608] This invention promotes interactive exchanges between stores and users, improving the user experience. In particular, by using virtual characters equipped with an emotion engine, it is possible to provide optimal services according to the user's emotional state. This system also provides great benefits to stores, as it allows users to gain a deeper understanding of store information and encourages continued use.

[1609] The processing flow will be explained below.

[1610] Step 1:

[1611] The server generates a unique QR code for each store. The generated QR code contains a specific store ID, which allows access to the store's unique information. After generation, this QR code and basic store information are stored in a database.

[1612] Step 2:

[1613] The server registers the product information, service information, and detailed store features provided by the store in a database, allowing the necessary information to be quickly retrieved when the QR code is scanned later.

[1614] Step 3:

[1615] The user uses the smartphone camera to scan the store's QR code, which then reads the store ID contained in the QR code and stores it on the user's device.

[1616] Step 4:

[1617] The user's device sends a request to the server containing the scanned store ID, which may also contain the user's identification information (e.g., user ID).

[1618] Step 5:

[1619] The server retrieves the store's unique information from the database based on the store ID received from the terminal, including the store name, product list, and service details.

[1620] Step 6:

[1621] The server uses a large-scale language model (LLM) to generate a virtual character based on the acquired store information, which reflects the store's atmosphere and characteristics.

[1622] Step 7:

[1623] Once the virtual character data is generated, the server connects the data to the emotion engine, which contains the algorithms and necessary data for recognizing the user's emotions.

[1624] Step 8:

[1625] The generated virtual character and emotion engine data are sent to the user's device, including the character's visual data, dialogue scripts, and emotion recognition algorithms.

[1626] Step 9:

[1627] The user's device displays the received virtual character in AR, allowing the user to see the virtual character in real space through the smartphone screen.

[1628] Step 10:

[1629] The user asks the virtual character a question about a product or service. For example, the user might ask, "What drink do you recommend?"

[1630] Step 11:

[1631] The user's device analyzes the user's question using voice recognition or text input, and sends the question content to the server. The emotion engine also analyzes the user's facial expressions and tone of voice, and sends this data to the server as appropriate.

[1632] Step 12:

[1633] The server receives and analyzes the user's question and transmitted emotion data, and uses a large-scale language model and emotion engine to generate appropriate answers and emotion-based responses to the user's question.

[1634] Step 13:

[1635] The server then sends the generated answers and responses to the user's device, including text and voice responses to questions, as well as special emotional responses.

[1636] Step 14:

[1637] The user's device displays the received answer through a virtual character. For example, the character may respond, "I recommend a lavender latte," and even make special suggestions, such as, "You look tired. A lavender latte will help you relax."

[1638] Step 15:

[1639] The server awards users points based on their usage of the system, which are recorded in their accounts and can be used to receive new services or rewards when certain conditions are met.

[1640] Step 16:

[1641] Users can use their accumulated points to receive new services and benefits. For example, 50 points will give you a free drink on your next visit.

[1642] Example 2

[1643] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1644] Conventional store guide systems lacked interactive methods for users to gain a deeper understanding of store information, resulting in a limited user experience. Furthermore, services were not provided that took user emotions into consideration, making it difficult to meet individual needs. Furthermore, point systems and rewards based on usage were insufficient, and there was a lack of methods to encourage continued use.

[1645] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1646] In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model, means for displaying the generated virtual character in augmented reality, means for a user to read a QR code using a camera on a terminal and transmit the information to the server, means for the virtual character to recognize the user's emotions and adjust responses based on the recognition results, and means for awarding points to the user based on usage. This allows users to interactively acquire store information and receive optimal services based on their emotions. Furthermore, the point system encourages continued usage and improves the user experience.

[1647] "Store-specific information" refers to individual information such as product information, service information, and store features related to a specific store.

[1648] A "database" refers to a system for efficiently storing, retrieving, and managing information.

[1649] A "large-scale language model" is an artificial intelligence model trained on a huge amount of text data, and is capable of performing natural language processing.

[1650] "Virtual character" refers to a computer-generated digital character that has the ability to interact and respond.

[1651] "Augmented reality display" refers to a technology that overlays digital information and objects onto the real world.

[1652] "Terminal" refers to a device such as a computer or smartphone that is directly operated by a user.

[1653] A "QR code" is a code that encodes information in two dimensions and can be read by a camera or other device.

[1654] "Emotion recognition" refers to technology that analyzes a user's facial expressions, tone of voice, word choice, etc. to identify their emotional state.

[1655] "Adjusting the response" refers to dynamically changing the information or suggestions provided by the virtual character based on the results of emotion recognition.

[1656] "Giving points" refers to giving points that are managed within the system to a user based on the user's usage record.

[1657] "Benefits" refers to additional services or benefits provided to users based on conditions such as points.

[1658] "Interactive" refers to the ability to communicate and operate two-way.

[1659] This invention is a system in which a user reads a QR code installed in a store with the camera of their device, and a virtual character corresponding to that store is generated, and the virtual character guides the user about products and services. This system is realized through the interaction of three parties: a server, a device, and a user.

[1660] The server first registers unique information corresponding to each store in a database. This information includes the store name, product information, service information, and store features. A relational database such as MySQL or PostgreSQL is used for the database.

[1661] The server then generates a unique QR code for each store. The QR code contains a specific store ID, which allows the server to access the store's unique information. The QR code is generated using the Python qrcode library.

[1662] When a user scans a store's QR code using the device's camera, the device sends the information to the server. This communication is securely carried out using the HTTPS protocol.

[1663] The server retrieves the store information from the database based on the QR code, and generates a virtual character based on that information using a large-scale language model (e.g., GPT-4). An example of a prompt is "Please generate a virtual character for the store based on the following information: {Store information}."

[1664] The generated virtual character is sent to the device and displayed in augmented reality (AR) on the device. This AR display uses Unity or Unreal Engine. The user can see the virtual character appear in real space through their smartphone.

[1665] The interaction between the user and the virtual character is carried out by the user asking questions about products and services, and the virtual character responding to them. The virtual character is equipped with an emotion recognition function that recognizes emotions by analyzing the user's facial expressions, tone of voice, and choice of words. This function uses Google Cloud's Vision API and Microsoft Azure's emotion analysis API.

[1666] The virtual character's responses are adjusted based on the emotion recognition technology used. For example, if the virtual character recognizes that the user is tired, it will suggest, "Why don't you take a break?"

[1667] Additionally, by using the system, users can earn points, which are recorded in their account and, when certain conditions are met, can unlock new services and rewards (such as a free drink on their next visit).

[1668] This allows users to interactively obtain store information and receive optimal service according to their emotions. Furthermore, the point system encourages continued use and improves the user experience.

[1669] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1670] Step 1:

[1671] Registering store-specific information

[1672] Subject: Server

[1673] Description: The server registers product information, service information, and store features provided by stores in a database.

[1674] Input: Product information, service information, and store features offered by the store

[1675] Data processing: This information is stored in a database in an appropriate format, for example, by inserting it into a relational database table.

[1676] Output: Store information saved in the database

[1677] Specific operation: The server registers the information received through the administrator interface in a database using an SQL query.

[1678] Step 2:

[1679] Generate a QR code

[1680] Subject: Server

[1681] Description: The server generates a unique QR code for each store. The QR code contains a specific store ID, which can be used to access store-specific information.

[1682] Input: Store ID

[1683] Data processing: Encode the store ID and generate a QR code.

[1684] Output: Generated QR code

[1685] Specific operation: Generate a QR code using Python's qrcode library and save it as an image file.

[1686] Step 3:

[1687] Scanning QR codes

[1688] Subject: User

[1689] Description: The user scans the QR code displayed in the store with the device's camera.

[1690] Input: QR code

[1691] Data processing: Decode the contents of the QR code to obtain the store ID.

[1692] Output: Store ID

[1693] What happens: A user scans a QR code using their smartphone's camera app and decodes the information.

[1694] Step 4:

[1695] Sending information

[1696] Subject: Terminal

[1697] Description: The device sends the information from the scanned QR code to the server.

[1698] Input: QR code content (store ID)

[1699] Data processing: Format the QR code content into an HTTP POST request format.

[1700] Output: Store ID sent to the server

[1701] Specific operation: The device sends data to the server using the HTTPS protocol.

[1702] Step 5:

[1703] Obtaining store information

[1704] Subject: Server

[1705] Description: The server retrieves the store information from the database.

[1706] Input: Store ID

[1707] Data processing: Retrieve store information from the database using an SQL query.

[1708] Output: Retrieved store information

[1709] Specific operation: Based on the store ID received by the server, a query is sent to the database to obtain information about the corresponding store.

[1710] Step 6:

[1711] Virtual character generation

[1712] Subject: Server

[1713] Description: The server uses a large-scale language model to generate a virtual character that corresponds to the store from the acquired store information.

[1714] Input: Store information

[1715] Data processing: Create prompt sentences for the generative AI model and input them into the AI ​​model.

[1716] Output: Data of the generated virtual character

[1717] Specific operation: Based on the store information, the server sends a prompt message to the AI ​​model saying, "Please generate a virtual character for the store based on the following information: {Store information}" to generate a virtual character.

[1718] Step 7:

[1719] Virtual character display

[1720] Subject: Terminal

[1721] Description: The device displays the virtual character information sent from the server in the AR application.

[1722] Input: Virtual character data

[1723] Data processing: Convert into a format suitable for AR display.

[1724] Output: Virtual character displayed on the device

[1725] Specific operation: The device uses Unity or Unreal Engine to display a virtual character overlaid on real space.

[1726] Step 8:

[1727] Starting a conversation

[1728] Subject: User

[1729] Description: The user asks the virtual character questions about products and services.

[1730] Input: User question

[1731] Data processing: The user's question is converted into text format and sent to the server.

[1732] Output: The question sent to the server

[1733] Specific operation: The user inputs a question using the input interface on the terminal and sends it to the server.

[1734] Step 9:

[1735] emotion recognition

[1736] Subject: Terminal

[1737] Description: The emotion engine recognizes emotions by analyzing the user's facial expressions, tone of voice, and word choice.

[1738] Input: User audio and video data

[1739] Data processing: Analyzing audio and video data using emotion recognition algorithms.

[1740] Output: Perceived emotional state

[1741] What it does: Evaluates the user's emotional state using Google Cloud's Vision API and Microsoft Azure's Sentiment Analysis API.

[1742] Step 10:

[1743] Response adjustment

[1744] Subject: Server

[1745] Description: Adjust the virtual character's response based on the analysis results.

[1746] Input: perceived emotional state, user question

[1747] Data processing: Send prompts to the response generation AI model to get the appropriate response.

[1748] Output: Adjusted response

[1749] Specific operation: Based on the emotion recognition results and the question content, the server sends a prompt to the AI ​​model saying, "If the user feels tired, please make suggestions to help them relax," and generates a response.

[1750] Step 11:

[1751] Points awarded

[1752] Subject: Server

[1753] Description: The server will give points to users for using this system.

[1754] Input: User action log

[1755] Data processing: Calculate points based on the algorithm of the points system and add them to the user's account.

[1756] Output: Updated points

[1757] Specific operation: The server continuously monitors the user's usage history and calculates the points for each specific action, such as "scanning a QR code gives 10 points," and assigns them to the user's account.

[1758] (Application example 2)

[1759] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1760] The user experience in traditional brick-and-mortar stores has been limited in the information provided, making it difficult to fully utilize the unique characteristics and services of each store.In addition, there is a lack of guidance that takes into account the user's emotional state, making it difficult to provide highly satisfying services that meet individual needs.

[1761] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for registering store-specific information in a database, means for acquiring store information from the database and generating a virtual character corresponding to the store using a generative AI model, means for displaying the generated virtual character in AR, means for the virtual character to respond to user inquiries in an interactive format, means for recognizing the user's emotions and changing the virtual character's response based on the user's emotional state, and means for awarding points to the user based on usage. This makes it possible to provide appropriate product information and service suggestions based on the user's emotional state, significantly improving the user experience in physical stores.

[1762] "Store-specific information" refers to information about a specific store, such as its location, product lineup, special services, and business hours.

[1763] "Means of registering in a database" refers to a method of storing and managing a store's unique information in digital format using a server or cloud computing.

[1764] A "generative AI model" is an artificial intelligence system that uses natural language processing and machine learning techniques to generate virtual characters based on input data.

[1765] "Virtual Character" means a digital character created by a generative AI model and displayed through a user interface.

[1766] "AR display" is a method of displaying digital content overlaid on the real environment using augmented reality technology.

[1767] "Interactive response means" refers to a method in which a virtual character uses voice or text to communicate two-way in response to a user's questions or requests.

[1768] "Means for recognizing emotions and varying response content based on emotional state" refers to a method for determining a user's emotions by analyzing the user's facial expressions, tone of voice, choice of words, etc., and adjusting the virtual character's response content based on the results.

[1769] "Means for awarding points to users" refers to a means for adding digital points to users based on their system usage or specific actions, and allowing those points to be used for future benefits or services.

[1770] The present invention is a system that allows users to receive product information tailored to their emotions in a physical store. This system is realized by the interaction of three main entities: a server, a terminal (smartphone), and the user.

[1771] Server processing

[1772] 1. Register store information

[1773] The server registers the unique information provided by each store in the database. This information includes location, product lineup, special services, business hours, etc. For example, product information such as "sneakers, business shoes, sandals" is registered as information for the shoe zone.

[1774] 2. Virtual Character Generation

[1775] When a user scans a QR code in a store with their device, the server retrieves the corresponding store information from the database and generates a virtual character corresponding to the store using a generative AI model (e.g., GPT-4). An example of a prompt sentence is as follows:

[1776] "You are a virtual character from the Shoe Zone. Please introduce the following products to users: sneakers, business shoes, and sandals."

[1777] 3. Emotion recognition

[1778] The virtual character is equipped with an emotion engine that recognizes the user's emotions. The server is equipped with algorithms (e.g., the FER library) to analyze the user's facial expressions, tone of voice, and word choice, etc. This allows it to diagnose the user's emotional state in real time and send that information back to the device.

[1779] Processing by device (smartphone)

[1780] 1. Scan the QR code

[1781] When a user scans a QR code located in each zone of the store (for example, the shoe department or the clothing department) with their smartphone, the corresponding store ID is sent to the server.

[1782] 2. Displaying and interacting with virtual characters

[1783] The virtual character sent from the server is displayed on the smartphone in AR. The user can directly interact with the virtual character displayed overlaid on the real environment through the smartphone screen. For example, if the user asks, "What shoes do you recommend?" the virtual character will respond, "I recommend sneakers."

[1784] 3. Emotion recognition and response changes

[1785] The device adjusts the virtual character's response based on the emotion diagnosis results received from the server. For example, if the device detects that the user is tired, the virtual character will suggest, "Would you like to try some relaxing shoes?"

[1786] User Action

[1787] 1. Scan the QR code

[1788] Users scan the QR code in the store with their smartphone, and this information is sent to the server, which retrieves information about participating stores.

[1789] 2. Dialogue and Questions

[1790] Users can ask the virtual character questions about products and services. For example, if the user asks, "What are the current sales information?", the virtual character will respond with, "Sneakers are 20% off today."

[1791] 3. Earning points and providing new services

[1792] Points are awarded each time the system is used, and new services and benefits are offered depending on the points accumulated. For example, "50 points will get you a free drink on your next visit."

[1793] In this way, it will be possible to provide appropriate product information and service suggestions based on the user's emotional state, significantly improving the user experience in physical stores.

[1794] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1795] Step 1:

[1796] The server registers the unique information provided by each store (location, product lineup, special services, business hours, etc.) in a database. Specifically, store staff enter the information through the management screen, and this is automatically saved in a digital format in the database. The input is in the form of text or image data, and is indexed as soon as it is registered in the database, allowing for quick subsequent searches and processing.

[1797] Step 2:

[1798] When a user scans a QR code in a physical store, the device sends the scanned QR code information (store ID) to the server. The device's camera function is used to read the QR code, and the scanned data is parsed to extract the store ID, which is then sent to the server as an API request. The input at this time is the QR code image data, and the output is the store ID.

[1799] Step 3:

[1800] When the server receives the store ID, it retrieves the corresponding store's unique information from the database and uses a generative AI model (e.g., GPT-4) to create a virtual character generation prompt. The server uses this prompt to drive a generative AI model based on the store information. An example of a virtual character generation prompt is: "You are a virtual character from the Shoe Zone. Please introduce the following products to the user: sneakers, business shoes, sandals." The input is the store ID and unique information, and the output is a dialogue script for the generated virtual character.

[1801] Step 4:

[1802] The server sends the generated dialogue script for the virtual character to the device. The device displays the virtual character in AR based on the received information and begins dialogue with the user. AR technology (such as ARCore or ARKit) is used to overlay the virtual character on the real world via the smartphone screen. The input is the dialogue script for the virtual character, and the output is the AR character displayed on the user's device screen.

[1803] Step 5:

[1804] When a user interacts with a virtual character and asks a question or makes a request, the device captures the voice and text data. The device then sends the captured data to a server, which analyzes it and generates an appropriate response. Natural language processing technology is used for the analysis, and text tokenization and semantic analysis are performed to understand the user's intent and the content of the question. The input is the user's voice and text data, and the output is the analyzed question.

[1805] Step 6:

[1806] The server uses an emotion engine to analyze the user's emotional state. Based on the facial expression information obtained from the voice data and camera footage, the server uses an emotion recognition algorithm (e.g., the FER library) to identify the user's emotional state. The input is the user's facial expression and voice data, and the output is the recognized emotional state.

[1807] Step 7:

[1808] The server determines the virtual character's response based on the recognized emotional state and the user's question, and sends it back to the device. For example, if the server recognizes that the user is tired, it generates a response such as "Would you like to try some relaxing shoes?" The input in this case is the emotional state and the question, and the output is a customized response.

[1809] Step 8:

[1810] When a user uses the information provided, tries on products, or makes a purchase, points are awarded according to usage. The device sends this information to the server, which adds points to the user's account based on the points system. The input is usage data, and the output is updated points information.

[1811] This series of processing steps allows users to significantly improve their shopping experience in physical stores.

[1812] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1813] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1814] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1815] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1816] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1817] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1818] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1819] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1820] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1821] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1822] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1823] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1824] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1825] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1826] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1827] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1828] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1829] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1830] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1831] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1832] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1833] The following is further disclosed regarding the above embodiment.

[1834] (Claim 1)

[1835] In a system for providing information corresponding to a store,

[1836] A means for registering store-specific information in a database;

[1837] A means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model;

[1838] means for AR displaying the generated virtual character;

[1839] a means for the virtual character to interactively respond to user inquiries;

[1840] A means of awarding points to users according to their usage,

[1841] A system including:

[1842] (Claim 2)

[1843] 10. The system of claim 1, further comprising means for the virtual character to communicate with individual virtual characters owned by the user.

[1844] (Claim 3)

[1845] 2. The system according to claim 1, further comprising means for providing new services or benefits when the points satisfy a specific condition.

[1846] "Example 1"

[1847] (Claim 1)

[1848] In a system for providing information corresponding to a store,

[1849] A means for registering store-specific information in a database;

[1850] A means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model;

[1851] means for displaying the generated virtual character in augmented reality;

[1852] A means for a user to read a store's QR code using a mobile device;

[1853] means for transmitting the read QR code information to a server;

[1854] a means for the virtual character to interactively respond to user inquiries;

[1855] A means of awarding points to users according to their usage,

[1856] A system including:

[1857] (Claim 2)

[1858] 10. The system of claim 1, further comprising means for the virtual character to communicate with individual virtual characters owned by the user.

[1859] (Claim 3)

[1860] 2. The system according to claim 1, further comprising means for providing new services or benefits when the points satisfy a specific condition.

[1861] "Application Example 1"

[1862] (Claim 1)

[1863] In a system for providing information corresponding to a store,

[1864] A means for registering store-specific information in a database;

[1865] A means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model;

[1866] means for AR displaying the generated virtual character;

[1867] a means for the virtual character to interactively respond to user inquiries;

[1868] A means of awarding points to users according to their usage,

[1869] By scanning a QR code in the store, you can access the server and generate a virtual character.

[1870] A system including:

[1871] (Claim 2)

[1872] 10. The system of claim 1, further comprising means for the virtual character to communicate with individual virtual characters owned by the user.

[1873] (Claim 3)

[1874] 2. The system according to claim 1, further comprising means for providing new services or benefits when the points satisfy a specific condition.

[1875] "Example 2: Combining Emotion Engines"

[1876] (Claim 1)

[1877] In a system for providing information corresponding to a store,

[1878] A means for registering store-specific information in a database;

[1879] A means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model;

[1880] means for displaying the generated virtual character in augmented reality;

[1881] A means for a user to read the QR code using a camera on the terminal and transmit the information to a server;

[1882] means for the virtual character to recognize the user's emotion and adjust its response based on the recognition result;

[1883] A means for granting points to users according to their usage;

[1884] A system including:

[1885] (Claim 2)

[1886] 10. The system of claim 1, further comprising means for the virtual character to recognize emotions by analyzing the user's facial expressions, tone of voice, and word choice.

[1887] (Claim 3)

[1888] 2. The system according to claim 1, further comprising means for providing new services or benefits when the points satisfy a specific condition.

[1889] "Application example 2 when combining emotion engines"

[1890] (Claim 1)

[1891] In a system for providing information corresponding to a store,

[1892] A means for registering store-specific information in a database;

[1893] A means for acquiring store information from the database and generating a virtual character corresponding to the store using a generative AI model;

[1894] means for AR displaying the generated virtual character;

[1895] a means for the virtual character to interactively respond to user inquiries;

[1896] means for recognizing the user's emotions and varying the virtual character's responses based on the user's emotional state;

[1897] A means of awarding points to users according to their usage,

[1898] A system including:

[1899] (Claim 2)

[1900] 10. The system of claim 1, further comprising means for the virtual character to communicate with individual virtual characters owned by the user.

[1901] (Claim 3)

[1902] 2. The system according to claim 1, further comprising means for providing new services or benefits when the points satisfy a specific condition. [Explanation of symbols]

[1903] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. In a system for providing information corresponding to a store, A means for registering store-specific information in a database; A means for acquiring store information from the database and generating a virtual character corresponding to the store using a large-scale language model; means for AR displaying the generated virtual character; a means for the virtual character to interactively respond to user inquiries; A means of awarding points to users according to their usage, A system including:

2. 10. The system of claim 1, further comprising means for the virtual character to communicate with individual virtual characters owned by the user.

3. The system according to claim 1 , further comprising means for providing new services or benefits when the points satisfy a specific condition.

Images