system

Abstract

A system is provided.SOLUTION: A system, comprising: means for receiving user credentials and verifying the credentials; means for retrieving product information from a database based on a search query entered by a user; means for accepting a user question and generating an answer to the question using a generative model; means for delivering a URL for corresponding VR content based on a VR content request from a user; and means for generating and managing a real-time chat session between users.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional chemical product exhibitions and sales meetings face high participation costs and regional restrictions, and many offline events have been canceled or postponed, especially due to the coronavirus pandemic. This has created a need for ways for manufacturers and visitors to exchange information without meeting face-to-face. Furthermore, traditional online platforms have struggled to provide real-time Q&A and interactive VR experiences. This has led to challenges such as inefficient information gathering and less accurate market insights. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention provides the following means. The system includes a means for receiving user authentication information and verifying the authentication information, a means for retrieving product information from a database based on a search query entered by the user, a means for accepting user questions and generating answers to the questions using a generative model, a means for delivering URLs of corresponding VR content based on VR content requests from users, and a means for generating and managing real-time chat sessions between users. This allows manufacturers and visitors to exchange information without geographical constraints, and provides real-time question and answering and interactive VR experiences, enabling efficient and detailed information collection. The system also analyzes data obtained through the digital platform and provides market insights, helping manufacturers optimize their market strategies.

[0006] "User authentication information" refers to the identification information required when a user accesses the platform, and typically consists of a user ID and password.

[0007] A "database" is a central system for systematically storing and managing product information, user information, etc., and is used to respond to user search queries and obtain product information.

[0008] "Generative models" refer to algorithms and systems that use artificial intelligence techniques to generate appropriate answers to user questions.

[0009] "VR Content" means virtual reality demonstrations or exhibits designed to be interactively experienced by users.

[0010] A "URL" is a unique address that identifies a resource on the Internet and is used to provide access to VR content.

[0011] A "chat session" is a communication session for exchanging messages between users in real time.

[0012] "System" refers to the overall mechanism or structure that integrates multiple means to provide a set of services. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] The system for implementing the present invention utilizes generative AI and a digital platform, including functions for user authentication, product information search and display, question answering, VR experiences, and networking.

[0035] User Authentication and Login

[0036] A user launches a messaging app and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information from the database, and if authentication is successful, it creates a user session and returns the main menu to the device along with a success message. The device then displays this main menu to the user.

[0037] Search and view product information

[0038] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[0039] For example, if a user searches for "polymeric materials," the server that receives this search query retrieves related product information from the database and displays it on the user's terminal.

[0040] Questions and Answers

[0041] A user enters a question about a particular product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[0042] For example, if a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[0043] Running a VR experience

[0044] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[0045] For example, if a user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[0046] Networking and Communication

[0047] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[0048] For example, if a user wants to talk to a representative of another manufacturer participating in an exhibition, the user can send a message to that representative via the chat function and have a conversation in real time.

[0049] The above is a concrete implementation of this system. The system is designed to enable users to exchange information without geographical constraints and to efficiently gather information through real-time question and answering and interactive VR experiences. This facilitates communication between manufacturers and visitors, contributing to product understanding and the optimization of market strategies.

[0050] The processing flow will be explained below.

[0051] User Authentication and Login

[0052] Step 1:

[0053] The user launches the messaging app and enters their user ID and password on the login screen.

[0054] Step 2:

[0055] The terminal sends the authentication information (user ID and password) entered by the user to the server.

[0056] Step 3:

[0057] The server checks the received authentication information against a database and, if authentication is successful, creates a user session.

[0058] Step 4:

[0059] The server returns the main menu data to the terminal along with an authentication success message.

[0060] Step 5:

[0061] The terminal displays the main menu to the user.

[0062] Search and view product information

[0063] Step 1:

[0064] A user selects the product search option on the main menu and enters a search query.

[0065] Step 2:

[0066] The terminal transmits the search query entered by the user to the server.

[0067] Step 3:

[0068] The server retrieves product information that matches the search query from a database.

[0069] Step 4:

[0070] The server returns the acquired search results to the terminal.

[0071] Step 5:

[0072] The terminal displays the search results to the user.

[0073] Questions and Answers

[0074] Step 1:

[0075] A user enters a question about a particular product.

[0076] Step 2:

[0077] The terminal sends this question to the server.

[0078] Step 3:

[0079] The server generates answers to questions using generative AI models.

[0080] Step 4:

[0081] The server returns the generated answer to the terminal.

[0082] Step 5:

[0083] The terminal displays the generated answer to the user.

[0084] Running a VR experience

[0085] Step 1:

[0086] The user selects the VR content for the product demo.

[0087] Step 2:

[0088] The terminal transmits the selected product ID to the server.

[0089] Step 3:

[0090] The server retrieves the URL of the corresponding VR content from the database.

[0091] Step 4:

[0092] The server returns the acquired URL to the terminal.

[0093] Step 5:

[0094] The device will use this URL to launch the VR viewer and provide the VR experience.

[0095] Networking and Communication

[0096] Step 1:

[0097] The user selects a target person to begin a networking session.

[0098] Step 2:

[0099] The terminal transmits the target person's information to the server and requests the initiation of a chat session.

[0100] Step 3:

[0101] The server creates a new chat session and generates a session ID.

[0102] Step 4:

[0103] The server returns the generated session ID to the terminal.

[0104] Step 5:

[0105] The terminal displays a real-time chat window and notifies the user that a session has begun.

[0106] Step 6:

[0107] The user types a message in the chat window.

[0108] Step 7:

[0109] The terminal transmits the input message to the server.

[0110] Step 8:

[0111] The server relays the received message to the target user.

[0112] Example 1

[0113] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0114] Currently, there is a lack of an integrated system that allows users to smoothly use multiple functions (authentication, product search, question and answering, VR experience, real-time chat, etc.) on a single platform. At the same time, there are limitations on the means to enrich users' interactive experiences and the means of real-time communication. To solve this issue, it is necessary to centrally manage and integrate all functions.

[0115] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0116] In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting user questions and generating answers to the questions using a generative AI model, means for delivering URLs of corresponding VR content based on VR content requests from users, means for generating and managing real-time chat sessions between users, and means for users to input messages into real-time chat windows and for transmitting and relaying the messages. This allows users to seamlessly use multiple functions and efficiently gather information and communicate.

[0117] "User authentication information" refers to information such as ID and password entered by a user to access a system.

[0118] A database is a collection of information that is organized and stored according to certain rules, allowing for efficient searching and retrieval.

[0119] "Product information" refers to detailed data, specifications, features, and other information about a product.

[0120] A "generative AI model" is a model that uses artificial intelligence technology to realize question-answering and data generation similar to those performed by humans.

[0121] "VR content" refers to digital content that provides a visual and auditory experience using virtual reality technology.

[0122] "URL" is a uniform resource identifier that indicates the location of an information resource on the Internet.

[0123] A "real-time chat session" is a communication session in which multiple users simultaneously send and receive messages and communicate in real time.

[0124] A "messaging application" is software that allows users to send and receive text messages, voice messages, images, etc.

[0125] A "session ID" is a unique identifier for identifying a particular user session.

[0126] A "server" is a computer system that provides services to other computers on a network.

[0127] The system for implementing the present invention utilizes generative AI models and a digital platform that includes functions for user authentication, product information search and display, question answering, VR experiences, and networking.

[0128] User Authentication and Login

[0129] A user launches a messaging application and enters authentication information (user ID and password). The terminal sends this authentication information to the server. The server verifies the authentication information from a database, and if authentication is successful, it creates a user session and returns a main menu to the terminal along with a success message. The terminal then displays this main menu to the user.

[0130] Examples:

[0131] A user opens a messaging application and enters "user123" as the user ID and "password321" as the password. The device sends this to the server, which verifies it and returns the message "Authentication successful. The main menu will be displayed" if authentication is successful.

[0132] Example prompt sentence:

[0133] "When a user enters their user ID and password, explain how the server validates that information."

[0134] Search and view product information

[0135] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[0136] Examples:

[0137] When a user searches for "polymer materials," the server receives this search query, retrieves related product information from the database, and displays it on the user's terminal.

[0138] Example prompt sentence:

[0139] "If a user searches for information about a particular polymeric material, explain how the system retrieves and displays that information."

[0140] Questions and Answers

[0141] A user enters a question about a specific product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[0142] Examples:

[0143] If a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[0144] Example prompt sentence:

[0145] "If a user asks a question about a specific product, explain how you can use a generative AI model to generate an answer to that question."

[0146] Running a VR experience

[0147] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[0148] Examples:

[0149] If the user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[0150] Example prompt sentence:

[0151] "When a user selects a VR experience, explain how the system delivers that VR content."

[0152] Networking and Communication

[0153] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[0154] Examples:

[0155] If a user wants to talk to a representative from another manufacturer participating in the exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[0156] Example prompt sentence:

[0157] "Please explain how the system works to allow users to communicate with other manufacturer representatives in real time."

[0158] This concludes the description of the embodiment of the present invention. This system is designed to allow users to seamlessly use multiple functions on an integrated platform, enabling users to efficiently gather information and communicate.

[0159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0160] User Authentication and Login

[0161] Processing Steps:

[0162] Step 1:

[0163] The user launches a messaging app and enters authentication information (user ID and password), which then becomes the input data.

[0164] Input: User ID and password

[0165] Output: None

[0166] Step 2:

[0167] The terminal transmits the entered authentication information to the server.

[0168] Input: User ID and password

[0169] Output: Authentication information sent to the server

[0170] Step 3:

[0171] The server validates the received authentication information against a database. The database check confirms the authenticity of the authentication information.

[0172] Input: The authentication information sent to the server

[0173] Output: Authentication success or failure

[0174] Step 4:

[0175] If authentication is successful, the server creates a user session and returns the main menu to the terminal along with a success message.

[0176] Input: Authentication success information

[0177] Output: Success message and main menu

[0178] Step 5:

[0179] The terminal displays the received main menu to the user.

[0180] Input: Success message and main menu

[0181] Output: Display main menu

[0182] Search and view product information

[0183] Processing Steps:

[0184] Step 1:

[0185] A user selects a product search option in the main menu and enters a search query, which becomes the input data.

[0186] Input: search query

[0187] Output: None

[0188] Step 2:

[0189] The terminal sends a search query to the server.

[0190] Input: search query

[0191] Output: The search query sent to the server

[0192] Step 3:

[0193] The server retrieves product information that matches the query from the database. Matching product information is collected by searching the database.

[0194] Input: The search query sent to the server

[0195] Output: Search results

[0196] Step 4:

[0197] The server returns the search results to the terminal.

[0198] Input: Search results

[0199] Output: Search results sent back to your device

[0200] Step 5:

[0201] The terminal displays the search results to the user.

[0202] Input: Search results

[0203] Output: Display of search results

[0204] Questions and Answers

[0205] Processing Steps:

[0206] Step 1:

[0207] A user enters a question about a particular product, which then becomes the input data.

[0208] Input: User question

[0209] Output: None

[0210] Step 2:

[0211] The terminal sends a query to the server.

[0212] Input: User question

[0213] Output: The question sent to the server

[0214] Step 3:

[0215] The server generates answers to questions using a generative AI model, which analyzes the content of the question and generates an answer.

[0216] Input: The question sent to the server

[0217] Output: The generated answer

[0218] Step 4:

[0219] The server generates a response and sends it back to the terminal.

[0220] Input: Generated answer

[0221] Output: Answer sent back to the terminal

[0222] Step 5:

[0223] The terminal displays the generated answer to the user.

[0224] Input: Generated answer

[0225] Output: Display the answer

[0226] Running a VR experience

[0227] Processing Steps:

[0228] Step 1:

[0229] The user selects the VR content for the product demo, and the selected product ID becomes the input data.

[0230] Input: Product ID

[0231] Output: None

[0232] Step 2:

[0233] The terminal transmits the selected product ID to the server.

[0234] Input: Product ID

[0235] Output: Product ID sent to the server

[0236] Step 3:

[0237] The server retrieves the URL of the corresponding VR content from the database and identifies the URL of the corresponding VR content by database matching.

[0238] Input: Product ID sent to the server

[0239] Output: URL of VR content

[0240] Step 4:

[0241] The server returns the URL to the device.

[0242] Input: VR content URL

[0243] Output: URL sent back to the device

[0244] Step 5:

[0245] The device uses the URL to launch the VR viewer and provide the VR experience.

[0246] Input: VR content URL

[0247] Output: Providing a VR experience

[0248] Networking and Communication

[0249] Processing Steps:

[0250] Step 1:

[0251] To start a networking session, a user selects a target person, and the target person's information becomes input data.

[0252] Input: Subject information

[0253] Output: None

[0254] Step 2:

[0255] The terminal transmits the target person's information to the server.

[0256] Input: Subject information

[0257] Output: The target information sent to the server

[0258] Step 3:

[0259] The server creates a new chat session and returns a session ID to the terminal. The server generates a session ID and starts a chat session.

[0260] Input: Subject information sent to the server

[0261] Output: The generated session ID.

[0262] Step 4:

[0263] The terminal displays a real-time chat window and notifies the user that a session has begun.

[0264] Input: Generated session ID

[0265] Output: Chat window display

[0266] Step 5:

[0267] When a user types a message, the device sends the message to the server, which relays it to the target user.

[0268] Input: Message from the user

[0269] Output: Message sent to server and relayed to target user

[0270] Specific working example:

[0271] If a user wants to talk to a representative from another manufacturer participating in the exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[0272] (Application example 1)

[0273] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0274] Currently, there are many online shopping platforms on the market that allow users to search for various product information. However, users often experience inconveniences when checking product information, answering questions, and experiencing the products. Furthermore, the user experience can be limited due to limited interactive display of product information and real-time communication with other users and staff. Conventional systems do not provide users with sufficient information and support, making it difficult to provide the comfortable and efficient shopping experience desired by users. Therefore, this invention aims to provide a more comprehensive and interactive online shopping environment by integrating the functions of user authentication, product information search and display, question answering, VR experience provision, and real-time chat.

[0275] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0276] In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting user questions and generating answers to the questions using a generative AI model, means for delivering URLs of corresponding VR content based on VR content requests from users, means for generating and managing real-time chat sessions between users, means for displaying searched product information as 3D models or images, means for providing generated answers by voice, and means for communicating with other users and staff in real time. This allows users to enjoy a comprehensive online shopping experience, from authentication to question and answering, interactive display of product information, VR experiences, and real-time communication.

[0277] "User Credentials" means information used to identify a user and authorize access to a system.

[0278] "Means for verifying" is a function that executes a process to confirm whether user authentication information is correct.

[0279] A "search query" is a request entered by a user to retrieve specific information from a database.

[0280] "Product information" is detailed information about a particular product, typically stored in a database.

[0281] A "generative AI model" is an artificial intelligence model used to automatically generate answers to user questions.

[0282] A "VR content request" is a request submitted by a user to initiate a virtual reality experience.

[0283] A "real-time chat session" is an interactive session for sending and receiving text messages between users in real time.

[0284] "3D Model or Image" means a three-dimensional model or still image used to visually display product information.

[0285] The "means for providing by voice" is a function for providing the generated answer to the user as voice.

[0286] "Means of communicating in real time" refers to a function that allows you to communicate with other users and staff in real time.

[0287] MODE FOR CARRYING OUT THE INVENTION

[0288] The basic program for realizing this system includes the following means: Specific hardware required is a server, a user device (a smartphone or head-mounted display), a generative AI model, and a VR viewer.

[0289] Receiving and verifying user credentials

[0290] A user launches an application and enters authentication information (user ID and password). The terminal sends this authentication information to the server, which verifies it using a database. If authentication is successful, the server creates a user session and returns a main menu to the terminal along with a success message. The terminal displays this main menu to the user.

[0291] Search and view product information

[0292] A user selects the product search option from the main menu and enters a search query. The device sends this search query to the server, which retrieves product information matching the query from its database. This information is returned to the device as a 3D model or image, and the device displays the search results to the user. For example, if a user searches for "polymer materials," the server retrieves and displays related product information.

[0293] Questions and Answers

[0294] When a user enters a question about a particular product, the device sends the question to the server. The server uses a generative AI model to generate an answer to the question and returns it to the device. The device then provides the generated answer via voice. For example, if a user asks, "What are the main components of this compound?" the generative AI model might respond, "The main components are aluminum and silicon."

[0295] Running a VR experience

[0296] When a user selects a product demo VR content, the device sends the selected product ID to the server, which then retrieves the URL of the corresponding VR content from the database. This URL is then returned to the device, which then launches the VR viewer to provide the VR experience. For example, if a user selects a VR demo of a specific chemical process, the device retrieves the URL of the VR content corresponding to that process, and the user can use this URL to experience virtual reality.

[0297] Networking and Communication

[0298] When a user selects a target person to start a networking session, the terminal sends the target person's information to the server, and the server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, and the server relays the message to the target user. For example, if a user wants to talk to another representative at an exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[0299] Examples of concrete examples and prompts

[0300] As a specific example of user authentication, a user attempts to log in with the ID "user123" and the password "password123." As a specific example of product search, a search is performed for the chemical product "polymer material," and related product information is displayed. As a specific example of question and answer, a user asks "What are the main components of this compound?" via voice, and the generative AI model provides the answer "The main components are aluminum and silicon." As a specific example of a VR experience, a user starts a VR experience to view the details of a specific product in 3D. As a specific example of networking, a user can chat with other personnel in real time.

[0301] An example of a prompt to input to a generative AI model is as follows:

[0302] When asked, "What is the main component of this compound?", be specific about the main component of the chemical.

[0303] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0304] Step 1:

[0305] The user starts the application and enters authentication information (user ID and password). The device sends this authentication information to the server, which then verifies it using a database.

[0306] Input: User ID and password

[0307] Data processing: The server retrieves the authentication information from the database and compares it with the user's input.

[0308] Output: If authentication is successful, create a user session and generate a success message.

[0309] Step 2:

[0310] If the authentication is successful, the server sends the main menu to the terminal, which then displays the main menu to the user.

[0311] Input: Authentication success message

[0312] Data processing: The server saves the user's session information and prepares the main menu data.

[0313] Output: The main menu is displayed on the terminal.

[0314] Step 3:

[0315] The user selects the product search option from the main menu and enters a search query. The device sends this search query to the server, which retrieves product information matching the query from its database.

[0316] Input: search query

[0317] Data processing: The server uses a database search algorithm to retrieve product information that matches the query.

[0318] Output: A list of products matching the query will be displayed on the terminal.

[0319] Step 4:

[0320] The server transmits the acquired product information as a 3D model or image to the terminal, which then displays this information to the user.

[0321] Input: Product information list

[0322] Data processing: Processing to convert product information into 3D models or image formats.

[0323] Output: Product information displayed as a 3D model or image.

[0324] Step 5:

[0325] A user enters a question about a particular product, and the device sends the question to a server, which uses a generative AI model to generate an answer to the question.

[0326] Input: Product Question

[0327] Data processing: Questions are input as prompts into the AI ​​model to generate answers.

[0328] Output: The answer is sent to the terminal and displayed.

[0329] Step 6:

[0330] The terminal presents the user with the option to provide the generated answer aloud, and if selected by the user, outputs it as speech using speech synthesis technology.

[0331] Input: Answer text

[0332] Data processing: Converting text into audio data (speech synthesis).

[0333] Output: A spoken answer is provided.

[0334] Step 7:

[0335] When a user selects a product demo VR content, the terminal sends the selected product ID to the server, and the server retrieves the URL of the corresponding VR content from the database.

[0336] Input: Product ID

[0337] Data processing: The server retrieves the URL of the VR content corresponding to the product ID from the database.

[0338] Output: The URL of the VR content is sent to the device.

[0339] Step 8:

[0340] The device uses the obtained URL to launch the VR viewer and provide the user with a VR experience.

[0341] Input: VR content URL

[0342] Data processing: The VR viewer loads the URL and displays the corresponding VR content.

[0343] Output: The VR experience is delivered to the user.

[0344] Step 9:

[0345] The user selects a target person to start a networking session, and the device sends the target person's information to the server, which creates a new chat session and returns a session ID to the device.

[0346] Input: Subject information

[0347] Data processing: Create a new chat session and issue a session ID.

[0348] Output: The session ID is sent to the terminal.

[0349] Step 10:

[0350] The terminal displays a real-time chat window and allows the user to input a message, which is then sent to the server, which relays the message to the target user.

[0351] Input: User's message

[0352] Data processing: Processes messages to be relayed in real time.

[0353] Output: A message is sent to the target user.

[0354] By performing these steps, users can achieve a comprehensive online shopping experience, from authentication to question and answering, interactive display of product information, VR experience, and real-time communication.

[0355] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0356] The system for implementing the present invention utilizes generative AI and a digital platform that includes user authentication, product information search and display, question answering, VR experiences, networking, and an emotion engine that recognizes user emotions.

[0357] User Authentication and Login

[0358] A user launches a messaging app and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information from the database, and if authentication is successful, it creates a user session and returns the main menu to the device along with a success message. The device then displays this main menu to the user.

[0359] Search and view product information

[0360] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[0361] For example, if a user searches for "polymeric materials," the server that receives this search query retrieves related product information from the database and displays it on the user's terminal.

[0362] Questions and Answers

[0363] A user enters a question about a particular product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[0364] For example, if a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[0365] Running a VR experience

[0366] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[0367] For example, if a user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[0368] Networking and Communication

[0369] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[0370] For example, if a user wants to talk to a representative of another manufacturer participating in an exhibition, the user can send a message to that representative via the chat function and have a conversation in real time.

[0371] Introducing the Emotion Engine

[0372] When a user sends a message or question, the emotion engine recognizes the user's emotional state. The device then sends this emotional information to the server, which then uses the emotion analysis results obtained by the emotion engine to generate an appropriate response using a generative AI model.

[0373] For example, if a user types a question and the emotion engine recognizes that the user is nervous, the server is designed to take that information into account and generate a reassuring answer.

[0374] Personalized VR Experience

[0375] The emotion engine recognizes the user's emotional state and personalizes the VR content experience: if the user is calm, a VR experience with detailed technical explanations is provided, while if the user is nervous, content with more general and simplified explanations is provided.

[0376] For example, if the emotion engine recognizes that the user is nervous, the server will prioritize providing more intuitive and visually appealing content.

[0377] The above is a concrete example of combining this system with an emotion engine. The system is designed to enable users to exchange information regardless of geographical constraints and to efficiently collect information through real-time question and answering, interactive VR experiences, and emotional responses. This facilitates communication between manufacturers and visitors, contributing to product understanding and the optimization of market strategies.

[0378] The processing flow will be explained below.

[0379] User Authentication and Login

[0380] Step 1:

[0381] The user launches the messaging app and enters their user ID and password on the login screen.

[0382] Step 2:

[0383] The terminal sends the authentication information (user ID and password) entered by the user to the server.

[0384] Step 3:

[0385] The server checks the received authentication information against a database and, if authentication is successful, creates a user session.

[0386] Step 4:

[0387] The server returns the main menu data to the terminal along with an authentication success message.

[0388] Step 5:

[0389] The terminal displays the main menu to the user.

[0390] Search and view product information

[0391] Step 1:

[0392] A user selects the product search option on the main menu and enters a search query.

[0393] Step 2:

[0394] The terminal transmits the search query entered by the user to the server.

[0395] Step 3:

[0396] The server retrieves product information that matches the search query from a database.

[0397] Step 4:

[0398] The server returns the acquired search results to the terminal.

[0399] Step 5:

[0400] The terminal displays the search results to the user.

[0401] Questions and Answers

[0402] Step 1:

[0403] A user enters a question about a particular product.

[0404] Step 2:

[0405] The terminal sends this question to the server.

[0406] Step 3:

[0407] The server generates answers to questions using generative AI models.

[0408] Step 4:

[0409] The server returns the generated answer to the terminal.

[0410] Step 5:

[0411] The terminal displays the generated answer to the user.

[0412] Running a VR experience

[0413] Step 1:

[0414] The user selects the VR content for the product demo.

[0415] Step 2:

[0416] The terminal transmits the selected product ID to the server.

[0417] Step 3:

[0418] The server retrieves the URL of the corresponding VR content from the database.

[0419] Step 4:

[0420] The server returns the acquired URL to the terminal.

[0421] Step 5:

[0422] The device will use this URL to launch the VR viewer and provide the VR experience.

[0423] Networking and Communication

[0424] Step 1:

[0425] The user selects a target person to begin a networking session.

[0426] Step 2:

[0427] The terminal transmits the target person's information to the server and requests the initiation of a chat session.

[0428] Step 3:

[0429] The server creates a new chat session and generates a session ID.

[0430] Step 4:

[0431] The server returns the generated session ID to the terminal.

[0432] Step 5:

[0433] The terminal displays a real-time chat window and notifies the user that a session has begun.

[0434] Step 6:

[0435] The user types a message in the chat window.

[0436] Step 7:

[0437] The terminal transmits the input message to the server.

[0438] Step 8:

[0439] The server relays the received message to the target user.

[0440] Introducing the Emotion Engine

[0441] Step 1:

[0442] The user enters a message or question.

[0443] Step 2:

[0444] The device sends the input message or question to the emotion engine, which analyzes the emotional state.

[0445] Step 3:

[0446] The emotion engine recognizes the user's emotional state (e.g., happy, sad, nervous, etc.).

[0447] Step 4:

[0448] The emotion engine sends the analysis results to the server.

[0449] Step 5:

[0450] Based on the results of the sentiment analysis, the server uses a generative AI model to generate an appropriate response.

[0451] Step 6:

[0452] The server returns the generated response to the terminal.

[0453] Step 7:

[0454] The terminal displays the generated response to the user.

[0455] Personalized VR Experience

[0456] Step 1:

[0457] The user selects the VR content for the product demo.

[0458] Step 2:

[0459] The terminal analyzes the user's emotional state using an emotion engine.

[0460] Step 3:

[0461] The emotion engine recognizes whether the user is calm or nervous.

[0462] Step 4:

[0463] The emotion engine sends the analysis results to the server.

[0464] Step 5:

[0465] The server obtains the URL of the personalized VR content based on the emotion analysis results.

[0466] Step 6:

[0467] The server returns the URL to the terminal.

[0468] Step 7:

[0469] The device uses the URL to launch a VR viewer to provide a personalized VR experience.

[0470] For example, if the emotion engine recognizes that the user is tense, the server can provide visually appealing relaxation content to enhance the user's experience.

[0471] Example 2

[0472] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0473] Conventional systems perform user authentication, product information search, question answering, virtual reality experience, real-time communication, and emotion recognition and analysis separately, making it difficult to provide a seamless user experience. Furthermore, because each function is independent, it is not possible to provide a consistent service. In particular, there is a lack of personalized responses and content based on the user's emotional state, and there is a need to improve user satisfaction.

[0474] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0475] In this invention, the server includes: means for receiving user authentication information and verifying the authentication information; means for retrieving product information from a database based on a search query entered by the user; means for accepting user questions and generating answers to the questions using a generative AI model; means for delivering URLs of corresponding virtual reality content based on virtual reality content requests from users; means for generating and managing real-time communication sessions between users; and means for analyzing the user's emotional state and personalizing responses and content based on the analysis results. This integration of each function enables a seamless and consistent user experience. Furthermore, providing personalized services based on the user's emotional state is expected to improve user satisfaction.

[0476] "User authentication information" is data used to identify a user and verify access permissions.

[0477] "Verification" is the process of checking whether the information received is correct.

[0478] A "search query" is a condition or keyword entered to retrieve specific information from a database.

[0479] A "database" is a system for efficiently managing large amounts of data and for quickly searching and manipulating it.

[0480] A "generative AI model" is an algorithm or machine learning model that uses artificial intelligence to generate new data and answers.

[0481] "Virtual Reality Content" means information or simulations that are visually and experientially presented to users using virtual reality technology.

[0482] A "URL" is an address used to specify a resource on the Internet.

[0483] A "real-time communication session" is a communication session in which information is exchanged between users in real time.

[0484] "Emotional state" refers to the user's current psychological and physiological state.

[0485] "Analysis results" refer to the conclusions or findings obtained after conducting data analysis.

[0486] "Personalization" means customizing services and content according to the needs and circumstances of individual users.

[0487] A system for implementing the present invention provides user authentication, product information search and display, question answering, virtual reality experiences, real-time communication, and emotion recognition and personalization. The system utilizes generative AI models and a digital platform to provide a seamless and consistent user experience.

[0488] User Authentication and Login

[0489] The user launches a messaging application and enters their user ID and password. The device sends this authentication information to the server. The server verifies the authentication information using a database, and if authentication is successful, creates a user session and returns a success message and the main menu to the device. The device then displays this main menu to the user.

[0490] Example: When a user enters "user123" and "password123" and the terminal sends this information to the server, the server verifies the information in the database and, after successful authentication, displays the message "Login successful" and the main menu.

[0491] Search and view product information

[0492] The user selects the product search option from the main menu and enters a search query. The device sends this query to the server. The server retrieves matching product information from the database and returns the search results to the device. The device displays the search results to the user.

[0493] Example: When a user types "polymeric materials" and the terminal sends this query to the server, the server retrieves relevant product information from the database and the terminal displays it to the user.

[0494] Questions and Answers

[0495] When a user enters a question about a particular product, the device sends the question to the server, which uses a generative AI model to generate an answer to the question and returns the answer to the device, which then displays the answer to the user.

[0496] Example: A user asks, "What are the main components of this compound?" and the device sends the question to a server. The server uses a generative AI model to generate the answer, "The main components are aluminum and silicon," and the device displays this answer to the user.

[0497] Running a virtual reality experience

[0498] When a user selects a product demo virtual reality content, the device sends the selected product ID to the server, and the server retrieves the URL of the corresponding virtual reality content from the database and returns it to the device. The device then uses this URL to launch the VR viewer and provide the user with a virtual reality experience.

[0499] Example: When a user selects "VR demo of chemical process" and the device sends the product ID to the server, the server returns the URL of the corresponding virtual reality content, the device uses the URL to launch the VR viewer, and the user can experience the virtual reality.

[0500] Realizing real-time communication

[0501] When a user selects a target user for networking and the terminal sends this information to the server, the server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and sends messages entered by the user to the server, which then relays the messages to the target user.

[0502] Example: When a user selects to chat with "Person B at Manufacturer A" and the device sends information to the server, the server creates a chat session and returns a session ID. When the device displays a chat window and the user types "Hello" and sends it, the server relays this message to Person B.

[0503] Emotion Recognition and Personalization

[0504] When a user inputs a message or question, the emotion engine analyzes the user's emotional state, and the device transmits the emotional state to the server. The server uses a generative AI model to generate an appropriate response based on the analysis results and returns it to the device. The device then displays this response to the user, personalizing the VR content based on the user's emotional state.

[0505] Example: If a user asks, "Tell me about the properties of this compound," and the emotion engine recognizes "tension," the device sends this information to the server, which then generates a response such as, "Relax and ask your question," and returns it to the device. Based on the emotional state, the server provides the device with a URL for more intuitive and visually appealing VR content.

[0506] Prompt Sentence Examples

[0507] "Please explain in detail the process by which users search for polymer materials."

[0508] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0509] Step 1:

[0510] The user launches a messaging application and enters their user ID and password, which the device then sends to the server.

[0511] Specific operation: The user enters "user123" and "password123", and the device sends this to the server.

[0512] Input: User ID and password

[0513] Output: Sending authentication information

[0514] Step 2:

[0515] The server receives the entered authentication information and validates it against a database.

[0516] Specific operation: The server retrieves the password corresponding to "user123" from the database and compares it with the entered password.

[0517] Input: Credentials

[0518] Output: Authentication result

[0519] Step 3:

[0520] If authentication is successful, the server creates a user session and returns a success message and the main menu to the terminal, otherwise it returns an error message.

[0521] Specific operation: When authentication is successful, a session ID is generated and "Login successful" and the main menu are sent to the terminal.

[0522] Input: Authentication result

[0523] Output: Success message and main menu, or failure message

[0524] Step 4:

[0525] The terminal displays the main menu to the user, who selects the product search option and enters a search query.

[0526] Specific action: The user enters "polymer material."

[0527] Input: Main menu, search query

[0528] Output: Submitting a search query

[0529] Step 5:

[0530] The terminal sends the entered search query to the server, which uses a database to retrieve product information that matches the query.

[0531] Specific operation: The server searches and retrieves product information matching "polymer material" from the database.

[0532] Input: search query

[0533] Output: Product information

[0534] Step 6:

[0535] The server returns the acquired product information to the terminal, which then displays this information to the user.

[0536] Specific behavior: The device presents the user with details about "polymer materials."

[0537] Input: Product Information

[0538] Output: Display product information

[0539] Step 7:

[0540] The user enters a question about a particular product and the terminal transmits the question to the server.

[0541] What happens: A user asks, "What are the main components of this compound?"

[0542] Input: User question

[0543] Output: Question sent

[0544] Step 8:

[0545] The server receives the question and uses a generative AI model to generate an answer to the question.

[0546] What it does: The server uses a generative AI model to generate the answer, "The main components are aluminum and silicon."

[0547] Input: Question

[0548] Output: Answer

[0549] Step 9:

[0550] The server returns the generated answer to the terminal, which displays the answer to the user.

[0551] What it does: The device displays "The main components are aluminum and silicon."

[0552] Input: Answer

[0553] Output: Display the answer

[0554] Step 10:

[0555] When a user selects a particular product demo virtual reality content, the terminal transmits the selected product ID to the server.

[0556] Specific action: The user selects "Chemical Process VR Demo."

[0557] Input: Product ID

[0558] Output: Send product ID

[0559] Step 11:

[0560] The server retrieves the URL of the corresponding virtual reality content from the database and returns it to the terminal.

[0561] Specific operation: The server retrieves the URL of the corresponding VR content from the database.

[0562] Input: Product ID

[0563] Output: URL of virtual reality content

[0564] Step 12:

[0565] The device uses the obtained URL to launch the VR viewer and provide the user with a virtual reality experience.

[0566] Specific operation: The device launches the VR viewer and the user experiences virtual reality.

[0567] Input: URL of virtual reality content

[0568] Output: Virtual reality experience

[0569] Step 13:

[0570] The user selects the target users for networking, and the terminal transmits this information to the server.

[0571] Specific operation: The user selects that he wants to chat with "Person B in charge of Manufacturer A."

[0572] Input: Networking target information

[0573] Output: Sending information

[0574] Step 14:

[0575] The server creates a new chat session and returns a session ID to the device.

[0576] Specific operation: The server creates a chat session and returns a session ID to the device.

[0577] Input: Networking target information

[0578] Output: Session ID

[0579] Step 15:

[0580] The terminal displays a real-time chat window and transmits messages entered by the user to the server.

[0581] What happens: The device displays a chat window and the user types "Hello."

[0582] Input: Session ID, user message

[0583] Output: Sending a message

[0584] Step 16:

[0585] The server relays the received message to the target user.

[0586] Specific action: The server relays the message "Hello" to Agent B.

[0587] Input: message

[0588] Output: Message relay

[0589] Step 17:

[0590] The emotion engine analyzes messages and questions entered by the user, and the device sends the user's emotional state to the server.

[0591] Specific operation: The emotion engine recognizes "tension" and sends that information to the server.

[0592] Input: Message or question

[0593] output: emotional state

[0594] Step 18:

[0595] The server uses the generative AI model to generate an appropriate response based on the analysis results and returns it to the device.

[0596] Specific operation: The server generates a response such as "Just relax and ask your questions" and returns it to the device.

[0597] Input: Emotional state

[0598] Output: Response

[0599] Step 19:

[0600] The device displays the obtained responses to the user and provides personalized content based on the emotional state.

[0601] What it does: The device displays personalized content and provides it to the user.

[0602] Input: Response, Emotional State

[0603] Output: personalized content

[0604] The above are the detailed processing steps of this system, which will enable users to enjoy real-time services based on authentication, search, question answering, VR experiences, networking, and emotion recognition.

[0605] (Application example 2)

[0606] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0607] Conventional online shopping systems have difficulty in taking user emotions into consideration, limiting the improvement of user experience. In addition, the limited methods of providing product information make it difficult for users to fully understand the products. This can lead to a decrease in user satisfaction and a negative impact on sales.

[0608] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting a user's question and generating an answer to the question using a generative model, means for delivering a URL of corresponding VR content based on a VR content request from the user, means for generating and managing real-time chat sessions between users, and means for recognizing the user's emotional state and having the generative model personalize responses based on that information. This enables personalized responses according to the user's emotions, deepening product understanding and significantly improving the user experience.

[0609] "User Credentials" means the credentials used by a User to access a System, generally including a User ID and password.

[0610] A "database" refers to a collection of data that stores product information, user information, etc. and can be searched and retrieved.

[0611] A "search query" is a keyword or phrase that a user enters to retrieve specific information.

[0612] A "generative model" refers to a system that uses AI algorithms to generate answers or responses to user questions.

[0613] "VR Content" means digital content that can be experienced by a user using virtual reality technology.

[0614] "URL" is an abbreviation for Universal Resource Locator, and is an address that points to a specific resource on the Internet.

[0615] "Real-time chat session" refers to a chat feature that allows users to exchange messages with each other in real time.

[0616] An "emotional state" refers to the emotional state that the user is feeling at that time, and includes, for example, joy, sadness, tension, and the like.

[0617] "Personalization" means tailoring services and content to a user's specific needs and circumstances.

[0618] This invention provides a system that combines user authentication, product information search and display, question answering, VR experience, networking, and an emotion engine. A specific embodiment of this system is described below.

[0619] System configuration

[0620] This system mainly consists of three elements: a server, a device, and a user. The hardware used is a smartphone and a head-mounted display, and the software uses Python, a generative AI model, and a facial recognition API.

[0621] User Authentication and Login

[0622] A user launches a messaging application and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information against a database, and if authentication is successful, returns a session token to the device. The device uses this session token to display the main menu.

[0623] Search and view product information

[0624] When a user selects the product search option on the main menu and enters a search query, the device sends the search query to the server, which retrieves product information that matches the query from the database and returns the search results to the device, which then displays the search results to the user.

[0625] Questions and Answers

[0626] When a user enters a question about a particular product, the device sends the question to the server, which uses a generative AI model to generate an answer to the question and returns the generated answer to the device, which then displays the generated answer to the user.

[0627] Running a VR experience

[0628] When a user selects a product demo VR content, the device sends the selected product ID to the server. The server retrieves the URL of the corresponding VR content from the database and returns it to the device. The device then uses this URL to launch the VR viewer and provide the VR experience.

[0629] Networking and Communication

[0630] When a user selects a target to start a networking session, the terminal sends the target's information to the server, and the server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[0631] Introducing the Emotion Engine

[0632] When a user sends a message or question, the emotion engine recognizes the user's emotional state. The device sends this emotional information to the server, which then generates an appropriate response using a generative AI model based on the emotion analysis results obtained by the emotion engine. For example, if the emotion engine recognizes that the user is nervous when the user enters a question, the server will take that information into account and generate a response that provides a sense of security.

[0633] Personalized VR Experience

[0634] The emotion engine recognizes the user's emotional state and personalizes the VR content experience. If the user is calm, a VR experience with detailed technical explanations is provided, while if the user is nervous, content with more general and simple explanations is provided. For example, if the emotion engine recognizes that the user is nervous, the server will prioritize providing more intuitive and visually appealing content.

[0635] Prompt Sentence Examples

[0636] If a user asks "What are the main components of this compound?" and submits an image:

[0637] Input: "What is the main component of this compound?", image file "path_to_image.jpg"

[0638] Prompt: In response to the user's question, the generated AI model based on the results of sentiment analysis replies, "The main ingredients are aluminum and silicon. There's nothing to be worried about; we'll explain the details in detail."

[0639] The above is a specific embodiment of the present invention. This system allows users to exchange information regardless of geographical constraints and efficiently collect information through real-time question and answering, interactive VR experiences, and emotional responses. This improves the user experience and deepens product understanding, contributing to the optimization of marketing strategies.

[0640] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0641] Step 1:

[0642] A user launches a messaging application and enters their user ID and password. The device receives this authentication information and sends it to the server. The server verifies the authentication information from the database, and if authentication is successful, generates a session token and sends it to the device. The device receives this session token and displays the main menu.

[0643] Input: User ID and password entered by the user

[0644] Output: Session token if successful, main menu displayed

[0645] What happens: The server authenticates the user by looking up the credentials in a database and verifying them.

[0646] Step 2:

[0647] The user selects a product search option and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[0648] Input: The search query entered by the user

[0649] Output: Product information search results

[0650] What happens: The server searches the database and returns matching product information.

[0651] Step 3:

[0652] A user enters a question about a specific product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[0653] Input: Product questions entered by the user

[0654] Output: Answer from the generative AI model

[0655] Specific operation: The server inputs a question into the generative AI model and generates an answer.

[0656] Step 4:

[0657] The user selects a product demo VR content. The device sends the selected product ID to the server. The server retrieves the URL of the corresponding VR content from the database and returns it to the device. The device uses this URL to launch the VR viewer and provide the VR experience.

[0658] Input: Product ID selected by the user

[0659] Output: URL of VR content, start of VR experience

[0660] Specific operation: The server searches the database, obtains the URL of the VR content, and returns it.

[0661] Step 5:

[0662] The user selects a target to start a networking session. The terminal sends the target's information to the server. The server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user. The user types a message and sends it to the server, which relays the message to the target user.

[0663] Input: Information of the target person selected by the user

[0664] Output: Chat session ID, chat window display

[0665] Specific operation: The server creates a chat session and relays messages.

[0666] Step 6:

[0667] When a user sends a message or question, the emotion engine recognizes the user's emotional state. The device sends the emotion engine's analysis results to the server. The server generates an appropriate response using a generative AI model based on the emotion analysis results obtained by the emotion engine and returns it to the device. The device then displays the response to the user.

[0668] Input: Messages and questions sent by users, and their emotional state

[0669] Output: The response of the generative AI model taking into account the emotional state

[0670] Specific operation: The emotion engine analyzes the user's emotional state, and the generative AI model generates a response based on the results.

[0671] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0672] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0673] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0674] [Second embodiment]

[0675] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0676] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0677] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0678] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0679] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0680] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0681] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0682] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0683] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0684] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0685] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0686] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0687] The system for implementing the present invention utilizes generative AI and a digital platform, including functions for user authentication, product information search and display, question answering, VR experiences, and networking.

[0688] User Authentication and Login

[0689] A user launches a messaging app and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information from the database, and if authentication is successful, it creates a user session and returns the main menu to the device along with a success message. The device then displays this main menu to the user.

[0690] Search and view product information

[0691] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[0692] For example, if a user searches for "polymeric materials," the server that receives this search query retrieves related product information from the database and displays it on the user's terminal.

[0693] Questions and Answers

[0694] A user enters a question about a particular product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[0695] For example, if a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[0696] Running a VR experience

[0697] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[0698] For example, if a user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[0699] Networking and Communication

[0700] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[0701] For example, if a user wants to talk to a representative of another manufacturer participating in an exhibition, the user can send a message to that representative via the chat function and have a conversation in real time.

[0702] The above is a concrete implementation of this system. The system is designed to enable users to exchange information without geographical constraints and to efficiently gather information through real-time question and answering and interactive VR experiences. This facilitates communication between manufacturers and visitors, contributing to product understanding and the optimization of market strategies.

[0703] The processing flow will be explained below.

[0704] User Authentication and Login

[0705] Step 1:

[0706] The user launches the messaging app and enters their user ID and password on the login screen.

[0707] Step 2:

[0708] The terminal sends the authentication information (user ID and password) entered by the user to the server.

[0709] Step 3:

[0710] The server checks the received authentication information against a database and, if authentication is successful, creates a user session.

[0711] Step 4:

[0712] The server returns the main menu data to the terminal along with an authentication success message.

[0713] Step 5:

[0714] The terminal displays the main menu to the user.

[0715] Search and view product information

[0716] Step 1:

[0717] A user selects the product search option on the main menu and enters a search query.

[0718] Step 2:

[0719] The terminal transmits the search query entered by the user to the server.

[0720] Step 3:

[0721] The server retrieves product information that matches the search query from a database.

[0722] Step 4:

[0723] The server returns the acquired search results to the terminal.

[0724] Step 5:

[0725] The terminal displays the search results to the user.

[0726] Questions and Answers

[0727] Step 1:

[0728] A user enters a question about a particular product.

[0729] Step 2:

[0730] The terminal sends this question to the server.

[0731] Step 3:

[0732] The server generates answers to questions using generative AI models.

[0733] Step 4:

[0734] The server returns the generated answer to the terminal.

[0735] Step 5:

[0736] The terminal displays the generated answer to the user.

[0737] Running a VR experience

[0738] Step 1:

[0739] The user selects the VR content for the product demo.

[0740] Step 2:

[0741] The terminal transmits the selected product ID to the server.

[0742] Step 3:

[0743] The server retrieves the URL of the corresponding VR content from the database.

[0744] Step 4:

[0745] The server returns the acquired URL to the terminal.

[0746] Step 5:

[0747] The device will use this URL to launch the VR viewer and provide the VR experience.

[0748] Networking and Communication

[0749] Step 1:

[0750] The user selects a target person to begin a networking session.

[0751] Step 2:

[0752] The terminal transmits the target person's information to the server and requests the initiation of a chat session.

[0753] Step 3:

[0754] The server creates a new chat session and generates a session ID.

[0755] Step 4:

[0756] The server returns the generated session ID to the terminal.

[0757] Step 5:

[0758] The terminal displays a real-time chat window and notifies the user that a session has begun.

[0759] Step 6:

[0760] The user types a message in the chat window.

[0761] Step 7:

[0762] The terminal transmits the input message to the server.

[0763] Step 8:

[0764] The server relays the received message to the target user.

[0765] Example 1

[0766] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0767] Currently, there is a lack of an integrated system that allows users to smoothly use multiple functions (authentication, product search, question and answering, VR experience, real-time chat, etc.) on a single platform. At the same time, there are limitations on the means to enrich users' interactive experiences and the means of real-time communication. To solve this issue, it is necessary to centrally manage and integrate all functions.

[0768] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0769] In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting user questions and generating answers to the questions using a generative AI model, means for delivering URLs of corresponding VR content based on VR content requests from users, means for generating and managing real-time chat sessions between users, and means for users to input messages into real-time chat windows and for transmitting and relaying the messages. This allows users to seamlessly use multiple functions and efficiently gather information and communicate.

[0770] "User authentication information" refers to information such as ID and password entered by a user to access a system.

[0771] A database is a collection of information that is organized and stored according to certain rules, allowing for efficient searching and retrieval.

[0772] "Product information" refers to detailed data, specifications, features, and other information about a product.

[0773] A "generative AI model" is a model that uses artificial intelligence technology to realize question-answering and data generation similar to those performed by humans.

[0774] "VR content" refers to digital content that provides a visual and auditory experience using virtual reality technology.

[0775] "URL" is a uniform resource identifier that indicates the location of an information resource on the Internet.

[0776] A "real-time chat session" is a communication session in which multiple users simultaneously send and receive messages and communicate in real time.

[0777] A "messaging application" is software that allows users to send and receive text messages, voice messages, images, etc.

[0778] A "session ID" is a unique identifier for identifying a particular user session.

[0779] A "server" is a computer system that provides services to other computers on a network.

[0780] The system for implementing the present invention utilizes generative AI models and a digital platform that includes functions for user authentication, product information search and display, question answering, VR experiences, and networking.

[0781] User Authentication and Login

[0782] A user launches a messaging application and enters authentication information (user ID and password). The terminal sends this authentication information to the server. The server verifies the authentication information from a database, and if authentication is successful, it creates a user session and returns a main menu to the terminal along with a success message. The terminal then displays this main menu to the user.

[0783] Examples:

[0784] A user opens a messaging application and enters "user123" as the user ID and "password321" as the password. The device sends this to the server, which verifies it and returns the message "Authentication successful. The main menu will be displayed" if authentication is successful.

[0785] Example prompt sentence:

[0786] "When a user enters their user ID and password, explain how the server validates that information."

[0787] Search and view product information

[0788] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[0789] Examples:

[0790] When a user searches for "polymer materials," the server receives this search query, retrieves related product information from the database, and displays it on the user's terminal.

[0791] Example prompt sentence:

[0792] "If a user searches for information about a particular polymeric material, explain how the system retrieves and displays that information."

[0793] Questions and Answers

[0794] A user enters a question about a specific product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[0795] Examples:

[0796] If a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[0797] Example prompt sentence:

[0798] "If a user asks a question about a specific product, explain how you can use a generative AI model to generate an answer to that question."

[0799] Running a VR experience

[0800] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[0801] Examples:

[0802] If the user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[0803] Example prompt sentence:

[0804] "When a user selects a VR experience, explain how the system delivers that VR content."

[0805] Networking and Communication

[0806] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[0807] Examples:

[0808] If a user wants to talk to a representative from another manufacturer participating in the exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[0809] Example prompt sentence:

[0810] "Please explain how the system works to allow users to communicate with other manufacturer representatives in real time."

[0811] This concludes the description of the embodiment of the present invention. This system is designed to allow users to seamlessly use multiple functions on an integrated platform, enabling users to efficiently gather information and communicate.

[0812] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0813] User Authentication and Login

[0814] Processing Steps:

[0815] Step 1:

[0816] The user launches a messaging app and enters authentication information (user ID and password), which then becomes the input data.

[0817] Input: User ID and password

[0818] Output: None

[0819] Step 2:

[0820] The terminal transmits the entered authentication information to the server.

[0821] Input: User ID and password

[0822] Output: Authentication information sent to the server

[0823] Step 3:

[0824] The server validates the received authentication information against a database. The database check confirms the authenticity of the authentication information.

[0825] Input: The authentication information sent to the server

[0826] Output: Authentication success or failure

[0827] Step 4:

[0828] If authentication is successful, the server creates a user session and returns the main menu to the terminal along with a success message.

[0829] Input: Authentication success information

[0830] Output: Success message and main menu

[0831] Step 5:

[0832] The terminal displays the received main menu to the user.

[0833] Input: Success message and main menu

[0834] Output: Display main menu

[0835] Search and view product information

[0836] Processing Steps:

[0837] Step 1:

[0838] A user selects a product search option in the main menu and enters a search query, which becomes the input data.

[0839] Input: search query

[0840] Output: None

[0841] Step 2:

[0842] The terminal sends a search query to the server.

[0843] Input: search query

[0844] Output: The search query sent to the server

[0845] Step 3:

[0846] The server retrieves product information that matches the query from the database. Matching product information is collected by searching the database.

[0847] Input: The search query sent to the server

[0848] Output: Search results

[0849] Step 4:

[0850] The server returns the search results to the terminal.

[0851] Input: Search results

[0852] Output: Search results sent back to your device

[0853] Step 5:

[0854] The terminal displays the search results to the user.

[0855] Input: Search results

[0856] Output: Display of search results

[0857] Questions and Answers

[0858] Processing Steps:

[0859] Step 1:

[0860] A user enters a question about a particular product, which then becomes the input data.

[0861] Input: User question

[0862] Output: None

[0863] Step 2:

[0864] The terminal sends a query to the server.

[0865] Input: User question

[0866] Output: The question sent to the server

[0867] Step 3:

[0868] The server generates answers to questions using a generative AI model, which analyzes the content of the question and generates an answer.

[0869] Input: The question sent to the server

[0870] Output: The generated answer

[0871] Step 4:

[0872] The server generates a response and sends it back to the terminal.

[0873] Input: Generated answer

[0874] Output: Answer sent back to the terminal

[0875] Step 5:

[0876] The terminal displays the generated answer to the user.

[0877] Input: Generated answer

[0878] Output: Display the answer

[0879] Running a VR experience

[0880] Processing Steps:

[0881] Step 1:

[0882] The user selects the VR content for the product demo, and the selected product ID becomes the input data.

[0883] Input: Product ID

[0884] Output: None

[0885] Step 2:

[0886] The terminal transmits the selected product ID to the server.

[0887] Input: Product ID

[0888] Output: Product ID sent to the server

[0889] Step 3:

[0890] The server retrieves the URL of the corresponding VR content from the database and identifies the URL of the corresponding VR content by database matching.

[0891] Input: Product ID sent to the server

[0892] Output: URL of VR content

[0893] Step 4:

[0894] The server returns the URL to the device.

[0895] Input: VR content URL

[0896] Output: URL sent back to the device

[0897] Step 5:

[0898] The device uses the URL to launch the VR viewer and provide the VR experience.

[0899] Input: VR content URL

[0900] Output: Providing a VR experience

[0901] Networking and Communication

[0902] Processing Steps:

[0903] Step 1:

[0904] To start a networking session, a user selects a target person, and the target person's information becomes input data.

[0905] Input: Subject information

[0906] Output: None

[0907] Step 2:

[0908] The terminal transmits the target person's information to the server.

[0909] Input: Subject information

[0910] Output: The target information sent to the server

[0911] Step 3:

[0912] The server creates a new chat session and returns a session ID to the terminal. The server generates a session ID and starts a chat session.

[0913] Input: Subject information sent to the server

[0914] Output: The generated session ID.

[0915] Step 4:

[0916] The terminal displays a real-time chat window and notifies the user that a session has begun.

[0917] Input: Generated session ID

[0918] Output: Chat window display

[0919] Step 5:

[0920] When a user types a message, the device sends the message to the server, which relays it to the target user.

[0921] Input: Message from the user

[0922] Output: Message sent to server and relayed to target user

[0923] Specific working example:

[0924] If a user wants to talk to a representative from another manufacturer participating in the exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[0925] (Application example 1)

[0926] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0927] Currently, there are many online shopping platforms on the market that allow users to search for various product information. However, users often experience inconveniences when checking product information, answering questions, and experiencing the products. Furthermore, the user experience can be limited due to limited interactive display of product information and real-time communication with other users and staff. Conventional systems do not provide users with sufficient information and support, making it difficult to provide the comfortable and efficient shopping experience desired by users. Therefore, this invention aims to provide a more comprehensive and interactive online shopping environment by integrating the functions of user authentication, product information search and display, question answering, VR experience provision, and real-time chat.

[0928] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0929] In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting user questions and generating answers to the questions using a generative AI model, means for delivering URLs of corresponding VR content based on VR content requests from users, means for generating and managing real-time chat sessions between users, means for displaying searched product information as 3D models or images, means for providing generated answers by voice, and means for communicating with other users and staff in real time. This allows users to enjoy a comprehensive online shopping experience, from authentication to question and answering, interactive display of product information, VR experiences, and real-time communication.

[0930] "User Credentials" means information used to identify a user and authorize access to a system.

[0931] "Means for verifying" is a function that executes a process to confirm whether user authentication information is correct.

[0932] A "search query" is a request entered by a user to retrieve specific information from a database.

[0933] "Product information" is detailed information about a particular product, typically stored in a database.

[0934] A "generative AI model" is an artificial intelligence model used to automatically generate answers to user questions.

[0935] A "VR content request" is a request submitted by a user to initiate a virtual reality experience.

[0936] A "real-time chat session" is an interactive session for sending and receiving text messages between users in real time.

[0937] "3D Model or Image" means a three-dimensional model or still image used to visually display product information.

[0938] The "means for providing by voice" is a function for providing the generated answer to the user as voice.

[0939] "Means of communicating in real time" refers to a function that allows you to communicate with other users and staff in real time.

[0940] MODE FOR CARRYING OUT THE INVENTION

[0941] The basic program for realizing this system includes the following means: Specific hardware required is a server, a user device (a smartphone or head-mounted display), a generative AI model, and a VR viewer.

[0942] Receiving and verifying user credentials

[0943] A user launches an application and enters authentication information (user ID and password). The terminal sends this authentication information to the server, which verifies it using a database. If authentication is successful, the server creates a user session and returns a main menu to the terminal along with a success message. The terminal displays this main menu to the user.

[0944] Search and view product information

[0945] A user selects the product search option from the main menu and enters a search query. The device sends this search query to the server, which retrieves product information matching the query from its database. This information is returned to the device as a 3D model or image, and the device displays the search results to the user. For example, if a user searches for "polymer materials," the server retrieves and displays related product information.

[0946] Questions and Answers

[0947] When a user enters a question about a particular product, the device sends the question to the server. The server uses a generative AI model to generate an answer to the question and returns it to the device. The device then provides the generated answer via voice. For example, if a user asks, "What are the main components of this compound?" the generative AI model might respond, "The main components are aluminum and silicon."

[0948] Running a VR experience

[0949] When a user selects a product demo VR content, the device sends the selected product ID to the server, which then retrieves the URL of the corresponding VR content from the database. This URL is then returned to the device, which then launches the VR viewer to provide the VR experience. For example, if a user selects a VR demo of a specific chemical process, the device retrieves the URL of the VR content corresponding to that process, and the user can use this URL to experience virtual reality.

[0950] Networking and Communication

[0951] When a user selects a target person to start a networking session, the terminal sends the target person's information to the server, and the server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, and the server relays the message to the target user. For example, if a user wants to talk to another representative at an exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[0952] Examples of concrete examples and prompts

[0953] As a specific example of user authentication, a user attempts to log in with the ID "user123" and the password "password123." As a specific example of product search, a search is performed for the chemical product "polymer material," and related product information is displayed. As a specific example of question and answer, a user asks "What are the main components of this compound?" via voice, and the generative AI model provides the answer "The main components are aluminum and silicon." As a specific example of a VR experience, a user starts a VR experience to view the details of a specific product in 3D. As a specific example of networking, a user can chat with other personnel in real time.

[0954] An example of a prompt to input to a generative AI model is as follows:

[0955] When asked, "What is the main component of this compound?", be specific about the main component of the chemical.

[0956] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0957] Step 1:

[0958] The user starts the application and enters authentication information (user ID and password). The device sends this authentication information to the server, which then verifies it using a database.

[0959] Input: User ID and password

[0960] Data processing: The server retrieves the authentication information from the database and compares it with the user's input.

[0961] Output: If authentication is successful, create a user session and generate a success message.

[0962] Step 2:

[0963] If the authentication is successful, the server sends the main menu to the terminal, which then displays the main menu to the user.

[0964] Input: Authentication success message

[0965] Data processing: The server saves the user's session information and prepares the main menu data.

[0966] Output: The main menu is displayed on the terminal.

[0967] Step 3:

[0968] The user selects the product search option from the main menu and enters a search query. The device sends this search query to the server, which retrieves product information matching the query from its database.

[0969] Input: search query

[0970] Data processing: The server uses a database search algorithm to retrieve product information that matches the query.

[0971] Output: A list of products matching the query will be displayed on the terminal.

[0972] Step 4:

[0973] The server transmits the acquired product information as a 3D model or image to the terminal, which then displays this information to the user.

[0974] Input: Product information list

[0975] Data processing: Processing to convert product information into 3D models or image formats.

[0976] Output: Product information displayed as a 3D model or image.

[0977] Step 5:

[0978] A user enters a question about a particular product, and the device sends the question to a server, which uses a generative AI model to generate an answer to the question.

[0979] Input: Product Question

[0980] Data processing: Questions are input as prompts into the AI ​​model to generate answers.

[0981] Output: The answer is sent to the terminal and displayed.

[0982] Step 6:

[0983] The terminal presents the user with the option to provide the generated answer aloud, and if selected by the user, outputs it as speech using speech synthesis technology.

[0984] Input: Answer text

[0985] Data processing: Converting text into audio data (speech synthesis).

[0986] Output: A spoken answer is provided.

[0987] Step 7:

[0988] When a user selects a product demo VR content, the terminal sends the selected product ID to the server, and the server retrieves the URL of the corresponding VR content from the database.

[0989] Input: Product ID

[0990] Data processing: The server retrieves the URL of the VR content corresponding to the product ID from the database.

[0991] Output: The URL of the VR content is sent to the device.

[0992] Step 8:

[0993] The device uses the obtained URL to launch the VR viewer and provide the user with a VR experience.

[0994] Input: VR content URL

[0995] Data processing: The VR viewer loads the URL and displays the corresponding VR content.

[0996] Output: The VR experience is delivered to the user.

[0997] Step 9:

[0998] The user selects a target person to start a networking session, and the device sends the target person's information to the server, which creates a new chat session and returns a session ID to the device.

[0999] Input: Subject information

[1000] Data processing: Create a new chat session and issue a session ID.

[1001] Output: The session ID is sent to the terminal.

[1002] Step 10:

[1003] The terminal displays a real-time chat window and allows the user to input a message, which is then sent to the server, which relays the message to the target user.

[1004] Input: User's message

[1005] Data processing: Processes messages to be relayed in real time.

[1006] Output: A message is sent to the target user.

[1007] By performing these steps, users can achieve a comprehensive online shopping experience, from authentication to question and answering, interactive display of product information, VR experience, and real-time communication.

[1008] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1009] The system for implementing the present invention utilizes generative AI and a digital platform that includes user authentication, product information search and display, question answering, VR experiences, networking, and an emotion engine that recognizes user emotions.

[1010] User Authentication and Login

[1011] A user launches a messaging app and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information from the database, and if authentication is successful, it creates a user session and returns the main menu to the device along with a success message. The device then displays this main menu to the user.

[1012] Search and view product information

[1013] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[1014] For example, if a user searches for "polymeric materials," the server that receives this search query retrieves related product information from the database and displays it on the user's terminal.

[1015] Questions and Answers

[1016] A user enters a question about a particular product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[1017] For example, if a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[1018] Running a VR experience

[1019] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[1020] For example, if a user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[1021] Networking and Communication

[1022] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[1023] For example, if a user wants to talk to a representative of another manufacturer participating in an exhibition, the user can send a message to that representative via the chat function and have a conversation in real time.

[1024] Introducing the Emotion Engine

[1025] When a user sends a message or question, the emotion engine recognizes the user's emotional state. The device then sends this emotional information to the server, which then uses the emotion analysis results obtained by the emotion engine to generate an appropriate response using a generative AI model.

[1026] For example, if a user types a question and the emotion engine recognizes that the user is nervous, the server is designed to take that information into account and generate a reassuring answer.

[1027] Personalized VR Experience

[1028] The emotion engine recognizes the user's emotional state and personalizes the VR content experience: if the user is calm, a VR experience with detailed technical explanations is provided, while if the user is nervous, content with more general and simplified explanations is provided.

[1029] For example, if the emotion engine recognizes that the user is nervous, the server will prioritize providing more intuitive and visually appealing content.

[1030] The above is a concrete example of combining this system with an emotion engine. The system is designed to enable users to exchange information regardless of geographical constraints and to efficiently collect information through real-time question and answering, interactive VR experiences, and emotional responses. This facilitates communication between manufacturers and visitors, contributing to product understanding and the optimization of market strategies.

[1031] The processing flow will be explained below.

[1032] User Authentication and Login

[1033] Step 1:

[1034] The user launches the messaging app and enters their user ID and password on the login screen.

[1035] Step 2:

[1036] The terminal sends the authentication information (user ID and password) entered by the user to the server.

[1037] Step 3:

[1038] The server checks the received authentication information against a database and, if authentication is successful, creates a user session.

[1039] Step 4:

[1040] The server returns the main menu data to the terminal along with an authentication success message.

[1041] Step 5:

[1042] The terminal displays the main menu to the user.

[1043] Search and view product information

[1044] Step 1:

[1045] A user selects the product search option on the main menu and enters a search query.

[1046] Step 2:

[1047] The terminal transmits the search query entered by the user to the server.

[1048] Step 3:

[1049] The server retrieves product information that matches the search query from a database.

[1050] Step 4:

[1051] The server returns the acquired search results to the terminal.

[1052] Step 5:

[1053] The terminal displays the search results to the user.

[1054] Questions and Answers

[1055] Step 1:

[1056] A user enters a question about a particular product.

[1057] Step 2:

[1058] The terminal sends this question to the server.

[1059] Step 3:

[1060] The server generates answers to questions using generative AI models.

[1061] Step 4:

[1062] The server returns the generated answer to the terminal.

[1063] Step 5:

[1064] The terminal displays the generated answer to the user.

[1065] Running a VR experience

[1066] Step 1:

[1067] The user selects the VR content for the product demo.

[1068] Step 2:

[1069] The terminal transmits the selected product ID to the server.

[1070] Step 3:

[1071] The server retrieves the URL of the corresponding VR content from the database.

[1072] Step 4:

[1073] The server returns the acquired URL to the terminal.

[1074] Step 5:

[1075] The device will use this URL to launch the VR viewer and provide the VR experience.

[1076] Networking and Communication

[1077] Step 1:

[1078] The user selects a target person to begin a networking session.

[1079] Step 2:

[1080] The terminal transmits the target person's information to the server and requests the initiation of a chat session.

[1081] Step 3:

[1082] The server creates a new chat session and generates a session ID.

[1083] Step 4:

[1084] The server returns the generated session ID to the terminal.

[1085] Step 5:

[1086] The terminal displays a real-time chat window and notifies the user that a session has begun.

[1087] Step 6:

[1088] The user types a message in the chat window.

[1089] Step 7:

[1090] The terminal transmits the input message to the server.

[1091] Step 8:

[1092] The server relays the received message to the target user.

[1093] Introducing the Emotion Engine

[1094] Step 1:

[1095] The user enters a message or question.

[1096] Step 2:

[1097] The device sends the input message or question to the emotion engine, which analyzes the emotional state.

[1098] Step 3:

[1099] The emotion engine recognizes the user's emotional state (e.g., happy, sad, nervous, etc.).

[1100] Step 4:

[1101] The emotion engine sends the analysis results to the server.

[1102] Step 5:

[1103] Based on the results of the sentiment analysis, the server uses a generative AI model to generate an appropriate response.

[1104] Step 6:

[1105] The server returns the generated response to the terminal.

[1106] Step 7:

[1107] The terminal displays the generated response to the user.

[1108] Personalized VR Experience

[1109] Step 1:

[1110] The user selects the VR content for the product demo.

[1111] Step 2:

[1112] The terminal analyzes the user's emotional state using an emotion engine.

[1113] Step 3:

[1114] The emotion engine recognizes whether the user is calm or nervous.

[1115] Step 4:

[1116] The emotion engine sends the analysis results to the server.

[1117] Step 5:

[1118] The server obtains the URL of the personalized VR content based on the emotion analysis results.

[1119] Step 6:

[1120] The server returns the URL to the terminal.

[1121] Step 7:

[1122] The device uses the URL to launch a VR viewer to provide a personalized VR experience.

[1123] For example, if the emotion engine recognizes that the user is tense, the server can provide visually appealing relaxation content to enhance the user's experience.

[1124] Example 2

[1125] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1126] Conventional systems perform user authentication, product information search, question answering, virtual reality experience, real-time communication, and emotion recognition and analysis separately, making it difficult to provide a seamless user experience. Furthermore, because each function is independent, it is not possible to provide a consistent service. In particular, there is a lack of personalized responses and content based on the user's emotional state, and there is a need to improve user satisfaction.

[1127] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1128] In this invention, the server includes: means for receiving user authentication information and verifying the authentication information; means for retrieving product information from a database based on a search query entered by the user; means for accepting user questions and generating answers to the questions using a generative AI model; means for delivering URLs of corresponding virtual reality content based on virtual reality content requests from users; means for generating and managing real-time communication sessions between users; and means for analyzing the user's emotional state and personalizing responses and content based on the analysis results. This integration of each function enables a seamless and consistent user experience. Furthermore, providing personalized services based on the user's emotional state is expected to improve user satisfaction.

[1129] "User authentication information" is data used to identify a user and verify access permissions.

[1130] "Verification" is the process of checking whether the information received is correct.

[1131] A "search query" is a condition or keyword entered to retrieve specific information from a database.

[1132] A "database" is a system for efficiently managing large amounts of data and for quickly searching and manipulating it.

[1133] A "generative AI model" is an algorithm or machine learning model that uses artificial intelligence to generate new data and answers.

[1134] "Virtual Reality Content" means information or simulations that are visually and experientially presented to users using virtual reality technology.

[1135] A "URL" is an address used to specify a resource on the Internet.

[1136] A "real-time communication session" is a communication session in which information is exchanged between users in real time.

[1137] "Emotional state" refers to the user's current psychological and physiological state.

[1138] "Analysis results" refer to the conclusions or findings obtained after conducting data analysis.

[1139] "Personalization" means customizing services and content according to the needs and circumstances of individual users.

[1140] A system for implementing the present invention provides user authentication, product information search and display, question answering, virtual reality experiences, real-time communication, and emotion recognition and personalization. The system utilizes generative AI models and a digital platform to provide a seamless and consistent user experience.

[1141] User Authentication and Login

[1142] The user launches a messaging application and enters their user ID and password. The device sends this authentication information to the server. The server verifies the authentication information using a database, and if authentication is successful, creates a user session and returns a success message and the main menu to the device. The device then displays this main menu to the user.

[1143] Example: When a user enters "user123" and "password123" and the terminal sends this information to the server, the server verifies the information in the database and, after successful authentication, displays the message "Login successful" and the main menu.

[1144] Search and view product information

[1145] The user selects the product search option from the main menu and enters a search query. The device sends this query to the server. The server retrieves matching product information from the database and returns the search results to the device. The device displays the search results to the user.

[1146] Example: When a user types "polymeric materials" and the terminal sends this query to the server, the server retrieves relevant product information from the database and the terminal displays it to the user.

[1147] Questions and Answers

[1148] When a user enters a question about a particular product, the device sends the question to the server, which uses a generative AI model to generate an answer to the question and returns the answer to the device, which then displays the answer to the user.

[1149] Example: A user asks, "What are the main components of this compound?" and the device sends the question to a server. The server uses a generative AI model to generate the answer, "The main components are aluminum and silicon," and the device displays this answer to the user.

[1150] Running a virtual reality experience

[1151] When a user selects a product demo virtual reality content, the device sends the selected product ID to the server, and the server retrieves the URL of the corresponding virtual reality content from the database and returns it to the device. The device then uses this URL to launch the VR viewer and provide the user with a virtual reality experience.

[1152] Example: When a user selects "VR demo of chemical process" and the device sends the product ID to the server, the server returns the URL of the corresponding virtual reality content, the device uses the URL to launch the VR viewer, and the user can experience the virtual reality.

[1153] Realizing real-time communication

[1154] When a user selects a target user for networking and the terminal sends this information to the server, the server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and sends messages entered by the user to the server, which then relays the messages to the target user.

[1155] Example: When a user selects to chat with "Person B at Manufacturer A" and the device sends information to the server, the server creates a chat session and returns a session ID. When the device displays a chat window and the user types "Hello" and sends it, the server relays this message to Person B.

[1156] Emotion Recognition and Personalization

[1157] When a user inputs a message or question, the emotion engine analyzes the user's emotional state, and the device transmits the emotional state to the server. The server uses a generative AI model to generate an appropriate response based on the analysis results and returns it to the device. The device then displays this response to the user, personalizing the VR content based on the user's emotional state.

[1158] Example: If a user asks, "Tell me about the properties of this compound," and the emotion engine recognizes "tension," the device sends this information to the server, which then generates a response such as, "Relax and ask your question," and returns it to the device. Based on the emotional state, the server provides the device with a URL for more intuitive and visually appealing VR content.

[1159] Prompt Sentence Examples

[1160] "Please explain in detail the process by which users search for polymer materials."

[1161] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1162] Step 1:

[1163] The user launches a messaging application and enters their user ID and password, which the device then sends to the server.

[1164] Specific operation: The user enters "user123" and "password123", and the device sends this to the server.

[1165] Input: User ID and password

[1166] Output: Sending authentication information

[1167] Step 2:

[1168] The server receives the entered authentication information and validates it against a database.

[1169] Specific operation: The server retrieves the password corresponding to "user123" from the database and compares it with the entered password.

[1170] Input: Credentials

[1171] Output: Authentication result

[1172] Step 3:

[1173] If authentication is successful, the server creates a user session and returns a success message and the main menu to the terminal, otherwise it returns an error message.

[1174] Specific operation: When authentication is successful, a session ID is generated and "Login successful" and the main menu are sent to the terminal.

[1175] Input: Authentication result

[1176] Output: Success message and main menu, or failure message

[1177] Step 4:

[1178] The terminal displays the main menu to the user, who selects the product search option and enters a search query.

[1179] Specific action: The user enters "polymer material."

[1180] Input: Main menu, search query

[1181] Output: Submitting a search query

[1182] Step 5:

[1183] The terminal sends the entered search query to the server, which uses a database to retrieve product information that matches the query.

[1184] Specific operation: The server searches and retrieves product information matching "polymer material" from the database.

[1185] Input: search query

[1186] Output: Product information

[1187] Step 6:

[1188] The server returns the acquired product information to the terminal, which then displays this information to the user.

[1189] Specific behavior: The device presents the user with details about "polymer materials."

[1190] Input: Product Information

[1191] Output: Display product information

[1192] Step 7:

[1193] The user enters a question about a particular product and the terminal transmits the question to the server.

[1194] What happens: A user asks, "What are the main components of this compound?"

[1195] Input: User question

[1196] Output: Question sent

[1197] Step 8:

[1198] The server receives the question and uses a generative AI model to generate an answer to the question.

[1199] What it does: The server uses a generative AI model to generate the answer, "The main components are aluminum and silicon."

[1200] Input: Question

[1201] Output: Answer

[1202] Step 9:

[1203] The server returns the generated answer to the terminal, which displays the answer to the user.

[1204] What it does: The device displays "The main components are aluminum and silicon."

[1205] Input: Answer

[1206] Output: Display the answer

[1207] Step 10:

[1208] When a user selects a particular product demo virtual reality content, the terminal transmits the selected product ID to the server.

[1209] Specific action: The user selects "Chemical Process VR Demo."

[1210] Input: Product ID

[1211] Output: Send product ID

[1212] Step 11:

[1213] The server retrieves the URL of the corresponding virtual reality content from the database and returns it to the terminal.

[1214] Specific operation: The server retrieves the URL of the corresponding VR content from the database.

[1215] Input: Product ID

[1216] Output: URL of virtual reality content

[1217] Step 12:

[1218] The device uses the obtained URL to launch the VR viewer and provide the user with a virtual reality experience.

[1219] Specific operation: The device launches the VR viewer and the user experiences virtual reality.

[1220] Input: URL of virtual reality content

[1221] Output: Virtual reality experience

[1222] Step 13:

[1223] The user selects the target users for networking, and the terminal transmits this information to the server.

[1224] Specific operation: The user selects that he wants to chat with "Person B in charge of Manufacturer A."

[1225] Input: Networking target information

[1226] Output: Sending information

[1227] Step 14:

[1228] The server creates a new chat session and returns a session ID to the device.

[1229] Specific operation: The server creates a chat session and returns a session ID to the device.

[1230] Input: Networking target information

[1231] Output: Session ID

[1232] Step 15:

[1233] The terminal displays a real-time chat window and transmits messages entered by the user to the server.

[1234] What happens: The device displays a chat window and the user types "Hello."

[1235] Input: Session ID, user message

[1236] Output: Sending a message

[1237] Step 16:

[1238] The server relays the received message to the target user.

[1239] Specific action: The server relays the message "Hello" to Agent B.

[1240] Input: message

[1241] Output: Message relay

[1242] Step 17:

[1243] The emotion engine analyzes messages and questions entered by the user, and the device sends the user's emotional state to the server.

[1244] Specific operation: The emotion engine recognizes "tension" and sends that information to the server.

[1245] Input: Message or question

[1246] output: emotional state

[1247] Step 18:

[1248] The server uses the generative AI model to generate an appropriate response based on the analysis results and returns it to the device.

[1249] Specific operation: The server generates a response such as "Just relax and ask your questions" and returns it to the device.

[1250] Input: Emotional state

[1251] Output: Response

[1252] Step 19:

[1253] The device displays the obtained responses to the user and provides personalized content based on the emotional state.

[1254] What it does: The device displays personalized content and provides it to the user.

[1255] Input: Response, Emotional State

[1256] Output: personalized content

[1257] The above are the detailed processing steps of this system, which will enable users to enjoy real-time services based on authentication, search, question answering, VR experiences, networking, and emotion recognition.

[1258] (Application example 2)

[1259] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1260] Conventional online shopping systems have difficulty in taking user emotions into consideration, limiting the improvement of user experience. In addition, the limited methods of providing product information make it difficult for users to fully understand the products. This can lead to a decrease in user satisfaction and a negative impact on sales.

[1261] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting a user's question and generating an answer to the question using a generative model, means for delivering a URL of corresponding VR content based on a VR content request from the user, means for generating and managing real-time chat sessions between users, and means for recognizing the user's emotional state and having the generative model personalize responses based on that information. This enables personalized responses according to the user's emotions, deepening product understanding and significantly improving the user experience.

[1262] "User Credentials" means the credentials used by a User to access a System, generally including a User ID and password.

[1263] A "database" refers to a collection of data that stores product information, user information, etc. and can be searched and retrieved.

[1264] A "search query" is a keyword or phrase that a user enters to retrieve specific information.

[1265] A "generative model" refers to a system that uses AI algorithms to generate answers or responses to user questions.

[1266] "VR Content" means digital content that can be experienced by a user using virtual reality technology.

[1267] "URL" is an abbreviation for Universal Resource Locator, and is an address that points to a specific resource on the Internet.

[1268] "Real-time chat session" refers to a chat feature that allows users to exchange messages with each other in real time.

[1269] An "emotional state" refers to the emotional state that the user is feeling at that time, and includes, for example, joy, sadness, tension, and the like.

[1270] "Personalization" means tailoring services and content to a user's specific needs and circumstances.

[1271] This invention provides a system that combines user authentication, product information search and display, question answering, VR experience, networking, and an emotion engine. A specific embodiment of this system is described below.

[1272] System configuration

[1273] This system mainly consists of three elements: a server, a device, and a user. The hardware used is a smartphone and a head-mounted display, and the software uses Python, a generative AI model, and a facial recognition API.

[1274] User Authentication and Login

[1275] A user launches a messaging application and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information against a database, and if authentication is successful, returns a session token to the device. The device uses this session token to display the main menu.

[1276] Search and view product information

[1277] When a user selects the product search option on the main menu and enters a search query, the device sends the search query to the server, which retrieves product information that matches the query from the database and returns the search results to the device, which then displays the search results to the user.

[1278] Questions and Answers

[1279] When a user enters a question about a particular product, the device sends the question to the server, which uses a generative AI model to generate an answer to the question and returns the generated answer to the device, which then displays the generated answer to the user.

[1280] Running a VR experience

[1281] When a user selects a product demo VR content, the device sends the selected product ID to the server. The server retrieves the URL of the corresponding VR content from the database and returns it to the device. The device then uses this URL to launch the VR viewer and provide the VR experience.

[1282] Networking and Communication

[1283] When a user selects a target to start a networking session, the terminal sends the target's information to the server, and the server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[1284] Introducing the Emotion Engine

[1285] When a user sends a message or question, the emotion engine recognizes the user's emotional state. The device sends this emotional information to the server, which then generates an appropriate response using a generative AI model based on the emotion analysis results obtained by the emotion engine. For example, if the emotion engine recognizes that the user is nervous when the user enters a question, the server will take that information into account and generate a response that provides a sense of security.

[1286] Personalized VR Experience

[1287] The emotion engine recognizes the user's emotional state and personalizes the VR content experience. If the user is calm, a VR experience with detailed technical explanations is provided, while if the user is nervous, content with more general and simple explanations is provided. For example, if the emotion engine recognizes that the user is nervous, the server will prioritize providing more intuitive and visually appealing content.

[1288] Prompt Sentence Examples

[1289] If a user asks "What are the main components of this compound?" and submits an image:

[1290] Input: "What is the main component of this compound?", image file "path_to_image.jpg"

[1291] Prompt: In response to the user's question, the generated AI model based on the results of sentiment analysis replies, "The main ingredients are aluminum and silicon. There's nothing to be worried about; we'll explain the details in detail."

[1292] The above is a specific embodiment of the present invention. This system allows users to exchange information regardless of geographical constraints and efficiently collect information through real-time question and answering, interactive VR experiences, and emotional responses. This improves the user experience and deepens product understanding, contributing to the optimization of marketing strategies.

[1293] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1294] Step 1:

[1295] A user launches a messaging application and enters their user ID and password. The device receives this authentication information and sends it to the server. The server verifies the authentication information from the database, and if authentication is successful, generates a session token and sends it to the device. The device receives this session token and displays the main menu.

[1296] Input: User ID and password entered by the user

[1297] Output: Session token if successful, main menu displayed

[1298] What happens: The server authenticates the user by looking up the credentials in a database and verifying them.

[1299] Step 2:

[1300] The user selects a product search option and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[1301] Input: The search query entered by the user

[1302] Output: Product information search results

[1303] What happens: The server searches the database and returns matching product information.

[1304] Step 3:

[1305] A user enters a question about a specific product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[1306] Input: Product questions entered by the user

[1307] Output: Answer from the generative AI model

[1308] Specific operation: The server inputs a question into the generative AI model and generates an answer.

[1309] Step 4:

[1310] The user selects a product demo VR content. The device sends the selected product ID to the server. The server retrieves the URL of the corresponding VR content from the database and returns it to the device. The device uses this URL to launch the VR viewer and provide the VR experience.

[1311] Input: Product ID selected by the user

[1312] Output: URL of VR content, start of VR experience

[1313] Specific operation: The server searches the database, obtains the URL of the VR content, and returns it.

[1314] Step 5:

[1315] The user selects a target to start a networking session. The terminal sends the target's information to the server. The server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user. The user types a message and sends it to the server, which relays the message to the target user.

[1316] Input: Information of the target person selected by the user

[1317] Output: Chat session ID, chat window display

[1318] Specific operation: The server creates a chat session and relays messages.

[1319] Step 6:

[1320] When a user sends a message or question, the emotion engine recognizes the user's emotional state. The device sends the emotion engine's analysis results to the server. The server generates an appropriate response using a generative AI model based on the emotion analysis results obtained by the emotion engine and returns it to the device. The device then displays the response to the user.

[1321] Input: Messages and questions sent by users, and their emotional state

[1322] Output: The response of the generative AI model taking into account the emotional state

[1323] Specific operation: The emotion engine analyzes the user's emotional state, and the generative AI model generates a response based on the results.

[1324] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1325] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1326] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1327] [Third embodiment]

[1328] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1329] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1330] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1331] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1332] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1333] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1334] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1335] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1336] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1337] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1338] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1339] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1340] The system for implementing the present invention utilizes generative AI and a digital platform, including functions for user authentication, product information search and display, question answering, VR experiences, and networking.

[1341] User Authentication and Login

[1342] A user launches a messaging app and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information from the database, and if authentication is successful, it creates a user session and returns the main menu to the device along with a success message. The device then displays this main menu to the user.

[1343] Search and view product information

[1344] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[1345] For example, if a user searches for "polymeric materials," the server that receives this search query retrieves related product information from the database and displays it on the user's terminal.

[1346] Questions and Answers

[1347] A user enters a question about a particular product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[1348] For example, if a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[1349] Running a VR experience

[1350] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[1351] For example, if a user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[1352] Networking and Communication

[1353] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[1354] For example, if a user wants to talk to a representative of another manufacturer participating in an exhibition, the user can send a message to that representative via the chat function and have a conversation in real time.

[1355] The above is a concrete implementation of this system. The system is designed to enable users to exchange information without geographical constraints and to efficiently gather information through real-time question and answering and interactive VR experiences. This facilitates communication between manufacturers and visitors, contributing to product understanding and the optimization of market strategies.

[1356] The processing flow will be explained below.

[1357] User Authentication and Login

[1358] Step 1:

[1359] The user launches the messaging app and enters their user ID and password on the login screen.

[1360] Step 2:

[1361] The terminal sends the authentication information (user ID and password) entered by the user to the server.

[1362] Step 3:

[1363] The server checks the received authentication information against a database and, if authentication is successful, creates a user session.

[1364] Step 4:

[1365] The server returns the main menu data to the terminal along with an authentication success message.

[1366] Step 5:

[1367] The terminal displays the main menu to the user.

[1368] Search and view product information

[1369] Step 1:

[1370] A user selects the product search option on the main menu and enters a search query.

[1371] Step 2:

[1372] The terminal transmits the search query entered by the user to the server.

[1373] Step 3:

[1374] The server retrieves product information that matches the search query from a database.

[1375] Step 4:

[1376] The server returns the acquired search results to the terminal.

[1377] Step 5:

[1378] The terminal displays the search results to the user.

[1379] Questions and Answers

[1380] Step 1:

[1381] A user enters a question about a particular product.

[1382] Step 2:

[1383] The terminal sends this question to the server.

[1384] Step 3:

[1385] The server generates answers to questions using generative AI models.

[1386] Step 4:

[1387] The server returns the generated answer to the terminal.

[1388] Step 5:

[1389] The terminal displays the generated answer to the user.

[1390] Running a VR experience

[1391] Step 1:

[1392] The user selects the VR content for the product demo.

[1393] Step 2:

[1394] The terminal transmits the selected product ID to the server.

[1395] Step 3:

[1396] The server retrieves the URL of the corresponding VR content from the database.

[1397] Step 4:

[1398] The server returns the acquired URL to the terminal.

[1399] Step 5:

[1400] The device will use this URL to launch the VR viewer and provide the VR experience.

[1401] Networking and Communication

[1402] Step 1:

[1403] The user selects a target person to begin a networking session.

[1404] Step 2:

[1405] The terminal transmits the target person's information to the server and requests the initiation of a chat session.

[1406] Step 3:

[1407] The server creates a new chat session and generates a session ID.

[1408] Step 4:

[1409] The server returns the generated session ID to the terminal.

[1410] Step 5:

[1411] The terminal displays a real-time chat window and notifies the user that a session has begun.

[1412] Step 6:

[1413] The user types a message in the chat window.

[1414] Step 7:

[1415] The terminal transmits the input message to the server.

[1416] Step 8:

[1417] The server relays the received message to the target user.

[1418] Example 1

[1419] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1420] Currently, there is a lack of an integrated system that allows users to smoothly use multiple functions (authentication, product search, question and answering, VR experience, real-time chat, etc.) on a single platform. At the same time, there are limitations on the means to enrich users' interactive experiences and the means of real-time communication. To solve this issue, it is necessary to centrally manage and integrate all functions.

[1421] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1422] In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting user questions and generating answers to the questions using a generative AI model, means for delivering URLs of corresponding VR content based on VR content requests from users, means for generating and managing real-time chat sessions between users, and means for users to input messages into real-time chat windows and for transmitting and relaying the messages. This allows users to seamlessly use multiple functions and efficiently gather information and communicate.

[1423] "User authentication information" refers to information such as ID and password entered by a user to access a system.

[1424] A database is a collection of information that is organized and stored according to certain rules, allowing for efficient searching and retrieval.

[1425] "Product information" refers to detailed data, specifications, features, and other information about a product.

[1426] A "generative AI model" is a model that uses artificial intelligence technology to realize question-answering and data generation similar to those performed by humans.

[1427] "VR content" refers to digital content that provides a visual and auditory experience using virtual reality technology.

[1428] "URL" is a uniform resource identifier that indicates the location of an information resource on the Internet.

[1429] A "real-time chat session" is a communication session in which multiple users simultaneously send and receive messages and communicate in real time.

[1430] A "messaging application" is software that allows users to send and receive text messages, voice messages, images, etc.

[1431] A "session ID" is a unique identifier for identifying a particular user session.

[1432] A "server" is a computer system that provides services to other computers on a network.

[1433] The system for implementing the present invention utilizes generative AI models and a digital platform that includes functions for user authentication, product information search and display, question answering, VR experiences, and networking.

[1434] User Authentication and Login

[1435] A user launches a messaging application and enters authentication information (user ID and password). The terminal sends this authentication information to the server. The server verifies the authentication information from a database, and if authentication is successful, it creates a user session and returns a main menu to the terminal along with a success message. The terminal then displays this main menu to the user.

[1436] Examples:

[1437] A user opens a messaging application and enters "user123" as the user ID and "password321" as the password. The device sends this to the server, which verifies it and returns the message "Authentication successful. The main menu will be displayed" if authentication is successful.

[1438] Example prompt sentence:

[1439] "When a user enters their user ID and password, explain how the server validates that information."

[1440] Search and view product information

[1441] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[1442] Examples:

[1443] When a user searches for "polymer materials," the server receives this search query, retrieves related product information from the database, and displays it on the user's terminal.

[1444] Example prompt sentence:

[1445] "If a user searches for information about a particular polymeric material, explain how the system retrieves and displays that information."

[1446] Questions and Answers

[1447] A user enters a question about a specific product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[1448] Examples:

[1449] If a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[1450] Example prompt sentence:

[1451] "If a user asks a question about a specific product, explain how you can use a generative AI model to generate an answer to that question."

[1452] Running a VR experience

[1453] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[1454] Examples:

[1455] If the user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[1456] Example prompt sentence:

[1457] "When a user selects a VR experience, explain how the system delivers that VR content."

[1458] Networking and Communication

[1459] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[1460] Examples:

[1461] If a user wants to talk to a representative from another manufacturer participating in the exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[1462] Example prompt sentence:

[1463] "Please explain how the system works to allow users to communicate with other manufacturer representatives in real time."

[1464] This concludes the description of the embodiment of the present invention. This system is designed to allow users to seamlessly use multiple functions on an integrated platform, enabling users to efficiently gather information and communicate.

[1465] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1466] User Authentication and Login

[1467] Processing Steps:

[1468] Step 1:

[1469] The user launches a messaging app and enters authentication information (user ID and password), which then becomes the input data.

[1470] Input: User ID and password

[1471] Output: None

[1472] Step 2:

[1473] The terminal transmits the entered authentication information to the server.

[1474] Input: User ID and password

[1475] Output: Authentication information sent to the server

[1476] Step 3:

[1477] The server validates the received authentication information against a database. The database check confirms the authenticity of the authentication information.

[1478] Input: The authentication information sent to the server

[1479] Output: Authentication success or failure

[1480] Step 4:

[1481] If authentication is successful, the server creates a user session and returns the main menu to the terminal along with a success message.

[1482] Input: Authentication success information

[1483] Output: Success message and main menu

[1484] Step 5:

[1485] The terminal displays the received main menu to the user.

[1486] Input: Success message and main menu

[1487] Output: Display main menu

[1488] Search and view product information

[1489] Processing Steps:

[1490] Step 1:

[1491] A user selects a product search option in the main menu and enters a search query, which becomes the input data.

[1492] Input: search query

[1493] Output: None

[1494] Step 2:

[1495] The terminal sends a search query to the server.

[1496] Input: search query

[1497] Output: The search query sent to the server

[1498] Step 3:

[1499] The server retrieves product information that matches the query from the database. Matching product information is collected by searching the database.

[1500] Input: The search query sent to the server

[1501] Output: Search results

[1502] Step 4:

[1503] The server returns the search results to the terminal.

[1504] Input: Search results

[1505] Output: Search results sent back to your device

[1506] Step 5:

[1507] The terminal displays the search results to the user.

[1508] Input: Search results

[1509] Output: Display of search results

[1510] Questions and Answers

[1511] Processing Steps:

[1512] Step 1:

[1513] A user enters a question about a particular product, which then becomes the input data.

[1514] Input: User question

[1515] Output: None

[1516] Step 2:

[1517] The terminal sends a query to the server.

[1518] Input: User question

[1519] Output: The question sent to the server

[1520] Step 3:

[1521] The server generates answers to questions using a generative AI model, which analyzes the content of the question and generates an answer.

[1522] Input: The question sent to the server

[1523] Output: The generated answer

[1524] Step 4:

[1525] The server generates a response and sends it back to the terminal.

[1526] Input: Generated answer

[1527] Output: Answer sent back to the terminal

[1528] Step 5:

[1529] The terminal displays the generated answer to the user.

[1530] Input: Generated answer

[1531] Output: Display the answer

[1532] Running a VR experience

[1533] Processing Steps:

[1534] Step 1:

[1535] The user selects the VR content for the product demo, and the selected product ID becomes the input data.

[1536] Input: Product ID

[1537] Output: None

[1538] Step 2:

[1539] The terminal transmits the selected product ID to the server.

[1540] Input: Product ID

[1541] Output: Product ID sent to the server

[1542] Step 3:

[1543] The server retrieves the URL of the corresponding VR content from the database and identifies the URL of the corresponding VR content by database matching.

[1544] Input: Product ID sent to the server

[1545] Output: URL of VR content

[1546] Step 4:

[1547] The server returns the URL to the device.

[1548] Input: VR content URL

[1549] Output: URL sent back to the device

[1550] Step 5:

[1551] The device uses the URL to launch the VR viewer and provide the VR experience.

[1552] Input: VR content URL

[1553] Output: Providing a VR experience

[1554] Networking and Communication

[1555] Processing Steps:

[1556] Step 1:

[1557] To start a networking session, a user selects a target person, and the target person's information becomes input data.

[1558] Input: Subject information

[1559] Output: None

[1560] Step 2:

[1561] The terminal transmits the target person's information to the server.

[1562] Input: Subject information

[1563] Output: The target information sent to the server

[1564] Step 3:

[1565] The server creates a new chat session and returns a session ID to the terminal. The server generates a session ID and starts a chat session.

[1566] Input: Subject information sent to the server

[1567] Output: The generated session ID.

[1568] Step 4:

[1569] The terminal displays a real-time chat window and notifies the user that a session has begun.

[1570] Input: Generated session ID

[1571] Output: Chat window display

[1572] Step 5:

[1573] When a user types a message, the device sends the message to the server, which relays it to the target user.

[1574] Input: Message from the user

[1575] Output: Message sent to server and relayed to target user

[1576] Specific working example:

[1577] If a user wants to talk to a representative from another manufacturer participating in the exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[1578] (Application example 1)

[1579] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1580] Currently, there are many online shopping platforms on the market that allow users to search for various product information. However, users often experience inconveniences when checking product information, answering questions, and experiencing the products. Furthermore, the user experience can be limited due to limited interactive display of product information and real-time communication with other users and staff. Conventional systems do not provide users with sufficient information and support, making it difficult to provide the comfortable and efficient shopping experience desired by users. Therefore, this invention aims to provide a more comprehensive and interactive online shopping environment by integrating the functions of user authentication, product information search and display, question answering, VR experience provision, and real-time chat.

[1581] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1582] In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting user questions and generating answers to the questions using a generative AI model, means for delivering URLs of corresponding VR content based on VR content requests from users, means for generating and managing real-time chat sessions between users, means for displaying searched product information as 3D models or images, means for providing generated answers by voice, and means for communicating with other users and staff in real time. This allows users to enjoy a comprehensive online shopping experience, from authentication to question and answering, interactive display of product information, VR experiences, and real-time communication.

[1583] "User Credentials" means information used to identify a user and authorize access to a system.

[1584] "Means for verifying" is a function that executes a process to confirm whether user authentication information is correct.

[1585] A "search query" is a request entered by a user to retrieve specific information from a database.

[1586] "Product information" is detailed information about a particular product, typically stored in a database.

[1587] A "generative AI model" is an artificial intelligence model used to automatically generate answers to user questions.

[1588] A "VR content request" is a request submitted by a user to initiate a virtual reality experience.

[1589] A "real-time chat session" is an interactive session for sending and receiving text messages between users in real time.

[1590] "3D Model or Image" means a three-dimensional model or still image used to visually display product information.

[1591] The "means for providing by voice" is a function for providing the generated answer to the user as voice.

[1592] "Means of communicating in real time" refers to a function that allows you to communicate with other users and staff in real time.

[1593] MODE FOR CARRYING OUT THE INVENTION

[1594] The basic program for realizing this system includes the following means: Specific hardware required is a server, a user device (a smartphone or head-mounted display), a generative AI model, and a VR viewer.

[1595] Receiving and verifying user credentials

[1596] A user launches an application and enters authentication information (user ID and password). The terminal sends this authentication information to the server, which verifies it using a database. If authentication is successful, the server creates a user session and returns a main menu to the terminal along with a success message. The terminal displays this main menu to the user.

[1597] Search and view product information

[1598] A user selects the product search option from the main menu and enters a search query. The device sends this search query to the server, which retrieves product information matching the query from its database. This information is returned to the device as a 3D model or image, and the device displays the search results to the user. For example, if a user searches for "polymer materials," the server retrieves and displays related product information.

[1599] Questions and Answers

[1600] When a user enters a question about a particular product, the device sends the question to the server. The server uses a generative AI model to generate an answer to the question and returns it to the device. The device then provides the generated answer via voice. For example, if a user asks, "What are the main components of this compound?" the generative AI model might respond, "The main components are aluminum and silicon."

[1601] Running a VR experience

[1602] When a user selects a product demo VR content, the device sends the selected product ID to the server, which then retrieves the URL of the corresponding VR content from the database. This URL is then returned to the device, which then launches the VR viewer to provide the VR experience. For example, if a user selects a VR demo of a specific chemical process, the device retrieves the URL of the VR content corresponding to that process, and the user can use this URL to experience virtual reality.

[1603] Networking and Communication

[1604] When a user selects a target person to start a networking session, the terminal sends the target person's information to the server, and the server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, and the server relays the message to the target user. For example, if a user wants to talk to another representative at an exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[1605] Examples of concrete examples and prompts

[1606] As a specific example of user authentication, a user attempts to log in with the ID "user123" and the password "password123." As a specific example of product search, a search is performed for the chemical product "polymer material," and related product information is displayed. As a specific example of question and answer, a user asks "What are the main components of this compound?" via voice, and the generative AI model provides the answer "The main components are aluminum and silicon." As a specific example of a VR experience, a user starts a VR experience to view the details of a specific product in 3D. As a specific example of networking, a user can chat with other personnel in real time.

[1607] An example of a prompt to input to a generative AI model is as follows:

[1608] When asked, "What is the main component of this compound?", be specific about the main component of the chemical.

[1609] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1610] Step 1:

[1611] The user starts the application and enters authentication information (user ID and password). The device sends this authentication information to the server, which then verifies it using a database.

[1612] Input: User ID and password

[1613] Data processing: The server retrieves the authentication information from the database and compares it with the user's input.

[1614] Output: If authentication is successful, create a user session and generate a success message.

[1615] Step 2:

[1616] If the authentication is successful, the server sends the main menu to the terminal, which then displays the main menu to the user.

[1617] Input: Authentication success message

[1618] Data processing: The server saves the user's session information and prepares the main menu data.

[1619] Output: The main menu is displayed on the terminal.

[1620] Step 3:

[1621] The user selects the product search option from the main menu and enters a search query. The device sends this search query to the server, which retrieves product information matching the query from its database.

[1622] Input: search query

[1623] Data processing: The server uses a database search algorithm to retrieve product information that matches the query.

[1624] Output: A list of products matching the query will be displayed on the terminal.

[1625] Step 4:

[1626] The server transmits the acquired product information as a 3D model or image to the terminal, which then displays this information to the user.

[1627] Input: Product information list

[1628] Data processing: Processing to convert product information into 3D models or image formats.

[1629] Output: Product information displayed as a 3D model or image.

[1630] Step 5:

[1631] A user enters a question about a particular product, and the device sends the question to a server, which uses a generative AI model to generate an answer to the question.

[1632] Input: Product Question

[1633] Data processing: Questions are input as prompts into the AI ​​model to generate answers.

[1634] Output: The answer is sent to the terminal and displayed.

[1635] Step 6:

[1636] The terminal presents the user with the option to provide the generated answer aloud, and if selected by the user, outputs it as speech using speech synthesis technology.

[1637] Input: Answer text

[1638] Data processing: Converting text into audio data (speech synthesis).

[1639] Output: A spoken answer is provided.

[1640] Step 7:

[1641] When a user selects a product demo VR content, the terminal sends the selected product ID to the server, and the server retrieves the URL of the corresponding VR content from the database.

[1642] Input: Product ID

[1643] Data processing: The server retrieves the URL of the VR content corresponding to the product ID from the database.

[1644] Output: The URL of the VR content is sent to the device.

[1645] Step 8:

[1646] The device uses the acquired URL to launch the VR viewer and provide the user with a VR experience.

[1647] Input: VR content URL

[1648] Data processing: The VR viewer loads the URL and displays the corresponding VR content.

[1649] Output: The VR experience is delivered to the user.

[1650] Step 9:

[1651] The user selects a target person to start a networking session, and the device sends the target person's information to the server, which creates a new chat session and returns a session ID to the device.

[1652] Input: Subject information

[1653] Data processing: Create a new chat session and issue a session ID.

[1654] Output: The session ID is sent to the terminal.

[1655] Step 10:

[1656] The terminal displays a real-time chat window and allows the user to input a message, which is then sent to the server, which relays the message to the target user.

[1657] Input: User's message

[1658] Data processing: Processes messages to be relayed in real time.

[1659] Output: A message is sent to the target user.

[1660] By performing these steps, users can achieve a comprehensive online shopping experience, from authentication to question and answering, interactive display of product information, VR experience, and real-time communication.

[1661] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1662] The system for implementing the present invention utilizes generative AI and a digital platform that includes user authentication, product information search and display, question answering, VR experiences, networking, and an emotion engine that recognizes user emotions.

[1663] User Authentication and Login

[1664] A user launches a messaging app and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information from the database, and if authentication is successful, it creates a user session and returns the main menu to the device along with a success message. The device then displays this main menu to the user.

[1665] Search and view product information

[1666] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[1667] For example, if a user searches for "polymeric materials," the server that receives this search query retrieves related product information from the database and displays it on the user's terminal.

[1668] Questions and Answers

[1669] A user enters a question about a particular product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[1670] For example, if a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[1671] Running a VR experience

[1672] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[1673] For example, if a user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[1674] Networking and Communication

[1675] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[1676] For example, if a user wants to talk to a representative of another manufacturer participating in an exhibition, the user can send a message to that representative via the chat function and have a conversation in real time.

[1677] Introducing the Emotion Engine

[1678] When a user sends a message or question, the emotion engine recognizes the user's emotional state. The device then sends this emotional information to the server, which then uses the emotion analysis results obtained by the emotion engine to generate an appropriate response using a generative AI model.

[1679] For example, if a user types a question and the emotion engine recognizes that the user is nervous, the server is designed to take that information into account and generate a reassuring answer.

[1680] Personalized VR Experience

[1681] The emotion engine recognizes the user's emotional state and personalizes the VR content experience: if the user is calm, a VR experience with detailed technical explanations is provided, while if the user is nervous, content with more general and simplified explanations is provided.

[1682] For example, if the emotion engine recognizes that the user is nervous, the server will prioritize providing more intuitive and visually appealing content.

[1683] The above is a concrete example of combining this system with an emotion engine. The system is designed to enable users to exchange information regardless of geographical constraints and to efficiently collect information through real-time question and answering, interactive VR experiences, and emotional responses. This facilitates communication between manufacturers and visitors, contributing to product understanding and the optimization of market strategies.

[1684] The processing flow will be explained below.

[1685] User Authentication and Login

[1686] Step 1:

[1687] The user launches the messaging app and enters their user ID and password on the login screen.

[1688] Step 2:

[1689] The terminal sends the authentication information (user ID and password) entered by the user to the server.

[1690] Step 3:

[1691] The server checks the received authentication information against a database and, if authentication is successful, creates a user session.

[1692] Step 4:

[1693] The server returns the main menu data to the terminal along with an authentication success message.

[1694] Step 5:

[1695] The terminal displays the main menu to the user.

[1696] Search and view product information

[1697] Step 1:

[1698] A user selects the product search option on the main menu and enters a search query.

[1699] Step 2:

[1700] The terminal transmits the search query entered by the user to the server.

[1701] Step 3:

[1702] The server retrieves product information that matches the search query from a database.

[1703] Step 4:

[1704] The server returns the acquired search results to the terminal.

[1705] Step 5:

[1706] The terminal displays the search results to the user.

[1707] Questions and Answers

[1708] Step 1:

[1709] A user enters a question about a particular product.

[1710] Step 2:

[1711] The terminal sends this question to the server.

[1712] Step 3:

[1713] The server generates answers to questions using generative AI models.

[1714] Step 4:

[1715] The server returns the generated answer to the terminal.

[1716] Step 5:

[1717] The terminal displays the generated answer to the user.

[1718] Running a VR experience

[1719] Step 1:

[1720] The user selects the VR content for the product demo.

[1721] Step 2:

[1722] The terminal transmits the selected product ID to the server.

[1723] Step 3:

[1724] The server retrieves the URL of the corresponding VR content from the database.

[1725] Step 4:

[1726] The server returns the acquired URL to the terminal.

[1727] Step 5:

[1728] The device will use this URL to launch the VR viewer and provide the VR experience.

[1729] Networking and Communication

[1730] Step 1:

[1731] The user selects a target person to begin a networking session.

[1732] Step 2:

[1733] The terminal transmits the target person's information to the server and requests the initiation of a chat session.

[1734] Step 3:

[1735] The server creates a new chat session and generates a session ID.

[1736] Step 4:

[1737] The server returns the generated session ID to the terminal.

[1738] Step 5:

[1739] The terminal displays a real-time chat window and notifies the user that a session has begun.

[1740] Step 6:

[1741] The user types a message in the chat window.

[1742] Step 7:

[1743] The terminal transmits the input message to the server.

[1744] Step 8:

[1745] The server relays the received message to the target user.

[1746] Introducing the Emotion Engine

[1747] Step 1:

[1748] The user enters a message or question.

[1749] Step 2:

[1750] The device sends the input message or question to the emotion engine, which analyzes the emotional state.

[1751] Step 3:

[1752] The emotion engine recognizes the user's emotional state (e.g., happy, sad, nervous, etc.).

[1753] Step 4:

[1754] The emotion engine sends the analysis results to the server.

[1755] Step 5:

[1756] Based on the results of the sentiment analysis, the server uses a generative AI model to generate an appropriate response.

[1757] Step 6:

[1758] The server returns the generated response to the terminal.

[1759] Step 7:

[1760] The terminal displays the generated response to the user.

[1761] Personalized VR Experience

[1762] Step 1:

[1763] The user selects the VR content for the product demo.

[1764] Step 2:

[1765] The terminal analyzes the user's emotional state using an emotion engine.

[1766] Step 3:

[1767] The emotion engine recognizes whether the user is calm or nervous.

[1768] Step 4:

[1769] The emotion engine sends the analysis results to the server.

[1770] Step 5:

[1771] The server obtains the URL of the personalized VR content based on the emotion analysis results.

[1772] Step 6:

[1773] The server returns the URL to the terminal.

[1774] Step 7:

[1775] The device uses the URL to launch a VR viewer to provide a personalized VR experience.

[1776] For example, if the emotion engine recognizes that the user is tense, the server can provide visually appealing relaxation content to enhance the user's experience.

[1777] Example 2

[1778] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1779] Conventional systems perform user authentication, product information search, question answering, virtual reality experience, real-time communication, and emotion recognition and analysis separately, making it difficult to provide a seamless user experience. Furthermore, because each function is independent, it is not possible to provide a consistent service. In particular, there is a lack of personalized responses and content based on the user's emotional state, and there is a need to improve user satisfaction.

[1780] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1781] In this invention, the server includes: means for receiving user authentication information and verifying the authentication information; means for retrieving product information from a database based on a search query entered by the user; means for accepting user questions and generating answers to the questions using a generative AI model; means for delivering URLs of corresponding virtual reality content based on virtual reality content requests from users; means for generating and managing real-time communication sessions between users; and means for analyzing the user's emotional state and personalizing responses and content based on the analysis results. This integration of each function enables a seamless and consistent user experience. Furthermore, providing personalized services based on the user's emotional state is expected to improve user satisfaction.

[1782] "User authentication information" is data used to identify a user and verify access permissions.

[1783] "Verification" is the process of checking whether the information received is correct.

[1784] A "search query" is a condition or keyword entered to retrieve specific information from a database.

[1785] A "database" is a system for efficiently managing large amounts of data and for quickly searching and manipulating it.

[1786] A "generative AI model" is an algorithm or machine learning model that uses artificial intelligence to generate new data and answers.

[1787] "Virtual Reality Content" means information or simulations that are visually and experientially presented to users using virtual reality technology.

[1788] A "URL" is an address used to specify a resource on the Internet.

[1789] A "real-time communication session" is a communication session in which information is exchanged between users in real time.

[1790] "Emotional state" refers to the user's current psychological and physiological state.

[1791] "Analysis results" refer to the conclusions or findings obtained after conducting data analysis.

[1792] "Personalization" means customizing services and content according to the needs and circumstances of individual users.

[1793] A system for implementing the present invention provides user authentication, product information search and display, question answering, virtual reality experiences, real-time communication, and emotion recognition and personalization. The system utilizes generative AI models and a digital platform to provide a seamless and consistent user experience.

[1794] User Authentication and Login

[1795] The user launches a messaging application and enters their user ID and password. The device sends this authentication information to the server. The server verifies the authentication information using a database, and if authentication is successful, creates a user session and returns a success message and the main menu to the device. The device then displays this main menu to the user.

[1796] Example: When a user enters "user123" and "password123" and the terminal sends this information to the server, the server verifies the information in the database and, after successful authentication, displays the message "Login successful" and the main menu.

[1797] Search and view product information

[1798] The user selects the product search option from the main menu and enters a search query. The device sends this query to the server. The server retrieves matching product information from the database and returns the search results to the device. The device displays the search results to the user.

[1799] Example: When a user types "polymeric materials" and the terminal sends this query to the server, the server retrieves relevant product information from the database and the terminal displays it to the user.

[1800] Questions and Answers

[1801] When a user enters a question about a particular product, the device sends the question to the server, which uses a generative AI model to generate an answer to the question and returns the answer to the device, which then displays the answer to the user.

[1802] Example: A user asks, "What are the main components of this compound?" and the device sends the question to a server. The server uses a generative AI model to generate the answer, "The main components are aluminum and silicon," and the device displays this answer to the user.

[1803] Running a virtual reality experience

[1804] When a user selects a product demo virtual reality content, the device sends the selected product ID to the server, and the server retrieves the URL of the corresponding virtual reality content from the database and returns it to the device. The device then uses this URL to launch the VR viewer and provide the user with a virtual reality experience.

[1805] Example: When a user selects "VR demo of chemical process" and the device sends the product ID to the server, the server returns the URL of the corresponding virtual reality content, the device uses the URL to launch the VR viewer, and the user can experience the virtual reality.

[1806] Realizing real-time communication

[1807] When a user selects a target user for networking and the terminal sends this information to the server, the server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and sends messages entered by the user to the server, which then relays the messages to the target user.

[1808] Example: When a user selects to chat with "Person B at Manufacturer A" and the device sends information to the server, the server creates a chat session and returns a session ID. When the device displays a chat window and the user types "Hello" and sends it, the server relays this message to Person B.

[1809] Emotion Recognition and Personalization

[1810] When a user inputs a message or question, the emotion engine analyzes the user's emotional state, and the device transmits the emotional state to the server. The server uses a generative AI model to generate an appropriate response based on the analysis results and returns it to the device. The device then displays this response to the user, personalizing the VR content based on the user's emotional state.

[1811] Example: If a user asks, "Tell me about the properties of this compound," and the emotion engine recognizes "tension," the device sends this information to the server, which then generates a response such as, "Relax and ask your question," and returns it to the device. Based on the emotional state, the server provides the device with a URL for more intuitive and visually appealing VR content.

[1812] Prompt Sentence Examples

[1813] "Please explain in detail the process by which users search for polymer materials."

[1814] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1815] Step 1:

[1816] The user launches a messaging application and enters their user ID and password, which the device then sends to the server.

[1817] Specific operation: The user enters "user123" and "password123", and the device sends this to the server.

[1818] Input: User ID and password

[1819] Output: Sending authentication information

[1820] Step 2:

[1821] The server receives the entered authentication information and validates it against a database.

[1822] Specific operation: The server retrieves the password corresponding to "user123" from the database and compares it with the entered password.

[1823] Input: Credentials

[1824] Output: Authentication result

[1825] Step 3:

[1826] If authentication is successful, the server creates a user session and returns a success message and the main menu to the terminal, otherwise it returns an error message.

[1827] Specific operation: When authentication is successful, a session ID is generated and "Login successful" and the main menu are sent to the terminal.

[1828] Input: Authentication result

[1829] Output: Success message and main menu, or failure message

[1830] Step 4:

[1831] The terminal displays the main menu to the user, who selects the product search option and enters a search query.

[1832] Specific action: The user enters "polymer material."

[1833] Input: Main menu, search query

[1834] Output: Submitting a search query

[1835] Step 5:

[1836] The terminal sends the entered search query to the server, which uses a database to retrieve product information that matches the query.

[1837] Specific operation: The server searches and retrieves product information matching "polymer material" from the database.

[1838] Input: search query

[1839] Output: Product information

[1840] Step 6:

[1841] The server returns the acquired product information to the terminal, which then displays this information to the user.

[1842] Specific behavior: The device presents the user with details about "polymer materials."

[1843] Input: Product Information

[1844] Output: Display product information

[1845] Step 7:

[1846] The user enters a question about a particular product and the terminal transmits the question to the server.

[1847] What happens: A user asks, "What are the main components of this compound?"

[1848] Input: User question

[1849] Output: Question sent

[1850] Step 8:

[1851] The server receives the question and uses a generative AI model to generate an answer to the question.

[1852] What it does: The server uses a generative AI model to generate the answer, "The main components are aluminum and silicon."

[1853] Input: Question

[1854] Output: Answer

[1855] Step 9:

[1856] The server returns the generated answer to the terminal, which displays the answer to the user.

[1857] What it does: The device displays "The main components are aluminum and silicon."

[1858] Input: Answer

[1859] Output: Display the answer

[1860] Step 10:

[1861] When a user selects a particular product demo virtual reality content, the terminal transmits the selected product ID to the server.

[1862] Specific action: The user selects "Chemical Process VR Demo."

[1863] Input: Product ID

[1864] Output: Send product ID

[1865] Step 11:

[1866] The server retrieves the URL of the corresponding virtual reality content from the database and returns it to the terminal.

[1867] Specific operation: The server retrieves the URL of the corresponding VR content from the database.

[1868] Input: Product ID

[1869] Output: URL of virtual reality content

[1870] Step 12:

[1871] The device uses the obtained URL to launch the VR viewer and provide the user with a virtual reality experience.

[1872] Specific operation: The device launches the VR viewer and the user experiences virtual reality.

[1873] Input: URL of virtual reality content

[1874] Output: Virtual reality experience

[1875] Step 13:

[1876] The user selects the target users for networking, and the terminal transmits this information to the server.

[1877] Specific operation: The user selects that he wants to chat with "Person B in charge of Manufacturer A."

[1878] Input: Networking target information

[1879] Output: Sending information

[1880] Step 14:

[1881] The server creates a new chat session and returns a session ID to the device.

[1882] Specific operation: The server creates a chat session and returns a session ID to the device.

[1883] Input: Networking target information

[1884] Output: Session ID

[1885] Step 15:

[1886] The terminal displays a real-time chat window and transmits messages entered by the user to the server.

[1887] What happens: The device displays a chat window and the user types "Hello."

[1888] Input: Session ID, user message

[1889] Output: Sending a message

[1890] Step 16:

[1891] The server relays the received message to the target user.

[1892] Specific action: The server relays the message "Hello" to Agent B.

[1893] Input: message

[1894] Output: Message relay

[1895] Step 17:

[1896] The emotion engine analyzes messages and questions entered by the user, and the device sends the user's emotional state to the server.

[1897] Specific operation: The emotion engine recognizes "tension" and sends that information to the server.

[1898] Input: Message or question

[1899] output: emotional state

[1900] Step 18:

[1901] The server uses the generative AI model to generate an appropriate response based on the analysis results and returns it to the device.

[1902] Specific operation: The server generates a response such as "Just relax and ask your questions" and returns it to the device.

[1903] Input: Emotional state

[1904] Output: Response

[1905] Step 19:

[1906] The device displays the obtained responses to the user and provides personalized content based on the emotional state.

[1907] What it does: The device displays personalized content and provides it to the user.

[1908] Input: Response, Emotional State

[1909] Output: personalized content

[1910] The above are the detailed processing steps of this system, which will enable users to enjoy real-time services based on authentication, search, question answering, VR experiences, networking, and emotion recognition.

[1911] (Application example 2)

[1912] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1913] Conventional online shopping systems have difficulty in taking user emotions into consideration, limiting the improvement of user experience. In addition, the limited methods of providing product information make it difficult for users to fully understand the products. This can lead to a decrease in user satisfaction and a negative impact on sales.

[1914] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting a user's question and generating an answer to the question using a generative model, means for delivering a URL of corresponding VR content based on a VR content request from the user, means for generating and managing real-time chat sessions between users, and means for recognizing the user's emotional state and having the generative model personalize responses based on that information. This enables personalized responses according to the user's emotions, deepening product understanding and significantly improving the user experience.

[1915] "User Credentials" means the credentials used by a User to access a System, generally including a User ID and password.

[1916] A "database" refers to a collection of data that stores product information, user information, etc. and can be searched and retrieved.

[1917] A "search query" is a keyword or phrase that a user enters to retrieve specific information.

[1918] A "generative model" refers to a system that uses AI algorithms to generate answers or responses to user questions.

[1919] "VR Content" means digital content that can be experienced by a user using virtual reality technology.

[1920] "URL" is an abbreviation for Universal Resource Locator, and is an address that points to a specific resource on the Internet.

[1921] "Real-time chat session" refers to a chat feature that allows users to exchange messages with each other in real time.

[1922] An "emotional state" refers to the emotional state that the user is feeling at that time, and includes, for example, joy, sadness, tension, and the like.

[1923] "Personalization" means tailoring services and content to a user's specific needs and circumstances.

[1924] This invention provides a system that combines user authentication, product information search and display, question answering, VR experience, networking, and an emotion engine. A specific embodiment of this system is described below.

[1925] System configuration

[1926] This system mainly consists of three elements: a server, a device, and a user. The hardware used is a smartphone and a head-mounted display, and the software uses Python, a generative AI model, and a facial recognition API.

[1927] User Authentication and Login

[1928] A user launches a messaging application and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information against a database, and if authentication is successful, returns a session token to the device. The device uses this session token to display the main menu.

[1929] Search and view product information

[1930] When a user selects the product search option on the main menu and enters a search query, the device sends the search query to the server, which retrieves product information that matches the query from the database and returns the search results to the device, which then displays the search results to the user.

[1931] Questions and Answers

[1932] When a user enters a question about a particular product, the device sends the question to the server, which uses a generative AI model to generate an answer to the question and returns the generated answer to the device, which then displays the generated answer to the user.

[1933] Running a VR experience

[1934] When a user selects a product demo VR content, the device sends the selected product ID to the server. The server retrieves the URL of the corresponding VR content from the database and returns it to the device. The device then uses this URL to launch the VR viewer and provide the VR experience.

[1935] Networking and Communication

[1936] When a user selects a target to start a networking session, the terminal sends the target's information to the server, and the server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[1937] Introducing the Emotion Engine

[1938] When a user sends a message or question, the emotion engine recognizes the user's emotional state. The device sends this emotional information to the server, which then generates an appropriate response using a generative AI model based on the emotion analysis results obtained by the emotion engine. For example, if the emotion engine recognizes that the user is nervous when the user enters a question, the server will take that information into account and generate a response that provides a sense of security.

[1939] Personalized VR Experience

[1940] The emotion engine recognizes the user's emotional state and personalizes the VR content experience. If the user is calm, a VR experience with detailed technical explanations is provided, while if the user is nervous, content with more general and simple explanations is provided. For example, if the emotion engine recognizes that the user is nervous, the server will prioritize providing more intuitive and visually appealing content.

[1941] Prompt Sentence Examples

[1942] If a user asks "What are the main components of this compound?" and submits an image:

[1943] Input: "What is the main component of this compound?", image file "path_to_image.jpg"

[1944] Prompt: In response to the user's question, the generated AI model based on the results of sentiment analysis replies, "The main ingredients are aluminum and silicon. There's nothing to be worried about; we'll explain the details in detail."

[1945] The above is a specific embodiment of the present invention. This system allows users to exchange information regardless of geographical constraints and efficiently collect information through real-time question and answering, interactive VR experiences, and emotional responses. This improves the user experience and deepens product understanding, contributing to the optimization of marketing strategies.

[1946] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1947] Step 1:

[1948] A user launches a messaging application and enters their user ID and password. The device receives this authentication information and sends it to the server. The server verifies the authentication information from the database, and if authentication is successful, generates a session token and sends it to the device. The device receives this session token and displays the main menu.

[1949] Input: User ID and password entered by the user

[1950] Output: Session token if successful, main menu displayed

[1951] What happens: The server authenticates the user by looking up the credentials in a database and verifying them.

[1952] Step 2:

[1953] The user selects a product search option and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[1954] Input: The search query entered by the user

[1955] Output: Product information search results

[1956] What happens: The server searches the database and returns matching product information.

[1957] Step 3:

[1958] A user enters a question about a specific product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[1959] Input: Product questions entered by the user

[1960] Output: Answer from the generative AI model

[1961] Specific operation: The server inputs a question into the generative AI model and generates an answer.

[1962] Step 4:

[1963] The user selects a product demo VR content. The device sends the selected product ID to the server. The server retrieves the URL of the corresponding VR content from the database and returns it to the device. The device uses this URL to launch the VR viewer and provide the VR experience.

[1964] Input: Product ID selected by the user

[1965] Output: URL of VR content, start of VR experience

[1966] Specific operation: The server searches the database, obtains the URL of the VR content, and returns it.

[1967] Step 5:

[1968] The user selects a target to start a networking session. The terminal sends the target's information to the server. The server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user. The user types a message and sends it to the server, which relays the message to the target user.

[1969] Input: Information of the target person selected by the user

[1970] Output: Chat session ID, chat window display

[1971] Specific operation: The server creates a chat session and relays messages.

[1972] Step 6:

[1973] When a user sends a message or question, the emotion engine recognizes the user's emotional state. The device sends the emotion engine's analysis results to the server. The server generates an appropriate response using a generative AI model based on the emotion analysis results obtained by the emotion engine and returns it to the device. The device then displays the response to the user.

[1974] Input: Messages and questions sent by users, and their emotional state

[1975] Output: The response of the generative AI model taking into account the emotional state

[1976] Specific operation: The emotion engine analyzes the user's emotional state, and the generative AI model generates a response based on the results.

[1977] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1978] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1979] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1980] [Fourth embodiment]

[1981] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1982] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1983] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1984] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1985] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1986] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1987] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1988] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1989] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1990] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1991] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1992] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1993] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1994] The system for implementing the present invention utilizes generative AI and a digital platform, including functions for user authentication, product information search and display, question answering, VR experiences, and networking.

[1995] User Authentication and Login

[1996] A user launches a messaging app and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information from the database, and if authentication is successful, it creates a user session and returns the main menu to the device along with a success message. The device then displays this main menu to the user.

[1997] Search and view product information

[1998] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[1999] For example, if a user searches for "polymeric materials," the server that receives this search query retrieves related product information from the database and displays it on the user's terminal.

[2000] Questions and Answers

[2001] A user enters a question about a particular product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[2002] For example, if a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[2003] Running a VR experience

[2004] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[2005] For example, if a user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[2006] Networking and Communication

[2007] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[2008] For example, if a user wants to talk to a representative of another manufacturer participating in an exhibition, the user can send a message to that representative via the chat function and have a conversation in real time.

[2009] The above is a concrete implementation of this system. The system is designed to enable users to exchange information without geographical constraints and to efficiently gather information through real-time question and answering and interactive VR experiences. This facilitates communication between manufacturers and visitors, contributing to product understanding and the optimization of market strategies.

[2010] The processing flow will be explained below.

[2011] User Authentication and Login

[2012] Step 1:

[2013] The user launches the messaging app and enters their user ID and password on the login screen.

[2014] Step 2:

[2015] The terminal sends the authentication information (user ID and password) entered by the user to the server.

[2016] Step 3:

[2017] The server checks the received authentication information against a database and, if authentication is successful, creates a user session.

[2018] Step 4:

[2019] The server returns the main menu data to the terminal along with an authentication success message.

[2020] Step 5:

[2021] The terminal displays the main menu to the user.

[2022] Search and view product information

[2023] Step 1:

[2024] A user selects the product search option on the main menu and enters a search query.

[2025] Step 2:

[2026] The terminal transmits the search query entered by the user to the server.

[2027] Step 3:

[2028] The server retrieves product information that matches the search query from a database.

[2029] Step 4:

[2030] The server returns the acquired search results to the terminal.

[2031] Step 5:

[2032] The terminal displays the search results to the user.

[2033] Questions and Answers

[2034] Step 1:

[2035] A user enters a question about a particular product.

[2036] Step 2:

[2037] The terminal sends this question to the server.

[2038] Step 3:

[2039] The server generates answers to questions using generative AI models.

[2040] Step 4:

[2041] The server returns the generated answer to the terminal.

[2042] Step 5:

[2043] The terminal displays the generated answer to the user.

[2044] Running a VR experience

[2045] Step 1:

[2046] The user selects the VR content for the product demo.

[2047] Step 2:

[2048] The terminal transmits the selected product ID to the server.

[2049] Step 3:

[2050] The server retrieves the URL of the corresponding VR content from the database.

[2051] Step 4:

[2052] The server returns the acquired URL to the terminal.

[2053] Step 5:

[2054] The device will use this URL to launch the VR viewer and provide the VR experience.

[2055] Networking and Communication

[2056] Step 1:

[2057] The user selects a target person to begin a networking session.

[2058] Step 2:

[2059] The terminal transmits the target person's information to the server and requests the initiation of a chat session.

[2060] Step 3:

[2061] The server creates a new chat session and generates a session ID.

[2062] Step 4:

[2063] The server returns the generated session ID to the terminal.

[2064] Step 5:

[2065] The terminal displays a real-time chat window and notifies the user that a session has begun.

[2066] Step 6:

[2067] The user types a message in the chat window.

[2068] Step 7:

[2069] The terminal transmits the input message to the server.

[2070] Step 8:

[2071] The server relays the received message to the target user.

[2072] Example 1

[2073] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2074] Currently, there is a lack of an integrated system that allows users to smoothly use multiple functions (authentication, product search, question and answering, VR experience, real-time chat, etc.) on a single platform. At the same time, there are limitations on the means to enrich users' interactive experiences and the means of real-time communication. To solve this issue, it is necessary to centrally manage and integrate all functions.

[2075] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2076] In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting user questions and generating answers to the questions using a generative AI model, means for delivering URLs of corresponding VR content based on VR content requests from users, means for generating and managing real-time chat sessions between users, and means for users to input messages into real-time chat windows and for transmitting and relaying the messages. This allows users to seamlessly use multiple functions and efficiently gather information and communicate.

[2077] "User authentication information" refers to information such as ID and password entered by a user to access a system.

[2078] A database is a collection of information that is organized and stored according to certain rules, allowing for efficient searching and retrieval.

[2079] "Product information" refers to detailed data, specifications, features, and other information about a product.

[2080] A "generative AI model" is a model that uses artificial intelligence technology to realize question-answering and data generation similar to those performed by humans.

[2081] "VR content" refers to digital content that provides a visual and auditory experience using virtual reality technology.

[2082] "URL" is a uniform resource identifier that indicates the location of an information resource on the Internet.

[2083] A "real-time chat session" is a communication session in which multiple users simultaneously send and receive messages and communicate in real time.

[2084] A "messaging application" is software that allows users to send and receive text messages, voice messages, images, etc.

[2085] A "session ID" is a unique identifier for identifying a particular user session.

[2086] A "server" is a computer system that provides services to other computers on a network.

[2087] The system for implementing the present invention utilizes generative AI models and a digital platform that includes functions for user authentication, product information search and display, question answering, VR experiences, and networking.

[2088] User Authentication and Login

[2089] A user launches a messaging application and enters authentication information (user ID and password). The terminal sends this authentication information to the server. The server verifies the authentication information from a database, and if authentication is successful, it creates a user session and returns a main menu to the terminal along with a success message. The terminal then displays this main menu to the user.

[2090] Examples:

[2091] A user opens a messaging application and enters "user123" as the user ID and "password321" as the password. The device sends this to the server, which verifies it and returns the message "Authentication successful. The main menu will be displayed" if authentication is successful.

[2092] Example prompt sentence:

[2093] "When a user enters their user ID and password, explain how the server validates that information."

[2094] Search and view product information

[2095] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[2096] Examples:

[2097] When a user searches for "polymer materials," the server receives this search query, retrieves related product information from the database, and displays it on the user's terminal.

[2098] Example prompt sentence:

[2099] "If a user searches for information about a particular polymeric material, explain how the system retrieves and displays that information."

[2100] Questions and Answers

[2101] A user enters a question about a specific product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[2102] Examples:

[2103] If a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[2104] Example prompt sentence:

[2105] "If a user asks a question about a specific product, explain how you can use a generative AI model to generate an answer to that question."

[2106] Running a VR experience

[2107] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[2108] Examples:

[2109] If the user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[2110] Example prompt sentence:

[2111] "When a user selects a VR experience, explain how the system delivers that VR content."

[2112] Networking and Communication

[2113] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[2114] Examples:

[2115] If a user wants to talk to a representative from another manufacturer participating in the exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[2116] Example prompt sentence:

[2117] "Please explain how the system works to allow users to communicate with other manufacturer representatives in real time."

[2118] This concludes the description of the embodiment of the present invention. This system is designed to allow users to seamlessly use multiple functions on an integrated platform, enabling users to efficiently gather information and communicate.

[2119] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2120] User Authentication and Login

[2121] Processing Steps:

[2122] Step 1:

[2123] The user launches a messaging app and enters authentication information (user ID and password), which then becomes the input data.

[2124] Input: User ID and password

[2125] Output: None

[2126] Step 2:

[2127] The terminal transmits the entered authentication information to the server.

[2128] Input: User ID and password

[2129] Output: Authentication information sent to the server

[2130] Step 3:

[2131] The server validates the received authentication information against a database. The database check confirms the authenticity of the authentication information.

[2132] Input: The authentication information sent to the server

[2133] Output: Authentication success or failure

[2134] Step 4:

[2135] If authentication is successful, the server creates a user session and returns the main menu to the terminal along with a success message.

[2136] Input: Authentication success information

[2137] Output: Success message and main menu

[2138] Step 5:

[2139] The terminal displays the received main menu to the user.

[2140] Input: Success message and main menu

[2141] Output: Display main menu

[2142] Search and view product information

[2143] Processing Steps:

[2144] Step 1:

[2145] A user selects a product search option in the main menu and enters a search query, which becomes the input data.

[2146] Input: search query

[2147] Output: None

[2148] Step 2:

[2149] The terminal sends a search query to the server.

[2150] Input: search query

[2151] Output: The search query sent to the server

[2152] Step 3:

[2153] The server retrieves product information that matches the query from the database. Matching product information is collected by searching the database.

[2154] Input: The search query sent to the server

[2155] Output: Search results

[2156] Step 4:

[2157] The server returns the search results to the terminal.

[2158] Input: Search results

[2159] Output: Search results sent back to your device

[2160] Step 5:

[2161] The terminal displays the search results to the user.

[2162] Input: Search results

[2163] Output: Display of search results

[2164] Questions and Answers

[2165] Processing Steps:

[2166] Step 1:

[2167] A user enters a question about a particular product, which then becomes the input data.

[2168] Input: User question

[2169] Output: None

[2170] Step 2:

[2171] The terminal sends a query to the server.

[2172] Input: User question

[2173] Output: The question sent to the server

[2174] Step 3:

[2175] The server generates answers to questions using a generative AI model, which analyzes the content of the question and generates an answer.

[2176] Input: The question sent to the server

[2177] Output: The generated answer

[2178] Step 4:

[2179] The server generates a response and sends it back to the terminal.

[2180] Input: Generated answer

[2181] Output: Answer sent back to the terminal

[2182] Step 5:

[2183] The terminal displays the generated answer to the user.

[2184] Input: Generated answer

[2185] Output: Display the answer

[2186] Running a VR experience

[2187] Processing Steps:

[2188] Step 1:

[2189] The user selects the VR content for the product demo, and the selected product ID becomes the input data.

[2190] Input: Product ID

[2191] Output: None

[2192] Step 2:

[2193] The terminal transmits the selected product ID to the server.

[2194] Input: Product ID

[2195] Output: Product ID sent to the server

[2196] Step 3:

[2197] The server retrieves the URL of the corresponding VR content from the database and identifies the URL of the corresponding VR content by database matching.

[2198] Input: Product ID sent to the server

[2199] Output: URL of VR content

[2200] Step 4:

[2201] The server returns the URL to the device.

[2202] Input: VR content URL

[2203] Output: URL sent back to the device

[2204] Step 5:

[2205] The device uses the URL to launch the VR viewer and provide the VR experience.

[2206] Input: VR content URL

[2207] Output: Providing a VR experience

[2208] Networking and Communication

[2209] Processing Steps:

[2210] Step 1:

[2211] To start a networking session, a user selects a target person, and the target person's information becomes input data.

[2212] Input: Subject information

[2213] Output: None

[2214] Step 2:

[2215] The terminal transmits the target person's information to the server.

[2216] Input: Subject information

[2217] Output: The target information sent to the server

[2218] Step 3:

[2219] The server creates a new chat session and returns a session ID to the terminal. The server generates a session ID and starts a chat session.

[2220] Input: Subject information sent to the server

[2221] Output: The generated session ID.

[2222] Step 4:

[2223] The terminal displays a real-time chat window and notifies the user that a session has begun.

[2224] Input: Generated session ID

[2225] Output: Chat window display

[2226] Step 5:

[2227] When a user types a message, the device sends the message to the server, which relays it to the target user.

[2228] Input: Message from the user

[2229] Output: Message sent to server and relayed to target user

[2230] Specific working example:

[2231] If a user wants to talk to a representative from another manufacturer participating in the exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[2232] (Application example 1)

[2233] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2234] Currently, there are many online shopping platforms on the market that allow users to search for various product information. However, users often experience inconveniences when checking product information, answering questions, and experiencing the products. Furthermore, the user experience can be limited due to limited interactive display of product information and real-time communication with other users and staff. Conventional systems do not provide users with sufficient information and support, making it difficult to provide the comfortable and efficient shopping experience desired by users. Therefore, this invention aims to provide a more comprehensive and interactive online shopping environment by integrating the functions of user authentication, product information search and display, question answering, VR experience provision, and real-time chat.

[2235] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2236] In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting user questions and generating answers to the questions using a generative AI model, means for delivering URLs of corresponding VR content based on VR content requests from users, means for generating and managing real-time chat sessions between users, means for displaying searched product information as 3D models or images, means for providing generated answers by voice, and means for communicating with other users and staff in real time. This allows users to enjoy a comprehensive online shopping experience, from authentication to question and answering, interactive display of product information, VR experiences, and real-time communication.

[2237] "User Credentials" means information used to identify a user and authorize access to a system.

[2238] "Means for verifying" is a function that executes a process to confirm whether user authentication information is correct.

[2239] A "search query" is a request entered by a user to retrieve specific information from a database.

[2240] "Product information" is detailed information about a particular product, typically stored in a database.

[2241] A "generative AI model" is an artificial intelligence model used to automatically generate answers to user questions.

[2242] A "VR content request" is a request submitted by a user to initiate a virtual reality experience.

[2243] A "real-time chat session" is an interactive session for sending and receiving text messages between users in real time.

[2244] "3D Model or Image" means a three-dimensional model or still image used to visually display product information.

[2245] The "means for providing by voice" is a function for providing the generated answer to the user as voice.

[2246] "Means of communicating in real time" refers to a function that allows you to communicate with other users and staff in real time.

[2247] MODE FOR CARRYING OUT THE INVENTION

[2248] The basic program for realizing this system includes the following means: Specific hardware required is a server, a user device (a smartphone or head-mounted display), a generative AI model, and a VR viewer.

[2249] Receiving and verifying user credentials

[2250] A user launches an application and enters authentication information (user ID and password). The terminal sends this authentication information to the server, which verifies it using a database. If authentication is successful, the server creates a user session and returns a main menu to the terminal along with a success message. The terminal displays this main menu to the user.

[2251] Search and view product information

[2252] A user selects the product search option from the main menu and enters a search query. The device sends this search query to the server, which retrieves product information matching the query from its database. This information is returned to the device as a 3D model or image, and the device displays the search results to the user. For example, if a user searches for "polymer materials," the server retrieves and displays related product information.

[2253] Questions and Answers

[2254] When a user enters a question about a particular product, the device sends the question to the server. The server uses a generative AI model to generate an answer to the question and returns it to the device. The device then provides the generated answer via voice. For example, if a user asks, "What are the main components of this compound?" the generative AI model might respond, "The main components are aluminum and silicon."

[2255] Running a VR experience

[2256] When a user selects a product demo VR content, the device sends the selected product ID to the server, which then retrieves the URL of the corresponding VR content from the database. This URL is then returned to the device, which then launches the VR viewer to provide the VR experience. For example, if a user selects a VR demo of a specific chemical process, the device retrieves the URL of the VR content corresponding to that process, and the user can use this URL to experience virtual reality.

[2257] Networking and Communication

[2258] When a user selects a target person to start a networking session, the terminal sends the target person's information to the server, and the server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, and the server relays the message to the target user. For example, if a user wants to talk to another representative at an exhibition, they can send a message to that representative via the chat function and have a real-time conversation.

[2259] Examples of concrete examples and prompts

[2260] As a specific example of user authentication, a user attempts to log in with the ID "user123" and the password "password123." As a specific example of product search, a search is performed for the chemical product "polymer material," and related product information is displayed. As a specific example of question and answer, a user asks "What are the main components of this compound?" via voice, and the generative AI model provides the answer "The main components are aluminum and silicon." As a specific example of a VR experience, a user starts a VR experience to view the details of a specific product in 3D. As a specific example of networking, a user can chat with other personnel in real time.

[2261] An example of a prompt to input to a generative AI model is as follows:

[2262] When asked, "What is the main component of this compound?", be specific about the main component of the chemical.

[2263] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2264] Step 1:

[2265] The user starts the application and enters authentication information (user ID and password). The device sends this authentication information to the server, which then verifies it using a database.

[2266] Input: User ID and password

[2267] Data processing: The server retrieves the authentication information from the database and compares it with the user's input.

[2268] Output: If authentication is successful, create a user session and generate a success message.

[2269] Step 2:

[2270] If the authentication is successful, the server sends the main menu to the terminal, which then displays the main menu to the user.

[2271] Input: Authentication success message

[2272] Data processing: The server saves the user's session information and prepares the main menu data.

[2273] Output: The main menu is displayed on the terminal.

[2274] Step 3:

[2275] The user selects the product search option from the main menu and enters a search query. The device sends this search query to the server, which retrieves product information matching the query from its database.

[2276] Input: search query

[2277] Data processing: The server uses a database search algorithm to retrieve product information that matches the query.

[2278] Output: A list of products matching the query will be displayed on the terminal.

[2279] Step 4:

[2280] The server transmits the acquired product information as a 3D model or image to the terminal, which then displays this information to the user.

[2281] Input: Product information list

[2282] Data processing: Processing to convert product information into 3D models or image formats.

[2283] Output: Product information displayed as a 3D model or image.

[2284] Step 5:

[2285] A user enters a question about a particular product, and the device sends the question to a server, which uses a generative AI model to generate an answer to the question.

[2286] Input: Product Question

[2287] Data processing: Questions are input as prompts into the AI ​​model to generate answers.

[2288] Output: The answer is sent to the terminal and displayed.

[2289] Step 6:

[2290] The terminal presents the user with the option to provide the generated answer aloud, and if selected by the user, outputs it as speech using speech synthesis technology.

[2291] Input: Answer text

[2292] Data processing: Converting text into audio data (speech synthesis).

[2293] Output: A spoken answer is provided.

[2294] Step 7:

[2295] When a user selects a product demo VR content, the terminal sends the selected product ID to the server, and the server retrieves the URL of the corresponding VR content from the database.

[2296] Input: Product ID

[2297] Data processing: The server retrieves the URL of the VR content corresponding to the product ID from the database.

[2298] Output: The URL of the VR content is sent to the device.

[2299] Step 8:

[2300] The device uses the acquired URL to launch the VR viewer and provide the user with a VR experience.

[2301] Input: VR content URL

[2302] Data processing: The VR viewer loads the URL and displays the corresponding VR content.

[2303] Output: The VR experience is delivered to the user.

[2304] Step 9:

[2305] The user selects a target person to start a networking session, and the device sends the target person's information to the server, which creates a new chat session and returns a session ID to the device.

[2306] Input: Subject information

[2307] Data processing: Create a new chat session and issue a session ID.

[2308] Output: The session ID is sent to the terminal.

[2309] Step 10:

[2310] The terminal displays a real-time chat window and allows the user to input a message, which is then sent to the server, which relays the message to the target user.

[2311] Input: User's message

[2312] Data processing: Processes messages to be relayed in real time.

[2313] Output: A message is sent to the target user.

[2314] By performing these steps, users can achieve a comprehensive online shopping experience, from authentication to question and answering, interactive display of product information, VR experience, and real-time communication.

[2315] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2316] The system for implementing the present invention utilizes generative AI and a digital platform that includes user authentication, product information search and display, question answering, VR experiences, networking, and an emotion engine that recognizes user emotions.

[2317] User Authentication and Login

[2318] A user launches a messaging app and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information from the database, and if authentication is successful, it creates a user session and returns the main menu to the device along with a success message. The device then displays this main menu to the user.

[2319] Search and view product information

[2320] The user selects the product search option on the main menu and enters a search query. The device sends this search query to the server. The server retrieves product information that matches the query from a database and returns the search results to the device. The device displays the search results to the user.

[2321] For example, if a user searches for "polymeric materials," the server that receives this search query retrieves related product information from the database and displays it on the user's terminal.

[2322] Questions and Answers

[2323] A user enters a question about a particular product. The device sends the question to a server. The server uses a generative AI model to generate an answer to the question and returns the generated answer to the device. The device displays the generated answer to the user.

[2324] For example, if a user asks, "What are the main components of this compound?", the server uses a generative AI model to respond with the answer, "The main components are aluminum and silicon."

[2325] Running a VR experience

[2326] The user selects a product demo VR content. The device sends the selected product ID to the server, which retrieves the URL of the corresponding VR content from the database. The server returns this URL to the device, which then uses it to launch the VR viewer and provide the VR experience.

[2327] For example, if a user selects a VR demo of a particular chemical process, the server returns a URL for the VR content corresponding to that process, which the user can use to experience the virtual reality.

[2328] Networking and Communication

[2329] To start a networking session, a user selects a target user. The terminal sends the target user's information to the server, which then creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and notifies the user that the session has started. When the user types a message, the terminal sends the message to the server, which relays the message to the target user.

[2330] For example, if a user wants to talk to a representative of another manufacturer participating in an exhibition, the user can send a message to that representative via the chat function and have a conversation in real time.

[2331] Introducing the Emotion Engine

[2332] When a user sends a message or question, the emotion engine recognizes the user's emotional state. The device then sends this emotional information to the server, which then uses the emotion analysis results obtained by the emotion engine to generate an appropriate response using a generative AI model.

[2333] For example, if a user types a question and the emotion engine recognizes that the user is nervous, the server is designed to take that information into account and generate a reassuring answer.

[2334] Personalized VR Experience

[2335] The emotion engine recognizes the user's emotional state and personalizes the VR content experience: if the user is calm, a VR experience with detailed technical explanations is provided, while if the user is nervous, content with more general and simplified explanations is provided.

[2336] For example, if the emotion engine recognizes that the user is nervous, the server will prioritize providing more intuitive and visually appealing content.

[2337] The above is a concrete example of combining this system with an emotion engine. The system is designed to enable users to exchange information regardless of geographical constraints and to efficiently collect information through real-time question and answering, interactive VR experiences, and emotional responses. This facilitates communication between manufacturers and visitors, contributing to product understanding and the optimization of market strategies.

[2338] The processing flow will be explained below.

[2339] User Authentication and Login

[2340] Step 1:

[2341] The user launches the messaging app and enters their user ID and password on the login screen.

[2342] Step 2:

[2343] The terminal sends the authentication information (user ID and password) entered by the user to the server.

[2344] Step 3:

[2345] The server checks the received authentication information against a database and, if authentication is successful, creates a user session.

[2346] Step 4:

[2347] The server returns the main menu data to the terminal along with an authentication success message.

[2348] Step 5:

[2349] The terminal displays the main menu to the user.

[2350] Search and view product information

[2351] Step 1:

[2352] A user selects the product search option on the main menu and enters a search query.

[2353] Step 2:

[2354] The terminal transmits the search query entered by the user to the server.

[2355] Step 3:

[2356] The server retrieves product information that matches the search query from a database.

[2357] Step 4:

[2358] The server returns the acquired search results to the terminal.

[2359] Step 5:

[2360] The terminal displays the search results to the user.

[2361] Questions and Answers

[2362] Step 1:

[2363] A user enters a question about a particular product.

[2364] Step 2:

[2365] The terminal sends this question to the server.

[2366] Step 3:

[2367] The server generates answers to questions using generative AI models.

[2368] Step 4:

[2369] The server returns the generated answer to the terminal.

[2370] Step 5:

[2371] The terminal displays the generated answer to the user.

[2372] Running a VR experience

[2373] Step 1:

[2374] The user selects the VR content for the product demo.

[2375] Step 2:

[2376] The terminal transmits the selected product ID to the server.

[2377] Step 3:

[2378] The server retrieves the URL of the corresponding VR content from the database.

[2379] Step 4:

[2380] The server returns the acquired URL to the terminal.

[2381] Step 5:

[2382] The device will use this URL to launch the VR viewer and provide the VR experience.

[2383] Networking and Communication

[2384] Step 1:

[2385] The user selects a target person to begin a networking session.

[2386] Step 2:

[2387] The terminal transmits the target person's information to the server and requests the initiation of a chat session.

[2388] Step 3:

[2389] The server creates a new chat session and generates a session ID.

[2390] Step 4:

[2391] The server returns the generated session ID to the terminal.

[2392] Step 5:

[2393] The terminal displays a real-time chat window and notifies the user that a session has begun.

[2394] Step 6:

[2395] The user types a message in the chat window.

[2396] Step 7:

[2397] The terminal transmits the input message to the server.

[2398] Step 8:

[2399] The server relays the received message to the target user.

[2400] Introducing the Emotion Engine

[2401] Step 1:

[2402] The user enters a message or question.

[2403] Step 2:

[2404] The device sends the input message or question to the emotion engine, which analyzes the emotional state.

[2405] Step 3:

[2406] The emotion engine recognizes the user's emotional state (e.g., happy, sad, nervous, etc.).

[2407] Step 4:

[2408] The emotion engine sends the analysis results to the server.

[2409] Step 5:

[2410] Based on the results of the sentiment analysis, the server uses a generative AI model to generate an appropriate response.

[2411] Step 6:

[2412] The server returns the generated response to the terminal.

[2413] Step 7:

[2414] The terminal displays the generated response to the user.

[2415] Personalized VR Experience

[2416] Step 1:

[2417] The user selects the VR content for the product demo.

[2418] Step 2:

[2419] The terminal analyzes the user's emotional state using an emotion engine.

[2420] Step 3:

[2421] The emotion engine recognizes whether the user is calm or nervous.

[2422] Step 4:

[2423] The emotion engine sends the analysis results to the server.

[2424] Step 5:

[2425] The server obtains the URL of the personalized VR content based on the emotion analysis results.

[2426] Step 6:

[2427] The server returns the URL to the terminal.

[2428] Step 7:

[2429] The device uses the URL to launch a VR viewer to provide a personalized VR experience.

[2430] For example, if the emotion engine recognizes that the user is tense, the server can provide visually appealing relaxation content to enhance the user's experience.

[2431] Example 2

[2432] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2433] Conventional systems perform user authentication, product information search, question answering, virtual reality experience, real-time communication, and emotion recognition and analysis separately, making it difficult to provide a seamless user experience. Furthermore, because each function is independent, it is not possible to provide a consistent service. In particular, there is a lack of personalized responses and content based on the user's emotional state, and there is a need to improve user satisfaction.

[2434] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2435] In this invention, the server includes: means for receiving user authentication information and verifying the authentication information; means for retrieving product information from a database based on a search query entered by the user; means for accepting user questions and generating answers to the questions using a generative AI model; means for delivering URLs of corresponding virtual reality content based on virtual reality content requests from users; means for generating and managing real-time communication sessions between users; and means for analyzing the user's emotional state and personalizing responses and content based on the analysis results. This integration of each function enables a seamless and consistent user experience. Furthermore, providing personalized services based on the user's emotional state is expected to improve user satisfaction.

[2436] "User authentication information" is data used to identify a user and verify access permissions.

[2437] "Verification" is the process of checking whether the information received is correct.

[2438] A "search query" is a condition or keyword entered to retrieve specific information from a database.

[2439] A "database" is a system for efficiently managing large amounts of data and for quickly searching and manipulating it.

[2440] A "generative AI model" is an algorithm or machine learning model that uses artificial intelligence to generate new data and answers.

[2441] "Virtual Reality Content" means information or simulations that are visually and experientially presented to users using virtual reality technology.

[2442] A "URL" is an address used to specify a resource on the Internet.

[2443] A "real-time communication session" is a communication session in which information is exchanged between users in real time.

[2444] "Emotional state" refers to the user's current psychological and physiological state.

[2445] "Analysis results" refer to the conclusions or findings obtained after conducting data analysis.

[2446] "Personalization" means customizing services and content according to the needs and circumstances of individual users.

[2447] A system for implementing the present invention provides user authentication, product information search and display, question answering, virtual reality experiences, real-time communication, and emotion recognition and personalization. The system utilizes generative AI models and a digital platform to provide a seamless and consistent user experience.

[2448] User Authentication and Login

[2449] The user launches a messaging application and enters their user ID and password. The device sends this authentication information to the server. The server verifies the authentication information using a database, and if authentication is successful, creates a user session and returns a success message and the main menu to the device. The device then displays this main menu to the user.

[2450] Example: When a user enters "user123" and "password123" and the terminal sends this information to the server, the server verifies the information in the database and, after successful authentication, displays the message "Login successful" and the main menu.

[2451] Search and view product information

[2452] The user selects the product search option from the main menu and enters a search query. The device sends this query to the server. The server retrieves matching product information from the database and returns the search results to the device. The device displays the search results to the user.

[2453] Example: When a user types "polymeric materials" and the terminal sends this query to the server, the server retrieves relevant product information from the database and the terminal displays it to the user.

[2454] Questions and Answers

[2455] When a user enters a question about a particular product, the device sends the question to the server, which uses a generative AI model to generate an answer to the question and returns the answer to the device, which then displays the answer to the user.

[2456] Example: A user asks, "What are the main components of this compound?" and the device sends the question to a server. The server uses a generative AI model to generate the answer, "The main components are aluminum and silicon," and the device displays this answer to the user.

[2457] Running a virtual reality experience

[2458] When a user selects a product demo virtual reality content, the device sends the selected product ID to the server, and the server retrieves the URL of the corresponding virtual reality content from the database and returns it to the device. The device then uses this URL to launch the VR viewer and provide the user with a virtual reality experience.

[2459] Example: When a user selects "VR demo of chemical process" and the device sends the product ID to the server, the server returns the URL of the corresponding virtual reality content, the device uses the URL to launch the VR viewer, and the user can experience the virtual reality.

[2460] Realizing real-time communication

[2461] When a user selects a target user for networking and the terminal sends this information to the server, the server creates a new chat session and returns a session ID to the terminal. The terminal displays a real-time chat window and sends messages entered by the user to the server, which then relays the messages to the target user.

[2462] Example: When a user selects to chat with "Person B at Manufacturer A" and the device sends information to the server, the server creates a chat session and returns a session ID. When the device displays a chat window and the user types "Hello" and sends it, the server relays this message to Person B.

[2463] Emotion Recognition and Personalization

[2464] When a user inputs a message or question, the emotion engine analyzes the user's emotional state, and the device transmits the emotional state to the server. The server uses a generative AI model to generate an appropriate response based on the analysis results and returns it to the device. The device then displays this response to the user, personalizing the VR content based on the user's emotional state.

[2465] Example: If a user asks, "Tell me about the properties of this compound," and the emotion engine recognizes "tension," the device sends this information to the server, which then generates a response such as, "Relax and ask your question," and returns it to the device. Based on the emotional state, the server provides the device with a URL for more intuitive and visually appealing VR content.

[2466] Prompt Sentence Examples

[2467] "Please explain in detail the process by which users search for polymer materials."

[2468] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2469] Step 1:

[2470] The user launches a messaging application and enters their user ID and password, which the device then sends to the server.

[2471] Specific operation: The user enters "user123" and "password123", and the device sends this to the server.

[2472] Input: User ID and password

[2473] Output: Sending authentication information

[2474] Step 2:

[2475] The server receives the entered authentication information and validates it against a database.

[2476] Specific operation: The server retrieves the password corresponding to "user123" from the database and compares it with the entered password.

[2477] Input: Credentials

[2478] Output: Authentication result

[2479] Step 3:

[2480] If authentication is successful, the server creates a user session and returns a success message and the main menu to the terminal, otherwise it returns an error message.

[2481] Specific operation: When authentication is successful, a session ID is generated and "Login successful" and the main menu are sent to the terminal.

[2482] Input: Authentication result

[2483] Output: Success message and main menu, or failure message

[2484] Step 4:

[2485] The terminal displays the main menu to the user, who selects the product search option and enters a search query.

[2486] Specific action: The user enters "polymer material."

[2487] Input: Main menu, search query

[2488] Output: Submitting a search query

[2489] Step 5:

[2490] The terminal sends the entered search query to the server, which uses a database to retrieve product information that matches the query.

[2491] Specific operation: The server searches and retrieves product information matching "polymer material" from the database.

[2492] Input: search query

[2493] Output: Product information

[2494] Step 6:

[2495] The server returns the acquired product information to the terminal, which then displays this information to the user.

[2496] Specific behavior: The device presents the user with details about "polymer materials."

[2497] Input: Product Information

[2498] Output: Display product information

[2499] Step 7:

[2500] The user enters a question about a particular product and the terminal transmits the question to the server.

[2501] What happens: A user asks, "What are the main components of this compound?"

[2502] Input: User question

[2503] Output: Question sent

[2504] Step 8:

[2505] The server receives the question and uses a generative AI model to generate an answer to the question.

[2506] What it does: The server uses a generative AI model to generate the answer, "The main components are aluminum and silicon."

[2507] Input: Question

[2508] Output: Answer

[2509] Step 9:

[2510] The server returns the generated answer to the terminal, which displays the answer to the user.

[2511] What it does: The device displays "The main components are aluminum and silicon."

[2512] Input: Answer

[2513] Output: Display the answer

[2514] Step 10:

[2515] When a user selects a particular product demo virtual reality content, the terminal transmits the selected product ID to the server.

[2516] Specific action: The user selects "Chemical Process VR Demo."

[2517] Input: Product ID

[2518] Output: Send product ID

[2519] Step 11:

[2520] The server retrieves the URL of the corresponding virtual reality content from the database and returns it to the terminal.

[2521] Specific operation: The server retrieves the URL of the corresponding VR content from the database.

[2522] Input: Product ID

[2523] Output: URL of virtual reality content

[2524] Step 12:

[2525] The device uses the obtained URL to launch the VR viewer and provide the user with a virtual reality experience.

[2526] Specific operation: The device launches the VR viewer and the user experiences virtual reality.

[2527] Input: URL of virtual reality content

[2528] Output: Virtual reality experience

[2529] Step 13:

[2530] The user selects the target users for networking, and the terminal transmits this information to the server.

[2531] Specific operation: The user selects that he wants to chat with "Person B in charge of Manufacturer A."

[2532] Input: Networking target information

[2533] Output: Sending information

[2534] Step 14:

[2535] The server creates a new chat session and returns a session ID to the device.

[2536] Specific operation: The server creates a chat session and returns a session ID to the device.

[2537] Input: Networking target information

[2538] Output: Session ID

[2539] Step 15:

[2540] The terminal displays a real-time chat window and transmits messages entered by the user to the server.

[2541] What happens: The device displays a chat window and the user types "Hello."

[2542] Input: Session ID, user message

[2543] Output: Sending a message

[2544] Step 16:

[2545] The server relays the received message to the target user.

[2546] Specific action: The server relays the message "Hello" to Agent B.

[2547] Input: message

[2548] Output: Message relay

[2549] Step 17:

[2550] The emotion engine analyzes messages and questions entered by the user, and the device sends the user's emotional state to the server.

[2551] Specific operation: The emotion engine recognizes "tension" and sends that information to the server.

[2552] Input: Message or question

[2553] output: emotional state

[2554] Step 18:

[2555] The server uses the generative AI model to generate an appropriate response based on the analysis results and returns it to the device.

[2556] Specific operation: The server generates a response such as "Just relax and ask your questions" and returns it to the device.

[2557] Input: Emotional state

[2558] Output: Response

[2559] Step 19:

[2560] The device displays the obtained responses to the user and provides personalized content based on the emotional state.

[2561] What it does: The device displays personalized content and provides it to the user.

[2562] Input: Response, Emotional State

[2563] Output: personalized content

[2564] The above are the detailed processing steps of this system, which will enable users to enjoy real-time services based on authentication, search, question answering, VR experiences, networking, and emotion recognition.

[2565] (Application example 2)

[2566] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2567] Conventional online shopping systems have difficulty in taking user emotions into consideration, limiting the improvement of user experience. In addition, the limited methods of providing product information make it difficult for users to fully understand the products. This can lead to a decrease in user satisfaction and a negative impact on sales.

[2568] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user authentication information and verifying the authentication information, means for retrieving product information from a database based on a search query entered by the user, means for accepting a user's question and generating an answer to the question using a generative model, means for delivering a URL of corresponding VR content based on a VR content request from the user, means for generating and managing real-time chat sessions between users, and means for recognizing the user's emotional state and having the generative model personalize responses based on that information. This enables personalized responses according to the user's emotions, deepening product understanding and significantly improving the user experience.

[2569] "User Credentials" means the credentials used by a User to access a System, generally including a User ID and password.

[2570] A "database" refers to a collection of data that stores product information, user information, etc. and can be searched and retrieved.

[2571] A "search query" is a keyword or phrase that a user enters to retrieve specific information.

[2572] A "generative model" refers to a system that uses AI algorithms to generate answers or responses to user questions.

[2573] "VR Content" means digital content that can be experienced by a user using virtual reality technology.

[2574] "URL" is an abbreviation for Universal Resource Locator, and is an address that points to a specific resource on the Internet.

[2575] "Real-time chat session" refers to a chat feature that allows users to exchange messages with each other in real time.

[2576] An "emotional state" refers to the emotional state that the user is feeling at that time, and includes, for example, joy, sadness, tension, and the like.

[2577] "Personalization" means tailoring services and content to a user's specific needs and circumstances.

[2578] This invention provides a system that combines user authentication, product information search and display, question answering, VR experience, networking, and an emotion engine. A specific embodiment of this system is described below.

[2579] System configuration

[2580] This system mainly consists of three elements: a server, a device, and a user. The hardware used is a smartphone and a head-mounted display, and the software uses Python, a generative AI model, and a facial recognition API.

[2581] User Authentication and Login

[2582] A user launches a messaging application and enters authentication information (user ID and password). The device sends this authentication information to the server. The server verifies the authentication information against a database, and if authentication is successful, returns a session token to the device. The device uses this session token to display the main menu.

[2583] Search and view product information

[2584] When a user selects the product search option on the main menu and enters a search query, the device sends the search query to the server, which retrieves product information that matches the query from the database and returns the search results to the device, which then displays the search results to the user.

[2585] Questions and Answers

[2586] When a user enters a question about a particular product, the device sends the question to the server, which uses a generative AI model to generate an answer to the question and returns the generated answer to the device, which then displays the generated answer to the user.

[2587] Running a VR experience

[2588] When a user selects a product demo VR content, the device sends the selected product ID to the server. The server retrieves the URL of the corresponding VR content from the database and returns it to the device. The device then uses this URL to launch the VR viewer and provide the VR experi...

Claims

1. means for receiving user authentication information and verifying the authentication information; means for retrieving product information from a database based on a search query entered by a user; means for accepting a user question and generating an answer to the question using a generative model; A means for delivering a URL of corresponding VR content based on a VR content request from a user; means for creating and managing real-time chat sessions between users; A system including:

2. 10. The system of claim 1, designed to be easily accessible to users via a messaging application.

3. 10. The system of claim 1, further comprising means for providing an interactive VR experience when displaying product information retrieved from the database.

Images