system

Abstract

Provide a system. A reception terminal includes a means for verifying the identity of a participant; A guide robot will explain the facilities along the tour route, A means for the guide robot to respond to questions from participants, A means of collecting impressions and opinions from participants after the tour has ended, A means to analyze the collected impressions and opinions and store them as a knowledge base, A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, corporate office tours and tours have become an important part of corporate PR and new graduate recruitment activities. However, traditional tours require a person to physically guide the tour, which is labor-intensive and time-consuming. Furthermore, it is difficult to respond immediately and appropriately to the various questions that participants may have during the tour, which can lead to inconsistencies in the quality of the tour. Another issue is the lack of a well-established process for effectively collecting participants' opinions and feedback after the tour and reflecting them as improvements for future tours. [Means for solving the problem]

[0005] This invention relates to a system that solves the problems of conventional office tours by having a reception terminal verify the identity of participants and then having a guide robot explain the facilities and answer questions from participants along the tour route. Specifically, the system is equipped with a means for the reception terminal to verify the identity of participants using a QR code (registered trademark) or ID input, and a means for the guide robot to automatically move along the tour route while explaining the facilities. Furthermore, the guide robot is equipped with a means for immediately answering questions from participants and collecting impressions and opinions from participants after the tour ends. The collected opinions and impressions are then analyzed and stored as a knowledge base, making it possible to improve the quality of future tours.

[0006] A "reception terminal" is a device or system used to verify the identity of participants and provide an overview of the tour when they join an office tour.

[0007] "Identity verification" is the process of verifying that participants are legitimate registrants, and is done using methods such as QR codes or ID entry.

[0008] A "tour route" is a set route for sequentially touring each facility and department within an office.

[0009] The "guide robot" is a robot that moves automatically along the tour route, explaining each facility and answering questions from participants.

[0010] "Facility explanation" refers to the act of the guide robot introducing the functions, roles, and features of each facility to participants along the tour route.

[0011] "Question response" is the process in which the guide robot immediately provides appropriate answers to questions or concerns from participants.

[0012] "After the tour ends" refers to the time after the entire sightseeing route has been completed and the guide robot has reached the final destination.

[0013] "Gathering feedback and opinions" is the process of collecting feedback and opinions about the tour from participants after the tour has ended.

[0014] "Collected impressions and opinions" refers to the feedback, evaluations, and opinions submitted by participants after the tour has ended.

[0015] "Analysis" is a technical process for organizing collected impressions and opinions and extracting problems and areas for improvement.

[0016] A "knowledge base" is a database of knowledge built based on analyzed information, which is referenced in subsequent tours and service provision. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention is a system including a reception terminal, a guide robot, and a server, which automates the implementation of office tours for participants and provides high-quality guide services.

[0039] 1. Reception process

[0040] Waiting for reception and identity verification

[0041] The terminal waits for the participant to approach the reception terminal.

[0042] The user presses the QR code reader button displayed on the device screen.

[0043] The device scans the user's QR code to verify their identity.

[0044] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[0045] The server is contacted to check against registered participant data.

[0046] Based on the matching results, identity verification is completed.

[0047] Tour overview

[0048] The terminal provides the user with an overview of the tour.

[0049] The terminal displays the sightseeing locations, important points to note, the tour schedule, etc.

[0050] The terminal checks whether the user understands.

[0051] 2. Office tours

[0052] Meeting and tour start

[0053] The user heads to the designated meeting point.

[0054] The robot will wait at the meeting point to welcome participants.

[0055] The robot begins the tour.

[0056] The robot begins to move along the tour route.

[0057] Facility Description

[0058] The robot will provide an explanation as it arrives at each facility along the tour route.

[0059] A description of the facility is provided based on information obtained from the server.

[0060] The robot provides instructions via voice or text.

[0061] Answering questions

[0062] The user inputs a question into the robot by voice or via a terminal.

[0063] The robot analyzes the question and provides an appropriate answer.

[0064] The question is sent to the server and the response data is obtained.

[0065] The robot provides the obtained answer to the user.

[0066] 3. Closing

[0067] Tour End

[0068] The robot ends the tour at the end of the route.

[0069] Ask for opinions

[0070] The robot asks the user for their impressions and opinions of the tour.

[0071] The robot displays an interface on the terminal for inputting impressions.

[0072] The terminal waits for input from the user.

[0073] Information gathering

[0074] The user inputs their thoughts and opinions on the device.

[0075] The terminal transmits the user's opinion to the server.

[0076] Knowledge

[0077] The server analyzes the collected opinions and impressions.

[0078] The server organizes the data and identifies trends.

[0079] The server accumulates useful information as a knowledge base and uses it to improve future tours.

[0080] Specific examples

[0081] Below is an example of a participant taking an office tour.

[0082] 1. Reception process

[0083] The user arrives at the reception terminal and scans the QR code.

[0084] The device analyzes the data in the QR code and verifies the user's identity.

[0085] After verifying your identity, the device will explain the tour outline.

[0086] 2. Office tour

[0087] When the user goes to the designated meeting place, the robot will be waiting.

[0088] The robot moves along the tour route and explains each facility.

[0089] Every time the user asks a question, the robot provides an appropriate answer.

[0090] 3. Closing Process

[0091] At the tour end point, the robot ends the tour.

[0092] The robot asks the user for their thoughts and opinions and has them enter them into a terminal.

[0093] The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[0094] In this way, this system automates all processes from reception to office tour guidance, answering questions, and closing, making it possible to provide a highly efficient, high-quality tour experience.

[0095] The processing flow will be explained below.

[0096] 1. Reception process

[0097] Step 1:

[0098] The terminal waits for the participant to access the reception terminal.

[0099] Step 2:

[0100] The user presses the QR code reader button displayed on the device screen.

[0101] Step 3:

[0102] The device scans the user's QR code and analyzes the data in the QR code.

[0103] Step 4:

[0104] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[0105] Step 5:

[0106] The server checks the participant data and returns the matching results to the terminal.

[0107] Step 6:

[0108] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[0109] 2. Office tours

[0110] Step 1:

[0111] The user heads to the designated meeting point.

[0112] Step 2:

[0113] The robot waits at the meeting point and checks the participants.

[0114] Step 3:

[0115] The robot begins moving along the tour route.

[0116] Step 4:

[0117] Each time the robot arrives at a facility, it provides an explanation of the facility based on information obtained from the server.

[0118] Step 5:

[0119] The user inputs a question into the robot by voice or via a terminal.

[0120] Step 6:

[0121] The robot analyzes the question and sends the question to the server.

[0122] Step 7:

[0123] The server obtains the appropriate answer data for the question and sends it back to the robot.

[0124] Step 8:

[0125] The robot provides the obtained answer to the user.

[0126] 3. Closing

[0127] Step 1:

[0128] The robot ends the tour at the end of the route.

[0129] Step 2:

[0130] The robot asks the user for their impressions and opinions of the tour.

[0131] Step 3:

[0132] The robot displays an interface on the terminal for inputting impressions.

[0133] Step 4:

[0134] The user inputs their thoughts and opinions on the device.

[0135] Step 5:

[0136] The terminal transmits the user's opinion to the server.

[0137] Step 6:

[0138] The server analyzes the collected opinions and impressions and organizes the data.

[0139] Step 7:

[0140] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[0141] Example 1

[0142] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0143] Current office tours require dedicated guides, which poses challenges in terms of efficiency and cost. It is also difficult to respond to participants' diverse questions and opinions on the spot and to gather appropriate feedback. This results in tour quality that is inconsistent or not optimized for each individual participant.

[0144] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0145] In this invention, the server includes means for verifying the identity of participants, means for a guide device moving along the tour route to explain the facility, means for the guide device to respond to questions from participants in cooperation with the server, means for collecting impressions and opinions from participants after the tour ends, and means for saving the collected impressions and opinions in a database, analyzing them with a text mining algorithm, and accumulating them as a knowledge base. This makes it possible to automate highly efficient, high-quality office tours without the need for a dedicated guide, and to appropriately respond to the diverse requests of participants.

[0146] A "reception terminal" is a device used to verify the identity of participants, and is a device that authenticates participants via QR code or ID input.

[0147] A "guide device" is a robot or device that moves automatically along the tour route and has the function of explaining the facility to participants.

[0148] The "server" is a computer system that works in conjunction with the information device and the reception terminal to manage data, collate data, generate responses, and collect and analyze opinions.

[0149] "Identity verification" is the process by which participants are matched with registered information to verify their identity.

[0150] The "tour route" is a specific route along which the guide device travels during an office tour, and is a route along which each facility is visited in sequence.

[0151] The "facility explanation" is an explanation that the guide device provides as it arrives at each facility along the tour route, and is given in the form of audio or text.

[0152] "Response to questions" refers to the guide device working in conjunction with the server to provide appropriate answers to questions posed by participants.

[0153] "Collecting impressions and opinions" refers to the process of receiving feedback from participants after the tour and saving it as data.

[0154] The "knowledge base" is a collection of data that has been collected, analyzed, and stored as useful information to be used to improve future tours.

[0155] The present invention relates to an office tour automation system that includes a reception terminal, a guide device, and a server. This system automates participant authentication, facility explanations, question handling, and feedback collection, providing a highly efficient and high-quality tour experience.

[0156] 1. Reception terminal

[0157] The terminal is a device equipped with a QR code reading function that reads the QR code when the user approaches. The hardware used is a tablet terminal with a built-in QR code scanner or a dedicated reception device. For example, devices running Android (registered trademark) or Windows can be used. A library such as Zxing is used to analyze the QR code.

[0158] When a user approaches the reception terminal and presses the QR code reading button on the screen, the terminal scans the QR code and obtains participant information. This information is sent to the server and verified against registered information for identity verification. For example, participants are authenticated by following the instruction to "scan the QR code."

[0159] 2. Guidance device

[0160] The guide device is a robot with autonomous mobility capabilities that explains the facilities while moving along the tour route. The guide device achieves autonomous movement using ROS (Robot Operating System). It also has speech recognition and synthesis functions, and provides facility explanations and answers questions. A speech synthesis engine such as Google (registered trademark) Text-to-Speech is used to explain the facilities.

[0161] For example, when a user approaches a meeting point, a guide device will be waiting and begin the tour. The guide device will explain each facility using prompts such as, "This is our newest office space. Do you have any questions?"

[0162] 3. Server

[0163] The server is a computer system that manages and processes data. The server works in conjunction with the reception terminal and the information device to perform the following tasks:

[0164] Verification of authentication data

[0165] Provision of facility description data

[0166] Generating answers to user questions

[0167] Collecting and analyzing feedback

[0168] As a specific example of how to respond to a question, if a user asks, "How many people can this conference room accommodate?", the guidance device will recognize the question through speech recognition and send it to the server. The server will then obtain the answer data and send it to the guidance device, thereby providing the user with an appropriate answer.

[0169] 4. Collecting feedback and opinions and creating a knowledge base

[0170] After the tour ends, the device displays an interface requesting the user's impressions and opinions. When the user enters their feedback into the device, the data is sent to the server. The server stores the collected opinions in a database and analyzes them using a text mining algorithm. This accumulates useful information as a knowledge base that can be used to improve future tours.

[0171] This system allows participants to enjoy a highly efficient and personalized tour experience, and allows operators to improve the quality of their services based on feedback.

[0172] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0173] Step 1:

[0174] The terminal waits at the reception location for the user to approach. When the user presses the "QR code reading button" displayed on the reception terminal's screen, the terminal switches to QR code scanning mode. Specifically, pressing the button displays a camera preview on the screen, allowing the QR code to be recognized. The user's QR code is acquired as input, and the output after analysis is participant information (name, ID, etc.). This data is then sent to the server.

[0175] Step 2:

[0176] The server compares the received participant information with the existing registration information in the database. The input is the participant information sent from the terminal, and the output is the authentication result (authentication successful or authentication failed). In concrete terms, the server performs a database search and returns the comparison result to the terminal. If authentication is successful, the terminal displays an "Authentication successful" message, and if it fails, it displays an "Authentication failed" message.

[0177] Step 3:

[0178] After successful authentication, the terminal displays an overview of the tour to the user. The input is the authentication result from the server, and the output is a display of the tour overview information. Specifically, the terminal screen displays the tour locations, important points to note, the tour flow, etc. in a presentation format. The user presses the "Next" button to confirm the contents.

[0179] Step 4:

[0180] The user heads to the designated meeting place. When the user arrives at the meeting place, a guide device is waiting. The input is guide information for the meeting place, and the output is the user's movement. In concrete terms, the guide device displays information about the meeting place on the terminal screen and guides the user.

[0181] Step 5:

[0182] The guide device welcomes participants who have gathered at the meeting point and starts moving along the tour route. The input is the gathering state of users at the meeting point, and the output is a signal to start moving. In concrete terms, the robot moves autonomously along a pre-programmed route, stopping at each point.

[0183] Step 6:

[0184] The guide device provides explanations when arriving at each facility. The input is the location information of each point on the tour route, and the output is a description of the facility. In concrete terms, the robot uses facility information obtained from the server to provide explanations in voice and text. For example, it uses prompts such as, "This is our latest office space. Do you have any questions?"

[0185] Step 7:

[0186] When a user asks a question, the guidance device recognizes the question through voice recognition and sends it to the server. The input is the user's question, and the output is answer data. Specifically, the robot analyzes the question using its voice recognition function and sends the question data to the server. The server analyzes the question, generates an answer, and returns it to the robot. The robot then provides the answer to the user.

[0187] Step 8:

[0188] The guide device ends the tour when it reaches the end point of the tour route. The input is the end point information of the tour route, and the output is a notification that the tour has ended. Specifically, it plays a voice message saying, "This is the end of the tour. Thank you for participating."

[0189] Step 9:

[0190] The guide device instructs the user to provide their impressions and opinions, and displays an input interface on the terminal. The input is the state after the tour ends, and the output is a screen display for entering impressions. Specifically, the terminal displays an impression form and prompts the user to enter their impressions.

[0191] Step 10:

[0192] When a user inputs their thoughts or opinions into the device, the device sends the data to the server. The input is the user's feedback, and the output is the data sent to the server. The specific operation is to store the thoughts in a database.

[0193] Step 11:

[0194] The server stores the collected impressions and opinions in a database and analyzes them using a text mining algorithm. The input is user impressions and opinions, and the output is the analysis results and a knowledge base. Specifically, the server classifies and analyzes the data and accumulates useful information as a knowledge base.

[0195] (Application example 1)

[0196] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0197] Conventional customer guidance services in brick-and-mortar stores often rely on human resources, which means that the quality of guidance and consistency of service cannot be guaranteed. Another issue is that when customers ask a variety of questions about the store's products, they are unable to receive appropriate answers immediately. Furthermore, there is a lack of a system for effectively collecting and analyzing customer feedback and using it to improve services. For these reasons, there is a need for efficient, high-quality guidance services in brick-and-mortar stores.

[0198] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0199] In this invention, the server includes a means for verifying the identity of participants at the reception terminal, a means for a guide robot to explain the facilities along the tour route, a means for the guide robot to respond to participants' questions, a means for collecting impressions and opinions from participants after the tour ends, a means for analyzing the collected impressions and opinions and storing them as a knowledge base, an automated means for guiding visitors and a means for controlling the guide robot and introducing products in each section of the physical store, and a means for answering questions using a generative AI model. This enables the physical store to provide automated, high-quality guidance services while also appropriately responding to a variety of questions from visitors. Furthermore, the efficiency of feedback collection and analysis can be improved, which can be used to improve services.

[0200] A "reception terminal" is a device that customers use to check in and verify their identity by scanning a QR code or entering their ID.

[0201] A "participant" is someone who uses the system to receive an office tour or a guided tour of a physical store.

[0202] "Identity verification" is the process of verifying that a system user is a person who has been pre-registered by scanning a QR code or entering an ID.

[0203] A "tour route" indicates a guided course within a specific facility, and is the path along which the robot will guide the visitor.

[0204] A "guide robot" is an autonomous mobile robot that automatically moves along a tour route, such as a physical store or office tour, and provides explanations about the facility.

[0205] "After the tour ends" refers to the time after the entire tour by the guide robot has been completed.

[0206] "Impressions and opinions" refers to user feedback regarding tours and guide services, which is collected by being entered into the terminal.

[0207] A "knowledge base" is a database of knowledge that is analyzed based on collected impressions and opinions and accumulated for future service improvements.

[0208] "Automatic means" refers to a method or mechanism by which a system performs a function automatically without human intervention.

[0209] A "guide robot" is a robot that moves around a physical store and provides guidance to customers by introducing products and answering their questions.

[0210] "Generative AI model" refers to an artificial intelligence model used to generate appropriate answers to user questions.

[0211] A "prompt" is a text input that provides context for the answer an artificial intelligence question-answering model generates.

[0212] This invention automates the guidance service for customers in physical stores by using a system including a reception terminal, guide robot, and server, and provides high-quality explanations and answers to questions. This system comprehensively implements processes from customer check-in to guiding them around the store, explaining products, answering questions, and collecting feedback.

[0213] 1. Reception process

[0214] Waiting for reception and identity verification

[0215] The terminal waits for a customer to approach the reception terminal.

[0216] The user presses the QR code reader button displayed on the device screen.

[0217] The device scans the user's QR code to verify their identity.

[0218] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[0219] The server is contacted to check against registered participant data.

[0220] Based on the matching results, identity verification is completed.

[0221] Guide overview

[0222] The terminal provides the user with an overview of the tour.

[0223] The terminal displays the sightseeing locations, important points to note, the tour schedule, etc.

[0224] The terminal checks whether the user understands.

[0225] 2. Tours using guide robots

[0226] Meeting and tour start

[0227] The user heads to the designated meeting point.

[0228] The guide robot waits at the meeting point to welcome customers.

[0229] The guide robot begins the tour.

[0230] The guide robot begins to move along the tour route.

[0231] Facility Description

[0232] The guide robot provides explanations as it arrives at each section along the tour route.

[0233] Provides descriptions of facilities and products based on information obtained from the server.

[0234] The guide robot provides explanations by voice or text.

[0235] 3. Answering questions

[0236] Accepting questions and providing answers

[0237] The user inputs questions to the guide robot by voice or via a terminal.

[0238] The guide robot analyzes the question and provides an appropriate answer.

[0239] The question is sent to the server and the answer data is obtained using a generative AI model.

[0240] The guide robot provides the obtained answers to the user.

[0241] 4. Closing

[0242] Tour End

[0243] The guide robot ends the tour at the end of the route.

[0244] Ask for opinions

[0245] The guide robot asks the user for their impressions and opinions on the tour.

[0246] The guide robot displays an interface for inputting impressions on the terminal.

[0247] The terminal waits for input from the user.

[0248] Information collection and analysis

[0249] The user inputs their thoughts and opinions on the device.

[0250] The terminal transmits the user's opinion to the server.

[0251] The server analyzes the collected opinions and impressions and stores them as a knowledge base.

[0252] The server organizes the data and identifies trends.

[0253] The server uses the useful information as a knowledge base to improve the next tour.

[0254] Specific examples

[0255] The following is an example of a customer receiving guidance services at a physical store.

[0256] 1. Reception process

[0257] The user arrives at the reception terminal and scans the QR code.

[0258] The device analyzes the data in the QR code and verifies the user's identity.

[0259] After verifying your identity, the device will explain the tour outline.

[0260] 2. Tours using guide robots

[0261] When the user heads to the designated meeting place, a guide robot will be waiting.

[0262] The guide robot moves along the tour route and explains each section.

[0263] Every time the user asks a question, the guide robot provides an appropriate answer.

[0264] 3. Closing Process

[0265] At the tour end point, the guide robot ends the tour.

[0266] The guide robot asks the user for their impressions and opinions and has them enter them into a terminal.

[0267] The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[0268] For example, in response to a frequently asked question such as "What are the new features of this smartphone?", the generative AI model generates an answer using the following prompt sentence:

[0269] Context: This smartphone has a brand new chip, making it even faster than previous models, and it also has a number of new features, including a higher-resolution camera and longer battery life.

[0270] Question: What are the new features of this smartphone?

[0271] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0272] Step 1:

[0273] Waiting for reception and identity verification

[0274] The terminal waits for a customer to approach the reception terminal.

[0275] The user presses the QR code reading button displayed on the device screen to scan the QR code.

[0276] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[0277] The terminal sends the information to the server and checks it against registered participant data.

[0278] The server compares the information with that in the participant database and returns the identity verification results to the terminal.

[0279] The device completes identity verification based on the matching results.

[0280] Input: User's QR code

[0281] Output: Identity verification result

[0282] Step 2:

[0283] Guide overview

[0284] The terminal provides the user with an overview of the tour (places to visit, things to note, tour flow, etc.).

[0285] The terminal confirms through the screen interface whether the user has understood.

[0286] Input: QR code scan information, identity verification results from the server

[0287] Output: Tour overview

[0288] Step 3:

[0289] Meeting and tour start

[0290] The user heads to the designated meeting point.

[0291] The guide robot waits at the meeting point to welcome customers.

[0292] The guide robot begins moving along the tour route.

[0293] Input: Specify the meeting place and time

[0294] Output: Notification of tour start

[0295] Step 4:

[0296] Facility Description

[0297] The guide robot will explain the facilities and products as it arrives at each section along the tour route.

[0298] The guide robot retrieves explanatory information from the server in real time.

[0299] The guide robot provides explanations in voice or text.

[0300] Input: Tour route and section information

[0301] Output: Facility description audio or text

[0302] Step 5:

[0303] Accepting questions and providing answers

[0304] The user inputs questions to the guide robot by voice or via a terminal.

[0305] The guide robot analyzes the question and sends the content to the server.

[0306] The server uses a generative AI model to construct prompts to generate answers to questions.

[0307] The server returns the generated answer to the guide robot.

[0308] The guide robot provides answers to the user.

[0309] Input: User question, prompt

[0310] Output: Answer to the question

[0311] Step 6:

[0312] Tour End

[0313] The guide robot ends the tour at the end of the route.

[0314] Input: End position of tour route

[0315] Output: Notification of tour completion

[0316] Step 7:

[0317] Ask for opinions

[0318] The guide robot asks the user for their impressions and opinions on the tour.

[0319] The guide robot displays an interface on the terminal for inputting impressions.

[0320] The terminal waits for input from the user.

[0321] Input: User state after tour ends

[0322] Output: Display of the impression input screen

[0323] Step 8:

[0324] Information collection and analysis

[0325] The user inputs their thoughts and opinions on the device.

[0326] The terminal transmits the user's opinion to the server.

[0327] The server analyzes the collected opinions and impressions and stores them as a knowledge base.

[0328] The server organizes the data and identifies trends.

[0329] Input: User feedback and opinions

[0330] Output: Knowledge base updates and analysis results

[0331] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0332] The present invention is a system including a reception terminal, a guide robot, a server, and an emotion engine, which automates the implementation of office tours for participants and provides high-quality guide services and the ability to recognize participants' emotions.

[0333] 1. Reception process

[0334] Waiting for reception and identity verification

[0335] The terminal waits for the participant to access the reception terminal.

[0336] The user presses the QR code reader button displayed on the device screen.

[0337] The device scans the user's QR code and analyzes the data in the QR code.

[0338] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[0339] The server checks the participant data and returns the matching results to the terminal.

[0340] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[0341] 2. Office tours

[0342] Meeting and tour start

[0343] The user heads to the designated meeting point.

[0344] The robot waits at the meeting point and checks the participants.

[0345] The robot begins moving along the tour route.

[0346] Facility Description

[0347] Each time the robot arrives at a facility along the tour route, it provides an explanation of the facility based on information obtained from the server.

[0348] The robot provides instructions via voice or text.

[0349] Answering questions

[0350] The user inputs a question into the robot by voice or via a terminal.

[0351] The robot analyzes the question and sends the question to the server.

[0352] The server obtains the appropriate answer data for the question and sends it back to the robot.

[0353] The robot provides the obtained answer to the user.

[0354] emotion recognition

[0355] The robot analyzes the user's voice and facial expressions and recognizes emotions using an emotion engine.

[0356] The robot adjusts its explanations and response methods based on the user's emotions.

[0357] 3. Closing

[0358] Tour End

[0359] The robot ends the tour at the end of the route.

[0360] Ask for opinions

[0361] The robot asks the user for their impressions and opinions of the tour.

[0362] The robot displays an interface on the terminal for inputting impressions.

[0363] Information gathering

[0364] The user inputs their thoughts and opinions on the device.

[0365] The terminal transmits the user's opinion to the server.

[0366] Collecting emotional information

[0367] The robot also uses the emotion engine to collect impressions and opinions after the tour ends, recording the user's emotions as well.

[0368] The server analyzes the collected opinions and impressions, including the emotional information.

[0369] Knowledge

[0370] The server analyzes the collected impressions, opinions and emotional information and organizes the data.

[0371] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[0372] Specific examples

[0373] Below is an example of a participant taking an office tour.

[0374] Reception process

[0375] 1. The user arrives at the reception terminal and scans the QR code.

[0376] 2. The device analyzes the QR code data and verifies the user's identity.

[0377] 3. The terminal will provide an overview of the tour.

[0378] Office tour

[0379] 1. When the user goes to the designated meeting place, the robot will be waiting.

[0380] 2. The robot moves along the tour route and explains each facility.

[0381] 3. Every time the user asks a question, the robot provides an appropriate answer.

[0382] 4. The robot recognizes the user's emotions and adjusts its explanations and response accordingly.

[0383] Closing process

[0384] 1. At the tour end point, the robot ends the tour.

[0385] 2. The robot asks the user for their thoughts and opinions and has them enter them into the terminal.

[0386] 3. The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[0387] This system automates all processes, from reception to office tours, answering questions, recognizing emotions, and closing the deal, making it possible to provide a highly efficient and high-quality tour experience. Furthermore, by recognizing users' emotions and responding accordingly, it is possible to provide a more personalized service.

[0388] The processing flow will be explained below.

[0389] 1. Reception process

[0390] Step 1:

[0391] The terminal waits for the participant to access the reception terminal.

[0392] Step 2:

[0393] The user presses the QR code reader button displayed on the device screen.

[0394] Step 3:

[0395] The device scans the user's QR code and analyzes the data in the QR code.

[0396] Step 4:

[0397] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[0398] Step 5:

[0399] The server checks the participant data and returns the matching results to the terminal.

[0400] Step 6:

[0401] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[0402] 2. Office tours

[0403] Step 1:

[0404] The user heads to the designated meeting point.

[0405] Step 2:

[0406] The robot waits at the meeting point and waits for the user to arrive.

[0407] Step 3:

[0408] The robot begins moving along the tour route.

[0409] Step 4:

[0410] Each time the robot arrives at a facility, it provides an explanation of the facility based on information obtained from the server.

[0411] Step 5:

[0412] The robot analyzes the user's emotions in real time through voice and facial recognition.

[0413] Step 6:

[0414] The user inputs a question into the robot by voice or via a terminal.

[0415] Step 7:

[0416] The robot analyzes the question and sends the question to the server.

[0417] Step 8:

[0418] The server obtains the appropriate answer data for the question and sends it back to the robot.

[0419] Step 9:

[0420] The robot provides the obtained answer to the user.

[0421] Step 10:

[0422] The emotion engine analyzes the user's emotions and adjusts the explanation and response method based on the results.

[0423] 3. Closing

[0424] Step 1:

[0425] The robot ends the tour at the end of the route.

[0426] Step 2:

[0427] The robot asks the user for their impressions and opinions of the tour.

[0428] Step 3:

[0429] The robot displays an interface on the terminal for inputting impressions.

[0430] Step 4:

[0431] The user inputs their thoughts and opinions on the device.

[0432] Step 5:

[0433] The terminal transmits the user's opinion to the server.

[0434] Step 6:

[0435] The emotion engine also analyzes the user's emotions when they enter their thoughts or opinions and records that data.

[0436] Step 7:

[0437] The server analyzes the collected impressions, opinions, and even emotional information.

[0438] Step 8:

[0439] The server organizes the data and identifies trends.

[0440] Step 9:

[0441] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[0442] Example 2

[0443] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0444] Conventional office tours have faced the challenge of efficiently carrying out the entire process from reception to conducting the tour, answering questions, collecting feedback, and then turning it into knowledge. It was also difficult to recognize participants' emotions and respond flexibly based on them, which led to problems with the quality of the tour experience being inconsistent.

[0445] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0446] In this invention, the server includes means for verifying the identity of participants at the reception terminal, means for a guide robot to explain the facilities along the tour route, means for the guide robot to respond to questions from participants, means for collecting impressions and opinions from participants after the tour ends, means for analyzing the collected impressions, opinions, and emotion data and storing them as a knowledge base, means including an emotion engine for recognizing participants' emotions, and means for adjusting response methods based on the recognized emotions. This makes it possible to automate the entire tour process, provide efficient, high-quality services, and further realize personalized responses based on participants' emotions.

[0447] The "reception terminal" is a device where participants enter information upon arrival and verify their identity.

[0448] "Identity verification" refers to the process by which participants prove their identity when joining a tour.

[0449] The "guide robot" is an autonomous machine that moves automatically along the tour route, explains the facilities, and answers participants' questions.

[0450] The "tour route" refers to the route that tour participants follow, guided by a guide robot.

[0451] "Facility explanation" refers to detailed information provided by the guide robot about each facility and point of interest along the tour route.

[0452] "Question handling" is the process in which the guide robot provides information in response to questions from participants.

[0453] "Collecting impressions and opinions" refers to the process of collecting feedback from participants after the tour has ended.

[0454] An "emotion engine" is software or hardware that analyzes participants' voices, facial expressions, etc. and recognizes their emotions.

[0455] "Emotional data" refers to data that contains information about the emotions of participants.

[0456] A "knowledge base" is a database that accumulates and organizes knowledge obtained from collected impressions, opinions, and emotional data.

[0457] "Adjusting responses based on perceived emotions" refers to the procedure of changing the explanations and responses provided depending on the participant's emotional state.

[0458] The present invention is a system that includes a reception terminal, a guide robot, a server, and an emotion engine, and automates the implementation of office tours for participants, providing high-quality guide services and the ability to recognize participants' emotions. Specific examples of this system are shown below.

[0459] First, the reception terminal is a device used by participants when they arrive. The terminal is equipped with a QR code reader and a touch screen, and participants scan their own QR code by operating the QR code reading button displayed on the terminal. The QR code data is sent from the terminal to the server, and the server compares it with the registered participant data. The result of this comparison is sent back to the terminal, and the terminal completes identity verification. This process uses HTTP requests and responses.

[0460] Next, guide robots play a major role in conducting office tours along a tour route. The guide robot waits at a designated meeting point, and after confirming the participants, moves automatically along the tour route. Each time the robot arrives at a facility, it provides explanations in voice or text based on information obtained from the server. When a participant asks a question, the robot analyzes the question and sends it to the server. The server retrieves the appropriate answer from a database and sends it back to the robot. The robot then provides this answer to the participant.

[0461] The robot also has an emotion engine that analyzes voice and facial expressions to recognize participants' emotions. The analyzed emotion data is used to adjust the robot's response. For example, if a participant is excited, the robot will provide more detailed explanations, while if the participant is tired, it will provide a brief explanation.

[0462] After the tour ends, the robot asks participants for their impressions and opinions. Participants enter their impressions and opinions using the input interface displayed on their devices. This data is sent from the devices to a server, and the collected opinions and emotional data is stored in a knowledge base. The server analyzes this data and uses it to improve future tours.

[0463] As a concrete example, detailed explanations can be obtained by inputting the following prompt sentence into the generative AI model:

[0464] Please explain the check-in process for users to join the office tour. For example, please include steps such as waiting at the reception terminal, verifying identity using a QR code, and receiving an explanation of the tour overview.

[0465] An example of a prompt sentence for collecting emotional information and storing it in the knowledge base is as follows:

[0466] Regarding the closing process after the office tour, please explain in detail the process from the robot collecting user opinions to the server storing them in the knowledge base.

[0467] In this way, the present invention provides a system that automates all processes from reception processing to office tours, answering questions, emotion recognition, and closing, making it possible to provide participants with an efficient, high-quality tour experience.

[0468] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0469] Step 1:

[0470] Waiting for reception and identity verification

[0471] The terminal waits for the user to access the reception terminal. The input is "standby state" and the output is "start screen display." Specifically, the terminal displays a screen indicating the standby state and puts the RFID reader and camera into standby state.

[0472] The user presses the QR code reader button displayed on the device screen. The input is "touch the QR code reader button." The output is "start QR code scanning."

[0473] The device scans the user's QR code and analyzes the QR code data. The input is "QR code image data" and the output is "QR code string data." Specifically, the camera reads the QR code and converts it into string data.

[0474] The device sends the QR code data to the server and requests that it be matched with the registered participant data. The input is the "QR code string data" and the output is a "matching request to the server." Specifically, an HTTP request is sent from the device to the server.

[0475] The server checks the participant data and returns the matching results to the device. The input is a "matching request" and the output is a "matching result response." Specifically, it searches the database for participant information and returns the matching results to the device as an HTTP response.

[0476] The terminal completes identity verification based on the matching results and explains an overview of the tour to the user. The input is "matching result" and the output is "identity verification completed and tour overview displayed." Specifically, it displays "identity verification completed," followed by an overview of the tour in voice or text.

[0477] Step 2:

[0478] Meeting and tour start

[0479] The user heads to the designated meeting place. The input is "meeting place instructions" and the output is "user movement." Specifically, the user moves according to the map and instructions of the meeting place displayed on the device screen.

[0480] The robot waits at the meeting point and checks for participants. The input is "Confirmation of participant arrival" and the output is "Preparation to start the tour is complete." Specifically, the robot starts up and goes into standby mode at the meeting point.

[0481] The robot starts moving along the tour route. The input is "tour route instructions" and the output is "movement along the route." Specifically, the robot moves automatically along the pre-set route.

[0482] Step 3:

[0483] Facility Description

[0484] Each time the robot arrives at a facility along the tour route, it provides an explanation of the facility based on information obtained from the server. The input is a "facility information acquisition request" and the output is a "facility explanation." Specifically, when the robot arrives at each point, it sends an explanation request to the server and explains the information it has obtained using a voice device.

[0485] The robot provides explanations by voice or text. The input is "explanatory data" and the output is "explanation by voice or text." Specifically, it provides explanations by voice using the built-in speaker or displays text on the display.

[0486] Step 4:

[0487] Answering questions

[0488] The user inputs questions to the robot by voice or via a terminal. The input is the "user question" and the output is "question data." Specifically, the user either speaks questions directly to the robot or inputs questions into a dedicated tablet.

[0489] The robot analyzes the question and sends it to the server. The input is "question data" and the output is a "question request to the server." Specifically, the robot converts the question into text using a voice recognition system and sends the analysis results to the server.

[0490] The server retrieves the appropriate answer data for the question and sends it back to the robot. The input is a "question request" and the output is "answer data." Specifically, it searches for relevant information from a database and sends it to the robot as an answer.

[0491] The robot provides the obtained answer to the user. The input is "answer data" and the output is "answer to the user." Specifically, the answer is conveyed to the user by voice or text.

[0492] Step 5:

[0493] emotion recognition

[0494] The robot analyzes the user's voice and facial expressions and uses an emotion engine to recognize emotions. The input is "voice data and facial expression data" and the output is "emotion data." Specifically, the robot uses a camera and a voice analyzer to input the user's facial expressions and tone of voice into the emotion engine.

[0495] The robot adjusts its explanations and response methods based on the user's emotions. The input is "emotional data" and the output is "adjusted explanations." Specifically, if the user expresses positive emotions, it provides a bright and detailed explanation, and if the user expresses negative emotions, it provides a concise explanation.

[0496] Step 6:

[0497] Tour end and closing

[0498] The robot ends the tour at the final point of the tour route. The input is "Arrived at final point" and the output is "Announcement that the tour has ended." Specifically, when the robot arrives at the final point, it makes an announcement and ends the tour.

[0499] The robot asks the user for their impressions and opinions of the tour. The input is an "announcement that the tour has ended" and the output is a "request to collect impressions." Specifically, the robot proposes questions in the form of a questionnaire and displays them on the terminal screen.

[0500] Step 7:

[0501] Collecting opinions and feedback

[0502] The user inputs their thoughts and opinions into the device. The input is "user thoughts and opinions" and the output is "collected opinion data." Specifically, the user inputs their thoughts and opinions using a touch screen.

[0503] The terminal sends the user's opinion to the server. The input is "opinion data" and the output is "data to send to the server." Specifically, the input data is sent to the server via an HTTP request.

[0504] Step 8:

[0505] Collecting emotional information

[0506] The robot also uses the emotion engine to collect impressions and opinions after the tour ends, recording the user's emotions as well. The input is "collected impression data and real-time emotion data," and the output is "comprehensive emotion information." Specifically, the robot analyzes the user's facial expressions while they are entering their impressions, and sends that emotion data to the server.

[0507] The server analyzes the collected opinions and impressions, including emotional information. The input is "opinion data and emotional data," and the output is "analysis results." Specifically, it uses machine learning algorithms to analyze user feedback and emotional information.

[0508] Step 9:

[0509] Knowledge

[0510] The server analyzes the collected impressions, opinions, and emotional information and organizes the data. The input is the "analysis results" and the output is "organized data." Specifically, the data is tagged, categorized, and organized.

[0511] The server accumulates useful information in a knowledge base and uses it to improve future tours. The input is "organized data" and the output is "additional data to the knowledge base." Specifically, new entries are added to the knowledge base, and improvements are reflected in the next tour.

[0512] (Application example 2)

[0513] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0514] Conventional food delivery services have faced challenges in improving delivery efficiency and customer satisfaction. In particular, there is a demand for more personalized service by combining the automation of the delivery process with customer emotion recognition. However, current systems lack a mechanism for measuring real-time customer satisfaction and efficiently collecting and utilizing feedback while reducing the workload of delivery personnel.

[0515] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for verifying the identity of participants at the reception terminal, a means for a guide robot to explain the facilities along the tour route, a means for the guide robot to respond to participants' questions, a means for collecting impressions and opinions from participants after the tour ends, a means for analyzing the collected impressions and opinions and storing them in a knowledge base, a means for recognizing participants' emotions using a smartphone, a means for collecting appropriate feedback based on the results of emotion recognition, and a means for transmitting the feedback to the server and storing it in the knowledge base as data for service improvement. This advances automation of the delivery process and enables the provision of efficient, high-quality delivery services. Customer emotion data can be analyzed in real time and quickly reflected in service improvements, which is expected to improve customer satisfaction.

[0516] A "reception terminal" is an electronic device that allows participants to check in and verify their identity.

[0517] "Identity verification" is the process of verifying that participants are legitimately registered individuals using a QR code or ID entry.

[0518] A "guide robot" is a robotic device that automatically moves along the tour route, explaining the facilities and answering participants' questions.

[0519] A "tour route" is a route set up to visit specific facilities or locations in order.

[0520] "Emotion recognition" is the process of analyzing emotions from participants' facial expressions and voice using smartphones or other devices.

[0521] "After the tour" refers to the time or situation after the entire sightseeing route has been completed.

[0522] "Impressions and opinions" collect participants' personal evaluations and thoughts about the tour they experienced.

[0523] A "knowledge base" is a database system that accumulates collected data and information, especially impressions and opinions, and stores the results of analysis.

[0524] "Feedback" refers to information about opinions and feelings collected from participants, which is used to improve the service.

[0525] A "smartphone" is a mobile terminal that has communication capabilities and is equipped with a multi-function processor.

[0526] A "server" is a computer system that processes, stores, and manages data, and interacts with other devices over a network.

[0527] This invention is a system for automating the delivery process and improving customer satisfaction in food delivery services. The system includes a reception terminal, a guide robot, a server, a smartphone, and an emotion engine, and provides high-quality service to participants who join the bot, as well as emotion recognition functions.

[0528] 1. Reception process

[0529] Identity verification

[0530] The terminal waits for the participant to access the reception terminal. The user presses the QR code reading button displayed on the terminal screen to scan the QR code. The terminal analyzes the QR code data, sends it to the server, and compares it with the registered participant data. The server verifies the participant data and returns the comparison result to the terminal. The terminal completes identity verification based on the comparison result and explains the delivery overview to the user.

[0531] 2. Delivery

[0532] Delivery Setup

[0533] The user heads to the designated delivery destination. The navigation system provides the optimal route and real-time guidance. When the delivery location is reached, the user is notified via smartphone.

[0534] 3. Emotion recognition and feedback collection

[0535] Customer Service

[0536] After the delivery is completed, the user uses the smartphone's camera and voice recognition function to analyze the customer's facial expressions and voice. An emotion engine is used to recognize the customer's emotions in real time. Based on the emotion recognition results, the smartphone collects feedback and sends it to the server. The feedback is stored in a knowledge base and used to improve service in future deliveries.

[0537] Hardware and software used

[0538] Hardware

[0539] Smartphone: Check customer delivery information, navigation, camera and voice recognition

[0540] software

[0541] OpenCV: Image processing library (used to handle camera images)

[0542] EmotionRecognizer: A library for recognizing emotions from facial expressions and voice

[0543] requests: A Python library for communicating with servers

[0544] Specific examples

[0545] When a delivery person makes a food delivery, they first check in by scanning a QR code with their smartphone. Then, when they head to the designated delivery destination, the app provides real-time guidance on the optimal route. After the delivery is completed, the smartphone camera captures the customer's facial expression, and an emotion engine analyzes whether the customer is satisfied. The analysis results are sent to the server as feedback and used to improve the service.

[0546] Prompt Sentence Examples

[0547] "You are a food delivery driver. Scan the QR code and confirm the delivery address information. Then, guide the customer to the delivery address with the best route, and recognize the customer's emotions when you arrive at the delivery address. You also collect customer feedback and send it to the server to improve the service."

[0548] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0549] Step 1:

[0550] The reception terminal scans the QR code used by the participant to check in. In this step, the user presses the QR code reading button on the terminal, which then scans the QR code and obtains the QR code data. The obtained data is sent to the server and compared with the registered participant data. The input data is the QR code scan result, and the output data is the comparison result. Through this comparison, the terminal completes participant identity verification.

[0551] Step 2:

[0552] The server analyzes the QR code data sent from the reception terminal and compares it with the registered information in the database. It then returns the comparison results to the terminal. The input data is the scanned data of the QR code, and the output data is the comparison results. This confirms that the participant is a legitimately registered person.

[0553] Step 3:

[0554] The terminal receives the matching results from the server and notifies the user of a delivery summary. The input data is the matching results from the server, and the output data is the notification to the user. This allows the user to check the delivery details.

[0555] Step 4:

[0556] The navigation system provides the optimal route to the delivery destination and guides the user in real time. The user receives real-time navigation instructions on their smartphone. The input data is the address information of the delivery destination, and the output data is the optimal route. This allows the user to reach the delivery destination efficiently.

[0557] Step 5:

[0558] The smartphone notifies the user after the delivery arrives at the destination. This notification informs the customer that the delivery has been completed. The input data is the arrival event at the delivery destination, and the output data is a notification containing the destination information. This allows the customer to prepare to receive the delivery in a timely manner.

[0559] Step 6:

[0560] Using the smartphone's camera and voice recognition function, the system collects the customer's facial expressions and voice when the delivery is complete. The input data is the camera footage and voice data, and the output data is emotion data based on these. The smartphone sends this to the emotion engine and receives the analysis results.

[0561] Step 7:

[0562] The emotion engine analyzes collected video and audio data to recognize customer emotions. The input data is camera video and audio data, and the output data is analyzed emotional information. This makes it possible to evaluate customer satisfaction in real time.

[0563] Step 8:

[0564] Based on the results of emotion recognition, feedback is collected and sent to the server. The input data is the emotion recognition results, and the output data is feedback data. This allows the server to efficiently collect customer feedback.

[0565] Step 9:

[0566] The server stores the collected feedback in a knowledge base and uses it to improve services in the future. The input data is the collected feedback data, and the output data is updated information in the knowledge base. This helps improve customer satisfaction.

[0567] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0568] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0569] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0570] [Second embodiment]

[0571] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0572] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0573] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0574] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0575] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0576] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0577] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0578] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0579] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0580] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0581] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0582] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0583] The present invention is a system including a reception terminal, a guide robot, and a server, which automates the implementation of office tours for participants and provides high-quality guide services.

[0584] 1. Reception process

[0585] Waiting for reception and identity verification

[0586] The terminal waits for the participant to approach the reception terminal.

[0587] The user presses the QR code reader button displayed on the device screen.

[0588] The device scans the user's QR code to verify their identity.

[0589] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[0590] The server is contacted to check against registered participant data.

[0591] Based on the matching results, identity verification is completed.

[0592] Tour overview

[0593] The terminal provides the user with an overview of the tour.

[0594] The terminal displays the sightseeing locations, important points to note, the tour schedule, etc.

[0595] The terminal checks whether the user understands.

[0596] 2. Office tours

[0597] Meeting and tour start

[0598] The user heads to the designated meeting point.

[0599] The robot will wait at the meeting point to welcome participants.

[0600] The robot begins the tour.

[0601] The robot begins to move along the tour route.

[0602] Facility Description

[0603] The robot will provide an explanation as it arrives at each facility along the tour route.

[0604] A description of the facility is provided based on information obtained from the server.

[0605] The robot provides instructions via voice or text.

[0606] Answering questions

[0607] The user inputs a question into the robot by voice or via a terminal.

[0608] The robot analyzes the question and provides an appropriate answer.

[0609] The question is sent to the server and the response data is obtained.

[0610] The robot provides the obtained answer to the user.

[0611] 3. Closing

[0612] Tour End

[0613] The robot ends the tour at the end of the route.

[0614] Ask for opinions

[0615] The robot asks the user for their impressions and opinions of the tour.

[0616] The robot displays an interface on the terminal for inputting impressions.

[0617] The terminal waits for input from the user.

[0618] Information gathering

[0619] The user inputs their thoughts and opinions on the device.

[0620] The terminal transmits the user's opinion to the server.

[0621] Knowledge

[0622] The server analyzes the collected opinions and impressions.

[0623] The server organizes the data and identifies trends.

[0624] The server accumulates useful information as a knowledge base and uses it to improve future tours.

[0625] Specific examples

[0626] Below is an example of a participant taking an office tour.

[0627] 1. Reception process

[0628] The user arrives at the reception terminal and scans the QR code.

[0629] The device analyzes the data in the QR code and verifies the user's identity.

[0630] After verifying your identity, the device will explain the tour outline.

[0631] 2. Office tour

[0632] When the user goes to the designated meeting place, the robot will be waiting.

[0633] The robot moves along the tour route and explains each facility.

[0634] Every time the user asks a question, the robot provides an appropriate answer.

[0635] 3. Closing Process

[0636] At the tour end point, the robot ends the tour.

[0637] The robot asks the user for their thoughts and opinions and has them enter them into a terminal.

[0638] The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[0639] In this way, this system automates all processes from reception to office tour guidance, answering questions, and closing, making it possible to provide a highly efficient, high-quality tour experience.

[0640] The processing flow will be explained below.

[0641] 1. Reception process

[0642] Step 1:

[0643] The terminal waits for the participant to access the reception terminal.

[0644] Step 2:

[0645] The user presses the QR code reader button displayed on the device screen.

[0646] Step 3:

[0647] The device scans the user's QR code and analyzes the data in the QR code.

[0648] Step 4:

[0649] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[0650] Step 5:

[0651] The server checks the participant data and returns the matching results to the terminal.

[0652] Step 6:

[0653] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[0654] 2. Office tours

[0655] Step 1:

[0656] The user heads to the designated meeting point.

[0657] Step 2:

[0658] The robot waits at the meeting point and checks the participants.

[0659] Step 3:

[0660] The robot begins moving along the tour route.

[0661] Step 4:

[0662] Each time the robot arrives at a facility, it provides an explanation of the facility based on information obtained from the server.

[0663] Step 5:

[0664] The user inputs a question into the robot by voice or via a terminal.

[0665] Step 6:

[0666] The robot analyzes the question and sends the question to the server.

[0667] Step 7:

[0668] The server obtains the appropriate answer data for the question and sends it back to the robot.

[0669] Step 8:

[0670] The robot provides the obtained answer to the user.

[0671] 3. Closing

[0672] Step 1:

[0673] The robot ends the tour at the end of the route.

[0674] Step 2:

[0675] The robot asks the user for their impressions and opinions of the tour.

[0676] Step 3:

[0677] The robot displays an interface on the terminal for inputting impressions.

[0678] Step 4:

[0679] The user inputs their thoughts and opinions on the device.

[0680] Step 5:

[0681] The terminal transmits the user's opinion to the server.

[0682] Step 6:

[0683] The server analyzes the collected opinions and impressions and organizes the data.

[0684] Step 7:

[0685] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[0686] Example 1

[0687] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0688] Current office tours require dedicated guides, which poses challenges in terms of efficiency and cost. It is also difficult to respond to participants' diverse questions and opinions on the spot and to gather appropriate feedback. This results in tour quality that is inconsistent or not optimized for each individual participant.

[0689] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0690] In this invention, the server includes means for verifying the identity of participants, means for a guide device moving along the tour route to explain the facility, means for the guide device to respond to questions from participants in cooperation with the server, means for collecting impressions and opinions from participants after the tour ends, and means for saving the collected impressions and opinions in a database, analyzing them with a text mining algorithm, and accumulating them as a knowledge base. This makes it possible to automate highly efficient, high-quality office tours without the need for a dedicated guide, and to appropriately respond to the diverse requests of participants.

[0691] A "reception terminal" is a device used to verify the identity of participants, and is a device that authenticates participants via QR code or ID input.

[0692] A "guide device" is a robot or device that moves automatically along the tour route and has the function of explaining the facility to participants.

[0693] The "server" is a computer system that works in conjunction with the information device and the reception terminal to manage data, collate data, generate responses, and collect and analyze opinions.

[0694] "Identity verification" is the process by which participants are matched with registered information to verify their identity.

[0695] The "tour route" is a specific route along which the guide device travels during an office tour, and is a route along which each facility is visited in sequence.

[0696] The "facility explanation" is an explanation that the guide device provides as it arrives at each facility along the tour route, and is given in the form of audio or text.

[0697] "Response to questions" refers to the guide device working in conjunction with the server to provide appropriate answers to questions posed by participants.

[0698] "Collecting impressions and opinions" refers to the process of receiving feedback from participants after the tour and saving it as data.

[0699] The "knowledge base" is a collection of data that has been collected, analyzed, and stored as useful information to be used to improve future tours.

[0700] The present invention relates to an office tour automation system that includes a reception terminal, a guide device, and a server. This system automates participant authentication, facility explanations, question handling, and feedback collection, providing a highly efficient and high-quality tour experience.

[0701] 1. Reception terminal

[0702] The terminal is a device equipped with a QR code reading function that reads the QR code when the user approaches. The hardware used is a tablet terminal with a built-in QR code scanner or a dedicated reception device. For example, devices running Android or Windows can be used. A library such as Zxing is used to analyze the QR code.

[0703] When a user approaches the reception terminal and presses the QR code reading button on the screen, the terminal scans the QR code and obtains participant information. This information is sent to the server and verified against registered information for identity verification. For example, participants are authenticated by following the instruction to "scan the QR code."

[0704] 2. Guidance device

[0705] The guide device is a robot with autonomous mobility capabilities that explains the facilities while moving along the tour route. The guide device achieves its autonomous movement using ROS (Robot Operating System). It also has speech recognition and synthesis functions, and provides facility explanations and answers questions. A speech synthesis engine such as Google Text-to-Speech is used to explain the facilities.

[0706] For example, when a user approaches a meeting point, a guide device will be waiting and begin the tour. The guide device will explain each facility using prompts such as, "This is our newest office space. Do you have any questions?"

[0707] 3. Server

[0708] The server is a computer system that manages and processes data. The server works in conjunction with the reception terminal and the information device to perform the following tasks:

[0709] Verification of authentication data

[0710] Provision of facility description data

[0711] Generating answers to user questions

[0712] Collecting and analyzing feedback

[0713] As a specific example of how to respond to a question, if a user asks, "How many people can this conference room accommodate?", the guidance device will recognize the question through speech recognition and send it to the server. The server will then obtain the answer data and send it to the guidance device, thereby providing the user with an appropriate answer.

[0714] 4. Collecting feedback and opinions and creating a knowledge base

[0715] After the tour ends, the device displays an interface requesting the user's impressions and opinions. When the user enters their feedback into the device, the data is sent to the server. The server stores the collected opinions in a database and analyzes them using a text mining algorithm. This accumulates useful information as a knowledge base that can be used to improve future tours.

[0716] This system allows participants to enjoy a highly efficient and personalized tour experience, and allows operators to improve the quality of their services based on feedback.

[0717] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0718] Step 1:

[0719] The terminal waits at the reception location for the user to approach. When the user presses the "QR code reading button" displayed on the reception terminal's screen, the terminal switches to QR code scanning mode. Specifically, pressing the button displays a camera preview on the screen, allowing the QR code to be recognized. The user's QR code is acquired as input, and the output after analysis is participant information (name, ID, etc.). This data is then sent to the server.

[0720] Step 2:

[0721] The server compares the received participant information with the existing registration information in the database. The input is the participant information sent from the terminal, and the output is the authentication result (authentication successful or authentication failed). In concrete terms, the server performs a database search and returns the comparison result to the terminal. If authentication is successful, the terminal displays an "Authentication successful" message, and if it fails, it displays an "Authentication failed" message.

[0722] Step 3:

[0723] After successful authentication, the terminal displays an overview of the tour to the user. The input is the authentication result from the server, and the output is a display of the tour overview information. Specifically, the terminal screen displays the tour locations, important points to note, the tour flow, etc. in a presentation format. The user presses the "Next" button to confirm the contents.

[0724] Step 4:

[0725] The user heads to the designated meeting place. When the user arrives at the meeting place, a guide device is waiting. The input is guide information for the meeting place, and the output is the user's movement. In concrete terms, the guide device displays information about the meeting place on the terminal screen and guides the user.

[0726] Step 5:

[0727] The guide device welcomes participants who have gathered at the meeting point and starts moving along the tour route. The input is the gathering state of users at the meeting point, and the output is a signal to start moving. In concrete terms, the robot moves autonomously along a pre-programmed route, stopping at each point.

[0728] Step 6:

[0729] The guide device provides explanations when arriving at each facility. The input is the location information of each point on the tour route, and the output is a description of the facility. In concrete terms, the robot uses facility information obtained from the server to provide explanations in voice and text. For example, it uses prompts such as, "This is our latest office space. Do you have any questions?"

[0730] Step 7:

[0731] When a user asks a question, the guidance device recognizes the question through voice recognition and sends it to the server. The input is the user's question, and the output is answer data. Specifically, the robot analyzes the question using its voice recognition function and sends the question data to the server. The server analyzes the question, generates an answer, and returns it to the robot. The robot then provides the answer to the user.

[0732] Step 8:

[0733] The guide device ends the tour when it reaches the end point of the tour route. The input is the end point information of the tour route, and the output is a notification that the tour has ended. Specifically, it plays a voice message saying, "This is the end of the tour. Thank you for participating."

[0734] Step 9:

[0735] The guide device instructs the user to provide their impressions and opinions, and displays an input interface on the terminal. The input is the state after the tour ends, and the output is a screen display for entering impressions. Specifically, the terminal displays an impression form and prompts the user to enter their impressions.

[0736] Step 10:

[0737] When a user inputs their thoughts or opinions into the device, the device sends the data to the server. The input is the user's feedback, and the output is the data sent to the server. The specific operation is to store the thoughts in a database.

[0738] Step 11:

[0739] The server stores the collected impressions and opinions in a database and analyzes them using a text mining algorithm. The input is user impressions and opinions, and the output is the analysis results and a knowledge base. Specifically, the server classifies and analyzes the data and accumulates useful information as a knowledge base.

[0740] (Application example 1)

[0741] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0742] Conventional customer guidance services in brick-and-mortar stores often rely on human resources, which means that the quality of guidance and consistency of service cannot be guaranteed. Another issue is that when customers ask a variety of questions about the store's products, they are unable to receive appropriate answers immediately. Furthermore, there is a lack of a system for effectively collecting and analyzing customer feedback and using it to improve services. For these reasons, there is a need for efficient, high-quality guidance services in brick-and-mortar stores.

[0743] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0744] In this invention, the server includes a means for verifying the identity of participants at the reception terminal, a means for a guide robot to explain the facilities along the tour route, a means for the guide robot to respond to participants' questions, a means for collecting impressions and opinions from participants after the tour ends, a means for analyzing the collected impressions and opinions and storing them as a knowledge base, an automated means for guiding visitors and a means for controlling the guide robot and introducing products in each section of the physical store, and a means for answering questions using a generative AI model. This enables the physical store to provide automated, high-quality guidance services while also appropriately responding to a variety of questions from visitors. Furthermore, the efficiency of feedback collection and analysis can be improved, which can be used to improve services.

[0745] A "reception terminal" is a device that customers use to check in and verify their identity by scanning a QR code or entering their ID.

[0746] A "participant" is someone who uses the system to receive an office tour or a guided tour of a physical store.

[0747] "Identity verification" is the process of verifying that a system user is a person who has been pre-registered by scanning a QR code or entering an ID.

[0748] A "tour route" indicates a guided course within a specific facility, and is the path along which the robot will guide the visitor.

[0749] A "guide robot" is an autonomous mobile robot that automatically moves along a tour route, such as a physical store or office tour, and provides explanations about the facility.

[0750] "After the tour ends" refers to the time after the entire tour by the guide robot has been completed.

[0751] "Impressions and opinions" refers to user feedback regarding tours and guide services, which is collected by being entered into the terminal.

[0752] A "knowledge base" is a database of knowledge that is analyzed based on collected impressions and opinions and accumulated for future service improvements.

[0753] "Automatic means" refers to a method or mechanism by which a system performs a function automatically without human intervention.

[0754] A "guide robot" is a robot that moves around a physical store and provides guidance to customers by introducing products and answering their questions.

[0755] "Generative AI model" refers to an artificial intelligence model used to generate appropriate answers to user questions.

[0756] A "prompt" is a text input that provides context for the answer an artificial intelligence question-answering model generates.

[0757] This invention automates the guidance service for customers in physical stores by using a system including a reception terminal, guide robot, and server, and provides high-quality explanations and answers to questions. This system comprehensively implements processes from customer check-in to guiding them around the store, explaining products, answering questions, and collecting feedback.

[0758] 1. Reception process

[0759] Waiting for reception and identity verification

[0760] The terminal waits for a customer to approach the reception terminal.

[0761] The user presses the QR code reader button displayed on the device screen.

[0762] The device scans the user's QR code to verify their identity.

[0763] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[0764] The server is contacted to check against registered participant data.

[0765] Based on the matching results, identity verification is completed.

[0766] Guide overview

[0767] The terminal provides the user with an overview of the tour.

[0768] The terminal displays the sightseeing locations, important points to note, the tour schedule, etc.

[0769] The terminal checks whether the user understands.

[0770] 2. Tours using guide robots

[0771] Meeting and tour start

[0772] The user heads to the designated meeting point.

[0773] The guide robot waits at the meeting point to welcome customers.

[0774] The guide robot begins the tour.

[0775] The guide robot begins to move along the tour route.

[0776] Facility Description

[0777] The guide robot provides explanations as it arrives at each section along the tour route.

[0778] Provides descriptions of facilities and products based on information obtained from the server.

[0779] The guide robot provides explanations by voice or text.

[0780] 3. Answering questions

[0781] Accepting questions and providing answers

[0782] The user inputs questions to the guide robot by voice or via a terminal.

[0783] The guide robot analyzes the question and provides an appropriate answer.

[0784] The question is sent to the server and the answer data is obtained using a generative AI model.

[0785] The guide robot provides the obtained answers to the user.

[0786] 4. Closing

[0787] Tour End

[0788] The guide robot ends the tour at the end of the route.

[0789] Ask for opinions

[0790] The guide robot asks the user for their impressions and opinions on the tour.

[0791] The guide robot displays an interface for inputting impressions on the terminal.

[0792] The terminal waits for input from the user.

[0793] Information collection and analysis

[0794] The user inputs their thoughts and opinions on the device.

[0795] The terminal transmits the user's opinion to the server.

[0796] The server analyzes the collected opinions and impressions and stores them as a knowledge base.

[0797] The server organizes the data and identifies trends.

[0798] The server uses the useful information as a knowledge base to improve the next tour.

[0799] Specific examples

[0800] The following is an example of a customer receiving guidance services at a physical store.

[0801] 1. Reception process

[0802] The user arrives at the reception terminal and scans the QR code.

[0803] The device analyzes the data in the QR code and verifies the user's identity.

[0804] After verifying your identity, the device will explain the tour outline.

[0805] 2. Tours using guide robots

[0806] When the user heads to the designated meeting place, a guide robot will be waiting.

[0807] The guide robot moves along the tour route and explains each section.

[0808] Every time the user asks a question, the guide robot provides an appropriate answer.

[0809] 3. Closing Process

[0810] At the tour end point, the guide robot ends the tour.

[0811] The guide robot asks the user for their impressions and opinions and has them enter them into a terminal.

[0812] The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[0813] For example, in response to a frequently asked question such as "What are the new features of this smartphone?", the generative AI model generates an answer using the following prompt sentence:

[0814] Context: This smartphone has a brand new chip, making it even faster than previous models, and it also has a number of new features, including a higher-resolution camera and longer battery life.

[0815] Question: What are the new features of this smartphone?

[0816] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0817] Step 1:

[0818] Waiting for reception and identity verification

[0819] The terminal waits for a customer to approach the reception terminal.

[0820] The user presses the QR code reading button displayed on the device screen to scan the QR code.

[0821] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[0822] The terminal sends the information to the server and checks it against registered participant data.

[0823] The server compares the information with that in the participant database and returns the identity verification results to the terminal.

[0824] The device completes identity verification based on the matching results.

[0825] Input: User's QR code

[0826] Output: Identity verification result

[0827] Step 2:

[0828] Guide overview

[0829] The terminal provides the user with an overview of the tour (places to visit, things to note, tour flow, etc.).

[0830] The terminal confirms through the screen interface whether the user has understood.

[0831] Input: QR code scan information, identity verification results from the server

[0832] Output: Tour overview

[0833] Step 3:

[0834] Meeting and tour start

[0835] The user heads to the designated meeting point.

[0836] The guide robot waits at the meeting point to welcome customers.

[0837] The guide robot begins moving along the tour route.

[0838] Input: Specify the meeting place and time

[0839] Output: Notification of tour start

[0840] Step 4:

[0841] Facility Description

[0842] The guide robot will explain the facilities and products as it arrives at each section along the tour route.

[0843] The guide robot retrieves explanatory information from the server in real time.

[0844] The guide robot provides explanations in voice or text.

[0845] Input: Tour route and section information

[0846] Output: Facility description audio or text

[0847] Step 5:

[0848] Accepting questions and providing answers

[0849] The user inputs questions to the guide robot by voice or via a terminal.

[0850] The guide robot analyzes the question and sends the content to the server.

[0851] The server uses a generative AI model to construct prompts to generate answers to questions.

[0852] The server returns the generated answer to the guide robot.

[0853] The guide robot provides answers to the user.

[0854] Input: User question, prompt

[0855] Output: Answer to the question

[0856] Step 6:

[0857] Tour End

[0858] The guide robot ends the tour at the end of the route.

[0859] Input: End position of tour route

[0860] Output: Notification of tour completion

[0861] Step 7:

[0862] Ask for opinions

[0863] The guide robot asks the user for their impressions and opinions on the tour.

[0864] The guide robot displays an interface on the terminal for inputting impressions.

[0865] The terminal waits for input from the user.

[0866] Input: User state after tour ends

[0867] Output: Display of the impression input screen

[0868] Step 8:

[0869] Information collection and analysis

[0870] The user inputs their thoughts and opinions on the device.

[0871] The terminal transmits the user's opinion to the server.

[0872] The server analyzes the collected opinions and impressions and stores them as a knowledge base.

[0873] The server organizes the data and identifies trends.

[0874] Input: User feedback and opinions

[0875] Output: Knowledge base updates and analysis results

[0876] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0877] The present invention is a system including a reception terminal, a guide robot, a server, and an emotion engine, which automates the implementation of office tours for participants and provides high-quality guide services and the ability to recognize participants' emotions.

[0878] 1. Reception process

[0879] Waiting for reception and identity verification

[0880] The terminal waits for the participant to access the reception terminal.

[0881] The user presses the QR code reader button displayed on the device screen.

[0882] The device scans the user's QR code and analyzes the data in the QR code.

[0883] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[0884] The server checks the participant data and returns the matching results to the terminal.

[0885] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[0886] 2. Office tours

[0887] Meeting and tour start

[0888] The user heads to the designated meeting point.

[0889] The robot waits at the meeting point and checks the participants.

[0890] The robot begins moving along the tour route.

[0891] Facility Description

[0892] Each time the robot arrives at a facility along the tour route, it provides an explanation of the facility based on information obtained from the server.

[0893] The robot provides instructions via voice or text.

[0894] Answering questions

[0895] The user inputs a question into the robot by voice or via a terminal.

[0896] The robot analyzes the question and sends the question to the server.

[0897] The server obtains the appropriate answer data for the question and sends it back to the robot.

[0898] The robot provides the obtained answer to the user.

[0899] emotion recognition

[0900] The robot analyzes the user's voice and facial expressions and recognizes emotions using an emotion engine.

[0901] The robot adjusts its explanations and response methods based on the user's emotions.

[0902] 3. Closing

[0903] Tour End

[0904] The robot ends the tour at the end of the route.

[0905] Ask for opinions

[0906] The robot asks the user for their impressions and opinions of the tour.

[0907] The robot displays an interface on the terminal for inputting impressions.

[0908] Information gathering

[0909] The user inputs their thoughts and opinions on the device.

[0910] The terminal transmits the user's opinion to the server.

[0911] Collecting emotional information

[0912] The robot also uses the emotion engine to collect impressions and opinions after the tour ends, recording the user's emotions as well.

[0913] The server analyzes the collected opinions and impressions, including the emotional information.

[0914] Knowledge

[0915] The server analyzes the collected impressions, opinions and emotional information and organizes the data.

[0916] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[0917] Specific examples

[0918] Below is an example of a participant taking an office tour.

[0919] Reception process

[0920] 1. The user arrives at the reception terminal and scans the QR code.

[0921] 2. The device analyzes the QR code data and verifies the user's identity.

[0922] 3. The terminal will provide an overview of the tour.

[0923] Office tour

[0924] 1. When the user goes to the designated meeting place, the robot will be waiting.

[0925] 2. The robot moves along the tour route and explains each facility.

[0926] 3. Every time the user asks a question, the robot provides an appropriate answer.

[0927] 4. The robot recognizes the user's emotions and adjusts its explanations and response accordingly.

[0928] Closing process

[0929] 1. At the tour end point, the robot ends the tour.

[0930] 2. The robot asks the user for their thoughts and opinions and has them enter them into the terminal.

[0931] 3. The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[0932] This system automates all processes, from reception to office tours, answering questions, recognizing emotions, and closing the deal, making it possible to provide a highly efficient and high-quality tour experience. Furthermore, by recognizing users' emotions and responding accordingly, it is possible to provide a more personalized service.

[0933] The processing flow will be explained below.

[0934] 1. Reception process

[0935] Step 1:

[0936] The terminal waits for the participant to access the reception terminal.

[0937] Step 2:

[0938] The user presses the QR code reader button displayed on the device screen.

[0939] Step 3:

[0940] The device scans the user's QR code and analyzes the data in the QR code.

[0941] Step 4:

[0942] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[0943] Step 5:

[0944] The server checks the participant data and returns the matching results to the terminal.

[0945] Step 6:

[0946] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[0947] 2. Office tours

[0948] Step 1:

[0949] The user heads to the designated meeting point.

[0950] Step 2:

[0951] The robot waits at the meeting point and waits for the user to arrive.

[0952] Step 3:

[0953] The robot begins moving along the tour route.

[0954] Step 4:

[0955] Each time the robot arrives at a facility, it provides an explanation of the facility based on information obtained from the server.

[0956] Step 5:

[0957] The robot analyzes the user's emotions in real time through voice and facial recognition.

[0958] Step 6:

[0959] The user inputs a question into the robot by voice or via a terminal.

[0960] Step 7:

[0961] The robot analyzes the question and sends the question to the server.

[0962] Step 8:

[0963] The server obtains the appropriate answer data for the question and sends it back to the robot.

[0964] Step 9:

[0965] The robot provides the obtained answer to the user.

[0966] Step 10:

[0967] The emotion engine analyzes the user's emotions and adjusts the explanation and response method based on the results.

[0968] 3. Closing

[0969] Step 1:

[0970] The robot ends the tour at the end of the route.

[0971] Step 2:

[0972] The robot asks the user for their impressions and opinions of the tour.

[0973] Step 3:

[0974] The robot displays an interface on the terminal for inputting impressions.

[0975] Step 4:

[0976] The user inputs their thoughts and opinions on the device.

[0977] Step 5:

[0978] The terminal transmits the user's opinion to the server.

[0979] Step 6:

[0980] The emotion engine also analyzes the user's emotions when they enter their thoughts or opinions and records that data.

[0981] Step 7:

[0982] The server analyzes the collected impressions, opinions, and even emotional information.

[0983] Step 8:

[0984] The server organizes the data and identifies trends.

[0985] Step 9:

[0986] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[0987] Example 2

[0988] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0989] Conventional office tours have faced the challenge of efficiently carrying out the entire process from reception to conducting the tour, answering questions, collecting feedback, and then turning it into knowledge. It was also difficult to recognize participants' emotions and respond flexibly based on them, which led to problems with the quality of the tour experience being inconsistent.

[0990] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0991] In this invention, the server includes means for verifying the identity of participants at the reception terminal, means for a guide robot to explain the facilities along the tour route, means for the guide robot to respond to questions from participants, means for collecting impressions and opinions from participants after the tour ends, means for analyzing the collected impressions, opinions, and emotion data and storing them as a knowledge base, means including an emotion engine for recognizing participants' emotions, and means for adjusting response methods based on the recognized emotions. This makes it possible to automate the entire tour process, provide efficient, high-quality services, and further realize personalized responses based on participants' emotions.

[0992] The "reception terminal" is a device where participants enter information upon arrival and verify their identity.

[0993] "Identity verification" refers to the process by which participants prove their identity when joining a tour.

[0994] The "guide robot" is an autonomous machine that moves automatically along the tour route, explains the facilities, and answers participants' questions.

[0995] The "tour route" refers to the route that tour participants follow, guided by a guide robot.

[0996] "Facility explanation" refers to detailed information provided by the guide robot about each facility and point of interest along the tour route.

[0997] "Question handling" is the process in which the guide robot provides information in response to questions from participants.

[0998] "Collecting impressions and opinions" refers to the process of collecting feedback from participants after the tour has ended.

[0999] An "emotion engine" is software or hardware that analyzes participants' voices, facial expressions, etc. and recognizes their emotions.

[1000] "Emotional data" refers to data that contains information about the emotions of participants.

[1001] A "knowledge base" is a database that accumulates and organizes knowledge obtained from collected impressions, opinions, and emotional data.

[1002] "Adjusting responses based on perceived emotions" refers to the procedure of changing the explanations and responses provided depending on the participant's emotional state.

[1003] The present invention is a system that includes a reception terminal, a guide robot, a server, and an emotion engine, and automates the implementation of office tours for participants, providing high-quality guide services and the ability to recognize participants' emotions. Specific examples of this system are shown below.

[1004] First, the reception terminal is a device used by participants when they arrive. The terminal is equipped with a QR code reader and a touch screen, and participants scan their own QR code by operating the QR code reading button displayed on the terminal. The QR code data is sent from the terminal to the server, and the server compares it with the registered participant data. The result of this comparison is sent back to the terminal, and the terminal completes identity verification. This process uses HTTP requests and responses.

[1005] Next, guide robots play a major role in conducting office tours along a tour route. The guide robot waits at a designated meeting point, and after confirming the participants, moves automatically along the tour route. Each time the robot arrives at a facility, it provides explanations in voice or text based on information obtained from the server. When a participant asks a question, the robot analyzes the question and sends it to the server. The server retrieves the appropriate answer from a database and sends it back to the robot. The robot then provides this answer to the participant.

[1006] The robot also has an emotion engine that analyzes voice and facial expressions to recognize participants' emotions. The analyzed emotion data is used to adjust the robot's response. For example, if a participant is excited, the robot will provide more detailed explanations, while if the participant is tired, it will provide a brief explanation.

[1007] After the tour ends, the robot asks participants for their impressions and opinions. Participants enter their impressions and opinions using the input interface displayed on their devices. This data is sent from the devices to a server, and the collected opinions and emotional data is stored in a knowledge base. The server analyzes this data and uses it to improve future tours.

[1008] As a concrete example, detailed explanations can be obtained by inputting the following prompt sentence into the generative AI model:

[1009] Please explain the check-in process for users to join the office tour. For example, please include steps such as waiting at the reception terminal, verifying identity using a QR code, and receiving an explanation of the tour overview.

[1010] An example of a prompt sentence for collecting emotional information and storing it in the knowledge base is as follows:

[1011] Regarding the closing process after the office tour, please explain in detail the process from the robot collecting user opinions to the server storing them in the knowledge base.

[1012] In this way, the present invention provides a system that automates all processes from reception processing to office tours, answering questions, emotion recognition, and closing, making it possible to provide participants with an efficient, high-quality tour experience.

[1013] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1014] Step 1:

[1015] Waiting for reception and identity verification

[1016] The terminal waits for the user to access the reception terminal. The input is "standby state" and the output is "start screen display." Specifically, the terminal displays a screen indicating the standby state and puts the RFID reader and camera into standby state.

[1017] The user presses the QR code reader button displayed on the device screen. The input is "touch the QR code reader button." The output is "start QR code scanning."

[1018] The device scans the user's QR code and analyzes the QR code data. The input is "QR code image data" and the output is "QR code string data." Specifically, the camera reads the QR code and converts it into string data.

[1019] The device sends the QR code data to the server and requests that it be matched with the registered participant data. The input is the "QR code string data" and the output is a "matching request to the server." Specifically, an HTTP request is sent from the device to the server.

[1020] The server checks the participant data and returns the matching results to the device. The input is a "matching request" and the output is a "matching result response." Specifically, it searches the database for participant information and returns the matching results to the device as an HTTP response.

[1021] The terminal completes identity verification based on the matching results and explains an overview of the tour to the user. The input is "matching result" and the output is "identity verification completed and tour overview displayed." Specifically, it displays "identity verification completed," followed by an overview of the tour in voice or text.

[1022] Step 2:

[1023] Meeting and tour start

[1024] The user heads to the designated meeting place. The input is "meeting place instructions" and the output is "user movement." Specifically, the user moves according to the map and instructions of the meeting place displayed on the device screen.

[1025] The robot waits at the meeting point and checks for participants. The input is "Confirmation of participant arrival" and the output is "Preparation to start the tour is complete." Specifically, the robot starts up and goes into standby mode at the meeting point.

[1026] The robot starts moving along the tour route. The input is "tour route instructions" and the output is "movement along the route." Specifically, the robot moves automatically along the pre-set route.

[1027] Step 3:

[1028] Facility Description

[1029] Each time the robot arrives at a facility along the tour route, it provides an explanation of the facility based on information obtained from the server. The input is a "facility information acquisition request" and the output is a "facility explanation." Specifically, when the robot arrives at each point, it sends an explanation request to the server and explains the information it has obtained using a voice device.

[1030] The robot provides explanations by voice or text. The input is "explanatory data" and the output is "explanation by voice or text." Specifically, it provides explanations by voice using the built-in speaker or displays text on the display.

[1031] Step 4:

[1032] Answering questions

[1033] The user inputs questions to the robot by voice or via a terminal. The input is the "user question" and the output is "question data." Specifically, the user either speaks questions directly to the robot or inputs questions into a dedicated tablet.

[1034] The robot analyzes the question and sends it to the server. The input is "question data" and the output is a "question request to the server." Specifically, the robot converts the question into text using a voice recognition system and sends the analysis results to the server.

[1035] The server retrieves the appropriate answer data for the question and sends it back to the robot. The input is a "question request" and the output is "answer data." Specifically, it searches for relevant information from a database and sends it to the robot as an answer.

[1036] The robot provides the obtained answer to the user. The input is "answer data" and the output is "answer to the user." Specifically, the answer is conveyed to the user by voice or text.

[1037] Step 5:

[1038] emotion recognition

[1039] The robot analyzes the user's voice and facial expressions and uses an emotion engine to recognize emotions. The input is "voice data and facial expression data" and the output is "emotion data." Specifically, the robot uses a camera and a voice analyzer to input the user's facial expressions and tone of voice into the emotion engine.

[1040] The robot adjusts its explanations and response methods based on the user's emotions. The input is "emotional data" and the output is "adjusted explanations." Specifically, if the user expresses positive emotions, it provides a bright and detailed explanation, and if the user expresses negative emotions, it provides a concise explanation.

[1041] Step 6:

[1042] Tour end and closing

[1043] The robot ends the tour at the final point of the tour route. The input is "Arrived at final point" and the output is "Announcement that the tour has ended." Specifically, when the robot arrives at the final point, it makes an announcement and ends the tour.

[1044] The robot asks the user for their impressions and opinions of the tour. The input is an "announcement that the tour has ended" and the output is a "request to collect impressions." Specifically, the robot proposes questions in the form of a questionnaire and displays them on the terminal screen.

[1045] Step 7:

[1046] Collecting opinions and feedback

[1047] The user inputs their thoughts and opinions into the device. The input is "user thoughts and opinions" and the output is "collected opinion data." Specifically, the user inputs their thoughts and opinions using a touch screen.

[1048] The terminal sends the user's opinion to the server. The input is "opinion data" and the output is "data to send to the server." Specifically, the input data is sent to the server via an HTTP request.

[1049] Step 8:

[1050] Collecting emotional information

[1051] The robot also uses the emotion engine to collect impressions and opinions after the tour ends, recording the user's emotions as well. The input is "collected impression data and real-time emotion data," and the output is "comprehensive emotion information." Specifically, the robot analyzes the user's facial expressions while they are entering their impressions, and sends that emotion data to the server.

[1052] The server analyzes the collected opinions and impressions, including emotional information. The input is "opinion data and emotional data," and the output is "analysis results." Specifically, it uses machine learning algorithms to analyze user feedback and emotional information.

[1053] Step 9:

[1054] Knowledge

[1055] The server analyzes the collected impressions, opinions, and emotional information and organizes the data. The input is the "analysis results" and the output is "organized data." Specifically, the data is tagged, categorized, and organized.

[1056] The server accumulates useful information in a knowledge base and uses it to improve future tours. The input is "organized data" and the output is "additional data to the knowledge base." Specifically, new entries are added to the knowledge base, and improvements are reflected in the next tour.

[1057] (Application example 2)

[1058] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1059] Conventional food delivery services have faced challenges in improving delivery efficiency and customer satisfaction. In particular, there is a demand for more personalized service by combining the automation of the delivery process with customer emotion recognition. However, current systems lack a mechanism for measuring real-time customer satisfaction and efficiently collecting and utilizing feedback while reducing the workload of delivery personnel.

[1060] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for verifying the identity of participants at the reception terminal, a means for a guide robot to explain the facilities along the tour route, a means for the guide robot to respond to participants' questions, a means for collecting impressions and opinions from participants after the tour ends, a means for analyzing the collected impressions and opinions and storing them in a knowledge base, a means for recognizing participants' emotions using a smartphone, a means for collecting appropriate feedback based on the results of emotion recognition, and a means for transmitting the feedback to the server and storing it in the knowledge base as data for service improvement. This advances automation of the delivery process and enables the provision of efficient, high-quality delivery services. Customer emotion data can be analyzed in real time and quickly reflected in service improvements, which is expected to improve customer satisfaction.

[1061] A "reception terminal" is an electronic device that allows participants to check in and verify their identity.

[1062] "Identity verification" is the process of verifying that participants are legitimately registered individuals using a QR code or ID entry.

[1063] A "guide robot" is a robotic device that automatically moves along the tour route, explaining the facilities and answering participants' questions.

[1064] A "tour route" is a route set up to visit specific facilities or locations in order.

[1065] "Emotion recognition" is the process of analyzing emotions from participants' facial expressions and voice using smartphones or other devices.

[1066] "After the tour" refers to the time or situation after the entire sightseeing route has been completed.

[1067] "Impressions and opinions" collect participants' personal evaluations and thoughts about the tour they experienced.

[1068] A "knowledge base" is a database system that accumulates collected data and information, especially impressions and opinions, and stores the results of analysis.

[1069] "Feedback" refers to information about opinions and feelings collected from participants, which is used to improve the service.

[1070] A "smartphone" is a mobile terminal that has communication capabilities and is equipped with a multi-function processor.

[1071] A "server" is a computer system that processes, stores, and manages data, and interacts with other devices over a network.

[1072] This invention is a system for automating the delivery process and improving customer satisfaction in food delivery services. The system includes a reception terminal, a guide robot, a server, a smartphone, and an emotion engine, and provides high-quality service to participants who join the bot, as well as emotion recognition functions.

[1073] 1. Reception process

[1074] Identity verification

[1075] The terminal waits for the participant to access the reception terminal. The user presses the QR code reading button displayed on the terminal screen to scan the QR code. The terminal analyzes the QR code data, sends it to the server, and compares it with the registered participant data. The server verifies the participant data and returns the comparison result to the terminal. The terminal completes identity verification based on the comparison result and explains the delivery overview to the user.

[1076] 2. Delivery

[1077] Delivery Setup

[1078] The user heads to the designated delivery destination. The navigation system provides the optimal route and real-time guidance. When the delivery location is reached, the user is notified via smartphone.

[1079] 3. Emotion recognition and feedback collection

[1080] Customer Service

[1081] After the delivery is completed, the user uses the smartphone's camera and voice recognition function to analyze the customer's facial expressions and voice. An emotion engine is used to recognize the customer's emotions in real time. Based on the emotion recognition results, the smartphone collects feedback and sends it to the server. The feedback is stored in a knowledge base and used to improve service in future deliveries.

[1082] Hardware and software used

[1083] Hardware

[1084] Smartphone: Check customer delivery information, navigation, camera and voice recognition

[1085] software

[1086] OpenCV: Image processing library (used to handle camera images)

[1087] EmotionRecognizer: A library for recognizing emotions from facial expressions and voice

[1088] requests: A Python library for communicating with servers

[1089] Specific examples

[1090] When a delivery person makes a food delivery, they first check in by scanning a QR code with their smartphone. Then, when they head to the designated delivery destination, the app provides real-time guidance on the optimal route. After the delivery is completed, the smartphone camera captures the customer's facial expression, and an emotion engine analyzes whether the customer is satisfied. The analysis results are sent to the server as feedback and used to improve the service.

[1091] Prompt Sentence Examples

[1092] "You are a food delivery driver. Scan the QR code and confirm the delivery address information. Then, guide the customer to the delivery address with the best route, and recognize the customer's emotions when you arrive at the delivery address. You also collect customer feedback and send it to the server to improve the service."

[1093] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1094] Step 1:

[1095] The reception terminal scans the QR code used by the participant to check in. In this step, the user presses the QR code reading button on the terminal, which then scans the QR code and obtains the QR code data. The obtained data is sent to the server and compared with the registered participant data. The input data is the QR code scan result, and the output data is the comparison result. Through this comparison, the terminal completes participant identity verification.

[1096] Step 2:

[1097] The server analyzes the QR code data sent from the reception terminal and compares it with the registered information in the database. It then returns the comparison results to the terminal. The input data is the scanned data of the QR code, and the output data is the comparison results. This confirms that the participant is a legitimately registered person.

[1098] Step 3:

[1099] The terminal receives the matching results from the server and notifies the user of a delivery summary. The input data is the matching results from the server, and the output data is the notification to the user. This allows the user to check the delivery details.

[1100] Step 4:

[1101] The navigation system provides the optimal route to the delivery destination and guides the user in real time. The user receives real-time navigation instructions on their smartphone. The input data is the address information of the delivery destination, and the output data is the optimal route. This allows the user to reach the delivery destination efficiently.

[1102] Step 5:

[1103] The smartphone notifies the user after the delivery arrives at the destination. This notification informs the customer that the delivery has been completed. The input data is the arrival event at the delivery destination, and the output data is a notification containing the destination information. This allows the customer to prepare to receive the delivery in a timely manner.

[1104] Step 6:

[1105] Using the smartphone's camera and voice recognition function, the system collects the customer's facial expressions and voice when the delivery is complete. The input data is the camera footage and voice data, and the output data is emotion data based on these. The smartphone sends this to the emotion engine and receives the analysis results.

[1106] Step 7:

[1107] The emotion engine analyzes collected video and audio data to recognize customer emotions. The input data is camera video and audio data, and the output data is analyzed emotional information. This makes it possible to evaluate customer satisfaction in real time.

[1108] Step 8:

[1109] Based on the results of emotion recognition, feedback is collected and sent to the server. The input data is the emotion recognition results, and the output data is feedback data. This allows the server to efficiently collect customer feedback.

[1110] Step 9:

[1111] The server stores the collected feedback in a knowledge base and uses it to improve services in the future. The input data is the collected feedback data, and the output data is updated information in the knowledge base. This helps improve customer satisfaction.

[1112] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1113] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1114] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1115] [Third embodiment]

[1116] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1117] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1118] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1119] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1120] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1121] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1122] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1123] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1124] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1125] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1126] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1127] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1128] The present invention is a system including a reception terminal, a guide robot, and a server, which automates the implementation of office tours for participants and provides high-quality guide services.

[1129] 1. Reception process

[1130] Waiting for reception and identity verification

[1131] The terminal waits for the participant to approach the reception terminal.

[1132] The user presses the QR code reader button displayed on the device screen.

[1133] The device scans the user's QR code to verify their identity.

[1134] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[1135] The server is contacted to check against registered participant data.

[1136] Based on the matching results, identity verification is completed.

[1137] Tour overview

[1138] The terminal provides the user with an overview of the tour.

[1139] The terminal displays the sightseeing locations, important points to note, the tour schedule, etc.

[1140] The terminal checks whether the user understands.

[1141] 2. Office tours

[1142] Meeting and tour start

[1143] The user heads to the designated meeting point.

[1144] The robot will wait at the meeting point to welcome participants.

[1145] The robot begins the tour.

[1146] The robot begins to move along the tour route.

[1147] Facility Description

[1148] The robot will provide an explanation as it arrives at each facility along the tour route.

[1149] A description of the facility is provided based on information obtained from the server.

[1150] The robot provides instructions via voice or text.

[1151] Answering questions

[1152] The user inputs a question into the robot by voice or via a terminal.

[1153] The robot analyzes the question and provides an appropriate answer.

[1154] The question is sent to the server and the response data is obtained.

[1155] The robot provides the obtained answer to the user.

[1156] 3. Closing

[1157] Tour End

[1158] The robot ends the tour at the end of the route.

[1159] Ask for opinions

[1160] The robot asks the user for their impressions and opinions of the tour.

[1161] The robot displays an interface on the terminal for inputting impressions.

[1162] The terminal waits for input from the user.

[1163] Information gathering

[1164] The user inputs their thoughts and opinions on the device.

[1165] The terminal transmits the user's opinion to the server.

[1166] Knowledge

[1167] The server analyzes the collected opinions and impressions.

[1168] The server organizes the data and identifies trends.

[1169] The server accumulates useful information as a knowledge base and uses it to improve future tours.

[1170] Specific examples

[1171] Below is an example of a participant taking an office tour.

[1172] 1. Reception process

[1173] The user arrives at the reception terminal and scans the QR code.

[1174] The device analyzes the data in the QR code and verifies the user's identity.

[1175] After verifying your identity, the device will explain the tour outline.

[1176] 2. Office tour

[1177] When the user goes to the designated meeting place, the robot will be waiting.

[1178] The robot moves along the tour route and explains each facility.

[1179] Every time the user asks a question, the robot provides an appropriate answer.

[1180] 3. Closing Process

[1181] At the tour end point, the robot ends the tour.

[1182] The robot asks the user for their thoughts and opinions and has them enter them into a terminal.

[1183] The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[1184] In this way, this system automates all processes from reception to office tour guidance, answering questions, and closing, making it possible to provide a highly efficient, high-quality tour experience.

[1185] The processing flow will be explained below.

[1186] 1. Reception process

[1187] Step 1:

[1188] The terminal waits for the participant to access the reception terminal.

[1189] Step 2:

[1190] The user presses the QR code reader button displayed on the device screen.

[1191] Step 3:

[1192] The device scans the user's QR code and analyzes the data in the QR code.

[1193] Step 4:

[1194] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[1195] Step 5:

[1196] The server checks the participant data and returns the matching results to the terminal.

[1197] Step 6:

[1198] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[1199] 2. Office tours

[1200] Step 1:

[1201] The user heads to the designated meeting point.

[1202] Step 2:

[1203] The robot waits at the meeting point and checks the participants.

[1204] Step 3:

[1205] The robot begins moving along the tour route.

[1206] Step 4:

[1207] Each time the robot arrives at a facility, it provides an explanation of the facility based on information obtained from the server.

[1208] Step 5:

[1209] The user inputs a question into the robot by voice or via a terminal.

[1210] Step 6:

[1211] The robot analyzes the question and sends the question to the server.

[1212] Step 7:

[1213] The server obtains the appropriate answer data for the question and sends it back to the robot.

[1214] Step 8:

[1215] The robot provides the obtained answer to the user.

[1216] 3. Closing

[1217] Step 1:

[1218] The robot ends the tour at the end of the route.

[1219] Step 2:

[1220] The robot asks the user for their impressions and opinions of the tour.

[1221] Step 3:

[1222] The robot displays an interface on the terminal for inputting impressions.

[1223] Step 4:

[1224] The user inputs their thoughts and opinions on the device.

[1225] Step 5:

[1226] The terminal transmits the user's opinion to the server.

[1227] Step 6:

[1228] The server analyzes the collected opinions and impressions and organizes the data.

[1229] Step 7:

[1230] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[1231] Example 1

[1232] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1233] Current office tours require dedicated guides, which poses challenges in terms of efficiency and cost. It is also difficult to respond to participants' diverse questions and opinions on the spot and to gather appropriate feedback. This results in tour quality that is inconsistent or not optimized for each individual participant.

[1234] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1235] In this invention, the server includes means for verifying the identity of participants, means for a guide device moving along the tour route to explain the facility, means for the guide device to respond to questions from participants in cooperation with the server, means for collecting impressions and opinions from participants after the tour ends, and means for saving the collected impressions and opinions in a database, analyzing them with a text mining algorithm, and accumulating them as a knowledge base. This makes it possible to automate highly efficient, high-quality office tours without the need for a dedicated guide, and to appropriately respond to the diverse requests of participants.

[1236] A "reception terminal" is a device used to verify the identity of participants, and is a device that authenticates participants via QR code or ID input.

[1237] A "guide device" is a robot or device that moves automatically along the tour route and has the function of explaining the facility to participants.

[1238] The "server" is a computer system that works in conjunction with the information device and the reception terminal to manage data, collate data, generate responses, and collect and analyze opinions.

[1239] "Identity verification" is the process by which participants are matched with registered information to verify their identity.

[1240] The "tour route" is a specific route along which the guide device travels during an office tour, and is a route along which each facility is visited in sequence.

[1241] The "facility explanation" is an explanation that the guide device provides as it arrives at each facility along the tour route, and is given in the form of audio or text.

[1242] "Response to questions" refers to the guide device working in conjunction with the server to provide appropriate answers to questions posed by participants.

[1243] "Collecting impressions and opinions" refers to the process of receiving feedback from participants after the tour and saving it as data.

[1244] The "knowledge base" is a collection of data that has been collected, analyzed, and stored as useful information to be used to improve future tours.

[1245] The present invention relates to an office tour automation system that includes a reception terminal, a guide device, and a server. This system automates participant authentication, facility explanations, question handling, and feedback collection, providing a highly efficient and high-quality tour experience.

[1246] 1. Reception terminal

[1247] The terminal is a device equipped with a QR code reading function that reads the QR code when the user approaches. The hardware used is a tablet terminal with a built-in QR code scanner or a dedicated reception device. For example, devices running Android or Windows can be used. A library such as Zxing is used to analyze the QR code.

[1248] When a user approaches the reception terminal and presses the QR code reading button on the screen, the terminal scans the QR code and obtains participant information. This information is sent to the server and verified against registered information for identity verification. For example, participants are authenticated by following the instruction to "scan the QR code."

[1249] 2. Guidance device

[1250] The guide device is a robot with autonomous mobility capabilities that explains the facilities while moving along the tour route. The guide device achieves its autonomous movement using ROS (Robot Operating System). It also has speech recognition and synthesis functions, and provides facility explanations and answers questions. A speech synthesis engine such as Google Text-to-Speech is used to explain the facilities.

[1251] For example, when a user approaches a meeting point, a guide device will be waiting and begin the tour. The guide device will explain each facility using prompts such as, "This is our newest office space. Do you have any questions?"

[1252] 3. Server

[1253] The server is a computer system that manages and processes data. The server works in conjunction with the reception terminal and the information device to perform the following tasks:

[1254] Verification of authentication data

[1255] Provision of facility description data

[1256] Generating answers to user questions

[1257] Collecting and analyzing feedback

[1258] As a specific example of how to respond to a question, if a user asks, "How many people can this conference room accommodate?", the guidance device will recognize the question through speech recognition and send it to the server. The server will then obtain the answer data and send it to the guidance device, thereby providing the user with an appropriate answer.

[1259] 4. Collecting feedback and opinions and creating a knowledge base

[1260] After the tour ends, the device displays an interface requesting the user's impressions and opinions. When the user enters their feedback into the device, the data is sent to the server. The server stores the collected opinions in a database and analyzes them using a text mining algorithm. This accumulates useful information as a knowledge base that can be used to improve future tours.

[1261] This system allows participants to enjoy a highly efficient and personalized tour experience, and allows operators to improve the quality of their services based on feedback.

[1262] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1263] Step 1:

[1264] The terminal waits at the reception location for the user to approach. When the user presses the "QR code reading button" displayed on the reception terminal's screen, the terminal switches to QR code scanning mode. Specifically, pressing the button displays a camera preview on the screen, allowing the QR code to be recognized. The user's QR code is acquired as input, and the output after analysis is participant information (name, ID, etc.). This data is then sent to the server.

[1265] Step 2:

[1266] The server compares the received participant information with the existing registration information in the database. The input is the participant information sent from the terminal, and the output is the authentication result (authentication successful or authentication failed). In concrete terms, the server performs a database search and returns the comparison result to the terminal. If authentication is successful, the terminal displays an "Authentication successful" message, and if it fails, it displays an "Authentication failed" message.

[1267] Step 3:

[1268] After successful authentication, the terminal displays an overview of the tour to the user. The input is the authentication result from the server, and the output is a display of the tour overview information. Specifically, the terminal screen displays the tour locations, important points to note, the tour flow, etc. in a presentation format. The user presses the "Next" button to confirm the contents.

[1269] Step 4:

[1270] The user heads to the designated meeting place. When the user arrives at the meeting place, a guide device is waiting. The input is guide information for the meeting place, and the output is the user's movement. In concrete terms, the guide device displays information about the meeting place on the terminal screen and guides the user.

[1271] Step 5:

[1272] The guide device welcomes participants who have gathered at the meeting point and starts moving along the tour route. The input is the gathering state of users at the meeting point, and the output is a signal to start moving. In concrete terms, the robot moves autonomously along a pre-programmed route, stopping at each point.

[1273] Step 6:

[1274] The guide device provides explanations when arriving at each facility. The input is the location information of each point on the tour route, and the output is a description of the facility. In concrete terms, the robot uses facility information obtained from the server to provide explanations in voice and text. For example, it uses prompts such as, "This is our latest office space. Do you have any questions?"

[1275] Step 7:

[1276] When a user asks a question, the guidance device recognizes the question through voice recognition and sends it to the server. The input is the user's question, and the output is answer data. Specifically, the robot analyzes the question using its voice recognition function and sends the question data to the server. The server analyzes the question, generates an answer, and returns it to the robot. The robot then provides the answer to the user.

[1277] Step 8:

[1278] The guide device ends the tour when it reaches the end point of the tour route. The input is the end point information of the tour route, and the output is a notification that the tour has ended. Specifically, it plays a voice message saying, "This is the end of the tour. Thank you for participating."

[1279] Step 9:

[1280] The guide device instructs the user to provide their impressions and opinions, and displays an input interface on the terminal. The input is the state after the tour ends, and the output is a screen display for entering impressions. Specifically, the terminal displays an impression form and prompts the user to enter their impressions.

[1281] Step 10:

[1282] When a user inputs their thoughts or opinions into the device, the device sends the data to the server. The input is the user's feedback, and the output is the data sent to the server. The specific operation is to store the thoughts in a database.

[1283] Step 11:

[1284] The server stores the collected impressions and opinions in a database and analyzes them using a text mining algorithm. The input is user impressions and opinions, and the output is the analysis results and a knowledge base. Specifically, the server classifies and analyzes the data and accumulates useful information as a knowledge base.

[1285] (Application example 1)

[1286] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1287] Conventional customer guidance services in brick-and-mortar stores often rely on human resources, which means that the quality of guidance and consistency of service cannot be guaranteed. Another issue is that when customers ask a variety of questions about the store's products, they are unable to receive appropriate answers immediately. Furthermore, there is a lack of a system for effectively collecting and analyzing customer feedback and using it to improve services. For these reasons, there is a need for efficient, high-quality guidance services in brick-and-mortar stores.

[1288] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1289] In this invention, the server includes a means for verifying the identity of participants at the reception terminal, a means for a guide robot to explain the facilities along the tour route, a means for the guide robot to respond to participants' questions, a means for collecting impressions and opinions from participants after the tour ends, a means for analyzing the collected impressions and opinions and storing them as a knowledge base, an automated means for guiding visitors and a means for controlling the guide robot and introducing products in each section of the physical store, and a means for answering questions using a generative AI model. This enables the physical store to provide automated, high-quality guidance services while also appropriately responding to a variety of questions from visitors. Furthermore, the efficiency of feedback collection and analysis can be improved, which can be used to improve services.

[1290] A "reception terminal" is a device that customers use to check in and verify their identity by scanning a QR code or entering their ID.

[1291] A "participant" is someone who uses the system to receive an office tour or a guided tour of a physical store.

[1292] "Identity verification" is the process of verifying that a system user is a person who has been pre-registered by scanning a QR code or entering an ID.

[1293] A "tour route" indicates a guided course within a specific facility, and is the path along which the robot will guide the visitor.

[1294] A "guide robot" is an autonomous mobile robot that automatically moves along a tour route, such as a physical store or office tour, and provides explanations about the facility.

[1295] "After the tour ends" refers to the time after the entire tour by the guide robot has been completed.

[1296] "Impressions and opinions" refers to user feedback regarding tours and guide services, which is collected by being entered into the terminal.

[1297] A "knowledge base" is a database of knowledge that is analyzed based on collected impressions and opinions and accumulated for future service improvements.

[1298] "Automatic means" refers to a method or mechanism by which a system performs a function automatically without human intervention.

[1299] A "guide robot" is a robot that moves around a physical store and provides guidance to customers by introducing products and answering their questions.

[1300] "Generative AI model" refers to an artificial intelligence model used to generate appropriate answers to user questions.

[1301] A "prompt" is a text input that provides context for the answer an artificial intelligence question-answering model generates.

[1302] This invention automates the guidance service for customers in physical stores by using a system including a reception terminal, guide robot, and server, and provides high-quality explanations and answers to questions. This system comprehensively implements processes from customer check-in to guiding them around the store, explaining products, answering questions, and collecting feedback.

[1303] 1. Reception process

[1304] Waiting for reception and identity verification

[1305] The terminal waits for a customer to approach the reception terminal.

[1306] The user presses the QR code reader button displayed on the device screen.

[1307] The device scans the user's QR code to verify their identity.

[1308] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[1309] The server is contacted to check against registered participant data.

[1310] Based on the matching results, identity verification is completed.

[1311] Guide overview

[1312] The terminal provides the user with an overview of the tour.

[1313] The terminal displays the sightseeing locations, important points to note, the tour schedule, etc.

[1314] The terminal checks whether the user understands.

[1315] 2. Tours using guide robots

[1316] Meeting and tour start

[1317] The user heads to the designated meeting point.

[1318] The guide robot waits at the meeting point to welcome customers.

[1319] The guide robot begins the tour.

[1320] The guide robot begins to move along the tour route.

[1321] Facility Description

[1322] The guide robot provides explanations as it arrives at each section along the tour route.

[1323] Provides descriptions of facilities and products based on information obtained from the server.

[1324] The guide robot provides explanations by voice or text.

[1325] 3. Answering questions

[1326] Accepting questions and providing answers

[1327] The user inputs questions to the guide robot by voice or via a terminal.

[1328] The guide robot analyzes the question and provides an appropriate answer.

[1329] The question is sent to the server and the answer data is obtained using a generative AI model.

[1330] The guide robot provides the obtained answers to the user.

[1331] 4. Closing

[1332] Tour End

[1333] The guide robot ends the tour at the end of the route.

[1334] Ask for opinions

[1335] The guide robot asks the user for their impressions and opinions on the tour.

[1336] The guide robot displays an interface for inputting impressions on the terminal.

[1337] The terminal waits for input from the user.

[1338] Information collection and analysis

[1339] The user inputs their thoughts and opinions on the device.

[1340] The terminal transmits the user's opinion to the server.

[1341] The server analyzes the collected opinions and impressions and stores them as a knowledge base.

[1342] The server organizes the data and identifies trends.

[1343] The server uses the useful information as a knowledge base to improve the next tour.

[1344] Specific examples

[1345] The following is an example of a customer receiving guidance services at a physical store.

[1346] 1. Reception process

[1347] The user arrives at the reception terminal and scans the QR code.

[1348] The device analyzes the data in the QR code and verifies the user's identity.

[1349] After verifying your identity, the device will explain the tour outline.

[1350] 2. Tours using guide robots

[1351] When the user heads to the designated meeting place, a guide robot will be waiting.

[1352] The guide robot moves along the tour route and explains each section.

[1353] Every time the user asks a question, the guide robot provides an appropriate answer.

[1354] 3. Closing Process

[1355] At the tour end point, the guide robot ends the tour.

[1356] The guide robot asks the user for their impressions and opinions and has them enter them into a terminal.

[1357] The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[1358] For example, in response to a frequently asked question such as "What are the new features of this smartphone?", the generative AI model generates an answer using the following prompt sentence:

[1359] Context: This smartphone has a brand new chip, making it even faster than previous models, and it also has a number of new features, including a higher-resolution camera and longer battery life.

[1360] Question: What are the new features of this smartphone?

[1361] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1362] Step 1:

[1363] Waiting for reception and identity verification

[1364] The terminal waits for a customer to approach the reception terminal.

[1365] The user presses the QR code reading button displayed on the device screen to scan the QR code.

[1366] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[1367] The terminal sends the information to the server and checks it against registered participant data.

[1368] The server compares the information with that in the participant database and returns the identity verification results to the terminal.

[1369] The device completes identity verification based on the matching results.

[1370] Input: User's QR code

[1371] Output: Identity verification result

[1372] Step 2:

[1373] Guide overview

[1374] The terminal provides the user with an overview of the tour (places to visit, things to note, tour flow, etc.).

[1375] The terminal confirms through the screen interface whether the user has understood.

[1376] Input: QR code scan information, identity verification results from the server

[1377] Output: Tour overview

[1378] Step 3:

[1379] Meeting and tour start

[1380] The user heads to the designated meeting point.

[1381] The guide robot waits at the meeting point to welcome customers.

[1382] The guide robot begins moving along the tour route.

[1383] Input: Specify the meeting place and time

[1384] Output: Notification of tour start

[1385] Step 4:

[1386] Facility Description

[1387] The guide robot will explain the facilities and products as it arrives at each section along the tour route.

[1388] The guide robot retrieves explanatory information from the server in real time.

[1389] The guide robot provides explanations in voice or text.

[1390] Input: Tour route and section information

[1391] Output: Facility description audio or text

[1392] Step 5:

[1393] Accepting questions and providing answers

[1394] The user inputs questions to the guide robot by voice or via a terminal.

[1395] The guide robot analyzes the question and sends the content to the server.

[1396] The server uses a generative AI model to construct prompts to generate answers to questions.

[1397] The server returns the generated answer to the guide robot.

[1398] The guide robot provides answers to the user.

[1399] Input: User question, prompt

[1400] Output: Answer to the question

[1401] Step 6:

[1402] Tour End

[1403] The guide robot ends the tour at the end of the route.

[1404] Input: End position of tour route

[1405] Output: Notification of tour completion

[1406] Step 7:

[1407] Ask for opinions

[1408] The guide robot asks the user for their impressions and opinions on the tour.

[1409] The guide robot displays an interface on the terminal for inputting impressions.

[1410] The terminal waits for input from the user.

[1411] Input: User state after tour ends

[1412] Output: Display of the impression input screen

[1413] Step 8:

[1414] Information collection and analysis

[1415] The user inputs their thoughts and opinions on the device.

[1416] The terminal transmits the user's opinion to the server.

[1417] The server analyzes the collected opinions and impressions and stores them as a knowledge base.

[1418] The server organizes the data and identifies trends.

[1419] Input: User feedback and opinions

[1420] Output: Knowledge base updates and analysis results

[1421] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1422] The present invention is a system including a reception terminal, a guide robot, a server, and an emotion engine, which automates the implementation of office tours for participants and provides high-quality guide services and the ability to recognize participants' emotions.

[1423] 1. Reception process

[1424] Waiting for reception and identity verification

[1425] The terminal waits for the participant to access the reception terminal.

[1426] The user presses the QR code reader button displayed on the device screen.

[1427] The device scans the user's QR code and analyzes the data in the QR code.

[1428] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[1429] The server checks the participant data and returns the matching results to the terminal.

[1430] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[1431] 2. Office tours

[1432] Meeting and tour start

[1433] The user heads to the designated meeting point.

[1434] The robot waits at the meeting point and checks the participants.

[1435] The robot begins moving along the tour route.

[1436] Facility Description

[1437] Each time the robot arrives at a facility along the tour route, it provides an explanation of the facility based on information obtained from the server.

[1438] The robot provides instructions via voice or text.

[1439] Answering questions

[1440] The user inputs a question into the robot by voice or via a terminal.

[1441] The robot analyzes the question and sends the question to the server.

[1442] The server obtains the appropriate answer data for the question and sends it back to the robot.

[1443] The robot provides the obtained answer to the user.

[1444] emotion recognition

[1445] The robot analyzes the user's voice and facial expressions and recognizes emotions using an emotion engine.

[1446] The robot adjusts its explanations and response methods based on the user's emotions.

[1447] 3. Closing

[1448] Tour End

[1449] The robot ends the tour at the end of the route.

[1450] Ask for opinions

[1451] The robot asks the user for their impressions and opinions of the tour.

[1452] The robot displays an interface on the terminal for inputting impressions.

[1453] Information gathering

[1454] The user inputs their thoughts and opinions on the device.

[1455] The terminal transmits the user's opinion to the server.

[1456] Collecting emotional information

[1457] The robot also uses the emotion engine to collect impressions and opinions after the tour ends, recording the user's emotions as well.

[1458] The server analyzes the collected opinions and impressions, including the emotional information.

[1459] Knowledge

[1460] The server analyzes the collected impressions, opinions and emotional information and organizes the data.

[1461] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[1462] Specific examples

[1463] Below is an example of a participant taking an office tour.

[1464] Reception process

[1465] 1. The user arrives at the reception terminal and scans the QR code.

[1466] 2. The device analyzes the QR code data and verifies the user's identity.

[1467] 3. The terminal will provide an overview of the tour.

[1468] Office tour

[1469] 1. When the user goes to the designated meeting place, the robot will be waiting.

[1470] 2. The robot moves along the tour route and explains each facility.

[1471] 3. Every time the user asks a question, the robot provides an appropriate answer.

[1472] 4. The robot recognizes the user's emotions and adjusts its explanations and response accordingly.

[1473] Closing process

[1474] 1. At the tour end point, the robot ends the tour.

[1475] 2. The robot asks the user for their thoughts and opinions and has them enter them into the terminal.

[1476] 3. The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[1477] This system automates all processes, from reception to office tours, answering questions, recognizing emotions, and closing the deal, making it possible to provide a highly efficient and high-quality tour experience. Furthermore, by recognizing users' emotions and responding accordingly, it is possible to provide a more personalized service.

[1478] The processing flow will be explained below.

[1479] 1. Reception process

[1480] Step 1:

[1481] The terminal waits for the participant to access the reception terminal.

[1482] Step 2:

[1483] The user presses the QR code reader button displayed on the device screen.

[1484] Step 3:

[1485] The device scans the user's QR code and analyzes the data in the QR code.

[1486] Step 4:

[1487] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[1488] Step 5:

[1489] The server checks the participant data and returns the matching results to the terminal.

[1490] Step 6:

[1491] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[1492] 2. Office tours

[1493] Step 1:

[1494] The user heads to the designated meeting point.

[1495] Step 2:

[1496] The robot waits at the meeting point and waits for the user to arrive.

[1497] Step 3:

[1498] The robot begins moving along the tour route.

[1499] Step 4:

[1500] Each time the robot arrives at a facility, it provides an explanation of the facility based on information obtained from the server.

[1501] Step 5:

[1502] The robot analyzes the user's emotions in real time through voice and facial recognition.

[1503] Step 6:

[1504] The user inputs a question into the robot by voice or via a terminal.

[1505] Step 7:

[1506] The robot analyzes the question and sends the question to the server.

[1507] Step 8:

[1508] The server obtains the appropriate answer data for the question and sends it back to the robot.

[1509] Step 9:

[1510] The robot provides the obtained answer to the user.

[1511] Step 10:

[1512] The emotion engine analyzes the user's emotions and adjusts the explanation and response method based on the results.

[1513] 3. Closing

[1514] Step 1:

[1515] The robot ends the tour at the end of the route.

[1516] Step 2:

[1517] The robot asks the user for their impressions and opinions of the tour.

[1518] Step 3:

[1519] The robot displays an interface on the terminal for inputting impressions.

[1520] Step 4:

[1521] The user inputs their thoughts and opinions on the device.

[1522] Step 5:

[1523] The terminal transmits the user's opinion to the server.

[1524] Step 6:

[1525] The emotion engine also analyzes the user's emotions when they enter their thoughts or opinions and records that data.

[1526] Step 7:

[1527] The server analyzes the collected impressions, opinions, and even emotional information.

[1528] Step 8:

[1529] The server organizes the data and identifies trends.

[1530] Step 9:

[1531] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[1532] Example 2

[1533] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1534] Conventional office tours have faced the challenge of efficiently carrying out the entire process from reception to conducting the tour, answering questions, collecting feedback, and then turning it into knowledge. It was also difficult to recognize participants' emotions and respond flexibly based on them, which led to problems with the quality of the tour experience being inconsistent.

[1535] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1536] In this invention, the server includes means for verifying the identity of participants at the reception terminal, means for a guide robot to explain the facilities along the tour route, means for the guide robot to respond to questions from participants, means for collecting impressions and opinions from participants after the tour ends, means for analyzing the collected impressions, opinions, and emotion data and storing them as a knowledge base, means including an emotion engine for recognizing participants' emotions, and means for adjusting response methods based on the recognized emotions. This makes it possible to automate the entire tour process, provide efficient, high-quality services, and further realize personalized responses based on participants' emotions.

[1537] The "reception terminal" is a device where participants enter information upon arrival and verify their identity.

[1538] "Identity verification" refers to the process by which participants prove their identity when joining a tour.

[1539] The "guide robot" is an autonomous machine that moves automatically along the tour route, explains the facilities, and answers participants' questions.

[1540] The "tour route" refers to the route that tour participants follow, guided by a guide robot.

[1541] "Facility explanation" refers to detailed information provided by the guide robot about each facility and point of interest along the tour route.

[1542] "Question handling" is the process in which the guide robot provides information in response to questions from participants.

[1543] "Collecting impressions and opinions" refers to the process of collecting feedback from participants after the tour has ended.

[1544] An "emotion engine" is software or hardware that analyzes participants' voices, facial expressions, etc. and recognizes their emotions.

[1545] "Emotional data" refers to data that contains information about the emotions of participants.

[1546] A "knowledge base" is a database that accumulates and organizes knowledge obtained from collected impressions, opinions, and emotional data.

[1547] "Adjusting responses based on perceived emotions" refers to the procedure of changing the explanations and responses provided depending on the participant's emotional state.

[1548] The present invention is a system that includes a reception terminal, a guide robot, a server, and an emotion engine, and automates the implementation of office tours for participants, providing high-quality guide services and the ability to recognize participants' emotions. Specific examples of this system are shown below.

[1549] First, the reception terminal is a device used by participants when they arrive. The terminal is equipped with a QR code reader and a touch screen, and participants scan their own QR code by operating the QR code reading button displayed on the terminal. The QR code data is sent from the terminal to the server, and the server compares it with the registered participant data. The result of this comparison is sent back to the terminal, and the terminal completes identity verification. This process uses HTTP requests and responses.

[1550] Next, guide robots play a major role in conducting office tours along a tour route. The guide robot waits at a designated meeting point, and after confirming the participants, moves automatically along the tour route. Each time the robot arrives at a facility, it provides explanations in voice or text based on information obtained from the server. When a participant asks a question, the robot analyzes the question and sends it to the server. The server retrieves the appropriate answer from a database and sends it back to the robot. The robot then provides this answer to the participant.

[1551] The robot also has an emotion engine that analyzes voice and facial expressions to recognize participants' emotions. The analyzed emotion data is used to adjust the robot's response. For example, if a participant is excited, the robot will provide more detailed explanations, while if the participant is tired, it will provide a brief explanation.

[1552] After the tour ends, the robot asks participants for their impressions and opinions. Participants enter their impressions and opinions using the input interface displayed on their devices. This data is sent from the devices to a server, and the collected opinions and emotional data is stored in a knowledge base. The server analyzes this data and uses it to improve future tours.

[1553] As a concrete example, detailed explanations can be obtained by inputting the following prompt sentence into the generative AI model:

[1554] Please explain the check-in process for users to join the office tour. For example, please include steps such as waiting at the reception terminal, verifying identity using a QR code, and receiving an explanation of the tour overview.

[1555] An example of a prompt sentence for collecting emotional information and storing it in the knowledge base is as follows:

[1556] Regarding the closing process after the office tour, please explain in detail the process from the robot collecting user opinions to the server storing them in the knowledge base.

[1557] In this way, the present invention provides a system that automates all processes from reception processing to office tours, answering questions, emotion recognition, and closing, making it possible to provide participants with an efficient, high-quality tour experience.

[1558] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1559] Step 1:

[1560] Waiting for reception and identity verification

[1561] The terminal waits for the user to access the reception terminal. The input is "standby state" and the output is "start screen display." Specifically, the terminal displays a screen indicating the standby state and puts the RFID reader and camera into standby state.

[1562] The user presses the QR code reader button displayed on the device screen. The input is "touch the QR code reader button." The output is "start QR code scanning."

[1563] The device scans the user's QR code and analyzes the QR code data. The input is "QR code image data" and the output is "QR code string data." Specifically, the camera reads the QR code and converts it into string data.

[1564] The device sends the QR code data to the server and requests that it be matched with the registered participant data. The input is the "QR code string data" and the output is a "matching request to the server." Specifically, an HTTP request is sent from the device to the server.

[1565] The server checks the participant data and returns the matching results to the device. The input is a "matching request" and the output is a "matching result response." Specifically, it searches the database for participant information and returns the matching results to the device as an HTTP response.

[1566] The terminal completes identity verification based on the matching results and explains an overview of the tour to the user. The input is "matching result" and the output is "identity verification completed and tour overview displayed." Specifically, it displays "identity verification completed," followed by an overview of the tour in voice or text.

[1567] Step 2:

[1568] Meeting and tour start

[1569] The user heads to the designated meeting place. The input is "meeting place instructions" and the output is "user movement." Specifically, the user moves according to the map and instructions of the meeting place displayed on the device screen.

[1570] The robot waits at the meeting point and checks for participants. The input is "Confirmation of participant arrival" and the output is "Preparation to start the tour is complete." Specifically, the robot starts up and goes into standby mode at the meeting point.

[1571] The robot starts moving along the tour route. The input is "tour route instructions" and the output is "movement along the route." Specifically, the robot moves automatically along the pre-set route.

[1572] Step 3:

[1573] Facility Description

[1574] Each time the robot arrives at a facility along the tour route, it provides an explanation of the facility based on information obtained from the server. The input is a "facility information acquisition request" and the output is a "facility explanation." Specifically, when the robot arrives at each point, it sends an explanation request to the server and explains the information it has obtained using a voice device.

[1575] The robot provides explanations by voice or text. The input is "explanatory data" and the output is "explanation by voice or text." Specifically, it provides explanations by voice using the built-in speaker or displays text on the display.

[1576] Step 4:

[1577] Answering questions

[1578] The user inputs questions to the robot by voice or via a terminal. The input is the "user question" and the output is "question data." Specifically, the user either speaks questions directly to the robot or inputs questions into a dedicated tablet.

[1579] The robot analyzes the question and sends it to the server. The input is "question data" and the output is a "question request to the server." Specifically, the robot converts the question into text using a voice recognition system and sends the analysis results to the server.

[1580] The server retrieves the appropriate answer data for the question and sends it back to the robot. The input is a "question request" and the output is "answer data." Specifically, it searches for relevant information from a database and sends it to the robot as an answer.

[1581] The robot provides the obtained answer to the user. The input is "answer data" and the output is "answer to the user." Specifically, the answer is conveyed to the user by voice or text.

[1582] Step 5:

[1583] emotion recognition

[1584] The robot analyzes the user's voice and facial expressions and uses an emotion engine to recognize emotions. The input is "voice data and facial expression data" and the output is "emotion data." Specifically, the robot uses a camera and a voice analyzer to input the user's facial expressions and tone of voice into the emotion engine.

[1585] The robot adjusts its explanations and response methods based on the user's emotions. The input is "emotional data" and the output is "adjusted explanations." Specifically, if the user expresses positive emotions, it provides a bright and detailed explanation, and if the user expresses negative emotions, it provides a concise explanation.

[1586] Step 6:

[1587] Tour end and closing

[1588] The robot ends the tour at the final point of the tour route. The input is "Arrived at final point" and the output is "Announcement that the tour has ended." Specifically, when the robot arrives at the final point, it makes an announcement and ends the tour.

[1589] The robot asks the user for their impressions and opinions of the tour. The input is an "announcement that the tour has ended" and the output is a "request to collect impressions." Specifically, the robot proposes questions in the form of a questionnaire and displays them on the terminal screen.

[1590] Step 7:

[1591] Collecting opinions and feedback

[1592] The user inputs their thoughts and opinions into the device. The input is "user thoughts and opinions" and the output is "collected opinion data." Specifically, the user inputs their thoughts and opinions using a touch screen.

[1593] The terminal sends the user's opinion to the server. The input is "opinion data" and the output is "data to send to the server." Specifically, the input data is sent to the server via an HTTP request.

[1594] Step 8:

[1595] Collecting emotional information

[1596] The robot also uses the emotion engine to collect impressions and opinions after the tour ends, recording the user's emotions as well. The input is "collected impression data and real-time emotion data," and the output is "comprehensive emotion information." Specifically, the robot analyzes the user's facial expressions while they are entering their impressions, and sends that emotion data to the server.

[1597] The server analyzes the collected opinions and impressions, including emotional information. The input is "opinion data and emotional data," and the output is "analysis results." Specifically, it uses machine learning algorithms to analyze user feedback and emotional information.

[1598] Step 9:

[1599] Knowledge

[1600] The server analyzes the collected impressions, opinions, and emotional information and organizes the data. The input is the "analysis results" and the output is "organized data." Specifically, the data is tagged, categorized, and organized.

[1601] The server accumulates useful information in a knowledge base and uses it to improve future tours. The input is "organized data" and the output is "additional data to the knowledge base." Specifically, new entries are added to the knowledge base, and improvements are reflected in the next tour.

[1602] (Application example 2)

[1603] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1604] Conventional food delivery services have faced challenges in improving delivery efficiency and customer satisfaction. In particular, there is a demand for more personalized service by combining the automation of the delivery process with customer emotion recognition. However, current systems lack a mechanism for measuring real-time customer satisfaction and efficiently collecting and utilizing feedback while reducing the workload of delivery personnel.

[1605] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for verifying the identity of participants at the reception terminal, a means for a guide robot to explain the facilities along the tour route, a means for the guide robot to respond to participants' questions, a means for collecting impressions and opinions from participants after the tour ends, a means for analyzing the collected impressions and opinions and storing them in a knowledge base, a means for recognizing participants' emotions using a smartphone, a means for collecting appropriate feedback based on the results of emotion recognition, and a means for transmitting the feedback to the server and storing it in the knowledge base as data for service improvement. This advances automation of the delivery process and enables the provision of efficient, high-quality delivery services. Customer emotion data can be analyzed in real time and quickly reflected in service improvements, which is expected to improve customer satisfaction.

[1606] A "reception terminal" is an electronic device that allows participants to check in and verify their identity.

[1607] "Identity verification" is the process of verifying that participants are legitimately registered individuals using a QR code or ID entry.

[1608] A "guide robot" is a robotic device that automatically moves along the tour route, explaining the facilities and answering participants' questions.

[1609] A "tour route" is a route set up to visit specific facilities or locations in order.

[1610] "Emotion recognition" is the process of analyzing emotions from participants' facial expressions and voice using smartphones or other devices.

[1611] "After the tour" refers to the time or situation after the entire sightseeing route has been completed.

[1612] "Impressions and opinions" collect participants' personal evaluations and thoughts about the tour they experienced.

[1613] A "knowledge base" is a database system that accumulates collected data and information, especially impressions and opinions, and stores the results of analysis.

[1614] "Feedback" refers to information about opinions and feelings collected from participants, which is used to improve the service.

[1615] A "smartphone" is a mobile terminal that has communication capabilities and is equipped with a multi-function processor.

[1616] A "server" is a computer system that processes, stores, and manages data, and interacts with other devices over a network.

[1617] This invention is a system for automating the delivery process and improving customer satisfaction in food delivery services. The system includes a reception terminal, a guide robot, a server, a smartphone, and an emotion engine, and provides high-quality service to participants who join the bot, as well as emotion recognition functions.

[1618] 1. Reception process

[1619] Identity verification

[1620] The terminal waits for the participant to access the reception terminal. The user presses the QR code reading button displayed on the terminal screen to scan the QR code. The terminal analyzes the QR code data, sends it to the server, and compares it with the registered participant data. The server verifies the participant data and returns the comparison result to the terminal. The terminal completes identity verification based on the comparison result and explains the delivery overview to the user.

[1621] 2. Delivery

[1622] Delivery Setup

[1623] The user heads to the designated delivery destination. The navigation system provides the optimal route and real-time guidance. When the delivery location is reached, the user is notified via smartphone.

[1624] 3. Emotion recognition and feedback collection

[1625] Customer Service

[1626] After the delivery is completed, the user uses the smartphone's camera and voice recognition function to analyze the customer's facial expressions and voice. An emotion engine is used to recognize the customer's emotions in real time. Based on the emotion recognition results, the smartphone collects feedback and sends it to the server. The feedback is stored in a knowledge base and used to improve service in future deliveries.

[1627] Hardware and software used

[1628] Hardware

[1629] Smartphone: Check customer delivery information, navigation, camera and voice recognition

[1630] software

[1631] OpenCV: Image processing library (used to handle camera images)

[1632] EmotionRecognizer: A library for recognizing emotions from facial expressions and voice

[1633] requests: A Python library for communicating with servers

[1634] Specific examples

[1635] When a delivery person makes a food delivery, they first check in by scanning a QR code with their smartphone. Then, when they head to the designated delivery destination, the app provides real-time guidance on the optimal route. After the delivery is completed, the smartphone camera captures the customer's facial expression, and an emotion engine analyzes whether the customer is satisfied. The analysis results are sent to the server as feedback and used to improve the service.

[1636] Prompt Sentence Examples

[1637] "You are a food delivery driver. Scan the QR code and confirm the delivery address information. Then, guide the customer to the delivery address with the best route, and recognize the customer's emotions when you arrive at the delivery address. You also collect customer feedback and send it to the server to improve the service."

[1638] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1639] Step 1:

[1640] The reception terminal scans the QR code used by the participant to check in. In this step, the user presses the QR code reading button on the terminal, which then scans the QR code and obtains the QR code data. The obtained data is sent to the server and compared with the registered participant data. The input data is the QR code scan result, and the output data is the comparison result. Through this comparison, the terminal completes participant identity verification.

[1641] Step 2:

[1642] The server analyzes the QR code data sent from the reception terminal and compares it with the registered information in the database. It then returns the comparison results to the terminal. The input data is the scanned data of the QR code, and the output data is the comparison results. This confirms that the participant is a legitimately registered person.

[1643] Step 3:

[1644] The terminal receives the matching results from the server and notifies the user of a delivery summary. The input data is the matching results from the server, and the output data is the notification to the user. This allows the user to check the delivery details.

[1645] Step 4:

[1646] The navigation system provides the optimal route to the delivery destination and guides the user in real time. The user receives real-time navigation instructions on their smartphone. The input data is the address information of the delivery destination, and the output data is the optimal route. This allows the user to reach the delivery destination efficiently.

[1647] Step 5:

[1648] The smartphone notifies the user after the delivery arrives at the destination. This notification informs the customer that the delivery has been completed. The input data is the arrival event at the delivery destination, and the output data is a notification containing the destination information. This allows the customer to prepare to receive the delivery in a timely manner.

[1649] Step 6:

[1650] Using the smartphone's camera and voice recognition function, the system collects the customer's facial expressions and voice when the delivery is complete. The input data is the camera footage and voice data, and the output data is emotion data based on these. The smartphone sends this to the emotion engine and receives the analysis results.

[1651] Step 7:

[1652] The emotion engine analyzes collected video and audio data to recognize customer emotions. The input data is camera video and audio data, and the output data is analyzed emotional information. This makes it possible to evaluate customer satisfaction in real time.

[1653] Step 8:

[1654] Based on the results of emotion recognition, feedback is collected and sent to the server. The input data is the emotion recognition results, and the output data is feedback data. This allows the server to efficiently collect customer feedback.

[1655] Step 9:

[1656] The server stores the collected feedback in a knowledge base and uses it to improve services in the future. The input data is the collected feedback data, and the output data is updated information in the knowledge base. This helps improve customer satisfaction.

[1657] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1658] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1659] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1660] [Fourth embodiment]

[1661] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1662] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1663] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1664] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1665] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1666] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1667] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1668] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1669] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1670] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1671] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1672] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1673] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1674] The present invention is a system including a reception terminal, a guide robot, and a server, which automates the implementation of office tours for participants and provides high-quality guide services.

[1675] 1. Reception process

[1676] Waiting for reception and identity verification

[1677] The terminal waits for the participant to approach the reception terminal.

[1678] The user presses the QR code reader button displayed on the device screen.

[1679] The device scans the user's QR code to verify their identity.

[1680] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[1681] The server is contacted to check against registered participant data.

[1682] Based on the matching results, identity verification is completed.

[1683] Tour overview

[1684] The terminal provides the user with an overview of the tour.

[1685] The terminal displays the sightseeing locations, important points to note, the tour schedule, etc.

[1686] The terminal checks whether the user understands.

[1687] 2. Office tours

[1688] Meeting and tour start

[1689] The user heads to the designated meeting point.

[1690] The robot will wait at the meeting point to welcome participants.

[1691] The robot begins the tour.

[1692] The robot begins to move along the tour route.

[1693] Facility Description

[1694] The robot will provide an explanation as it arrives at each facility along the tour route.

[1695] A description of the facility is provided based on information obtained from the server.

[1696] The robot provides instructions via voice or text.

[1697] Answering questions

[1698] The user inputs a question into the robot by voice or via a terminal.

[1699] The robot analyzes the question and provides an appropriate answer.

[1700] The question is sent to the server and the response data is obtained.

[1701] The robot provides the obtained answer to the user.

[1702] 3. Closing

[1703] Tour End

[1704] The robot ends the tour at the end of the route.

[1705] Ask for opinions

[1706] The robot asks the user for their impressions and opinions of the tour.

[1707] The robot displays an interface on the terminal for inputting impressions.

[1708] The terminal waits for input from the user.

[1709] Information gathering

[1710] The user inputs their thoughts and opinions on the device.

[1711] The terminal transmits the user's opinion to the server.

[1712] Knowledge

[1713] The server analyzes the collected opinions and impressions.

[1714] The server organizes the data and identifies trends.

[1715] The server accumulates useful information as a knowledge base and uses it to improve future tours.

[1716] Specific examples

[1717] Below is an example of a participant taking an office tour.

[1718] 1. Reception process

[1719] The user arrives at the reception terminal and scans the QR code.

[1720] The device analyzes the data in the QR code and verifies the user's identity.

[1721] After verifying your identity, the device will explain the tour outline.

[1722] 2. Office tour

[1723] When the user goes to the designated meeting place, the robot will be waiting.

[1724] The robot moves along the tour route and explains each facility.

[1725] Every time the user asks a question, the robot provides an appropriate answer.

[1726] 3. Closing Process

[1727] At the tour end point, the robot ends the tour.

[1728] The robot asks the user for their thoughts and opinions and has them enter them into a terminal.

[1729] The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[1730] In this way, this system automates all processes from reception to office tour guidance, answering questions, and closing, making it possible to provide a highly efficient, high-quality tour experience.

[1731] The processing flow will be explained below.

[1732] 1. Reception process

[1733] Step 1:

[1734] The terminal waits for the participant to access the reception terminal.

[1735] Step 2:

[1736] The user presses the QR code reader button displayed on the device screen.

[1737] Step 3:

[1738] The device scans the user's QR code and analyzes the data in the QR code.

[1739] Step 4:

[1740] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[1741] Step 5:

[1742] The server checks the participant data and returns the matching results to the terminal.

[1743] Step 6:

[1744] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[1745] 2. Office tours

[1746] Step 1:

[1747] The user heads to the designated meeting point.

[1748] Step 2:

[1749] The robot waits at the meeting point and checks the participants.

[1750] Step 3:

[1751] The robot begins moving along the tour route.

[1752] Step 4:

[1753] Each time the robot arrives at a facility, it provides an explanation of the facility based on information obtained from the server.

[1754] Step 5:

[1755] The user inputs a question into the robot by voice or via a terminal.

[1756] Step 6:

[1757] The robot analyzes the question and sends the question to the server.

[1758] Step 7:

[1759] The server obtains the appropriate answer data for the question and sends it back to the robot.

[1760] Step 8:

[1761] The robot provides the obtained answer to the user.

[1762] 3. Closing

[1763] Step 1:

[1764] The robot ends the tour at the end of the route.

[1765] Step 2:

[1766] The robot asks the user for their impressions and opinions of the tour.

[1767] Step 3:

[1768] The robot displays an interface on the terminal for inputting impressions.

[1769] Step 4:

[1770] The user inputs their thoughts and opinions on the device.

[1771] Step 5:

[1772] The terminal transmits the user's opinion to the server.

[1773] Step 6:

[1774] The server analyzes the collected opinions and impressions and organizes the data.

[1775] Step 7:

[1776] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[1777] Example 1

[1778] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1779] Current office tours require dedicated guides, which poses challenges in terms of efficiency and cost. It is also difficult to respond to participants' diverse questions and opinions on the spot and to gather appropriate feedback. This results in tour quality that is inconsistent or not optimized for each individual participant.

[1780] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1781] In this invention, the server includes means for verifying the identity of participants, means for a guide device moving along the tour route to explain the facility, means for the guide device to respond to questions from participants in cooperation with the server, means for collecting impressions and opinions from participants after the tour ends, and means for saving the collected impressions and opinions in a database, analyzing them with a text mining algorithm, and accumulating them as a knowledge base. This makes it possible to automate highly efficient, high-quality office tours without the need for a dedicated guide, and to appropriately respond to the diverse requests of participants.

[1782] A "reception terminal" is a device used to verify the identity of participants, and is a device that authenticates participants via QR code or ID input.

[1783] A "guide device" is a robot or device that moves automatically along the tour route and has the function of explaining the facility to participants.

[1784] The "server" is a computer system that works in conjunction with the information device and the reception terminal to manage data, collate data, generate responses, and collect and analyze opinions.

[1785] "Identity verification" is the process by which participants are matched with registered information to verify their identity.

[1786] The "tour route" is a specific route along which the guide device travels during an office tour, and is a route along which each facility is visited in sequence.

[1787] The "facility explanation" is an explanation that the guide device provides as it arrives at each facility along the tour route, and is given in the form of audio or text.

[1788] "Response to questions" refers to the guide device working in conjunction with the server to provide appropriate answers to questions posed by participants.

[1789] "Collecting impressions and opinions" refers to the process of receiving feedback from participants after the tour and saving it as data.

[1790] The "knowledge base" is a collection of data that has been collected, analyzed, and stored as useful information to be used to improve future tours.

[1791] The present invention relates to an office tour automation system that includes a reception terminal, a guide device, and a server. This system automates participant authentication, facility explanations, question handling, and feedback collection, providing a highly efficient and high-quality tour experience.

[1792] 1. Reception terminal

[1793] The terminal is a device equipped with a QR code reading function that reads the QR code when the user approaches. The hardware used is a tablet terminal with a built-in QR code scanner or a dedicated reception device. For example, devices running Android or Windows can be used. A library such as Zxing is used to analyze the QR code.

[1794] When a user approaches the reception terminal and presses the QR code reading button on the screen, the terminal scans the QR code and obtains participant information. This information is sent to the server and verified against registered information for identity verification. For example, participants are authenticated by following the instruction to "scan the QR code."

[1795] 2. Guidance device

[1796] The guide device is a robot with autonomous mobility capabilities that explains the facilities while moving along the tour route. The guide device achieves its autonomous movement using ROS (Robot Operating System). It also has speech recognition and synthesis functions, and provides facility explanations and answers questions. A speech synthesis engine such as Google Text-to-Speech is used to explain the facilities.

[1797] For example, when a user approaches a meeting point, a guide device will be waiting and begin the tour. The guide device will explain each facility using prompts such as, "This is our newest office space. Do you have any questions?"

[1798] 3. Server

[1799] The server is a computer system that manages and processes data. The server works in conjunction with the reception terminal and the information device to perform the following tasks:

[1800] Verification of authentication data

[1801] Provision of facility description data

[1802] Generating answers to user questions

[1803] Collecting and analyzing feedback

[1804] As a specific example of how to respond to a question, if a user asks, "How many people can this conference room accommodate?", the guidance device will recognize the question through speech recognition and send it to the server. The server will then obtain the answer data and send it to the guidance device, thereby providing the user with an appropriate answer.

[1805] 4. Collecting feedback and opinions and creating a knowledge base

[1806] After the tour ends, the device displays an interface requesting the user's impressions and opinions. When the user enters their feedback into the device, the data is sent to the server. The server stores the collected opinions in a database and analyzes them using a text mining algorithm. This accumulates useful information as a knowledge base that can be used to improve future tours.

[1807] This system allows participants to enjoy a highly efficient and personalized tour experience, and allows operators to improve the quality of their services based on feedback.

[1808] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1809] Step 1:

[1810] The terminal waits at the reception location for the user to approach. When the user presses the "QR code reading button" displayed on the reception terminal's screen, the terminal switches to QR code scanning mode. Specifically, pressing the button displays a camera preview on the screen, allowing the QR code to be recognized. The user's QR code is acquired as input, and the output after analysis is participant information (name, ID, etc.). This data is then sent to the server.

[1811] Step 2:

[1812] The server compares the received participant information with the existing registration information in the database. The input is the participant information sent from the terminal, and the output is the authentication result (authentication successful or authentication failed). In concrete terms, the server performs a database search and returns the comparison result to the terminal. If authentication is successful, the terminal displays an "Authentication successful" message, and if it fails, it displays an "Authentication failed" message.

[1813] Step 3:

[1814] After successful authentication, the terminal displays an overview of the tour to the user. The input is the authentication result from the server, and the output is a display of the tour overview information. Specifically, the terminal screen displays the tour locations, important points to note, the tour flow, etc. in a presentation format. The user presses the "Next" button to confirm the contents.

[1815] Step 4:

[1816] The user heads to the designated meeting place. When the user arrives at the meeting place, a guide device is waiting. The input is guide information for the meeting place, and the output is the user's movement. In concrete terms, the guide device displays information about the meeting place on the terminal screen and guides the user.

[1817] Step 5:

[1818] The guide device welcomes participants who have gathered at the meeting point and starts moving along the tour route. The input is the gathering state of users at the meeting point, and the output is a signal to start moving. In concrete terms, the robot moves autonomously along a pre-programmed route, stopping at each point.

[1819] Step 6:

[1820] The guide device provides explanations when arriving at each facility. The input is the location information of each point on the tour route, and the output is a description of the facility. In concrete terms, the robot uses facility information obtained from the server to provide explanations in voice and text. For example, it uses prompts such as, "This is our latest office space. Do you have any questions?"

[1821] Step 7:

[1822] When a user asks a question, the guidance device recognizes the question through voice recognition and sends it to the server. The input is the user's question, and the output is answer data. Specifically, the robot analyzes the question using its voice recognition function and sends the question data to the server. The server analyzes the question, generates an answer, and returns it to the robot. The robot then provides the answer to the user.

[1823] Step 8:

[1824] The guide device ends the tour when it reaches the end point of the tour route. The input is the end point information of the tour route, and the output is a notification that the tour has ended. Specifically, it plays a voice message saying, "This is the end of the tour. Thank you for participating."

[1825] Step 9:

[1826] The guide device instructs the user to provide their impressions and opinions, and displays an input interface on the terminal. The input is the state after the tour ends, and the output is a screen display for entering impressions. Specifically, the terminal displays an impression form and prompts the user to enter their impressions.

[1827] Step 10:

[1828] When a user inputs their thoughts or opinions into the device, the device sends the data to the server. The input is the user's feedback, and the output is the data sent to the server. The specific operation is to store the thoughts in a database.

[1829] Step 11:

[1830] The server stores the collected impressions and opinions in a database and analyzes them using a text mining algorithm. The input is user impressions and opinions, and the output is the analysis results and a knowledge base. Specifically, the server classifies and analyzes the data and accumulates useful information as a knowledge base.

[1831] (Application example 1)

[1832] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1833] Conventional customer guidance services in brick-and-mortar stores often rely on human resources, which means that the quality of guidance and consistency of service cannot be guaranteed. Another issue is that when customers ask a variety of questions about the store's products, they are unable to receive appropriate answers immediately. Furthermore, there is a lack of a system for effectively collecting and analyzing customer feedback and using it to improve services. For these reasons, there is a need for efficient, high-quality guidance services in brick-and-mortar stores.

[1834] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1835] In this invention, the server includes a means for verifying the identity of participants at the reception terminal, a means for a guide robot to explain the facilities along the tour route, a means for the guide robot to respond to participants' questions, a means for collecting impressions and opinions from participants after the tour ends, a means for analyzing the collected impressions and opinions and storing them as a knowledge base, an automated means for guiding visitors and a means for controlling the guide robot and introducing products in each section of the physical store, and a means for answering questions using a generative AI model. This enables the physical store to provide automated, high-quality guidance services while also appropriately responding to a variety of questions from visitors. Furthermore, the efficiency of feedback collection and analysis can be improved, which can be used to improve services.

[1836] A "reception terminal" is a device that customers use to check in and verify their identity by scanning a QR code or entering their ID.

[1837] A "participant" is someone who uses the system to receive an office tour or a guided tour of a physical store.

[1838] "Identity verification" is the process of verifying that a system user is a person who has been pre-registered by scanning a QR code or entering an ID.

[1839] A "tour route" indicates a guided course within a specific facility, and is the path along which the robot will guide the visitor.

[1840] A "guide robot" is an autonomous mobile robot that automatically moves along a tour route, such as a physical store or office tour, and provides explanations about the facility.

[1841] "After the tour ends" refers to the time after the entire tour by the guide robot has been completed.

[1842] "Impressions and opinions" refers to user feedback regarding tours and guide services, which is collected by being entered into the terminal.

[1843] A "knowledge base" is a database of knowledge that is analyzed based on collected impressions and opinions and accumulated for future service improvements.

[1844] "Automatic means" refers to a method or mechanism by which a system performs a function automatically without human intervention.

[1845] A "guide robot" is a robot that moves around a physical store and provides guidance to customers by introducing products and answering their questions.

[1846] "Generative AI model" refers to an artificial intelligence model used to generate appropriate answers to user questions.

[1847] A "prompt" is a text input that provides context for the answer an artificial intelligence question-answering model generates.

[1848] This invention automates the guidance service for customers in physical stores by using a system including a reception terminal, guide robot, and server, and provides high-quality explanations and answers to questions. This system comprehensively implements processes from customer check-in to guiding them around the store, explaining products, answering questions, and collecting feedback.

[1849] 1. Reception process

[1850] Waiting for reception and identity verification

[1851] The terminal waits for a customer to approach the reception terminal.

[1852] The user presses the QR code reader button displayed on the device screen.

[1853] The device scans the user's QR code to verify their identity.

[1854] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[1855] The server is contacted to check against registered participant data.

[1856] Based on the matching results, identity verification is completed.

[1857] Guide overview

[1858] The terminal provides the user with an overview of the tour.

[1859] The terminal displays the sightseeing locations, important points to note, the tour schedule, etc.

[1860] The terminal checks whether the user understands.

[1861] 2. Tours using guide robots

[1862] Meeting and tour start

[1863] The user heads to the designated meeting point.

[1864] The guide robot waits at the meeting point to welcome customers.

[1865] The guide robot begins the tour.

[1866] The guide robot begins to move along the tour route.

[1867] Facility Description

[1868] The guide robot provides explanations as it arrives at each section along the tour route.

[1869] Provides descriptions of facilities and products based on information obtained from the server.

[1870] The guide robot provides explanations by voice or text.

[1871] 3. Answering questions

[1872] Accepting questions and providing answers

[1873] The user inputs questions to the guide robot by voice or via a terminal.

[1874] The guide robot analyzes the question and provides an appropriate answer.

[1875] The question is sent to the server and the answer data is obtained using a generative AI model.

[1876] The guide robot provides the obtained answers to the user.

[1877] 4. Closing

[1878] Tour End

[1879] The guide robot ends the tour at the end of the route.

[1880] Ask for opinions

[1881] The guide robot asks the user for their impressions and opinions on the tour.

[1882] The guide robot displays an interface for inputting impressions on the terminal.

[1883] The terminal waits for input from the user.

[1884] Information collection and analysis

[1885] The user inputs their thoughts and opinions on the device.

[1886] The terminal transmits the user's opinion to the server.

[1887] The server analyzes the collected opinions and impressions and stores them as a knowledge base.

[1888] The server organizes the data and identifies trends.

[1889] The server uses the useful information as a knowledge base to improve the next tour.

[1890] Specific examples

[1891] The following is an example of a customer receiving guidance services at a physical store.

[1892] 1. Reception process

[1893] The user arrives at the reception terminal and scans the QR code.

[1894] The device analyzes the data in the QR code and verifies the user's identity.

[1895] After verifying your identity, the device will explain the tour outline.

[1896] 2. Tours using guide robots

[1897] When the user heads to the designated meeting place, a guide robot will be waiting.

[1898] The guide robot moves along the tour route and explains each section.

[1899] Every time the user asks a question, the guide robot provides an appropriate answer.

[1900] 3. Closing Process

[1901] At the tour end point, the guide robot ends the tour.

[1902] The guide robot asks the user for their impressions and opinions and has them enter them into a terminal.

[1903] The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[1904] For example, in response to a frequently asked question such as "What are the new features of this smartphone?", the generative AI model generates an answer using the following prompt sentence:

[1905] Context: This smartphone has a brand new chip, making it even faster than previous models, and it also has a number of new features, including a higher-resolution camera and longer battery life.

[1906] Question: What are the new features of this smartphone?

[1907] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1908] Step 1:

[1909] Waiting for reception and identity verification

[1910] The terminal waits for a customer to approach the reception terminal.

[1911] The user presses the QR code reading button displayed on the device screen to scan the QR code.

[1912] The device analyzes the data in the QR code and confirms the participant's information (name, ID, etc.).

[1913] The terminal sends the information to the server and checks it against registered participant data.

[1914] The server compares the information with that in the participant database and returns the identity verification results to the terminal.

[1915] The device completes identity verification based on the matching results.

[1916] Input: User's QR code

[1917] Output: Identity verification result

[1918] Step 2:

[1919] Guide overview

[1920] The terminal provides the user with an overview of the tour (places to visit, things to note, tour flow, etc.).

[1921] The terminal confirms through the screen interface whether the user has understood.

[1922] Input: QR code scan information, identity verification results from the server

[1923] Output: Tour overview

[1924] Step 3:

[1925] Meeting and tour start

[1926] The user heads to the designated meeting point.

[1927] The guide robot waits at the meeting point to welcome customers.

[1928] The guide robot begins moving along the tour route.

[1929] Input: Specify the meeting place and time

[1930] Output: Notification of tour start

[1931] Step 4:

[1932] Facility Description

[1933] The guide robot will explain the facilities and products as it arrives at each section along the tour route.

[1934] The guide robot retrieves explanatory information from the server in real time.

[1935] The guide robot provides explanations in voice or text.

[1936] Input: Tour route and section information

[1937] Output: Facility description audio or text

[1938] Step 5:

[1939] Accepting questions and providing answers

[1940] The user inputs questions to the guide robot by voice or via a terminal.

[1941] The guide robot analyzes the question and sends the content to the server.

[1942] The server uses a generative AI model to construct prompts to generate answers to questions.

[1943] The server returns the generated answer to the guide robot.

[1944] The guide robot provides answers to the user.

[1945] Input: User question, prompt

[1946] Output: Answer to the question

[1947] Step 6:

[1948] Tour End

[1949] The guide robot ends the tour at the end of the route.

[1950] Input: End position of tour route

[1951] Output: Notification of tour completion

[1952] Step 7:

[1953] Ask for opinions

[1954] The guide robot asks the user for their impressions and opinions on the tour.

[1955] The guide robot displays an interface on the terminal for inputting impressions.

[1956] The terminal waits for input from the user.

[1957] Input: User state after tour ends

[1958] Output: Display of the impression input screen

[1959] Step 8:

[1960] Information collection and analysis

[1961] The user inputs their thoughts and opinions on the device.

[1962] The terminal transmits the user's opinion to the server.

[1963] The server analyzes the collected opinions and impressions and stores them as a knowledge base.

[1964] The server organizes the data and identifies trends.

[1965] Input: User feedback and opinions

[1966] Output: Knowledge base updates and analysis results

[1967] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1968] The present invention is a system including a reception terminal, a guide robot, a server, and an emotion engine, which automates the implementation of office tours for participants and provides high-quality guide services and the ability to recognize participants' emotions.

[1969] 1. Reception process

[1970] Waiting for reception and identity verification

[1971] The terminal waits for the participant to access the reception terminal.

[1972] The user presses the QR code reader button displayed on the device screen.

[1973] The device scans the user's QR code and analyzes the data in the QR code.

[1974] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[1975] The server checks the participant data and returns the matching results to the terminal.

[1976] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[1977] 2. Office tours

[1978] Meeting and tour start

[1979] The user heads to the designated meeting point.

[1980] The robot waits at the meeting point and checks the participants.

[1981] The robot begins moving along the tour route.

[1982] Facility Description

[1983] Each time the robot arrives at a facility along the tour route, it provides an explanation of the facility based on information obtained from the server.

[1984] The robot provides instructions via voice or text.

[1985] Answering questions

[1986] The user inputs a question into the robot by voice or via a terminal.

[1987] The robot analyzes the question and sends the question to the server.

[1988] The server obtains the appropriate answer data for the question and sends it back to the robot.

[1989] The robot provides the obtained answer to the user.

[1990] emotion recognition

[1991] The robot analyzes the user's voice and facial expressions and recognizes emotions using an emotion engine.

[1992] The robot adjusts its explanations and response methods based on the user's emotions.

[1993] 3. Closing

[1994] Tour End

[1995] The robot ends the tour at the end of the route.

[1996] Ask for opinions

[1997] The robot asks the user for their impressions and opinions of the tour.

[1998] The robot displays an interface on the terminal for inputting impressions.

[1999] Information gathering

[2000] The user inputs their thoughts and opinions on the device.

[2001] The terminal transmits the user's opinion to the server.

[2002] Collecting emotional information

[2003] The robot also uses the emotion engine to collect impressions and opinions after the tour ends, recording the user's emotions as well.

[2004] The server analyzes the collected opinions and impressions, including the emotional information.

[2005] Knowledge

[2006] The server analyzes the collected impressions, opinions and emotional information and organizes the data.

[2007] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[2008] Specific examples

[2009] Below is an example of a participant taking an office tour.

[2010] Reception process

[2011] 1. The user arrives at the reception terminal and scans the QR code.

[2012] 2. The device analyzes the QR code data and verifies the user's identity.

[2013] 3. The terminal will provide an overview of the tour.

[2014] Office tour

[2015] 1. When the user goes to the designated meeting place, the robot will be waiting.

[2016] 2. The robot moves along the tour route and explains each facility.

[2017] 3. Every time the user asks a question, the robot provides an appropriate answer.

[2018] 4. The robot recognizes the user's emotions and adjusts its explanations and response accordingly.

[2019] Closing process

[2020] 1. At the tour end point, the robot ends the tour.

[2021] 2. The robot asks the user for their thoughts and opinions and has them enter them into the terminal.

[2022] 3. The terminal sends the collected opinions to the server, which analyzes them and stores them in a knowledge base.

[2023] This system automates all processes, from reception to office tours, answering questions, recognizing emotions, and closing the deal, making it possible to provide a highly efficient and high-quality tour experience. Furthermore, by recognizing users' emotions and responding accordingly, it is possible to provide a more personalized service.

[2024] The processing flow will be explained below.

[2025] 1. Reception process

[2026] Step 1:

[2027] The terminal waits for the participant to access the reception terminal.

[2028] Step 2:

[2029] The user presses the QR code reader button displayed on the device screen.

[2030] Step 3:

[2031] The device scans the user's QR code and analyzes the data in the QR code.

[2032] Step 4:

[2033] The device sends the QR code data to the server and requests that it be matched with the registered participant data.

[2034] Step 5:

[2035] The server checks the participant data and returns the matching results to the terminal.

[2036] Step 6:

[2037] The terminal completes identity verification based on the matching results and provides the user with an overview of the tour.

[2038] 2. Office tours

[2039] Step 1:

[2040] The user heads to the designated meeting point.

[2041] Step 2:

[2042] The robot waits at the meeting point and waits for the user to arrive.

[2043] Step 3:

[2044] The robot begins moving along the tour route.

[2045] Step 4:

[2046] Each time the robot arrives at a facility, it provides an explanation of the facility based on information obtained from the server.

[2047] Step 5:

[2048] The robot analyzes the user's emotions in real time through voice and facial recognition.

[2049] Step 6:

[2050] The user inputs a question into the robot by voice or via a terminal.

[2051] Step 7:

[2052] The robot analyzes the question and sends the question to the server.

[2053] Step 8:

[2054] The server obtains the appropriate answer data for the question and sends it back to the robot.

[2055] Step 9:

[2056] The robot provides the obtained answer to the user.

[2057] Step 10:

[2058] The emotion engine analyzes the user's emotions and adjusts the explanation and response method based on the results.

[2059] 3. Closing

[2060] Step 1:

[2061] The robot ends the tour at the end of the route.

[2062] Step 2:

[2063] The robot asks the user for their impressions and opinions of the tour.

[2064] Step 3:

[2065] The robot displays an interface on the terminal for inputting impressions.

[2066] Step 4:

[2067] The user inputs their thoughts and opinions on the device.

[2068] Step 5:

[2069] The terminal transmits the user's opinion to the server.

[2070] Step 6:

[2071] The emotion engine also analyzes the user's emotions when they enter their thoughts or opinions and records that data.

[2072] Step 7:

[2073] The server analyzes the collected impressions, opinions, and even emotional information.

[2074] Step 8:

[2075] The server organizes the data and identifies trends.

[2076] Step 9:

[2077] The server accumulates useful information in a knowledge base and uses it to improve future tours.

[2078] Example 2

[2079] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2080] Conventional office tours have faced the challenge of efficiently carrying out the entire process from reception to conducting the tour, answering questions, collecting feedback, and then turning it into knowledge. It was also difficult to recognize participants' emotions and respond flexibly based on them, which led to problems with the quality of the tour experience being inconsistent.

[2081] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2082] In this invention, the server includes means for verifying the identity of participants at the reception terminal, means for a guide robot to explain the facilities along the tour route, means for the guide robot to respond to questions from participants, means for collecting impressions and opinions from participants after the tour ends, means for analyzing the collected impressions, opinions, and emotion data and storing them as a knowledge base, means including an emotion engine for recognizing participants' emotions, and means for adjusting response methods based on the recognized emotions. This makes it possible to automate the entire tour process, provide efficient, high-quality services, and further realize personalized responses based on participants' emotions.

[2083] The "reception terminal" is a device where participants enter information upon arrival and verify their identity.

[2084] "Identity verification" refers to the process by which participants prove their identity when joining a tour.

[2085] The "guide robot" is an autonomous machine that moves automatically along the tour route, explains the facilities, and answers participants' questions.

[2086] The "tour route" refers to the route that tour participants follow, guided by a guide robot.

[2087] "Facility explanation" refers to detailed information provided by the guide robot about each facility and point of interest along the tour route.

[2088] "Question handling" is the process in which the guide robot provides information in response to questions from participants.

[2089] "Collecting impressions and opinions" refers to the process of collecting feedback from participants after the tour has ended.

[2090] An "emotion engine" is software or hardware that analyzes participants' voices, facial expressions, etc. and recognizes their emotions.

[2091] "Emotional data" refers to data that contains information about the emotions of participants.

[2092] A "knowledge base" is a database that accumulates and organizes knowledge obtained from collected impressions, opinions, and emotional data.

[2093] "Adjusting responses based on perceived emotions" refers to the procedure of changing the explanations and responses provided depending on the participant's emotional state.

[2094] The present invention is a system that includes a reception terminal, a guide robot, a server, and an emotion engine, and automates the implementation of office tours for participants, providing high-quality guide services and the ability to recognize participants' emotions. Specific examples of this system are shown below.

[2095] First, the reception terminal is a device used by participants when they arrive. The terminal is equipped with a QR code reader and a touch screen, and participants scan their own QR code by operating the QR code reading button displayed on the terminal. The QR code data is sent from the terminal to the server, and the server compares it with the registered participant data. The result of this comparison is sent back to the terminal, and the terminal completes identity verification. This process uses HTTP requests and responses.

[2096] Next, guide robots play a major role in conducting office tours along a tour route. The guide robot waits at a designated meeting point, and after confirming the participants, moves automatically along the tour route. Each time the robot arrives at a facility, it provides explanations in voice or text based on information obtained from the server. When a participant asks a question, the robot analyzes the question and sends it to the server. The server retrieves the appropriate answer from a database and sends it back to the robot. The robot then provides this answer to the participant.

[2097] The robot also has an emotion engine that analyzes voice and facial expressions to recognize participants' emotions. The analyzed emotion data is used to adjust the robot's response. For example, if a participant is excited, the robot will provide more detailed explanations, while if the participant is tired, it will provide a brief explanation.

[2098] After the tour ends, the robot asks participants for their impressions and opinions. Participants enter their impressions and opinions using the input interface displayed on their devices. This data is sent from the devices to a server, and the collected opinions and emotional data is stored in a knowledge base. The server analyzes this data and uses it to improve future tours.

[2099] As a concrete example, detailed explanations can be obtained by inputting the following prompt sentence into the generative AI model:

[2100] Please explain the check-in process for users to join the office tour. For example, please include steps such as waiting at the reception terminal, verifying identity using a QR code, and receiving an explanation of the tour overview.

[2101] An example of a prompt sentence for collecting emotional information and storing it in the knowledge base is as follows:

[2102] Regarding the closing process after the office tour, please explain in detail the process from the robot collecting user opinions to the server storing them in the knowledge base.

[2103] In this way, the present invention provides a system that automates all processes from reception processing to office tours, answering questions, emotion recognition, and closing, making it possible to provide participants with an efficient, high-quality tour experience.

[2104] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2105] Step 1:

[2106] Waiting for reception and identity verification

[2107] The terminal waits for the user to access the reception terminal. The input is "standby state" and the output is "start screen display." Specifically, the terminal displays a screen indicating the standby state and puts the RFID reader and camera into standby state.

[2108] The user presses the QR code reader button displayed on the device screen. The input is "touch the QR code reader button." The output is "start QR code scanning."

[2109] The device scans the user's QR code and analyzes the QR code data. The input is "QR code image data" and the output is "QR code string data." Specifically, the camera reads the QR code and converts it into string data.

[2110] The device sends the QR code data to the server and requests that it be matched with the registered participant data. The input is the "QR code string data" and the output is a "matching request to the server." Specifically, an HTTP request is sent from the device to the server.

[2111] The server checks the participant data and returns the matching results to the device. The input is a "matching request" and the output is a "matching result response." Specifically, it searches the database for participant information and returns the matching results to the device as an HTTP response.

[2112] The terminal completes identity verification based on the matching results and explains an overview of the tour to the user. The input is "matching result" and the output is "identity verification completed and tour overview displayed." Specifically, it displays "identity verification completed," followed by an overview of the tour in voice or text.

[2113] Step 2:

[2114] Meeting and tour start

[2115] The user heads to the designated meeting place. The input is "meeting place instructions" and the output is "user movement." Specifically, the user moves according to the map and instructions of the meeting place displayed on the device screen.

[2116] The robot waits at the meeting point and checks for participants. The input is "Confirmation of participant arrival" and the output is "Preparation to start the tour is complete." Specifically, the robot starts up and goes into standby mode at the meeting point.

[2117] The robot starts moving along the tour route. The input is "tour route instructions" and the output is "movement along the route." Specifically, the robot moves automatically along the pre-set route.

[2118] Step 3:

[2119] Facility Description

[2120] Each time the robot arrives at a facility along the tour route, it provides an explanation of the facility based on information obtained from the server. The input is a "facility information acquisition request" and the output is a "facility explanation." Specifically, when the robot arrives at each point, it sends an explanation request to the server and explains the information it has obtained using a voice device.

[2121] The robot provides explanations by voice or text. The input is "explanatory data" and the output is "explanation by voice or text." Specifically, it provides explanations by voice using the built-in speaker or displays text on the display.

[2122] Step 4:

[2123] Answering questions

[2124] The user inputs questions to the robot by voice or via a terminal. The input is the "user question" and the output is "question data." Specifically, the user either speaks questions directly to the robot or inputs questions into a dedicated tablet.

[2125] The robot analyzes the question and sends it to the server. The input is "question data" and the output is a "question request to the server." Specifically, the robot converts the question into text using a voice recognition system and sends the analysis results to the server.

[2126] The server retrieves the appropriate answer data for the question and sends it back to the robot. The input is a "question request" and the output is "answer data." Specifically, it searches for relevant information from a database and sends it to the robot as an answer.

[2127] The robot provides the obtained answer to the user. The input is "answer data" and the output is "answer to the user." Specifically, the answer is conveyed to the user by voice or text.

[2128] Step 5:

[2129] emotion recognition

[2130] The robot analyzes the user's voice and facial expressions and uses an emotion engine to recognize emotions. The input is "voice data and facial expression data" and the output is "emotion data." Specifically, the robot uses a camera and a voice analyzer to input the user's facial expressions and tone of voice into the emotion engine.

[2131] The robot adjusts its explanations and response methods based on the user's emotions. The input is "emotional data" and the output is "adjusted explanations." Specifically, if the user expresses positive emotions, it provides a bright and detailed explanation, and if the user expresses negative emotions, it provides a concise explanation.

[2132] Step 6:

[2133] Tour end and closing

[2134] The robot ends the tour at the final point of the tour route. The input is "Arrived at final point" and the output is "Announcement that the tour has ended." Specifically, when the robot arrives at the final point, it makes an announcement and ends the tour.

[2135] The robot asks the user for their impressions and opinions of the tour. The input is an "announcement that the tour has ended" and the output is a "request to collect impressions." Specifically, the robot proposes questions in the form of a questionnaire and displays them on the terminal screen.

[2136] Step 7:

[2137] Collecting opinions and feedback

[2138] The user inputs their thoughts and opinions into the device. The input is "user thoughts and opinions" and the output is "collected opinion data." Specifically, the user inputs their thoughts and opinions using a touch screen.

[2139] The terminal sends the user's opinion to the server. The input is "opinion data" and the output is "data to send to the server." Specifically, the input data is sent to the server via an HTTP request.

[2140] Step 8:

[2141] Collecting emotional information

[2142] The robot also uses the emotion engine to collect impressions and opinions after the tour ends, recording the user's emotions as well. The input is "collected impression data and real-time emotion data," and the output is "comprehensive emotion information." Specifically, the robot analyzes the user's facial expressions while they are entering their impressions, and sends that emotion data to the server.

[2143] The server analyzes the collected opinions and impressions, including emotional information. The input is "opinion data and emotional data," and the output is "analysis results." Specifically, it uses machine learning algorithms to analyze user feedback and emotional information.

[2144] Step 9:

[2145] Knowledge

[2146] The server analyzes the collected impressions, opinions, and emotional information and organizes the data. The input is the "analysis results" and the output is "organized data." Specifically, the data is tagged, categorized, and organized.

[2147] The server accumulates useful information in a knowledge base and uses it to improve future tours. The input is "organized data" and the output is "additional data to the knowledge base." Specifically, new entries are added to the knowledge base, and improvements are reflected in the next tour.

[2148] (Application example 2)

[2149] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2150] Conventional food delivery services have faced challenges in improving delivery efficiency and customer satisfaction. In particular, there is a demand for more personalized service by combining the automation of the delivery process with customer emotion recognition. However, current systems lack a mechanism for measuring real-time customer satisfaction and efficiently collecting and utilizing feedback while reducing the workload of delivery personnel.

[2151] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for verifying the identity of participants at the reception terminal, a means for a guide robot to explain the facilities along the tour route, a means for the guide robot to respond to participants' questions, a means for collecting impressions and opinions from participants after the tour ends, a means for analyzing the collected impressions and opinions and storing them in a knowledge base, a means for recognizing participants' emotions using a smartphone, a means for collecting appropriate feedback based on the results of emotion recognition, and a means for transmitting the feedback to the server and storing it in the knowledge base as data for service improvement. This advances automation of the delivery process and enables the provision of efficient, high-quality delivery services. Customer emotion data can be analyzed in real time and quickly reflected in service improvements, which is expected to improve customer satisfaction.

[2152] A "reception terminal" is an electronic device that allows participants to check in and verify their identity.

[2153] "Identity verification" is the process of verifying that participants are legitimately registered individuals using a QR code or ID entry.

[2154] A "guide robot" is a robotic device that automatically moves along the tour route, explaining the facilities and answering participants' questions.

[2155] A "tour route" is a route set up to visit specific facilities or locations in order.

[2156] "Emotion recognition" is the process of analyzing emotions from participants' facial expressions and voice using smartphones or other devices.

[2157] "After the tour" refers to the time or situation after the entire sightseeing route has been completed.

[2158] "Impressions and opinions" collect participants' personal evaluations and thoughts about the tour they experienced.

[2159] A "knowledge base" is a database system that accumulates collected data and information, especially impressions and opinions, and stores the results of analysis.

[2160] "Feedback" refers to information about opinions and feelings collected from participants, which is used to improve the service.

[2161] A "smartphone" is a mobile terminal that has communication capabilities and is equipped with a multi-function processor.

[2162] A "server" is a computer system that processes, stores, and manages data, and interacts with other devices over a network.

[2163] This invention is a system for automating the delivery process and improving customer satisfaction in food delivery services. The system includes a reception terminal, a guide robot, a server, a smartphone, and an emotion engine, and provides high-quality service to participants who join the bot, as well as emotion recognition functions.

[2164] 1. Reception process

[2165] Identity verification

[2166] The terminal waits for the participant to access the reception terminal. The user presses the QR code reading button displayed on the terminal screen to scan the QR code. The terminal analyzes the QR code data, sends it to the server, and compares it with the registered participant data. The server verifies the participant data and returns the comparison result to the terminal. The terminal completes identity verification based on the comparison result and explains the delivery overview to the user.

[2167] 2. Delivery

[2168] Delivery Setup

[2169] The user heads to the designated delivery destination. The navigation system provides the optimal route and real-time guidance. When the delivery location is reached, the user is notified via smartphone.

[2170] 3. Emotion recognition and feedback collection

[2171] Customer Service

[2172] After the delivery is completed, the user uses the smartphone's camera and voice recognition function to analyze the customer's facial expressions and voice. An emotion engine is used to recognize the customer's emotions in real time. Based on the emotion recognition results, the smartphone collects feedback and sends it to the server. The feedback is stored in a knowledge base and used to improve service in future deliveries.

[2173] Hardware and software used

[2174] Hardware

[2175] Smartphone: Check customer delivery information, navigation, camera and voice recognition

[2176] software

[2177] OpenCV: Image processing library (used to handle camera images)

[2178] EmotionRecognizer: A library for recognizing emotions from facial expressions and voice

[2179] requests: A Python library for communicating with servers

[2180] Specific examples

[2181] When a delivery person makes a food delivery, they first check in by scanning a QR code with their smartphone. Then, when they head to the designated delivery destination, the app provides real-time guidance on the optimal route. After the delivery is completed, the smartphone camera captures the customer's facial expression, and an emotion engine analyzes whether the customer is satisfied. The analysis results are sent to the server as feedback and used to improve the service.

[2182] Prompt Sentence Examples

[2183] "You are a food delivery driver. Scan the QR code and confirm the delivery address information. Then, guide the customer to the delivery address with the best route, and recognize the customer's emotions when you arrive at the delivery address. You also collect customer feedback and send it to the server to improve the service."

[2184] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2185] Step 1:

[2186] The reception terminal scans the QR code used by the participant to check in. In this step, the user presses the QR code reading button on the terminal, which then scans the QR code and obtains the QR code data. The obtained data is sent to the server and compared with the registered participant data. The input data is the QR code scan result, and the output data is the comparison result. Through this comparison, the terminal completes participant identity verification.

[2187] Step 2:

[2188] The server analyzes the QR code data sent from the reception terminal and compares it with the registered information in the database. It then returns the comparison results to the terminal. The input data is the scanned data of the QR code, and the output data is the comparison results. This confirms that the participant is a legitimately registered person.

[2189] Step 3:

[2190] The terminal receives the matching results from the server and notifies the user of a delivery summary. The input data is the matching results from the server, and the output data is the notification to the user. This allows the user to check the delivery details.

[2191] Step 4:

[2192] The navigation system provides the optimal route to the delivery destination and guides the user in real time. The user receives real-time navigation instructions on their smartphone. The input data is the address information of the delivery destination, and the output data is the optimal route. This allows the user to reach the delivery destination efficiently.

[2193] Step 5:

[2194] The smartphone notifies the user after the delivery arrives at the destination. This notification informs the customer that the delivery has been completed. The input data is the arrival event at the delivery destination, and the output data is a notification containing the destination information. This allows the customer to prepare to receive the delivery in a timely manner.

[2195] Step 6:

[2196] Using the smartphone's camera and voice recognition function, the system collects the customer's facial expressions and voice when the delivery is complete. The input data is the camera footage and voice data, and the output data is emotion data based on these. The smartphone sends this to the emotion engine and receives the analysis results.

[2197] Step 7:

[2198] The emotion engine analyzes collected video and audio data to recognize customer emotions. The input data is camera video and audio data, and the output data is analyzed emotional information. This makes it possible to evaluate customer satisfaction in real time.

[2199] Step 8:

[2200] Based on the results of emotion recognition, feedback is collected and sent to the server. The input data is the emotion recognition results, and the output data is feedback data. This allows the server to efficiently collect customer feedback.

[2201] Step 9:

[2202] The server stores the collected feedback in a knowledge base and uses it to improve services in the future. The input data is the collected feedback data, and the output data is updated information in the knowledge base. This helps improve customer satisfaction.

[2203] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2204] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2205] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2206] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2207] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2208] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2209] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2210] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2211] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2212] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2213] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2214] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2215] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2216] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2217] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2218] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2219] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2220] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2221] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2222] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2223] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2224] The following is further disclosed regarding the above embodiment.

[2225] (Claim 1)

[2226] A means for verifying the identity of participants at the reception terminal;

[2227] A guide robot will explain the facilities along the tour route,

[2228] A means for the guide robot to respond to questions from participants,

[2229] A means of collecting impressions and opinions from participants after the tour has ended,

[2230] A means to analyze the collected impressions and opinions and store them as a knowledge base,

[2231] A system including:

[2232] (Claim 2)

[2233] The system according to claim 1, further comprising a means for performing the identity verification by inputting a QR code or an ID.

[2234] (Claim 3)

[2235] 2. The system according to claim 1, wherein the guide robot has a means for explaining the facility while automatically moving along the tour route.

[2236] "Example 1"

[2237] (Claim 1)

[2238] A means for verifying the identity of participants at the reception terminal;

[2239] a means for a guide device moving along a tour route to provide an explanation of the facility;

[2240] a means for the guide device to respond to questions from participants in cooperation with the server;

[2241] A means of collecting impressions and opinions from participants after the tour has ended,

[2242] The collected impressions and opinions are stored in a database, analyzed using text mining algorithms, and accumulated as a knowledge base.

[2243] A system including:

[2244] (Claim 2)

[2245] The system according to claim 1, further comprising a means for performing the identity verification by inputting a QR code or an ID.

[2246] (Claim 3)

[2247] 2. The system according to claim 1, wherein the guide device has means for explaining the facility while automatically moving along the tour route.

[2248] "Application Example 1"

[2249] (Claim 1)

[2250] A means for verifying the identity of participants at the reception terminal;

[2251] A guide robot will explain the facilities along the tour route,

[2252] A means for the guide robot to respond to questions from participants,

[2253] A means of collecting impressions and opinions from participants after the tour has ended,

[2254] A means to analyze the collected impressions and opinions and store them as a knowledge base,

[2255] an automatic means for guiding customers and a means for controlling a guide robot to introduce products in each section of the physical store;

[2256] a means for answering questions using a generative AI model; and

[2257] A system including:

[2258] (Claim 2)

[2259] The system according to claim 1, further comprising a means for performing the identity verification by inputting a QR code or an ID.

[2260] (Claim 3)

[2261] The system according to claim 1, wherein the guide robot has a means for explaining the facilities while automatically moving along the tour route and providing appropriate answers using prompt sentences generated by a generative AI model.

[2262] "Example 2: Combining Emotion Engines"

[2263] (Claim 1)

[2264] A means for verifying the identity of participants at the reception terminal;

[2265] A guide robot will explain the facilities along the tour route,

[2266] A means for the guide robot to respond to questions from participants,

[2267] A means of collecting impressions and opinions from participants after the tour has ended,

[2268] A means for analyzing the collected impressions, opinions, and emotional data and storing it as a knowledge base;

[2269] means including an emotion engine for recognizing emotions of participants;

[2270] a means of adjusting responses based on perceived emotions;

[2271] A system including:

[2272] (Claim 2)

[2273] The system according to claim 1, further comprising a means for performing the identity verification by inputting a QR code or an ID.

[2274] (Claim 3)

[2275] 2. The system according to claim 1, wherein the guide robot has a means for explaining the facility while automatically moving along the tour route.

[2276] "Application example 2 when combining emotion engines"

[2277] (Claim 1)

[2278] A means for verifying the identity of participants at the reception terminal;

[2279] A guide robot will explain the facilities along the tour route,

[2280] A means for the guide robot to respond to questions from participants,

[2281] A means of collecting impressions and opinions from participants after the tour has ended,

[2282] A means to analyze the collected impressions and opinions and store them as a knowledge base,

[2283] A method for recognizing participants' emotions using a smartphone;

[2284] A means for collecting appropriate feedback based on the results of emotion recognition;

[2285] A means for transm...

Claims

1. A means for verifying the identity of participants at the reception terminal; A guide robot will explain the facilities along the tour route, A means for the guide robot to respond to questions from participants, A means of collecting impressions and opinions from participants after the tour has ended, A means to analyze the collected impressions and opinions and store them as a knowledge base, A system including:

2. The system according to claim 1, further comprising means for performing the identity verification by inputting a QR code or an ID.

3. 2. The system according to claim 1, wherein the guide robot has a means for explaining the facility while automatically moving along the tour route.

Images