System

Abstract

To provide a system according to an embodiment that estimates the emotion of a customer and grasps evaluation on a service in real time.SOLUTION: A system according to an embodiment comprises a collection unit, an estimation unit, a determination unit, and an output unit. The collection unit collects data from sensors installed on table seats. The estimation unit estimates the emotions of customers at the table seats on the basis of the data collected by the collection unit. The determination unit determines the customers' evaluation on a service on the basis of the emotions estimated by the estimation unit. The output unit outputs a result of the determination made by the determination unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, it is difficult to grasp customer sentiment and evaluations of services in real time, leaving room for improvement.

[0005] The system according to the embodiment aims to estimate the emotions of customers and grasp their evaluations of services in real time. [Means for solving the problem]

[0006] The system according to the embodiment includes a collection unit, an estimation unit, a determination unit, and an output unit. The collection unit collects data from sensors installed at each table seat. The estimation unit estimates the emotion of a customer at each table seat based on the data collected by the collection unit. The determination unit determines the customer's evaluation of the service based on the emotion estimated by the estimation unit. The output unit outputs the determination result by the determination unit. Effect of the Invention

[0007] The system according to the embodiment can estimate the emotions of customers and grasp their evaluations of services in real time. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The system according to the embodiment of the present invention is a system that collects data from sensors installed at each table and grasps the customer's emotions in real time. This system collects data from sensors installed at each table and estimates the customer's emotions based on the collected data. Based on the estimated emotions, the customer's evaluation of the service is determined. For example, the emotions within a certain time from the serving of food are recorded as an evaluation of the food, and the emotions from the time of sitting down to leaving the seat are recorded as an overall satisfaction evaluation. Furthermore, if the negative emotions are large, a notification is issued to check whether the customer is dissatisfied or has any trouble. The judgment result is output through the output unit and provided to the wait staff in real time. This allows the wait staff to grasp the customer's satisfaction in real time and take necessary measures quickly. For example, the system includes a collection unit that collects data from sensors installed at each table, an estimation unit that estimates the customer's emotions based on the collected data, a judgment unit that judges the customer's evaluation of the service based on the estimated emotions, and an output unit that outputs the judgment result. This allows the system to grasp the customer's emotions in real time and judge the customer's evaluation of the service.

[0029] The system according to the embodiment includes a collection unit, an estimation unit, a determination unit, and an output unit. The collection unit collects data from a sensor installed at each table seat. The collection unit can collect data such as facial expressions, tone of voice, voice volume, and movement of customers using, for example, a camera sensor, a microphone sensor, a motion sensor, and the like. For example, the camera sensor captures the facial expressions of customers in real time and collects facial expression data. The microphone sensor captures the tone of voice and voice volume of customers in real time and collects voice data. The motion sensor captures the movement of customers in real time and collects movement data. The estimation unit estimates the emotion of the customer based on the data collected by the collection unit. The estimation unit analyzes the collected data and estimates the emotion of the customer using, for example, AI. For example, the AI ​​analyzes the facial expression data and estimates the emotion from the facial expression of the customer. The voice data is analyzed and the emotion is estimated from the tone of voice and voice volume of the customer. The movement data is analyzed and the emotion is estimated from the movement of the customer. The determination unit determines the customer's evaluation of the service based on the emotion estimated by the estimation unit. The judgment unit, for example, uses AI to analyze the estimated emotions based on evaluation criteria and judges the customer's evaluation of the service. For example, it analyzes emotions within a certain time from when the food is served and judges the evaluation of the food. It analyzes emotions from when the customer sits down to when they leave and judges the overall level of satisfaction. If there are a lot of negative emotions, it judges the possibility of dissatisfaction or trouble. The output unit outputs the judgment result by the judgment unit. The output unit, for example, uses AI to notify the customer service staff of the judgment result in real time. For example, it displays the judgment result to the customer service staff via a web application or mobile application. This allows the system to grasp the customer's emotions in real time and judge their evaluation of the service.

[0030] The collection unit collects data from sensors installed at each table. The collection unit can collect data such as facial expressions, tone of voice, voice volume, and movements of customers using, for example, a camera sensor, a microphone sensor, a motion sensor, and the like. Specifically, the camera sensor can capture high-resolution video in real time and capture subtle changes in the customer's facial expressions. This provides data for detailed analysis of emotions such as smiles, surprise, and anger. The microphone sensor uses a highly sensitive microphone with a noise canceling function to accurately capture the tone and volume of the customer's voice while eliminating ambient noise. This makes it possible to capture changes in emotions from the tone and strength of the customer's voice. The motion sensor uses 3D motion capture technology to record the customer's body movements and postures in detail. For example, it is possible to infer from the customer's movements whether the customer is relaxed, tense, or dissatisfied. These sensor data are centrally managed by the collection unit and transmitted to a central database in real time. In addition, the collection unit can dynamically adjust the frequency and resolution of data collection, allowing for flexible responses according to specific situations and conditions. For example, when a customer performs a specific action, the frame rate of the camera can be increased to obtain detailed images, allowing the collection unit to collect data efficiently and effectively and improving the performance of the entire system.

[0031] The estimation unit estimates the customer's emotions based on the data collected by the collection unit. The estimation unit, for example, uses AI to analyze the collected data and estimate the customer's emotions. Specifically, the AI ​​uses deep learning technology to analyze facial expression data and estimate emotions from the customer's facial expressions. For example, subtle changes in facial expressions, such as the degree of smiling and the depth of wrinkles between the eyebrows, are captured, and emotions such as joy, anger, and surprise are estimated with high accuracy. When analyzing voice data, a voice recognition technology is combined with an emotion analysis algorithm to estimate emotions from the customer's tone of voice and voice volume. For example, changes in the tone, pitch, and volume of the voice are analyzed to estimate whether the customer is relaxed, excited, or dissatisfied. When analyzing movement data, emotions are estimated from the customer's movements and postures based on motion capture data. For example, if the customer moves frequently, it indicates tension or anxiety, and if the customer has a relaxed posture, it indicates satisfaction. This allows the estimation unit to analyze the collected data from multiple angles and estimate the customer's emotions with high accuracy. Furthermore, the estimation unit can utilize past data and statistical information to learn patterns of customer emotional fluctuations and perform more accurate emotion estimation. For example, it can more accurately estimate current emotions based on the emotions a specific customer has shown in the past. This enables the estimation unit to handle not only real-time emotion estimation, but also long-term trend analysis of emotion fluctuations, improving the reliability and accuracy of the entire system.

[0032] The determination unit determines the customer's evaluation of the service based on the emotion estimated by the estimation unit. The determination unit, for example, uses AI to analyze the estimated emotion based on the evaluation criteria and determines the customer's evaluation of the service. Specifically, the AI ​​compares the estimated emotion data with the evaluation criteria to quantify the customer's satisfaction or dissatisfaction. For example, the emotion within a certain time from the serving of the food is analyzed to determine the evaluation of the food. If the customer smiles immediately after receiving the food, it is determined that the customer has a high evaluation, and conversely, if the customer shows a dissatisfied expression, it is determined that the customer has a low evaluation. The emotion from when the customer sits down to when the customer leaves the seat is analyzed to determine the overall satisfaction. For example, if the customer stays for a long time in a relaxed posture, it is determined that the customer has a high level of satisfaction. If the negative emotion is large, it is determined that there is a possibility of dissatisfaction or trouble. For example, if the customer frequently shows a dissatisfied expression or has a low tone of voice, it is determined that the customer is dissatisfied with the service. This allows the determination unit to determine the evaluation of the service with high accuracy based on the customer's emotion data. Furthermore, the determination unit can dynamically adjust the evaluation criteria by utilizing past evaluation data and statistical information. For example, the evaluation trends during a specific time period or day of the week are analyzed, and the evaluation criteria are optimized based on the results. This allows the evaluation unit to always provide highly accurate evaluation decisions based on the latest information, thereby improving the quality of service.

[0033] The output unit outputs the judgment result by the judgment unit. The output unit notifies the customer service staff of the judgment result in real time, for example, using AI. Specifically, the judgment result is displayed to the customer service staff through a web application or a mobile application. For example, a notification is sent to the customer service staff's smartphone or tablet to display the customer's evaluation and emotional state in real time. This allows the customer service staff to immediately grasp the customer's state and take appropriate action. For example, if a customer shows dissatisfaction, the customer service staff can respond quickly and take action to solve the problem. Also, if a customer shows a high evaluation, the customer service staff can consider an approach to provide even better service based on that information. Furthermore, the output unit also notifies the manager or owner of the judgment result and provides data for improving the quality of service. For example, the manager can monitor the evaluation status of each table in real time and adjust the staff placement and service content as necessary. As a result, the output unit provides information for the customer service staff and the manager to take prompt and appropriate action based on the customer evaluation data, and can improve the quality of service. Furthermore, the output unit can accumulate past evaluation data and perform analysis for long-term service improvement. For example, the evaluation trend in a specific period is analyzed, and improvements to the service are identified based on the results. This enables the output section to contribute not only to real-time notifications but also to long-term service improvements, maximizing the effectiveness of the entire system.

[0034] The determination unit can determine the customer's satisfaction with the service based on the emotions at each stage. The determination unit determines the customer's satisfaction with the service based on, for example, specific criteria for emotions at each stage. For example, the determination unit places emphasis on the emotions at the time of serving the food, thereby improving the evaluation of the food. The determination unit places emphasis on the emotions at the time of sitting down, thereby reflecting the initial impression in the evaluation. The determination unit places emphasis on the emotions at the time of leaving the seat, thereby reflecting the overall satisfaction in the evaluation. This makes it possible to determine the customer's satisfaction in detail based on the emotions at each stage. Some or all of the above-mentioned processing in the determination unit may be performed using, for example, AI, or may be performed without using AI. For example, the determination unit can input emotion data at each stage to AI, which can then determine the satisfaction level.

[0035] The determination unit can determine the customer's satisfaction with the service based on the emotions at each stage in the food and beverage service. The determination unit determines the customer's satisfaction with the service, for example, based on specific criteria for emotions at each stage in the food and beverage service. For example, the determination unit places emphasis on the emotions at the time of serving food, thereby improving the evaluation of the food. The determination unit places emphasis on the emotions at the time of sitting down, thereby reflecting the initial impression in the evaluation. The determination unit places emphasis on the emotions at the time of leaving the seat, thereby reflecting the overall satisfaction in the evaluation. This makes it possible to determine the customer's satisfaction in detail based on the emotions at each stage in the food and beverage service. Some or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input emotion data at each stage into AI, which can then determine the satisfaction level.

[0036] The determination unit has a different specific weight for each service stage, and can determine the customer's satisfaction with the service based on a satisfaction score obtained based on the emotion and weight at each stage. For example, the emotion at the time of serving is emphasized to increase the evaluation of the food. The emotion at the time of sitting is emphasized to reflect the initial impression in the evaluation. The emotion at the time of leaving the seat is emphasized to reflect the overall satisfaction in the evaluation. In this way, by setting different weights for each service stage, a more accurate satisfaction evaluation is possible. Some or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the emotion data and weight at each stage to AI, which can calculate the satisfaction score.

[0037] The estimation unit includes a detection processing unit that specifically detects the customer's facial expression, tone of voice, voice volume, and movement based on the data collected by the collection unit. The detection processing unit, for example, uses AI to analyze the collected data and detect the customer's facial expression, tone of voice, voice volume, and movement. For example, the AI ​​analyzes facial expression data and detects the customer's facial expression. It analyzes voice data and detects the customer's tone of voice and voice volume. It analyzes movement data and detects the customer's movement. As a result, the accuracy of emotion estimation is improved by detecting the customer's facial expression, tone of voice, voice volume, and movement. A part or all of the above-mentioned processing in the detection processing unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the detection processing unit inputs the collected data to the generation AI, and the generation AI can detect the facial expression, tone of voice, voice volume, and movement.

[0038] The estimation unit includes an estimation processing unit that estimates the customer's emotions based on the facial expression, tone of voice, voice volume, and movement of the customer detected by the detection processing unit. The estimation processing unit, for example, uses AI to analyze the detected data and estimate the customer's emotions. For example, the AI ​​analyzes facial expression data and estimates emotions from the customer's facial expression. It analyzes voice data and estimates emotions from the customer's tone of voice and voice volume. It analyzes movement data and estimates emotions from the customer's movement. This enables more accurate emotion estimation by estimating emotions based on the detected data. A part or all of the above-mentioned processing in the estimation processing unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the estimation processing unit inputs the detected data to the generation AI, which can estimate the emotions.

[0039] The system includes a notification unit that notifies a responsible customer service staff member when the level of evaluation up to a specific stage in the service is below a threshold. The notification unit, for example, uses AI to analyze the evaluation data and issues a notification when the level is below the threshold. For example, the AI ​​analyzes the evaluation data and notifies the customer service staff member when the level of evaluation is below the threshold. This allows for a quick response when the evaluation is low. Some or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the evaluation data into the generation AI and issue a notification when the generation AI is below the threshold.

[0040] The determination unit can tally up the customer satisfaction level for each customer service staff member in real time, and the output unit can provide the tallying result by the determination unit to each customer service staff member in real time. The determination unit uses AI to tally up the evaluation data of each customer service staff member and calculate the satisfaction level in real time. For example, the AI ​​analyzes the evaluation data of each customer service staff member and calculates a satisfaction score. The output unit provides the tallying result by the determination unit to each customer service staff member in real time. The output unit uses AI to notify the customer service staff member of the tallying result in real time. For example, the output unit displays the tallying result to the customer service staff member through a web application or a mobile application. This makes it possible to grasp the satisfaction level of each customer service staff member in real time and to respond quickly. Some or all of the above-mentioned processes in the determination unit and the output unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the determination unit can input the evaluation data to the generation AI, and the generation AI can calculate the satisfaction level. The output unit can input the tallying result to the generation AI, and the generation AI can notify.

[0041] The determination unit can determine the customer's evaluation based on the customer's past emotion estimation history. The determination unit uses AI to analyze past emotion data and reflect it in the current evaluation. For example, the AI ​​analyzes past emotion data, understands the customer's emotion fluctuation pattern, and reflects it in the current evaluation. This enables a more accurate evaluation based on the past emotion history. Some or all of the above-mentioned processing in the determination unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the determination unit can input past emotion data to the generation AI, and the generation AI can determine the evaluation.

[0042] The system includes a generation unit that compares the emotions of customers with the action logs of the service staff and generates a report showing areas for improvement. The generation unit uses AI to analyze the emotion data and the action logs and extract areas for improvement. For example, the AI ​​compares the emotion data of customers with the action logs of the service staff and identifies areas for improvement in the service. In this way, by comparing the emotions of customers with the action logs, it is possible to clarify areas for improvement in the service. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the generation unit can input the emotion data and the action logs to the generation AI, which can then generate a report showing areas for improvement.

[0043] The collection unit can collect data from sensors installed at each table seat. The collection unit can collect data such as facial expressions, tone of voice, volume of voice, and movement of customers using a camera sensor, a microphone sensor, a motion sensor, and the like. For example, the camera sensor captures the facial expressions of customers in real time and collects facial expression data. The microphone sensor captures the tone of voice and volume of voice of customers in real time and collects voice data. The motion sensor captures the movements of customers in real time and collects movement data. In this way, by collecting data from each table seat, it is possible to accurately grasp the emotions of customers. Some or all of the above-mentioned processing in the collection unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the collection unit can input sensor data to a generation AI, which then collects the data.

[0044] The collection unit can process the collected data in real time. The collection unit uses AI to analyze the collected data in real time and estimate the customer's emotions. For example, facial expression data collected from a camera sensor is analyzed in real time to estimate the customer's emotions. Tone of voice and volume data collected from a microphone sensor are analyzed in real time to estimate the customer's emotions. Movement data collected from a motion sensor are analyzed in real time to estimate the customer's emotions. This enables rapid emotion estimation by processing data in real time. Some or all of the above-mentioned processing in the collection unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the collection unit can input sensor data to a generation AI, which processes the data in real time.

[0045] The collection unit can transmit the collected data to the cloud and centrally manage it. The collection unit uses AI to transmit the collected data to the cloud and centrally manage it. For example, facial expression data collected from each table seat is transmitted to the cloud and centrally managed. Voice tone and volume data collected from each table seat is transmitted to the cloud and centrally managed. Movement data collected from each table seat is transmitted to the cloud and centrally managed. As a result, data can be centrally managed by transmitting data to the cloud and centrally managing it. Some or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input sensor data to the generation AI, and the generation AI can transmit the data to the cloud.

[0046] The collection unit can preprocess the collected data and remove noise. The collection unit uses AI to preprocess the collected data and remove noise. For example, noise is removed from facial expression data collected from a camera sensor to improve analysis accuracy. Noise is removed from voice tone and voice volume data collected from a microphone sensor to improve analysis accuracy. Noise is removed from movement data collected from a motion sensor to improve analysis accuracy. In this way, analysis accuracy is improved by preprocessing the data. Some or all of the above-mentioned processing in the collection unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the collection unit can input sensor data to a generation AI, which then preprocesses the data.

[0047] The collection unit can encrypt the collected data to enhance security. The collection unit can use AI to encrypt the collected data to enhance security. For example, facial expression data collected from a camera sensor is encrypted to enhance security. Voice tone and voice volume data collected from a microphone sensor is encrypted to enhance security. Movement data collected from a motion sensor is encrypted to enhance security. In this way, security is enhanced by encrypting data. Some or all of the above-mentioned processing in the collection unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the collection unit can input sensor data to a generation AI, which then encrypts the data.

[0048] The collection unit can periodically back up the collected data. The collection unit periodically backs up the collected data using AI. For example, facial expression data collected from a camera sensor is periodically backed up to prevent data loss. Voice tone and voice volume data collected from a microphone sensor is periodically backed up to prevent data loss. Movement data collected from a motion sensor is periodically backed up to prevent data loss. In this way, data loss can be prevented by periodically backing up data. Some or all of the above-mentioned processing in the collection unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the collection unit can input sensor data to a generation AI, which then periodically backs up the data.

[0049] The estimation unit can estimate the customer's emotions based on the collected data. The estimation unit uses AI to analyze the collected data and estimate the customer's emotions. For example, the estimation unit analyzes facial expression data collected from a camera sensor to estimate the customer's emotions. The estimation unit analyzes tone of voice and volume data collected from a microphone sensor to estimate the customer's emotions. The estimation unit analyzes movement data collected from a motion sensor to estimate the customer's emotions. This makes it possible to estimate the customer's emotions based on the collected data, thereby enabling accurate emotion estimation. Some or all of the above-mentioned processing in the estimation unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the estimation unit inputs sensor data to a generation AI, which can estimate the emotions.

[0050] The estimation unit can adjust the estimation algorithm to improve the accuracy of emotion estimation. The estimation unit uses AI to adjust the estimation algorithm to improve the accuracy of emotion estimation. For example, the analysis algorithm of facial expression data is adjusted to improve the accuracy of emotion estimation. The analysis algorithm of tone of voice and volume data is adjusted to improve the accuracy of emotion estimation. The analysis algorithm of movement data is adjusted to improve the accuracy of emotion estimation. In this way, the accuracy of emotion estimation is improved by adjusting the estimation algorithm. Some or all of the above-mentioned processing in the estimation unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the estimation unit can input the adjustment of the algorithm to the generation AI, and the generation AI can adjust the algorithm.

[0051] The estimation unit can update the estimation result in real time to reflect the latest emotional state. The estimation unit uses AI to update the estimation result in real time to reflect the latest emotional state. For example, the analysis result of facial expression data is updated in real time to reflect the latest emotional state. The analysis result of tone of voice and volume data is updated in real time to reflect the latest emotional state. The analysis result of movement data is updated in real time to reflect the latest emotional state. In this way, the estimation result can be updated in real time to reflect the latest emotional state. Some or all of the above-mentioned processing in the estimation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the estimation unit can input the analysis result to the generation AI, and the generation AI can update it in real time.

[0052] The estimation unit can compare the estimation result with past data and detect anomalies. The estimation unit uses AI to compare the estimation result with past data and detect anomalies. For example, the estimation result of facial expression data is compared with past data and detects anomalies. The estimation result of tone of voice and volume data is compared with past data and detects anomalies. The estimation result of movement data is compared with past data and detects anomalies. In this way, anomalies can be detected by comparing the estimation result with past data. Some or all of the above-mentioned processing in the estimation unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the estimation unit can input the estimation result to a generation AI, which can compare it with past data to detect anomalies.

[0053] The estimation unit can transmit the estimation result to the cloud and manage it centrally. The estimation unit uses AI to transmit the estimation result to the cloud and manage it centrally. For example, the estimation result of facial expression data is transmitted to the cloud and managed centrally. The estimation result of tone of voice and volume data is transmitted to the cloud and managed centrally. The estimation result of movement data is transmitted to the cloud and managed centrally. In this way, by transmitting the estimation result to the cloud and managing it centrally, centralized management of data is possible. Some or all of the above-mentioned processing in the estimation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the estimation unit can input the estimation result to the generation AI, which transmits it to the cloud.

[0054] The estimation unit can encrypt the estimation result to enhance security. The estimation unit uses AI to encrypt the estimation result to enhance security. For example, the estimation result of facial expression data is encrypted to enhance security. The estimation result of voice tone and voice volume data is encrypted to enhance security. The estimation result of movement data is encrypted to enhance security. In this way, security is enhanced by encrypting the estimation result. Some or all of the above-mentioned processing in the estimation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the estimation unit can input the estimation result to the generation AI, which then encrypts it.

[0055] The judgment unit can judge the evaluation of the service based on the estimated emotion. The judgment unit uses AI to analyze the estimated emotion based on the evaluation criteria and judges the evaluation of the service. For example, the judgment unit analyzes emotions within a certain time from the serving of food and judges the evaluation of the food. The judgment unit analyzes emotions from the time of sitting down to the time of leaving the seat and judges the overall satisfaction level. If the negative emotion is large, it judges the possibility of dissatisfaction or trouble. In this way, the evaluation of the service can be accurately judged based on the estimated emotion. Some or all of the above-mentioned processing in the judgment unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the judgment unit can input the estimated emotion data to the generation AI, and the generation AI can judge the evaluation.

[0056] The judgment unit can weight the emotions at each service stage and calculate the overall satisfaction level. The judgment unit uses AI to weight the emotions at each service stage and calculate the overall satisfaction level. For example, the emotions at the time of serving are emphasized to increase the evaluation of the food. The emotions at the time of sitting are emphasized to reflect the initial impression in the evaluation. The emotions at the time of leaving the seat are emphasized to reflect the overall satisfaction level in the evaluation. In this way, by weighting the emotions at each service stage, the overall satisfaction level can be accurately calculated. Some or all of the above-mentioned processing in the judgment unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the judgment unit can input the emotion data and weights to the generation AI, which can calculate the overall satisfaction level.

[0057] The judgment unit can refer to the past evaluation history and reflect it in the current evaluation. The judgment unit can use AI to analyze the past evaluation history and reflect it in the current evaluation. For example, the judgment unit can understand the customer's preferences and tendencies from the past evaluation history and reflect them in the current evaluation. The judgment unit can refer to the evaluation of a specific service from the past evaluation history and reflect them in the current evaluation. The judgment unit can understand the fluctuations in customer satisfaction from the past evaluation history and reflect them in the current evaluation. In this way, the judgment unit can refer to the past evaluation history and reflect them in the current evaluation. Some or all of the above-mentioned processing in the judgment unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the judgment unit can input the evaluation history data to the generation AI, which can reflect it in the current evaluation.

[0058] The determination unit can adjust the evaluation criteria based on the estimated emotion. The determination unit uses AI to analyze the estimated emotion and adjust the evaluation criteria. For example, if the customer is relaxed, the evaluation criteria are relaxed. If the customer is nervous, the evaluation criteria are tightened. If the customer is excited, the evaluation criteria are adjusted. This allows for more accurate evaluation by adjusting the evaluation criteria based on the estimated emotion. Some or all of the above-mentioned processing in the determination unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the determination unit can input emotion data to the generation AI, which then adjusts the evaluation criteria.

[0059] The judgment unit can update the evaluation result in real time to reflect the latest evaluation state. The judgment unit uses AI to update the evaluation result in real time to reflect the latest evaluation state. For example, the emotions at the time of serving the food are updated in real time to reflect the evaluation of the food. The emotions at the time of sitting down are updated in real time to reflect the initial impression in the evaluation. The emotions at the time of leaving the seat are updated in real time to reflect the overall satisfaction in the evaluation. In this way, the evaluation result can be updated in real time to reflect the latest evaluation state. Some or all of the above-mentioned processing in the judgment unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the judgment unit can input the evaluation data to the generation AI, which can update it in real time.

[0060] The judgment unit can transmit the evaluation results to the cloud and manage them centrally. The judgment unit uses AI to transmit the evaluation results to the cloud and manage them centrally. For example, the evaluation results at the time of serving food are transmitted to the cloud and managed centrally. The evaluation results at the time of sitting down are transmitted to the cloud and managed centrally. The evaluation results at the time of leaving the seat are transmitted to the cloud and managed centrally. In this way, by transmitting the evaluation results to the cloud and managing them centrally, centralized management of data is possible. Some or all of the above-mentioned processing in the judgment unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the judgment unit can input the evaluation results to the generation AI, which then transmits the evaluation results to the cloud.

[0061] The output unit can notify the wait staff of the judgment result in real time. The output unit uses AI to notify the wait staff of the judgment result in real time. For example, the evaluation result at the time of serving food is notified to the wait staff in real time. The evaluation result at the time of sitting down is notified to the wait staff in real time. The evaluation result at the time of leaving the seat is notified to the wait staff in real time. By notifying the judgment result in real time, a quick response is possible. Some or all of the above-mentioned processing in the output unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the output unit can input the judgment result to the generation AI, which notifies in real time.

[0062] The output unit can compare the judgment result with past data and analyze trends. The output unit uses AI to compare the judgment result with past data and analyze trends. For example, the evaluation result at the time of serving is compared with past data and analyze trends. The evaluation result at the time of sitting down is compared with past data and analyze trends. The evaluation result at the time of leaving the seat is compared with past data and analyze trends. In this way, trends can be analyzed by comparing the judgment result with past data. Some or all of the above-mentioned processing in the output unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the output unit can input the judgment result to a generation AI, which then compares it with past data to analyze trends.

[0063] The output unit can link the judgment result with other systems to improve overall service. The output unit uses AI to link the judgment result with other systems to improve overall service. For example, the evaluation result at the time of serving food is linked with other systems to improve overall service. The evaluation result at the time of sitting down is linked with other systems to improve overall service. The evaluation result at the time of leaving the seat is linked with other systems to improve overall service. In this way, by linking the judgment result with other systems, it becomes possible to improve overall service. Some or all of the above-mentioned processing in the output unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the output unit can input the judgment result to a generation AI, which can link with other systems.

[0064] The output unit can feed back the judgment result to the customer and collect opinions for improving the service. The output unit uses AI to feed back the judgment result to the customer and collect opinions for improving the service. For example, the evaluation result at the time of serving is fed back to the customer and opinions for improving the service are collected. The evaluation result at the time of sitting down is fed back to the customer and opinions for improving the service are collected. The evaluation result at the time of leaving the seat is fed back to the customer and opinions for improving the service are collected. In this way, opinions for improving the service can be collected by feeding back the judgment result to the customer. A part or all of the above-mentioned processing in the output unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the output unit can input the judgment result to the generation AI, and the generation AI can feed back the judgment result to the customer.

[0065] The output unit can visually display the judgment result and provide it to the service staff. The output unit can visually display the judgment result using AI and provide it to the service staff. For example, the evaluation result at the time of serving food can be visually displayed and provided to the service staff. The evaluation result at the time of sitting down can be visually displayed and provided to the service staff. The evaluation result at the time of leaving the seat can be visually displayed and provided to the service staff. In this way, by visually displaying the judgment result, the service staff can intuitively understand. Some or all of the above-mentioned processing in the output unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the output unit can input the judgment result to the generation AI, which then visually displays it.

[0066] The output unit can notify the judgment result by voice and provide it to the service staff. The output unit can use AI to notify the judgment result by voice and provide it to the service staff. For example, the evaluation result at the time of serving food can be notified by voice and provided to the service staff. The evaluation result at the time of sitting down can be notified by voice and provided to the service staff. The evaluation result at the time of leaving the seat can be notified by voice and provided to the service staff. In this way, the judgment result can be notified by voice, allowing the service staff to respond quickly. Some or all of the above-mentioned processing in the output unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the output unit can input the judgment result to the generation AI, which can notify by voice.

[0067] The detection processing unit can detect the facial expression of the customer and use it as auxiliary information for emotion estimation. The detection processing unit detects the facial expression of the customer using AI and uses it as auxiliary information for emotion estimation. For example, the detection processing unit analyzes facial expression data collected from a camera sensor and estimates the emotion of the customer. Changes in facial expression are detected in real time and used as auxiliary information for emotion estimation. A smiling or angry facial expression of the customer is detected and reflected in emotion estimation. In this way, the accuracy of emotion estimation is improved by detecting the facial expression of the customer. Some or all of the above-mentioned processing in the detection processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the detection processing unit can input facial expression data to a generation AI, which can detect the facial expression.

[0068] The detection processing unit can detect the tone of voice of the customer and use it as auxiliary information for emotion estimation. The detection processing unit detects the tone of voice of the customer using AI and uses it as auxiliary information for emotion estimation. For example, the detection processing unit analyzes tone of voice data collected from a microphone sensor and estimates the emotion of the customer. Changes in tone of voice are detected in real time and used as auxiliary information for emotion estimation. The tone and pitch of the customer's voice are detected and reflected in emotion estimation. In this way, the accuracy of emotion estimation is improved by detecting the tone of voice of the customer. Some or all of the above-mentioned processing in the detection processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the detection processing unit can input tone of voice data to a generation AI, which can detect the tone of voice.

[0069] The detection processing unit can detect the voice volume of the customer and use it as auxiliary information for emotion estimation. The detection processing unit detects the voice volume of the customer using AI and uses it as auxiliary information for emotion estimation. For example, the detection processing unit analyzes voice volume data collected from a microphone sensor and estimates the emotion of the customer. The change in voice volume is detected in real time and used as auxiliary information for emotion estimation. The loudness of the customer's voice is detected and reflected in emotion estimation. Thereby, the accuracy of emotion estimation is improved by detecting the voice volume of the customer. A part or all of the above-mentioned processing in the detection processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the detection processing unit can input voice volume data to the generation AI, and the generation AI can detect the voice volume.

[0070] The detection processing unit can detect the customer's movements and use them as auxiliary information for emotion estimation. The detection processing unit detects the customer's movements using AI and uses them as auxiliary information for emotion estimation. For example, the detection processing unit analyzes the movement data collected from a motion sensor and estimates the customer's emotions. Changes in movement are detected in real time and used as auxiliary information for emotion estimation. The customer's actions and postures are detected and reflected in emotion estimation. In this way, the accuracy of emotion estimation is improved by detecting the customer's movements. Some or all of the above-mentioned processing in the detection processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the detection processing unit can input the movement data to a generation AI, which detects the movement.

[0071] The detection processing unit can transmit customer motion data to the cloud and centrally manage it. The detection processing unit uses AI to transmit customer motion data to the cloud and centrally manage it. For example, motion data collected from a motion sensor is transmitted to the cloud and centrally manages it. Motion data is transmitted to the cloud in real time and centrally manages it. Motion data is transmitted to the cloud periodically and centrally manages it. Thereby, by transmitting customer motion data to the cloud and centrally managing it, centralized management of data is possible. Some or all of the above-mentioned processing in the detection processing unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the detection processing unit can input motion data to a generation AI, which transmits it to the cloud.

[0072] The detection processing unit can encrypt the customer's motion data to enhance security. The detection processing unit can use AI to encrypt the customer's motion data to enhance security. For example, the detection processing unit can encrypt motion data collected from a motion sensor to enhance security. The detection processing unit can encrypt motion data in real time to enhance security. The detection processing unit can encrypt motion data periodically to enhance security. In this way, security is enhanced by encrypting the customer's motion data. Some or all of the above-mentioned processing in the detection processing unit can be performed using, for example, a generation AI, or can be performed without using a generation AI. For example, the detection processing unit can input motion data to a generation AI, which then encrypts the motion data.

[0073] The estimation processing unit can analyze the facial expression data of the customer and estimate the emotion. The estimation processing unit uses AI to analyze the facial expression data of the customer and estimate the emotion. For example, the estimation processing unit analyzes facial expression data collected from a camera sensor and estimates the emotion of the customer. The estimation processing unit analyzes changes in facial expression in real time and estimates the emotion. The estimation processing unit analyzes the customer's smiling or angry facial expression and estimates the emotion. In this way, the emotion can be accurately estimated by analyzing the customer's facial expression data. Some or all of the above-mentioned processing in the estimation processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the estimation processing unit can input the facial expression data to a generation AI, which can estimate the emotion.

[0074] The estimation processing unit can analyze the voice tone data of the customer and estimate the emotion. The estimation processing unit uses AI to analyze the voice tone data of the customer and estimate the emotion. For example, the estimation processing unit analyzes voice tone data collected from a microphone sensor and estimates the emotion of the customer. The estimation processing unit analyzes changes in voice tone in real time and estimates the emotion. The estimation processing unit analyzes the tone of voice data and estimates the emotion. The estimation processing unit analyzes the tone of voice data and estimates the emotion. In this way, the emotion can be accurately estimated by analyzing the voice tone data of the customer. Some or all of the above-mentioned processing in the estimation processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the estimation processing unit inputs voice tone data to a generation AI, which can estimate the emotion.

[0075] The estimation processing unit can analyze the voice volume data of the customer and estimate the emotion. The estimation processing unit uses AI to analyze the voice volume data of the customer and estimate the emotion. For example, the estimation processing unit analyzes voice volume data collected from a microphone sensor and estimates the emotion of the customer. The estimation processing unit analyzes changes in voice volume in real time and estimates the emotion. The estimation processing unit analyzes the loudness of the customer's voice and estimates the emotion. In this way, the emotion can be accurately estimated by analyzing the voice volume data of the customer. A part or all of the above-mentioned processing in the estimation processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the estimation processing unit inputs voice volume data to a generation AI, which can estimate the emotion.

[0076] The estimation processing unit can analyze the customer's motion data and estimate the emotion. The estimation processing unit uses AI to analyze the customer's motion data and estimate the emotion. For example, the estimation processing unit analyzes motion data collected from a motion sensor and estimates the customer's emotion. The estimation processing unit analyzes changes in motion in real time and estimates the emotion. The estimation processing unit analyzes the customer's actions and posture and estimates the emotion. In this way, the emotion can be accurately estimated by analyzing the customer's motion data. Some or all of the above-mentioned processing in the estimation processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the estimation processing unit can input the motion data to a generation AI, which can estimate the emotion.

[0077] The estimation processing unit can transmit the customer's motion data to the cloud and centrally manage it. The estimation processing unit uses AI to transmit the customer's motion data to the cloud and centrally manage it. For example, the motion data collected from a motion sensor is transmitted to the cloud and centrally managed. The motion data is transmitted to the cloud in real time and centrally managed. The motion data is transmitted to the cloud periodically and centrally managed. This enables centralized management of data by transmitting the customer's motion data to the cloud and centrally managing it. Some or all of the above-mentioned processing in the estimation processing unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the estimation processing unit can input the motion data to the generation AI, which then transmits it to the cloud.

[0078] The estimation processing unit can encrypt the customer's motion data to enhance security. The estimation processing unit can use AI to encrypt the customer's motion data to enhance security. For example, the estimation processing unit can encrypt motion data collected from a motion sensor to enhance security. The estimation processing unit can encrypt motion data in real time to enhance security. The estimation processing unit can encrypt motion data periodically to enhance security. In this way, security is enhanced by encrypting the customer's motion data. Some or all of the above-mentioned processing in the estimation processing unit can be performed using, for example, a generation AI, or can be performed without using a generation AI. For example, the estimation processing unit can input motion data to a generation AI, which can then encrypt the data.

[0079] At the time of notification, the notification unit can refer to the customer's past evaluation history and encourage an appropriate response. The notification unit uses AI to refer to the customer's past evaluation history and encourage an appropriate response. For example, the notification unit refers to the customer's past evaluation history and notifies the customer of an appropriate response. The notification unit notifies the customer of a response to a specific service from the customer's past evaluation history. The notification unit notifies the customer of an optimal response based on the customer's past evaluation history. In this way, an appropriate response can be encouraged by referring to the customer's past evaluation history. Some or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input evaluation history data into the generation AI, which then notifies the customer of an appropriate response.

[0080] The notification unit can determine the priority of the notification based on the customer's current situation at the time of notification. The notification unit uses AI to determine the priority of the notification based on the customer's current situation. For example, if the customer is relaxed, the notification priority is set low. If the customer is nervous, the notification priority is set high. If the customer is excited, the notification priority is adjusted. This enables more appropriate notification by determining the priority of the notification based on the customer's current situation. Some or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input situation data into the generation AI, which can determine the priority of the notification.

[0081] The notification unit can select a notification format and notify in an optimal manner. The notification unit uses AI to select a notification format and notify in an optimal manner. For example, if the customer is relaxed, a gentle voice notification is provided. If the customer is nervous, a quick and concise text notification is provided. If the customer is excited, a detailed visual notification is provided. This allows for more effective notification by selecting the notification format. Some or all of the above-mentioned processing in the notification unit may be performed, for example, using a generation AI, or may be performed without using the generation AI. For example, the notification unit can input notification format data to the generation AI, which can then notify in an optimal manner.

[0082] The notification unit can link the notification result with other systems to improve the overall service. The notification unit uses AI to link the notification result with other systems to improve the overall service. For example, the notification result is linked with other systems to improve the overall service. The notification result is linked with other systems in real time to improve the overall service. The notification result is linked with other systems periodically to improve the overall service. This makes it possible to improve the overall service by linking the notification result with other systems. Some or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the notification result to the generation AI, which can then link with other systems.

[0083] The notification unit can feed back the notification result to the customer and collect opinions for improving the service. The notification unit uses AI to feed back the notification result to the customer and collect opinions for improving the service. For example, the notification result is fed back to the customer and opinions for improving the service. The notification result is fed back to the customer in real time and opinions for improving the service are collected. The notification result is periodically fed back to the customer and opinions for improving the service are collected. In this way, opinions for improving the service can be collected by feeding back the notification result to the customer. A part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the notification result to the generation AI, and the generation AI can feed back the notification result to the customer.

[0084] When generating a report, the generation unit can refer to the customer's past evaluation history and indicate areas for improvement. The generation unit uses AI to refer to the customer's past evaluation history and indicate areas for improvement. For example, the generation unit refers to the customer's past evaluation history and generates a report indicating areas for improvement. A report indicating areas for improvement for a specific service is generated from the customer's past evaluation history. A report indicating overall areas for improvement is generated based on the customer's past evaluation history. In this way, a report indicating areas for improvement is generated by referring to the customer's past evaluation history. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the generation unit can input evaluation history data into the generation AI, which generates a report indicating areas for improvement.

[0085] The generation unit can determine the priority of the report based on the customer's current situation when generating the report. The generation unit uses AI to determine the priority of the report based on the customer's current situation. For example, if the customer is relaxed, the priority of the report is set low. If the customer is nervous, the priority of the report is set high. If the customer is excited, the priority of the report is adjusted. In this way, a more appropriate report is generated by determining the priority of the report based on the customer's current situation. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the generation unit can input situation data into the generation AI, which can determine the priority of the report.

[0086] The generation unit can select a report format and generate a report in an optimal manner. The generation unit can use AI to select a report format and generate a report in an optimal manner. For example, if a customer is relaxed, a report is generated in a calm tone. If a customer is nervous, a quick and concise report is generated. If a customer is excited, a report including detailed information is generated. In this way, a more effective report is generated by selecting a report format. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the generation unit can input report format data to the generation AI, which can generate a report in an optimal manner.

[0087] The generation unit can link the report results with other systems to improve services comprehensively. The generation unit uses AI to link the report results with other systems to improve services comprehensively. For example, the report results are linked with other systems to improve services comprehensively. The report results are linked with other systems in real time to improve services comprehensively. The report results are linked with other systems periodically to improve services comprehensively. This makes it possible to improve services comprehensively by linking the report results with other systems. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the generation unit can input the report results to the generation AI, which can then link with other systems.

[0088] The generation unit can feed back the report result to the customer and collect opinions for improving the service. The generation unit uses AI to feed back the report result to the customer and collect opinions for improving the service. For example, the report result is fed back to the customer and opinions for improving the service are collected. The report result is fed back to the customer in real time and opinions for improving the service are collected. The report result is periodically fed back to the customer and opinions for improving the service are collected. In this way, opinions for improving the service can be collected by feeding back the report result to the customer. A part or all of the above-mentioned processing in the generation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the generation unit can input the report result to the generation AI, and the generation AI can feed back the report result to the customer.

[0089] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0090] The collection unit can not only collect data from sensors installed on the customer's table, but also from the customer's smartphone or wearable device. For example, it collects location information and acceleration sensor data from the smartphone to understand the customer's movements and time spent there. It collects heart rate and skin temperature from the wearable device to estimate the customer's stress level and relaxation level. This enables more multifaceted data collection and improves the accuracy of customer emotion estimation.

[0091] The collection unit can collect data not only from customers' tables, but also from sensors installed throughout the store. For example, it can grasp the congestion level and noise level in the store in real time and analyze the impact on customer emotions. This makes it possible to estimate emotions taking into account the entire store environment.

[0092] The output unit can provide a visual interface when notifying the customer service staff of the result of the judgment. For example, the emotion data can be displayed in a graph or chart so that the customer service staff can intuitively understand the emotion data. This allows the customer service staff to receive information that will allow them to quickly respond.

[0093] The collection unit can collect data not only from customers' tables, but also from other areas of the store, such as restrooms and waiting areas. For example, it can grasp the usage of restrooms and the time spent in waiting areas, and evaluate the customer's overall experience. This can improve the service quality throughout the store.

[0094] The process flow of the first embodiment will be briefly described below.

[0095] Step 1: The collection unit collects data from sensors installed at each table. The collection unit uses a camera sensor, microphone sensor, motion sensor, etc. to collect data such as customer facial expressions, tone of voice, voice volume, and movements. For example, the camera sensor captures the customer's facial expressions in real time and collects facial expression data. The microphone sensor captures the customer's voice tone and voice volume in real time and collects voice data. The motion sensor captures the customer's movements in real time and collects movement data. Step 2: The estimation unit estimates the customer's emotions based on the data collected by the collection unit. The estimation unit uses AI to analyze the collected data and estimates emotions from the customer's facial expressions, tone of voice, voice volume, and movements. Step 3: The judgment unit judges the customer's evaluation of the service based on the emotions estimated by the estimation unit. The judgment unit analyzes the emotions estimated using AI based on evaluation criteria and judges the customer's evaluation of the service. For example, it analyzes emotions within a certain time after the food is served and judges the evaluation of the food. It analyzes emotions from the time the customer sits down to the time they leave the table and judges the overall level of satisfaction. If negative emotions are strong, it judges the possibility of dissatisfaction or trouble. Step 4: The output unit outputs the judgment result by the judgment unit. The output unit notifies the customer service staff of the judgment result in real time using AI. For example, the judgment result is displayed to the customer service staff via a web application or a mobile application.

[0096] (Example 2) The system according to the embodiment of the present invention is a system that collects data from sensors installed at each table and grasps the customer's emotions in real time. This system collects data from sensors installed at each table and estimates the customer's emotions based on the collected data. Based on the estimated emotions, the customer's evaluation of the service is determined. For example, the emotions within a certain time from the serving of food are recorded as an evaluation of the food, and the emotions from the time of sitting down to leaving the seat are recorded as an overall satisfaction evaluation. Furthermore, if the negative emotions are large, a notification is issued to check whether the customer is dissatisfied or has any trouble. The judgment result is output through the output unit and provided to the wait staff in real time. This allows the wait staff to grasp the customer's satisfaction in real time and take necessary measures quickly. For example, the system includes a collection unit that collects data from sensors installed at each table, an estimation unit that estimates the customer's emotions based on the collected data, a judgment unit that judges the customer's evaluation of the service based on the estimated emotions, and an output unit that outputs the judgment result. This allows the system to grasp the customer's emotions in real time and judge the customer's evaluation of the service.

[0097] The system according to the embodiment includes a collection unit, an estimation unit, a determination unit, and an output unit. The collection unit collects data from a sensor installed at each table seat. The collection unit can collect data such as facial expressions, tone of voice, voice volume, and movement of customers using, for example, a camera sensor, a microphone sensor, a motion sensor, and the like. For example, the camera sensor captures the facial expressions of customers in real time and collects facial expression data. The microphone sensor captures the tone of voice and voice volume of customers in real time and collects voice data. The motion sensor captures the movement of customers in real time and collects movement data. The estimation unit estimates the emotion of the customer based on the data collected by the collection unit. The estimation unit analyzes the collected data and estimates the emotion of the customer using, for example, AI. For example, the AI ​​analyzes the facial expression data and estimates the emotion from the facial expression of the customer. The voice data is analyzed and the emotion is estimated from the tone of voice and voice volume of the customer. The movement data is analyzed and the emotion is estimated from the movement of the customer. The determination unit determines the customer's evaluation of the service based on the emotion estimated by the estimation unit. The judgment unit, for example, uses AI to analyze the estimated emotions based on evaluation criteria and judges the customer's evaluation of the service. For example, it analyzes emotions within a certain time from when the food is served and judges the evaluation of the food. It analyzes emotions from when the customer sits down to when they leave and judges the overall level of satisfaction. If there are a lot of negative emotions, it judges the possibility of dissatisfaction or trouble. The output unit outputs the judgment result by the judgment unit. The output unit, for example, uses AI to notify the customer service staff of the judgment result in real time. For example, it displays the judgment result to the customer service staff via a web application or mobile application. This allows the system to grasp the customer's emotions in real time and judge their evaluation of the service.

[0098] The collection unit collects data from sensors installed at each table. The collection unit can collect data such as facial expressions, tone of voice, voice volume, and movements of customers using, for example, a camera sensor, a microphone sensor, a motion sensor, and the like. Specifically, the camera sensor can capture high-resolution video in real time and capture subtle changes in the customer's facial expressions. This provides data for detailed analysis of emotions such as smiles, surprise, and anger. The microphone sensor uses a highly sensitive microphone with a noise canceling function to accurately capture the tone and volume of the customer's voice while eliminating ambient noise. This makes it possible to capture changes in emotions from the tone and strength of the customer's voice. The motion sensor uses 3D motion capture technology to record the customer's body movements and postures in detail. For example, it is possible to infer from the customer's movements whether the customer is relaxed, tense, or dissatisfied. These sensor data are centrally managed by the collection unit and transmitted to a central database in real time. In addition, the collection unit can dynamically adjust the frequency and resolution of data collection, allowing for flexible responses according to specific situations and conditions. For example, when a customer performs a specific action, the frame rate of the camera can be increased to obtain detailed images, allowing the collection unit to collect data efficiently and effectively and improving the performance of the entire system.

[0099] The estimation unit estimates the customer's emotions based on the data collected by the collection unit. The estimation unit, for example, uses AI to analyze the collected data and estimate the customer's emotions. Specifically, the AI ​​uses deep learning technology to analyze facial expression data and estimate emotions from the customer's facial expressions. For example, subtle changes in facial expressions, such as the degree of smiling and the depth of wrinkles between the eyebrows, are captured, and emotions such as joy, anger, and surprise are estimated with high accuracy. When analyzing voice data, a voice recognition technology is combined with an emotion analysis algorithm to estimate emotions from the customer's tone of voice and voice volume. For example, changes in the tone, pitch, and volume of the voice are analyzed to estimate whether the customer is relaxed, excited, or dissatisfied. When analyzing movement data, emotions are estimated from the customer's movements and postures based on motion capture data. For example, if the customer moves frequently, it indicates tension or anxiety, and if the customer has a relaxed posture, it indicates satisfaction. This allows the estimation unit to analyze the collected data from multiple angles and estimate the customer's emotions with high accuracy. Furthermore, the estimation unit can utilize past data and statistical information to learn patterns of customer emotional fluctuations and perform more accurate emotion estimation. For example, it can more accurately estimate current emotions based on the emotions a specific customer has shown in the past. This enables the estimation unit to handle not only real-time emotion estimation, but also long-term trend analysis of emotion fluctuations, improving the reliability and accuracy of the entire system.

[0100] The determination unit determines the customer's evaluation of the service based on the emotion estimated by the estimation unit. The determination unit, for example, uses AI to analyze the estimated emotion based on the evaluation criteria and determines the customer's evaluation of the service. Specifically, the AI ​​compares the estimated emotion data with the evaluation criteria to quantify the customer's satisfaction or dissatisfaction. For example, the emotion within a certain time from the serving of the food is analyzed to determine the evaluation of the food. If the customer smiles immediately after receiving the food, it is determined that the customer has a high evaluation, and conversely, if the customer shows a dissatisfied expression, it is determined that the customer has a low evaluation. The emotion from when the customer sits down to when the customer leaves the seat is analyzed to determine the overall satisfaction. For example, if the customer stays for a long time in a relaxed posture, it is determined that the customer has a high level of satisfaction. If the negative emotion is large, it is determined that there is a possibility of dissatisfaction or trouble. For example, if the customer frequently shows a dissatisfied expression or has a low tone of voice, it is determined that the customer is dissatisfied with the service. This allows the determination unit to determine the evaluation of the service with high accuracy based on the customer's emotion data. Furthermore, the determination unit can dynamically adjust the evaluation criteria by utilizing past evaluation data and statistical information. For example, the evaluation trends during a specific time period or day of the week are analyzed, and the evaluation criteria are optimized based on the results. This allows the evaluation unit to always provide highly accurate evaluation decisions based on the latest information, thereby improving the quality of service.

[0101] The output unit outputs the judgment result by the judgment unit. The output unit notifies the customer service staff of the judgment result in real time, for example, using AI. Specifically, the judgment result is displayed to the customer service staff through a web application or a mobile application. For example, a notification is sent to the customer service staff's smartphone or tablet to display the customer's evaluation and emotional state in real time. This allows the customer service staff to immediately grasp the customer's state and take appropriate action. For example, if a customer shows dissatisfaction, the customer service staff can respond quickly and take action to solve the problem. Also, if a customer shows a high evaluation, the customer service staff can consider an approach to provide even better service based on that information. Furthermore, the output unit also notifies the manager or owner of the judgment result and provides data for improving the quality of service. For example, the manager can monitor the evaluation status of each table in real time and adjust the staff placement and service content as necessary. As a result, the output unit provides information for the customer service staff and the manager to take prompt and appropriate action based on the customer evaluation data, and can improve the quality of service. Furthermore, the output unit can accumulate past evaluation data and perform analysis for long-term service improvement. For example, the evaluation trend in a specific period is analyzed, and improvements to the service are identified based on the results. This enables the output section to contribute not only to real-time notifications but also to long-term service improvements, maximizing the effectiveness of the entire system.

[0102] The determination unit can determine the customer's satisfaction with the service based on the emotions at each stage. The determination unit determines the customer's satisfaction with the service based on, for example, specific criteria for emotions at each stage. For example, the determination unit places emphasis on the emotions at the time of serving the food, thereby improving the evaluation of the food. The determination unit places emphasis on the emotions at the time of sitting down, thereby reflecting the initial impression in the evaluation. The determination unit places emphasis on the emotions at the time of leaving the seat, thereby reflecting the overall satisfaction in the evaluation. This makes it possible to determine the customer's satisfaction in detail based on the emotions at each stage. Some or all of the above-mentioned processing in the determination unit may be performed using, for example, AI, or may be performed without using AI. For example, the determination unit can input emotion data at each stage to AI, which can then determine the satisfaction level.

[0103] The determination unit can determine the customer's satisfaction with the service based on the emotions at each stage in the food and beverage service. The determination unit determines the customer's satisfaction with the service, for example, based on specific criteria for emotions at each stage in the food and beverage service. For example, the determination unit places emphasis on the emotions at the time of serving food, thereby improving the evaluation of the food. The determination unit places emphasis on the emotions at the time of sitting down, thereby reflecting the initial impression in the evaluation. The determination unit places emphasis on the emotions at the time of leaving the seat, thereby reflecting the overall satisfaction in the evaluation. This makes it possible to determine the customer's satisfaction in detail based on the emotions at each stage in the food and beverage service. Some or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input emotion data at each stage into AI, which can then determine the satisfaction level.

[0104] The determination unit has a different specific weight for each service stage, and can determine the customer's satisfaction with the service based on a satisfaction score obtained based on the emotion and weight at each stage. For example, the emotion at the time of serving is emphasized to increase the evaluation of the food. The emotion at the time of sitting is emphasized to reflect the initial impression in the evaluation. The emotion at the time of leaving the seat is emphasized to reflect the overall satisfaction in the evaluation. In this way, by setting different weights for each service stage, a more accurate satisfaction evaluation is possible. Some or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the emotion data and weight at each stage to AI, which can calculate the satisfaction score.

[0105] The estimation unit includes a detection processing unit that specifically detects the customer's facial expression, tone of voice, voice volume, and movement based on the data collected by the collection unit. The detection processing unit, for example, uses AI to analyze the collected data and detect the customer's facial expression, tone of voice, voice volume, and movement. For example, the AI ​​analyzes facial expression data and detects the customer's facial expression. It analyzes voice data and detects the customer's tone of voice and voice volume. It analyzes movement data and detects the customer's movement. As a result, the accuracy of emotion estimation is improved by detecting the customer's facial expression, tone of voice, voice volume, and movement. A part or all of the above-mentioned processing in the detection processing unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the detection processing unit inputs the collected data to the generation AI, and the generation AI can detect the facial expression, tone of voice, voice volume, and movement.

[0106] The estimation unit includes an estimation processing unit that estimates the customer's emotions based on the facial expression, tone of voice, voice volume, and movement of the customer detected by the detection processing unit. The estimation processing unit, for example, uses AI to analyze the detected data and estimate the customer's emotions. For example, the AI ​​analyzes facial expression data and estimates emotions from the customer's facial expression. It analyzes voice data and estimates emotions from the customer's tone of voice and voice volume. It analyzes movement data and estimates emotions from the customer's movement. This enables more accurate emotion estimation by estimating emotions based on the detected data. A part or all of the above-mentioned processing in the estimation processing unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the estimation processing unit inputs the detected data to the generation AI, which can estimate the emotions.

[0107] The system includes a notification unit that notifies a responsible customer service staff member when the level of evaluation up to a specific stage in the service is below a threshold. The notification unit, for example, uses AI to analyze the evaluation data and issues a notification when the level is below the threshold. For example, the AI ​​analyzes the evaluation data and notifies the customer service staff member when the level of evaluation is below the threshold. This allows for a quick response when the evaluation is low. Some or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the evaluation data into the generation AI and issue a notification when the generation AI is below the threshold.

[0108] The determination unit can tally up the customer satisfaction level for each customer service staff member in real time, and the output unit can provide the tallying result by the determination unit to each customer service staff member in real time. The determination unit uses AI to tally up the evaluation data of each customer service staff member and calculate the satisfaction level in real time. For example, the AI ​​analyzes the evaluation data of each customer service staff member and calculates a satisfaction score. The output unit provides the tallying result by the determination unit to each customer service staff member in real time. The output unit uses AI to notify the customer service staff member of the tallying result in real time. For example, the output unit displays the tallying result to the customer service staff member through a web application or a mobile application. This makes it possible to grasp the satisfaction level of each customer service staff member in real time and to respond quickly. Some or all of the above-mentioned processes in the determination unit and the output unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the determination unit can input the evaluation data to the generation AI, and the generation AI can calculate the satisfaction level. The output unit can input the tallying result to the generation AI, and the generation AI can notify.

[0109] The determination unit can determine the customer's evaluation based on the customer's past emotion estimation history. The determination unit uses AI to analyze past emotion data and reflect it in the current evaluation. For example, the AI ​​analyzes past emotion data, understands the customer's emotion fluctuation pattern, and reflects it in the current evaluation. This enables a more accurate evaluation based on the past emotion history. Some or all of the above-mentioned processing in the determination unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the determination unit can input past emotion data to the generation AI, and the generation AI can determine the evaluation.

[0110] The system includes a generation unit that compares the emotions of customers with the action logs of the service staff and generates a report showing areas for improvement. The generation unit uses AI to analyze the emotion data and the action logs and extract areas for improvement. For example, the AI ​​compares the emotion data of customers with the action logs of the service staff and identifies areas for improvement in the service. In this way, by comparing the emotions of customers with the action logs, it is possible to clarify areas for improvement in the service. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the generation unit can input the emotion data and the action logs to the generation AI, which can then generate a report showing areas for improvement.

[0111] The collection unit can collect data from sensors installed at each table seat. The collection unit can collect data such as facial expressions, tone of voice, volume of voice, and movement of customers using a camera sensor, a microphone sensor, a motion sensor, and the like. For example, the camera sensor captures the facial expressions of customers in real time and collects facial expression data. The microphone sensor captures the tone of voice and volume of voice of customers in real time and collects voice data. The motion sensor captures the movements of customers in real time and collects movement data. In this way, by collecting data from each table seat, it is possible to accurately grasp the emotions of customers. Some or all of the above-mentioned processing in the collection unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the collection unit can input sensor data to a generation AI, which then collects the data.

[0112] The collection unit can process the collected data in real time. The collection unit uses AI to analyze the collected data in real time and estimate the customer's emotions. For example, facial expression data collected from a camera sensor is analyzed in real time to estimate the customer's emotions. Tone of voice and volume data collected from a microphone sensor are analyzed in real time to estimate the customer's emotions. Movement data collected from a motion sensor are analyzed in real time to estimate the customer's emotions. This enables rapid emotion estimation by processing data in real time. Some or all of the above-mentioned processing in the collection unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the collection unit can input sensor data to a generation AI, which processes the data in real time.

[0113] The collection unit can transmit the collected data to the cloud and centrally manage it. The collection unit uses AI to transmit the collected data to the cloud and centrally manage it. For example, facial expression data collected from each table seat is transmitted to the cloud and centrally managed. Voice tone and volume data collected from each table seat is transmitted to the cloud and centrally managed. Movement data collected from each table seat is transmitted to the cloud and centrally managed. As a result, data can be centrally managed by transmitting data to the cloud and centrally managing it. Some or all of the above-mentioned processing in the collection unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the collection unit can input sensor data to the generation AI, and the generation AI can transmit the data to the cloud.

[0114] The collection unit can preprocess the collected data and remove noise. The collection unit uses AI to preprocess the collected data and remove noise. For example, noise is removed from facial expression data collected from a camera sensor to improve analysis accuracy. Noise is removed from voice tone and voice volume data collected from a microphone sensor to improve analysis accuracy. Noise is removed from movement data collected from a motion sensor to improve analysis accuracy. In this way, analysis accuracy is improved by preprocessing the data. Some or all of the above-mentioned processing in the collection unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the collection unit can input sensor data to a generation AI, which then preprocesses the data.

[0115] The collection unit can encrypt the collected data to enhance security. The collection unit can use AI to encrypt the collected data to enhance security. For example, facial expression data collected from a camera sensor is encrypted to enhance security. Voice tone and voice volume data collected from a microphone sensor is encrypted to enhance security. Movement data collected from a motion sensor is encrypted to enhance security. In this way, security is enhanced by encrypting data. Some or all of the above-mentioned processing in the collection unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the collection unit can input sensor data to a generation AI, which then encrypts the data.

[0116] The collection unit can periodically back up the collected data. The collection unit periodically backs up the collected data using AI. For example, facial expression data collected from a camera sensor is periodically backed up to prevent data loss. Voice tone and voice volume data collected from a microphone sensor is periodically backed up to prevent data loss. Movement data collected from a motion sensor is periodically backed up to prevent data loss. In this way, data loss can be prevented by periodically backing up data. Some or all of the above-mentioned processing in the collection unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the collection unit can input sensor data to a generation AI, which then periodically backs up the data.

[0117] The estimation unit can estimate the customer's emotions based on the collected data. The estimation unit uses AI to analyze the collected data and estimate the customer's emotions. For example, the estimation unit analyzes facial expression data collected from a camera sensor to estimate the customer's emotions. The estimation unit analyzes tone of voice and volume data collected from a microphone sensor to estimate the customer's emotions. The estimation unit analyzes movement data collected from a motion sensor to estimate the customer's emotions. This makes it possible to estimate the customer's emotions based on the collected data, thereby enabling accurate emotion estimation. Some or all of the above-mentioned processing in the estimation unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the estimation unit inputs sensor data to a generation AI, which can estimate the emotions.

[0118] The estimation unit can adjust the estimation algorithm to improve the accuracy of emotion estimation. The estimation unit uses AI to adjust the estimation algorithm to improve the accuracy of emotion estimation. For example, the analysis algorithm of facial expression data is adjusted to improve the accuracy of emotion estimation. The analysis algorithm of tone of voice and volume data is adjusted to improve the accuracy of emotion estimation. The analysis algorithm of movement data is adjusted to improve the accuracy of emotion estimation. In this way, the accuracy of emotion estimation is improved by adjusting the estimation algorithm. Some or all of the above-mentioned processing in the estimation unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the estimation unit can input the adjustment of the algorithm to the generation AI, and the generation AI can adjust the algorithm.

[0119] The estimation unit can update the estimation result in real time to reflect the latest emotional state. The estimation unit uses AI to update the estimation result in real time to reflect the latest emotional state. For example, the analysis result of facial expression data is updated in real time to reflect the latest emotional state. The analysis result of tone of voice and volume data is updated in real time to reflect the latest emotional state. The analysis result of movement data is updated in real time to reflect the latest emotional state. In this way, the estimation result can be updated in real time to reflect the latest emotional state. Some or all of the above-mentioned processing in the estimation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the estimation unit can input the analysis result to the generation AI, and the generation AI can update it in real time.

[0120] The estimation unit can compare the estimation result with past data and detect anomalies. The estimation unit uses AI to compare the estimation result with past data and detect anomalies. For example, the estimation result of facial expression data is compared with past data and detects anomalies. The estimation result of tone of voice and volume data is compared with past data and detects anomalies. The estimation result of movement data is compared with past data and detects anomalies. In this way, anomalies can be detected by comparing the estimation result with past data. Some or all of the above-mentioned processing in the estimation unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the estimation unit can input the estimation result to a generation AI, which can compare it with past data to detect anomalies.

[0121] The estimation unit can transmit the estimation result to the cloud and manage it centrally. The estimation unit uses AI to transmit the estimation result to the cloud and manage it centrally. For example, the estimation result of facial expression data is transmitted to the cloud and managed centrally. The estimation result of tone of voice and volume data is transmitted to the cloud and managed centrally. The estimation result of movement data is transmitted to the cloud and managed centrally. In this way, by transmitting the estimation result to the cloud and managing it centrally, centralized management of data is possible. Some or all of the above-mentioned processing in the estimation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the estimation unit can input the estimation result to the generation AI, which transmits it to the cloud.

[0122] The estimation unit can encrypt the estimation result to enhance security. The estimation unit uses AI to encrypt the estimation result to enhance security. For example, the estimation result of facial expression data is encrypted to enhance security. The estimation result of voice tone and voice volume data is encrypted to enhance security. The estimation result of movement data is encrypted to enhance security. In this way, security is enhanced by encrypting the estimation result. Some or all of the above-mentioned processing in the estimation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the estimation unit can input the estimation result to the generation AI, which then encrypts it.

[0123] The judgment unit can judge the evaluation of the service based on the estimated emotion. The judgment unit uses AI to analyze the estimated emotion based on the evaluation criteria and judges the evaluation of the service. For example, the judgment unit analyzes emotions within a certain time from the serving of food and judges the evaluation of the food. The judgment unit analyzes emotions from the time of sitting down to the time of leaving the seat and judges the overall satisfaction level. If the negative emotion is large, it judges the possibility of dissatisfaction or trouble. In this way, the evaluation of the service can be accurately judged based on the estimated emotion. Some or all of the above-mentioned processing in the judgment unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the judgment unit can input the estimated emotion data to the generation AI, and the generation AI can judge the evaluation.

[0124] The judgment unit can weight the emotions at each service stage and calculate the overall satisfaction level. The judgment unit uses AI to weight the emotions at each service stage and calculate the overall satisfaction level. For example, the emotions at the time of serving are emphasized to increase the evaluation of the food. The emotions at the time of sitting are emphasized to reflect the initial impression in the evaluation. The emotions at the time of leaving the seat are emphasized to reflect the overall satisfaction level in the evaluation. In this way, by weighting the emotions at each service stage, the overall satisfaction level can be accurately calculated. Some or all of the above-mentioned processing in the judgment unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the judgment unit can input the emotion data and weights to the generation AI, which can calculate the overall satisfaction level.

[0125] The judgment unit can refer to the past evaluation history and reflect it in the current evaluation. The judgment unit can use AI to analyze the past evaluation history and reflect it in the current evaluation. For example, the judgment unit can understand the customer's preferences and tendencies from the past evaluation history and reflect them in the current evaluation. The judgment unit can refer to the evaluation of a specific service from the past evaluation history and reflect them in the current evaluation. The judgment unit can understand the fluctuations in customer satisfaction from the past evaluation history and reflect them in the current evaluation. In this way, the judgment unit can refer to the past evaluation history and reflect them in the current evaluation. Some or all of the above-mentioned processing in the judgment unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the judgment unit can input the evaluation history data to the generation AI, which can reflect it in the current evaluation.

[0126] The determination unit can adjust the evaluation criteria based on the estimated emotion. The determination unit uses AI to analyze the estimated emotion and adjust the evaluation criteria. For example, if the customer is relaxed, the evaluation criteria are relaxed. If the customer is nervous, the evaluation criteria are tightened. If the customer is excited, the evaluation criteria are adjusted. This allows for more accurate evaluation by adjusting the evaluation criteria based on the estimated emotion. Some or all of the above-mentioned processing in the determination unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the determination unit can input emotion data to the generation AI, which then adjusts the evaluation criteria.

[0127] The judgment unit can update the evaluation result in real time to reflect the latest evaluation state. The judgment unit uses AI to update the evaluation result in real time to reflect the latest evaluation state. For example, the emotions at the time of serving the food are updated in real time to reflect the evaluation of the food. The emotions at the time of sitting down are updated in real time to reflect the initial impression in the evaluation. The emotions at the time of leaving the seat are updated in real time to reflect the overall satisfaction in the evaluation. In this way, the evaluation result can be updated in real time to reflect the latest evaluation state. Some or all of the above-mentioned processing in the judgment unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the judgment unit can input the evaluation data to the generation AI, which can update it in real time.

[0128] The judgment unit can transmit the evaluation results to the cloud and manage them centrally. The judgment unit uses AI to transmit the evaluation results to the cloud and manage them centrally. For example, the evaluation results at the time of serving food are transmitted to the cloud and managed centrally. The evaluation results at the time of sitting down are transmitted to the cloud and managed centrally. The evaluation results at the time of leaving the seat are transmitted to the cloud and managed centrally. In this way, by transmitting the evaluation results to the cloud and managing them centrally, centralized management of data is possible. Some or all of the above-mentioned processing in the judgment unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the judgment unit can input the evaluation results to the generation AI, which then transmits the evaluation results to the cloud.

[0129] The output unit can notify the wait staff of the judgment result in real time. The output unit uses AI to notify the wait staff of the judgment result in real time. For example, the evaluation result at the time of serving food is notified to the wait staff in real time. The evaluation result at the time of sitting down is notified to the wait staff in real time. The evaluation result at the time of leaving the seat is notified to the wait staff in real time. By notifying the judgment result in real time, a quick response is possible. Some or all of the above-mentioned processing in the output unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the output unit can input the judgment result to the generation AI, which notifies in real time.

[0130] The output unit can compare the judgment result with past data and analyze trends. The output unit uses AI to compare the judgment result with past data and analyze trends. For example, the evaluation result at the time of serving is compared with past data and analyze trends. The evaluation result at the time of sitting down is compared with past data and analyze trends. The evaluation result at the time of leaving the seat is compared with past data and analyze trends. In this way, trends can be analyzed by comparing the judgment result with past data. Some or all of the above-mentioned processing in the output unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the output unit can input the judgment result to a generation AI, which then compares it with past data to analyze trends.

[0131] The output unit can link the judgment result with other systems to improve overall service. The output unit uses AI to link the judgment result with other systems to improve overall service. For example, the evaluation result at the time of serving food is linked with other systems to improve overall service. The evaluation result at the time of sitting down is linked with other systems to improve overall service. The evaluation result at the time of leaving the seat is linked with other systems to improve overall service. In this way, by linking the judgment result with other systems, it becomes possible to improve overall service. Some or all of the above-mentioned processing in the output unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the output unit can input the judgment result to a generation AI, which can link with other systems.

[0132] The output unit can feed back the judgment result to the customer and collect opinions for improving the service. The output unit uses AI to feed back the judgment result to the customer and collect opinions for improving the service. For example, the evaluation result at the time of serving is fed back to the customer and opinions for improving the service are collected. The evaluation result at the time of sitting down is fed back to the customer and opinions for improving the service are collected. The evaluation result at the time of leaving the seat is fed back to the customer and opinions for improving the service are collected. In this way, opinions for improving the service can be collected by feeding back the judgment result to the customer. A part or all of the above-mentioned processing in the output unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the output unit can input the judgment result to the generation AI, and the generation AI can feed back the judgment result to the customer.

[0133] The output unit can visually display the judgment result and provide it to the service staff. The output unit can visually display the judgment result using AI and provide it to the service staff. For example, the evaluation result at the time of serving food can be visually displayed and provided to the service staff. The evaluation result at the time of sitting down can be visually displayed and provided to the service staff. The evaluation result at the time of leaving the seat can be visually displayed and provided to the service staff. In this way, by visually displaying the judgment result, the service staff can intuitively understand. Some or all of the above-mentioned processing in the output unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the output unit can input the judgment result to the generation AI, which then visually displays it.

[0134] The output unit can notify the judgment result by voice and provide it to the service staff. The output unit can use AI to notify the judgment result by voice and provide it to the service staff. For example, the evaluation result at the time of serving food can be notified by voice and provided to the service staff. The evaluation result at the time of sitting down can be notified by voice and provided to the service staff. The evaluation result at the time of leaving the seat can be notified by voice and provided to the service staff. In this way, the judgment result can be notified by voice, allowing the service staff to respond quickly. Some or all of the above-mentioned processing in the output unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the output unit can input the judgment result to the generation AI, which can notify by voice.

[0135] The detection processing unit can detect the facial expression of the customer and use it as auxiliary information for emotion estimation. The detection processing unit detects the facial expression of the customer using AI and uses it as auxiliary information for emotion estimation. For example, the detection processing unit analyzes facial expression data collected from a camera sensor and estimates the emotion of the customer. Changes in facial expression are detected in real time and used as auxiliary information for emotion estimation. A smiling or angry facial expression of the customer is detected and reflected in emotion estimation. In this way, the accuracy of emotion estimation is improved by detecting the facial expression of the customer. Some or all of the above-mentioned processing in the detection processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the detection processing unit can input facial expression data to a generation AI, which can detect the facial expression.

[0136] The detection processing unit can detect the tone of voice of the customer and use it as auxiliary information for emotion estimation. The detection processing unit detects the tone of voice of the customer using AI and uses it as auxiliary information for emotion estimation. For example, the detection processing unit analyzes tone of voice data collected from a microphone sensor and estimates the emotion of the customer. Changes in tone of voice are detected in real time and used as auxiliary information for emotion estimation. The tone and pitch of the customer's voice are detected and reflected in emotion estimation. In this way, the accuracy of emotion estimation is improved by detecting the tone of voice of the customer. Some or all of the above-mentioned processing in the detection processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the detection processing unit can input tone of voice data to a generation AI, which can detect the tone of voice.

[0137] The detection processing unit can detect the voice volume of the customer and use it as auxiliary information for emotion estimation. The detection processing unit detects the voice volume of the customer using AI and uses it as auxiliary information for emotion estimation. For example, the detection processing unit analyzes voice volume data collected from a microphone sensor and estimates the emotion of the customer. The change in voice volume is detected in real time and used as auxiliary information for emotion estimation. The loudness of the customer's voice is detected and reflected in emotion estimation. Thereby, the accuracy of emotion estimation is improved by detecting the voice volume of the customer. A part or all of the above-mentioned processing in the detection processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the detection processing unit can input voice volume data to the generation AI, and the generation AI can detect the voice volume.

[0138] The detection processing unit can detect the customer's movements and use them as auxiliary information for emotion estimation. The detection processing unit detects the customer's movements using AI and uses them as auxiliary information for emotion estimation. For example, the detection processing unit analyzes the movement data collected from a motion sensor and estimates the customer's emotions. Changes in movement are detected in real time and used as auxiliary information for emotion estimation. The customer's actions and postures are detected and reflected in emotion estimation. In this way, the accuracy of emotion estimation is improved by detecting the customer's movements. Some or all of the above-mentioned processing in the detection processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the detection processing unit can input the movement data to a generation AI, which detects the movement.

[0139] The detection processing unit can transmit customer motion data to the cloud and centrally manage it. The detection processing unit uses AI to transmit customer motion data to the cloud and centrally manage it. For example, motion data collected from a motion sensor is transmitted to the cloud and centrally manages it. Motion data is transmitted to the cloud in real time and centrally manages it. Motion data is transmitted to the cloud periodically and centrally manages it. Thereby, by transmitting customer motion data to the cloud and centrally managing it, centralized management of data is possible. Some or all of the above-mentioned processing in the detection processing unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the detection processing unit can input motion data to a generation AI, which transmits it to the cloud.

[0140] The detection processing unit can encrypt the customer's motion data to enhance security. The detection processing unit can use AI to encrypt the customer's motion data to enhance security. For example, the detection processing unit can encrypt motion data collected from a motion sensor to enhance security. The detection processing unit can encrypt motion data in real time to enhance security. The detection processing unit can encrypt motion data periodically to enhance security. In this way, security is enhanced by encrypting the customer's motion data. Some or all of the above-mentioned processing in the detection processing unit can be performed using, for example, a generation AI, or can be performed without using a generation AI. For example, the detection processing unit can input motion data to a generation AI, which then encrypts the motion data.

[0141] The estimation processing unit can analyze the facial expression data of the customer and estimate the emotion. The estimation processing unit uses AI to analyze the facial expression data of the customer and estimate the emotion. For example, the estimation processing unit analyzes facial expression data collected from a camera sensor and estimates the emotion of the customer. The estimation processing unit analyzes changes in facial expression in real time and estimates the emotion. The estimation processing unit analyzes the customer's smiling or angry facial expression and estimates the emotion. In this way, the emotion can be accurately estimated by analyzing the customer's facial expression data. Some or all of the above-mentioned processing in the estimation processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the estimation processing unit can input the facial expression data to a generation AI, which can estimate the emotion.

[0142] The estimation processing unit can analyze the voice tone data of the customer and estimate the emotion. The estimation processing unit uses AI to analyze the voice tone data of the customer and estimate the emotion. For example, the estimation processing unit analyzes voice tone data collected from a microphone sensor and estimates the emotion of the customer. The estimation processing unit analyzes changes in voice tone in real time and estimates the emotion. The estimation processing unit analyzes the tone of voice and pitch of the customer's voice and estimates the emotion. In this way, the emotion can be accurately estimated by analyzing the voice tone data of the customer. Some or all of the above-mentioned processing in the estimation processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the estimation processing unit inputs voice tone data to a generation AI, which can estimate the emotion.

[0143] The estimation processing unit can analyze the voice volume data of the customer and estimate the emotion. The estimation processing unit uses AI to analyze the voice volume data of the customer and estimate the emotion. For example, the estimation processing unit analyzes voice volume data collected from a microphone sensor and estimates the emotion of the customer. The estimation processing unit analyzes changes in voice volume in real time and estimates the emotion. The estimation processing unit analyzes the loudness of the customer's voice and estimates the emotion. In this way, the emotion can be accurately estimated by analyzing the voice volume data of the customer. A part or all of the above-mentioned processing in the estimation processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the estimation processing unit inputs voice volume data to a generation AI, which can estimate the emotion.

[0144] The estimation processing unit can analyze the customer's motion data and estimate the emotion. The estimation processing unit uses AI to analyze the customer's motion data and estimate the emotion. For example, the estimation processing unit analyzes motion data collected from a motion sensor and estimates the customer's emotion. The estimation processing unit analyzes changes in motion in real time and estimates the emotion. The estimation processing unit analyzes the customer's actions and posture and estimates the emotion. In this way, the emotion can be accurately estimated by analyzing the customer's motion data. Some or all of the above-mentioned processing in the estimation processing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the estimation processing unit can input the motion data to a generation AI, which can estimate the emotion.

[0145] The estimation processing unit can transmit the customer's motion data to the cloud and centrally manage it. The estimation processing unit uses AI to transmit the customer's motion data to the cloud and centrally manage it. For example, the motion data collected from a motion sensor is transmitted to the cloud and centrally managed. The motion data is transmitted to the cloud in real time and centrally managed. The motion data is transmitted to the cloud periodically and centrally managed. This enables centralized management of data by transmitting the customer's motion data to the cloud and centrally managing it. Some or all of the above-mentioned processing in the estimation processing unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the estimation processing unit can input the motion data to the generation AI, which then transmits it to the cloud.

[0146] The estimation processing unit can encrypt the customer's motion data to enhance security. The estimation processing unit can use AI to encrypt the customer's motion data to enhance security. For example, the estimation processing unit can encrypt motion data collected from a motion sensor to enhance security. The estimation processing unit can encrypt motion data in real time to enhance security. The estimation processing unit can encrypt motion data periodically to enhance security. In this way, security is enhanced by encrypting the customer's motion data. Some or all of the above-mentioned processing in the estimation processing unit can be performed using, for example, a generation AI, or can be performed without using a generation AI. For example, the estimation processing unit can input motion data to a generation AI, which can then encrypt the data.

[0147] The notification unit can estimate the customer's emotions and adjust the notification content based on the estimated customer's emotions. The notification unit can estimate the customer's emotions using AI and adjust the notification content based on the estimated customer's emotions. For example, if the customer is relaxed, gentle notification content is provided. If the customer is nervous, quick and concise notification content is provided. If the customer is excited, detailed notification content is provided. This allows for more appropriate notification by adjusting the notification content based on the customer's emotions. Some or all of the above-mentioned processing in the notification unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the notification unit can input emotion data to the generation AI, which can adjust the notification content.

[0148] At the time of notification, the notification unit can refer to the customer's past evaluation history and encourage an appropriate response. The notification unit uses AI to refer to the customer's past evaluation history and encourage an appropriate response. For example, the notification unit refers to the customer's past evaluation history and notifies the customer of an appropriate response. The notification unit notifies the customer of a response to a specific service from the customer's past evaluation history. The notification unit notifies the customer of an optimal response based on the customer's past evaluation history. In this way, an appropriate response can be encouraged by referring to the customer's past evaluation history. Some or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input evaluation history data into the generation AI, which then notifies the customer of an appropriate response.

[0149] The notification unit can determine the priority of the notification based on the customer's current situation at the time of notification. The notification unit uses AI to determine the priority of the notification based on the customer's current situation. For example, if the customer is relaxed, the notification priority is set low. If the customer is nervous, the notification priority is set high. If the customer is excited, the notification priority is adjusted. This enables more appropriate notification by determining the priority of the notification based on the customer's current situation. Some or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input situation data into the generation AI, which can determine the priority of the notification.

[0150] The notification unit can select a notification format and notify in an optimal manner. The notification unit uses AI to select a notification format and notify in an optimal manner. For example, if the customer is relaxed, a gentle voice notification is provided. If the customer is nervous, a quick and concise text notification is provided. If the customer is excited, a detailed visual notification is provided. This allows for more effective notification by selecting the notification format. Some or all of the above-mentioned processing in the notification unit may be performed, for example, using a generation AI, or may be performed without using the generation AI. For example, the notification unit can input notification format data to the generation AI, which can then notify in an optimal manner.

[0151] The notification unit can link the notification result with other systems to improve the overall service. The notification unit uses AI to link the notification result with other systems to improve the overall service. For example, the notification result is linked with other systems to improve the overall service. The notification result is linked with other systems in real time to improve the overall service. The notification result is linked with other systems periodically to improve the overall service. This makes it possible to improve the overall service by linking the notification result with other systems. Some or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the notification result to the generation AI, which can then link with other systems.

[0152] The notification unit can feed back the notification result to the customer and collect opinions for improving the service. The notification unit uses AI to feed back the notification result to the customer and collect opinions for improving the service. For example, the notification result is fed back to the customer and opinions for improving the service. The notification result is fed back to the customer in real time and opinions for improving the service are collected. The notification result is periodically fed back to the customer and opinions for improving the service are collected. In this way, opinions for improving the service can be collected by feeding back the notification result to the customer. A part or all of the above-mentioned processing in the notification unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the notification unit can input the notification result to the generation AI, and the generation AI can feed back the notification result to the customer.

[0153] The generation unit can estimate the customer's emotions and adjust the report content based on the estimated customer's emotions. The generation unit uses AI to estimate the customer's emotions and adjust the report content based on the estimated customer's emotions. For example, if the customer is relaxed, a report is generated in a calm tone. If the customer is nervous, a quick and concise report is generated. If the customer is excited, a report including detailed information is generated. In this way, a more appropriate report is generated by adjusting the report content based on the customer's emotions. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the generation unit can input emotion data into the generation AI, which then adjusts the report content.

[0154] When generating a report, the generation unit can refer to the customer's past evaluation history and indicate areas for improvement. The generation unit uses AI to refer to the customer's past evaluation history and indicate areas for improvement. For example, the generation unit refers to the customer's past evaluation history and generates a report indicating areas for improvement. A report indicating areas for improvement for a specific service is generated from the customer's past evaluation history. A report indicating overall areas for improvement is generated based on the customer's past evaluation history. In this way, a report indicating areas for improvement is generated by referring to the customer's past evaluation history. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the generation unit can input evaluation history data into the generation AI, which generates a report indicating areas for improvement.

[0155] The generation unit can determine the priority of the report based on the customer's current situation when generating the report. The generation unit uses AI to determine the priority of the report based on the customer's current situation. For example, if the customer is relaxed, the priority of the report is set low. If the customer is nervous, the priority of the report is set high. If the customer is excited, the priority of the report is adjusted. In this way, a more appropriate report is generated by determining the priority of the report based on the customer's current situation. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the generation unit can input situation data into the generation AI, which can determine the priority of the report.

[0156] The generation unit can select a report format and generate a report in an optimal manner. The generation unit can use AI to select a report format and generate a report in an optimal manner. For example, if a customer is relaxed, a report is generated in a calm tone. If a customer is nervous, a quick and concise report is generated. If a customer is excited, a report including detailed information is generated. In this way, a more effective report is generated by selecting a report format. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the generation unit can input report format data to the generation AI, which can generate a report in an optimal manner.

[0157] The generation unit can link the report results with other systems to improve services comprehensively. The generation unit uses AI to link the report results with other systems to improve services comprehensively. For example, the report results are linked with other systems to improve services comprehensively. The report results are linked with other systems in real time to improve services comprehensively. The report results are linked with other systems periodically to improve services comprehensively. This makes it possible to improve services comprehensively by linking the report results with other systems. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the generation unit can input the report results to the generation AI, which can then link with other systems.

[0158] The generation unit can feed back the report result to the customer and collect opinions for improving the service. The generation unit uses AI to feed back the report result to the customer and collect opinions for improving the service. For example, the report result is fed back to the customer and opinions for improving the service are collected. The report result is fed back to the customer in real time and opinions for improving the service are collected. The report result is periodically fed back to the customer and opinions for improving the service are collected. In this way, opinions for improving the service can be collected by feeding back the report result to the customer. A part or all of the above-mentioned processing in the generation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the generation unit can input the report result to the generation AI, and the generation AI can feed back the report result to the customer.

[0159] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0160] The collection unit can not only collect data from sensors installed on the customer's table, but also from the customer's smartphone or wearable device. For example, it collects location information and acceleration sensor data from the smartphone to understand the customer's movements and time spent there. It collects heart rate and skin temperature from the wearable device to estimate the customer's stress level and relaxation level. This enables more multifaceted data collection and improves the accuracy of customer emotion estimation.

[0161] The estimation unit can take into account the customer's past behavioral patterns and preferences when estimating the customer's emotions. For example, it can refer to emotion data from when a particular dish was ordered in the past and predict the emotion when the same dish is ordered again. If a customer tends to visit during a particular time period, estimation is made based on emotion data from that time period. This makes it possible to estimate emotions based on the customer's individual preferences and behavioral patterns.

[0162] The judgment unit can suggest improvements to specific services or menu items based on customer emotion data. For example, if there are a lot of negative emotions about a specific dish, it can suggest improvements to the recipe or serving method for that dish. If there are a lot of negative emotions during a specific time period, it can suggest a review of staff deployment or service content during that time period. This makes it possible to make specific improvement suggestions based on customer emotion data.

[0163] The output unit not only notifies the customer service staff of the results of the judgment, but can also provide feedback to the customer themselves. For example, if a customer expresses positive emotions, a message of thanks for those emotions can be sent. If a customer expresses negative emotions, a survey asking about the cause can be sent. This enables two-way communication with customers, which can be used to improve services.

[0164] The collection unit can collect data not only from customers' tables, but also from sensors installed throughout the store. For example, it can grasp the congestion level and noise level in the store in real time and analyze the impact on customer emotions. This makes it possible to estimate emotions taking into account the entire store environment.

[0165] The estimation unit can also take into account the emotion data of other customers when estimating the emotion of a customer. For example, it can refer to the emotion data of other customers sitting at the same table and estimate the emotion of the entire group. This makes it possible to grasp the emotion of not only individual customers but the entire group.

[0166] The judgment unit can support the improvement of the customer service skills of a specific staff member based on the customer emotion data. For example, if there are many negative emotions at a table that a specific staff member is in charge of, the judgment unit can suggest specific improvements to that staff member. This can improve the customer service skills of the staff member.

[0167] The output unit can provide a visual interface when notifying the customer service staff of the result of the judgment. For example, the emotion data can be displayed in a graph or chart so that the customer service staff can intuitively understand the emotion data. This allows the customer service staff to receive information that will allow them to quickly respond.

[0168] The collection unit can collect data not only from customers' tables, but also from other areas of the store, such as restrooms and waiting areas. For example, it can grasp the usage of restrooms and the time spent in waiting areas, and evaluate the customer's overall experience. This can improve the service quality throughout the store.

[0169] The estimation unit can also take into account external factors such as season and weather when estimating a customer's emotions. For example, if negative emotions tend to increase on rainy days, the estimation unit can take these influences into account when estimating emotions. This allows for more accurate emotion estimation that takes external factors into account.

[0170] The process flow of the second embodiment will be briefly described below.

[0171] Step 1: The collection unit collects data from sensors installed at each table. The collection unit uses a camera sensor, microphone sensor, motion sensor, etc. to collect data such as customer facial expressions, tone of voice, voice volume, and movements. For example, the camera sensor captures the customer's facial expressions in real time and collects facial expression data. The microphone sensor captures the customer's voice tone and voice volume in real time and collects voice data. The motion sensor captures the customer's movements in real time and collects movement data. Step 2: The estimation unit estimates the customer's emotions based on the data collected by the collection unit. The estimation unit uses AI to analyze the collected data and estimates emotions from the customer's facial expressions, tone of voice, voice volume, and movements. Step 3: The judgment unit judges the customer's evaluation of the service based on the emotions estimated by the estimation unit. The judgment unit analyzes the emotions estimated using AI based on evaluation criteria and judges the customer's evaluation of the service. For example, it analyzes emotions within a certain time after the food is served and judges the evaluation of the food. It analyzes emotions from the time the customer sits down to the time they leave the table and judges the overall level of satisfaction. If negative emotions are strong, it judges the possibility of dissatisfaction or trouble. Step 4: The output unit outputs the judgment result by the judgment unit. The output unit notifies the customer service staff of the judgment result in real time using AI. For example, the judgment result is displayed to the customer service staff via a web application or a mobile application.

[0172] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0173] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0174] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0175] For example, the collection unit is realized by either the smart device 14 or the data processing device 12. For example, the facial expression and tone of voice of the customer are collected using a camera sensor or a microphone sensor of the smart device 14, and the specific processing unit 290 of the data processing device 12 analyzes these data. The estimation unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and estimates the customer's emotions based on the collected data. The determination unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and determines an evaluation of the service based on the estimated emotions. The output unit is realized by, for example, the control unit 46A of the smart device 14, and notifies the customer service staff of the determination result in real time. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0176] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0177] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0178] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0179] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0180] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0181] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0182] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0183] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0184] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0185] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0186] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0187] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0188] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0189] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0190] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0191] For example, the collection unit is realized by either the smart glasses 214 or the data processing device 12. For example, the facial expression and tone of voice of the customer are collected using a camera sensor or a microphone sensor of the smart glasses 214, and the specific processing unit 290 of the data processing device 12 analyzes these data. The estimation unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and estimates the customer's emotion based on the collected data. The determination unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and determines an evaluation of the service based on the estimated emotion. The output unit is realized by, for example, the control unit 46A of the smart glasses 214, and notifies the customer staff of the determination result in real time. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0192] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0193] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0194] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0195] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0196] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0197] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0198] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0199] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0200] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0201] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0202] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0203] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0204] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0205] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0206] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0207] For example, the collection unit is realized by either the headset terminal 314 or the data processing device 12. For example, the facial expression and tone of voice of the customer are collected using a camera sensor or a microphone sensor of the headset terminal 314, and the specific processing unit 290 of the data processing device 12 analyzes these data. The estimation unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and estimates the customer's emotion based on the collected data. The determination unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and determines the evaluation of the service based on the estimated emotion. The output unit is realized by, for example, the control unit 46A of the headset terminal 314, and notifies the customer staff of the determination result in real time. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0208] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0209] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0210] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0211] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0212] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0213] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0214] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0215] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0216] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0217] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0218] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0219] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0220] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0221] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0222] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0223] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0224] For example, the collection unit is realized by either the robot 414 or the data processing device 12. For example, the facial expression and tone of voice of the customer are collected using a camera sensor or a microphone sensor of the robot 414, and the specific processing unit 290 of the data processing device 12 analyzes these data. The estimation unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and estimates the customer's emotions based on the collected data. The determination unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and determines an evaluation of the service based on the estimated emotions. The output unit is realized by, for example, the control unit 46A of the robot 414, and notifies the customer staff of the determination result in real time. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0225] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0226] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0227] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0228] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0229] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0230] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0231] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0232] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0233] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0234] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0235] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0236] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0237] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0238] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0239] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0240] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0241] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0242] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0243] (Appendix 1) A collection unit that collects data from sensors installed at each table; an estimation unit that estimates the emotion of a customer at each table based on the data collected by the collection unit; a determination unit that determines an evaluation of the customer with respect to a service based on the emotion estimated by the estimation unit; an output unit that outputs a determination result by the determination unit; A system characterized by: (Appendix 2) The determination unit is Determining the customer's satisfaction with the service based on the emotions at each stage 2. The system of claim 1, (Appendix 3) The determination unit is Determining the customer's level of satisfaction with the service based on the emotions at each stage of the food and beverage service 2. The system of claim 1, (Appendix 4) The determination unit is Each customer service stage has a different specific weight, and the customer's satisfaction with the service is determined based on the emotion at each stage and the weight. 2. The system of claim 1, (Appendix 5) The estimation unit is A detection processing unit is provided for specifically detecting the facial expression, tone of voice, volume of voice, and movement of the customer based on the data collected by the collection unit. 2. The system of claim 1, (Appendix 6) The estimation unit is An estimation processing unit is provided for estimating the emotion of the customer based on the facial expression, tone of voice, volume of voice, and movement of the customer detected by the detection processing unit. 2. The system of claim 1, (Appendix 7) A notification unit is provided for notifying a responsible wait staff member when the level of the evaluation up to a specific stage in the service is less than a threshold value. 2. The system of claim 1, (Appendix 8) The determination unit is The degree of customer satisfaction is collected for each customer service staff member in real time, and the output unit provides the collected result by the determination unit to each customer service staff member in real time. 2. The system of claim 1, (Appendix 9) The determination unit is A rating of the customer is determined based on a past emotion estimation history of the customer. 2. The system of claim 1, (Appendix 10) A generating unit compares the emotions of the customers with the action logs of the wait staff and generates a report showing areas for improvement. 2. The system of claim 1, (Appendix 11) The collecting unit includes: Data is collected from sensors installed at each table. 2. The system of claim 1, (Appendix 12) The collecting unit includes: Process collected data in real time 2. The system of claim 1, (Appendix 13) The collecting unit includes: Collected data is sent to the cloud for centralized management 2. The system of claim 1, (Appendix 14) The collecting unit includes: Preprocessing collected data and removing noise 2. The system of claim 1, (Appendix 15) The collecting unit includes: Encrypting collected data for added security 2. The system of claim 1, (Appendix 16) The collecting unit includes: Regularly back up collected data 2. The system of claim 1, (Appendix 17) The estimation unit is Infer customer sentiment based on collected data 2. The system of claim 1, (Appendix 18) The estimation unit is Adjust the estimation algorithm to improve the accuracy of emotion estimation. 2. The system of claim 1, (Appendix 19) The estimation unit is Updates inference results in real time to reflect the latest emotional state 2. The system of claim 1, (Appendix 20) The estimation unit is Compare the inference results with past data to detect anomalies 2. The system of claim 1, (Appendix 21) The estimation unit is Send the estimation results to the cloud and manage them centrally 2. The system of claim 1, (Appendix 22) The estimation unit is Encrypting the estimation results to enhance security 2. The system of claim 1, (Appendix 23) The determination unit is Determine a rating for a service based on the estimated sentiment 2. The system of claim 1, (Appendix 24) The determination unit is Weighting emotions at each service stage to calculate overall satisfaction 2. The system of claim 1, (Appendix 25) The determination unit is Refer to past evaluation history and reflect it in the current evaluation 2. The system of claim 1, (Appendix 26) The determination unit is Adjusting evaluation criteria based on estimated emotions 2. The system of claim 1, (Appendix 27) The determination unit is Real-time updates of evaluation results to reflect the latest evaluation status 2. The system of claim 1, (Appendix 28) The determination unit is Evaluation results are sent to the cloud for centralized management 2. The system of claim 1, (Appendix 29) The output unit is Notify customer service staff of the results in real time 2. The system of claim 1, (Appendix 30) The output unit is Compare the results with past data and analyze trends 2. The system of claim 1, (Appendix 31) The output unit is The results will be linked to other systems to improve overall services. 2. The system of claim 1, (Appendix 32) The output unit is Feedback the results to customers and collect their opinions to improve services. 2. The system of claim 1, (Appendix 33) The output unit is The results are visually displayed and provided to the customer service staff. 2. The system of claim 1, (Appendix 34) The output unit is The result is notified by voice and provided to the customer service staff. 2. The system of claim 1, (Appendix 35) The detection processing unit is Detecting customer facial expressions and using them as auxiliary information for emotion estimation 2. The system of claim 1, (Appendix 36) The detection processing unit is Detecting the tone of a customer's voice and using it as auxiliary information for emotion estimation 2. The system of claim 1, (Appendix 37) The detection processing unit is Detecting the volume of a customer's voice and using it as auxiliary information for emotion estimation 2. The system of claim 1, (Appendix 38) The detection processing unit is Detecting customer movements and using them as auxiliary information for emotion estimation 2. The system of claim 1, (Appendix 39) The detection processing unit is Customer movement data is sent to the cloud and centrally managed 2. The system of claim 1, (Appendix 40) The detection processing unit is Encrypt customer movement data for added security 2. The system of claim 1, (Appendix 41) The estimation processing unit: Analyzing customer facial expression data to estimate emotions 2. The system of claim 1, (Appendix 42) The estimation processing unit: Analyzing customer tone of voice data and estimating emotions 2. The system of claim 1, (Appendix 43) The estimation processing unit: Analyze customer voice volume data and estimate emotions 2. The system of claim 1, (Appendix 44) The estimation processing unit: Analyzing customer movement data and inferring emotions 2. The system of claim 1, (Appendix 45) The estimation processing unit: Customer movement data is sent to the cloud and centrally managed 2. The system of claim 1, (Appendix 46) The estimation processing unit: Encrypt customer movement data for added security 2. The system of claim 1, (Appendix 47) The notification unit is Infer customer sentiment and tailor notifications based on the inferred sentiment 2. The system of claim 1, (Appendix 48) The notification unit is When notifying customers, refer to their past evaluation history and encourage them to take appropriate action. 2. The system of claim 1, (Appendix 49) The notification unit is At the time of notification, prioritize notifications based on the customer's current situation 2. The system of claim 1, (Appendix 50) The notification unit is Select the notification format and notify in the most optimal way 2. The system of claim 1, (Appendix 51) The notification unit is Linking notification results with other systems to improve overall services 2. The system of claim 1, (Appendix 52) The notification unit is Feedback the notification results to customers and collect their opinions to improve our services 2. The system of claim 1, (Appendix 53) The generation unit is Estimate customer sentiment and adjust report content based on estimated customer sentiment 2. The system of claim 1, (Appendix 54) The generation unit is When generating reports, refer to the customer's past evaluation history and indicate areas for improvement. 2. The system of claim 1, (Appendix 55) The generation unit is Prioritize reports based on the customer's current situation when generating reports 2. The system of claim 1, (Appendix 56) The generation unit is Select a report format and generate the report in the most optimal way 2. The system of claim 1, (Appendix 57) The generation unit is Link report results with other systems to improve overall services 2. The system of claim 1, (Appendix 58) The generation unit is Provide feedback on report results to customers and collect their opinions to improve services 2. The system of claim 1, [Explanation of symbols]

[0244] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A collection unit that collects data from sensors installed at each table; an estimation unit that estimates the emotion of a customer at each table based on the data collected by the collection unit; a determination unit that determines an evaluation of the customer with respect to a service based on the emotion estimated by the estimation unit; an output unit that outputs a determination result by the determination unit; A system characterized by:

2. The determination unit is Determining the customer's satisfaction with the service based on the emotions at each stage 2. The system of claim 1 .

3. The determination unit is Determining the customer's level of satisfaction with the service based on the emotions at each stage of the food and beverage service 2. The system of claim 1 .

4. The determination unit is Each customer service stage has a different specific weight, and the customer's satisfaction with the service is determined based on the emotion at each stage and the weight.

2. The system of claim 1 .

5. The estimation unit is A detection processing unit is provided for specifically detecting the facial expression, tone of voice, volume of voice, and movement of the customer based on the data collected by the collection unit.

2. The system of claim 1 .

6. The estimation unit is an estimation processing unit that estimates the emotion of the customer based on the facial expression, tone of voice, volume of voice, and movement of the customer detected by the detection processing unit; 6. The system of claim 5.

7. A notification unit is provided for notifying a responsible customer service staff member when the level of the evaluation up to a specific stage of the service is less than a threshold value.

2. The system of claim 1 .

8. The determination unit is The degree of customer satisfaction is collected for each customer service staff member in real time, and the output unit provides the collected result by the determination unit to each customer service staff member in real time.

2. The system of claim 1 .

Images