system

Abstract

The system according to this embodiment aims to control the venue's equipment in response to the emotions of the audience. [Solution] The system according to this embodiment comprises an acquisition unit, an estimation unit, and an equipment control unit. The acquisition unit acquires emotion estimation information, which is information used to estimate the emotions of the audience in the venue. The estimation unit estimates the emotions of the audience based on the emotion estimation information acquired by the acquisition unit. The equipment control unit controls the equipment in the venue based on the emotions estimated by the estimation unit.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, it is impossible to control the equipment in the venue according to the emotions of the audience, and there are limitations in improving the viewing experience.

[0005] The system according to the embodiment aims to control the equipment in the venue according to the emotions of the audience.

Means for Solving the Problems

[0006] The system according to the embodiment includes an acquisition unit, an estimation unit, and a device control unit. The acquisition unit acquires emotion estimation information, which is information used to estimate the emotions of the audience in the venue. The estimation unit estimates the emotions of the audience based on the emotion estimation information acquired by the acquisition unit. The device control unit controls the equipment in the venue based on the emotions estimated by the estimation unit. [Effects of the Invention]

[0007] The system according to this embodiment can control the venue's equipment in response to the audience's emotions. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. Also, the reception device 38, the output device 40, and the camera 42 are connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The spectator support robot according to an embodiment of the present invention is a system for improving the spectator experience at sporting events. This system uses an emotion engine to estimate the emotions of spectators and supports them in enjoying the spectator experience more. Specifically, it consists of the following steps. First, it acquires information used to estimate the emotions of spectators in the venue (e.g., facial expressions, tone of voice, heart rate, etc.). Next, it estimates the emotions of spectators based on the acquired information. Based on the estimated emotions, it controls the venue's equipment (lighting, sound, displays, etc.) and provides information and replay scenes to enhance the excitement of the spectators. It also suggests the optimal viewing position within the venue and a route to avoid congestion based on the spectator's emotion data. First, it acquires information used to estimate the emotions of spectators in the venue. This information is obtained from the spectators' facial expressions, tone of voice, heart rate, etc. For example, it collects information from spectators using cameras, microphones, heart rate sensors, etc. This provides data for estimating the emotions of spectators. Next, it estimates the emotions of spectators based on the acquired information. The emotion engine analyzes the collected data and estimates the emotions of the spectators. For example, it can estimate emotions such as joy, excitement, and surprise from the facial expressions of the audience. This allows the emotional state of the audience to be understood. Based on the estimated emotions, the venue's equipment is controlled. For example, if the audience is excited, the lighting can be brightened or the sound can be enhanced to further increase their excitement. Conversely, if the audience is relaxed, the lighting can be dimmed or the sound can be quieted to create a relaxed atmosphere. This provides an optimal viewing environment tailored to the audience's emotions. Furthermore, based on the audience's emotional data, the optimal viewing position within the venue and routes to avoid congestion can also be suggested. For example, if the audience is excited, a more exciting position can be suggested. Conversely, if the audience is relaxed, a quieter position can be suggested. This allows the audience to enjoy watching the event from an optimal viewing position that matches their emotional state. In this way, the spectator support robot of the present invention supports the audience in enjoying the viewing experience more by estimating their emotions and providing an optimal viewing environment tailored to those emotions.This allows the spectator support robot to control venue equipment based on the emotions of the spectators, thereby improving the viewing experience.

[0029] The spectator support robot according to this embodiment comprises an acquisition unit, an estimation unit, and an equipment control unit. The acquisition unit acquires emotion estimation information, which is information used to estimate the emotions of spectators in the venue. Emotion estimation information includes, for example, the spectators' facial expressions, tone of voice, and heart rate. The acquisition unit can acquire the spectators' facial expressions using a camera, for example. The acquisition unit can also acquire the spectators' tone of voice using a microphone. Furthermore, the acquisition unit can acquire the spectators' heart rate using a heart rate sensor. The estimation unit estimates the emotions of the spectators based on the emotion estimation information acquired by the acquisition unit. The estimation unit can, for example, analyze the spectators' facial expression data to estimate emotions such as joy, excitement, and surprise. Furthermore, the estimation unit can analyze the spectators' tone of voice data to estimate emotions. Furthermore, the estimation unit can analyze the spectators' heart rate data to estimate emotions. The equipment control unit controls the venue's equipment based on the emotions estimated by the estimation unit. For example, the equipment control unit can brighten the lighting if the spectators are excited. The equipment control unit can also enhance the sound. Furthermore, the equipment control unit can also display replay scenes using a display device. This can further enhance the excitement of the audience. The equipment control unit can dim the lights when the audience is relaxed. It can also lower the volume of sound. This can create a relaxed atmosphere. As a result, the spectator support robot according to the embodiment can control the venue's equipment based on the audience's emotions and improve the spectator experience.

[0030] The acquisition unit acquires emotion estimation information, which is used to estimate the emotions of the audience in the venue. Emotion estimation information includes, for example, the audience's facial expressions, tone of voice, and heart rate. The acquisition unit can acquire the audience's facial expressions using a camera, for example. The camera has high resolution and can capture even subtle changes in the audience's facial expressions. This makes it possible to understand the audience's emotions such as joy, surprise, and excitement in detail. The acquisition unit can also acquire the audience's tone of voice using a microphone. The microphone has high sensitivity and can accurately capture changes in the tone and volume of the audience's voice. This makes it possible to understand the audience's heightened or calmer emotions. Furthermore, the acquisition unit can acquire the audience's heart rate using a heart rate sensor. The heart rate sensor is attached to the audience's wrist or chest and monitors their heart rate in real time. This makes it possible to accurately understand the audience's level of excitement and tension. The acquisition unit collects this data centrally and transmits it to the analysis unit in real time. This makes it possible to quickly and accurately understand the emotional state of the audience. Furthermore, the data acquisition unit can adjust the frequency and accuracy of data collection, enabling flexible responses to specific situations and conditions. This allows the data acquisition unit to collect data efficiently and effectively, improving the overall system performance.

[0031] The estimation unit estimates the audience's emotions based on emotion estimation information acquired by the acquisition unit. For example, the estimation unit can analyze the audience's facial expression data to estimate emotions such as joy, excitement, and surprise. Specifically, it utilizes AI-based image recognition technology to extract feature points from the audience's faces and analyze changes in facial expressions. This allows it to capture subtle changes in the audience's facial expressions and estimate emotions with high accuracy. The estimation unit can also analyze the audience's voice tone data to estimate emotions. Using voice analysis technology, it extracts features such as voice pitch, volume, and rhythm to estimate the audience's emotional state. Furthermore, the estimation unit can analyze the audience's heart rate data to estimate emotions. It analyzes the fluctuation patterns of heart rate to evaluate the audience's level of excitement and tension. This allows the estimation unit to integrate multiple pieces of emotion estimation information and comprehensively evaluate the audience's emotional state. In addition, the estimation unit can utilize past data and statistical information to predict long-term emotional fluctuations. For example, based on past viewing data, the system can predict fluctuations in audience emotions during specific match developments or events, optimizing real-time responses. Furthermore, the estimation unit can use anomaly detection algorithms to detect unusual emotional states early and respond quickly. This allows the estimation unit to handle not only real-time emotion estimation but also long-term emotion management and anomaly detection, improving the overall reliability and security of the system.

[0032] The equipment control unit controls the venue's equipment based on the emotions estimated by the estimation unit. For example, if the audience is excited, the equipment control unit can brighten the lights. Specifically, it controls the lighting system in real time, adjusting the brightness and color of the lights according to the audience's level of excitement. This can further enhance the audience's excitement. The equipment control unit can also enhance the sound. It controls the sound system, adjusting the volume and sound quality according to the audience's emotions. This can further heighten the audience's excitement. Furthermore, the equipment control unit can display replay scenes using a display device. When the audience is excited, important scenes or replays can be displayed on a large screen to maintain their excitement. If the audience is relaxed, the equipment control unit can dim the lights. It adjusts the brightness of the lights to create a relaxed atmosphere. The equipment control unit can also quiet the sound. By lowering the volume and providing a quiet environment, it creates a space where the audience can relax. In this way, the equipment control unit can flexibly control the venue's equipment based on the audience's emotions and provide an optimal viewing experience. Furthermore, the equipment control unit can control multiple devices in coordination. For example, by integrating and controlling lighting, sound, and display devices, it is possible to create a comprehensive performance that responds to the emotions of the audience. This allows the equipment control unit to realize sophisticated performances based on the emotions of the audience, thereby improving the viewing experience.

[0033] The viewing position suggestion unit can suggest viewing positions in the venue based on emotion data estimated by the estimation unit. For example, if the audience is excited, the viewing position suggestion unit can suggest a visually stimulating position. For example, if the audience is relaxed, the viewing position suggestion unit can suggest a quiet position. For example, if the audience is feeling anxious, the viewing position suggestion unit can suggest a reassuring position. In this way, the viewing experience can be improved by suggesting the optimal viewing position based on the audience's emotions. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can suggest viewing positions using an AI model that takes emotion data estimated by the estimation unit as input and outputs viewing positions.

[0034] The travel route suggestion unit can suggest a travel route based on the emotion data estimated by the estimation unit. For example, if the audience is excited, the travel route suggestion unit can suggest a visually stimulating route. For example, if the audience is relaxed, the travel route suggestion unit can suggest a quiet route. For example, if the audience is feeling anxious, the travel route suggestion unit can suggest a reassuring route. This allows for avoiding congestion by suggesting the optimal travel route based on the audience's emotions. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can suggest a travel route using an AI model that takes emotion data estimated by the estimation unit as input and outputs a travel route.

[0035] The acquisition unit can acquire audience members' facial expressions, voice tone, and heart rate as information for emotion estimation. For example, the acquisition unit can acquire audience members' facial expressions using a camera. For example, the acquisition unit can acquire audience members' voice tone using a microphone. For example, the acquisition unit can acquire audience members' heart rate using a heart rate sensor. By acquiring diverse information about the audience, the accuracy of emotion estimation is improved. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input audience facial expression data acquired by the camera into the generative AI and have the generative AI perform the acquisition of emotion estimation information.

[0036] The equipment control unit can control venue equipment such as lighting, sound, and displays based on the emotions estimated by the estimation unit. For example, the equipment control unit can brighten the lights if the audience is excited. For example, the equipment control unit can enhance the sound. For example, the equipment control unit can display replay scenes using a display device. This can further increase the audience's excitement. For example, the equipment control unit can dim the lights if the audience is relaxed. For example, the equipment control unit can quiet the sound. This can create a relaxed atmosphere. This can provide an optimal viewing environment that responds to the audience's emotions. Some or all of the above processing in the equipment control unit may be performed using AI, for example, or without AI. For example, the equipment control unit can control equipment using an AI model that takes emotion data estimated by the estimation unit as input and outputs a method for controlling the equipment.

[0037] The acquisition unit can estimate the audience's emotions and adjust the timing of acquiring emotion estimation information based on the estimated audience emotions. For example, if the audience is excited, the acquisition unit can acquire emotion estimation information frequently in real time. For example, if the audience is relaxed, the acquisition unit can widen the interval between acquiring emotion estimation information. For example, if the audience is feeling anxious, the acquisition unit can periodically acquire emotion estimation information and monitor changes. This allows for the acquisition of more appropriate information by adjusting the timing of information acquisition according to the audience's emotions. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input audience emotion data into a generating AI and have the generating AI adjust the timing of acquiring emotion estimation information.

[0038] The data acquisition unit can analyze past emotional data of the audience and select an appropriate acquisition method. For example, the data acquisition unit can identify the timing of heightened emotions during a specific event from past emotional data and acquire information at that timing. For example, the data acquisition unit can analyze the emotional change patterns of a specific audience member based on past emotional data and select the optimal acquisition method. For example, the data acquisition unit can refer to past emotional data and adjust the frequency of information acquisition at specific emotional states. This makes it possible to acquire information more effectively by analyzing past emotional data. Some or all of the above processing in the data acquisition unit may be performed using AI, for example, or without AI. For example, the data acquisition unit can input past emotional data into a generating AI and have the generating AI select the optimal acquisition method.

[0039] The data acquisition unit can filter the information for sentiment estimation based on the audience's current activity status and areas of interest. For example, if an audience member is interested in a particular sport, the data acquisition unit can prioritize acquiring information related to that sport. For example, if an audience member is taking a break, the data acquisition unit can prioritize acquiring information that helps them relax. For example, if an audience member is paying attention to a particular player, the data acquisition unit can prioritize acquiring information about that player. This allows for the provision of more appropriate information by prioritizing the acquisition of information that matches the audience's interests. Some or all of the above processing in the data acquisition unit may be performed using AI, for example, or without AI. For example, the data acquisition unit can input data on the audience's activity status and areas of interest into a generating AI and have the generating AI perform the filtering.

[0040] The acquisition unit can estimate the audience's emotions and determine the priority of emotion estimation information to acquire based on the estimated audience emotions. For example, if the audience is excited, the acquisition unit can prioritize acquiring information that increases excitement. For example, if the audience is relaxed, the acquisition unit can prioritize acquiring information that promotes relaxation. For example, if the audience is feeling anxious, the acquisition unit can prioritize acquiring information that alleviates anxiety. By prioritizing information according to the audience's emotions, more effective information provision becomes possible. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input audience emotion data into a generating AI and have the generating AI perform the determination of information prioritization.

[0041] The acquisition unit can prioritize acquiring highly relevant information based on the geographical location information of spectators when acquiring information for emotion estimation. For example, if a spectator is in a specific area of ​​the stadium, the acquisition unit can prioritize acquiring information related to that area. For example, if a spectator is moving, the acquisition unit can prioritize acquiring information related to their destination. For example, if a spectator is in a specific seat, the acquisition unit can prioritize acquiring information related to the viewpoint from that seat. This makes it possible to provide more appropriate information by acquiring highly relevant information based on the spectator's location information. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input the geographical location information of spectators into a generating AI and have the generating AI perform the acquisition of highly relevant information.

[0042] The acquisition unit can analyze the audience's social media activity and acquire relevant information when acquiring information for sentiment estimation. For example, if an audience member mentions a specific player on social media, the acquisition unit can prioritize acquiring information about that player. For example, if an audience member mentions a specific match on social media, the acquisition unit can prioritize acquiring information about that match. For example, if an audience member mentions a specific event on social media, the acquisition unit can prioritize acquiring information about that event. This makes it possible to provide more appropriate information by acquiring relevant information based on the audience member's social media activity. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input audience social media activity data into a generating AI and have the generating AI acquire relevant information.

[0043] The estimation unit can estimate the audience's emotions and adjust the emotion estimation algorithm based on the estimated emotions. For example, if the audience is excited, the estimation unit can speed up the emotion estimation algorithm to estimate emotions in real time. For example, if the audience is relaxed, the estimation unit can loosen the emotion estimation algorithm to perform a more detailed emotional analysis. For example, if the audience is anxious, the estimation unit can make the emotion estimation algorithm more sensitive to capture subtle emotional changes. By adjusting the algorithm according to the audience's emotions, the accuracy of emotion estimation is improved. Some or all of the above processing in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input audience emotion data into a generating AI and have the generating AI adjust the emotion estimation algorithm.

[0044] The estimation unit can adjust the level of detail of the estimation based on the importance of the information used for sentiment estimation during the estimation process. For example, the estimation unit can perform detailed sentiment estimation based on information of high importance. For example, the estimation unit can perform simplified sentiment estimation based on information of low importance. For example, the estimation unit can perform sentiment estimation with a moderate level of detail based on information of moderate importance. By adjusting the level of detail of the estimation according to the importance of the information, efficient sentiment estimation becomes possible. Some or all of the above-described processes in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input importance data of the information used for sentiment estimation into a generating AI and have the generating AI perform the adjustment of the level of detail of the estimation.

[0045] The estimation unit can apply different estimation algorithms depending on the category of information used for emotion estimation during estimation. For example, the estimation unit can apply a facial expression recognition algorithm based on facial expression data. For example, the estimation unit can apply a voice emotion recognition algorithm based on voice tone data. For example, the estimation unit can apply a physiological emotion recognition algorithm based on heart rate data. By applying an algorithm according to the category of information, the accuracy of emotion estimation is improved. Some or all of the above processing in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input category data of information used for emotion estimation into a generating AI and cause the generating AI to apply different estimation algorithms.

[0046] The estimation unit can estimate the audience's emotions and adjust the display method of the estimation results based on the estimated emotions. For example, if the audience is excited, the estimation unit can provide a visually stimulating display method. For example, if the audience is relaxed, the estimation unit can provide a calming display method. For example, if the audience is feeling anxious, the estimation unit can provide a reassuring display method. By providing a display method that corresponds to the audience's emotions, it becomes possible to provide more appropriate information. Some or all of the above processing in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input audience emotion data into a generating AI and have the generating AI adjust the display method of the estimation results.

[0047] The estimation unit can determine the estimation priority based on the timing of acquisition of sentiment estimation information during estimation. For example, the estimation unit can perform estimation by prioritizing the use of the most recent sentiment estimation information. For example, the estimation unit can perform estimation by referring to past sentiment estimation information. For example, the estimation unit can perform estimation by prioritizing the use of sentiment estimation information acquired during a specific event. This enables efficient sentiment estimation by determining the estimation priority based on the timing of information acquisition. Some or all of the above processing in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input data on the timing of acquisition of sentiment estimation information into a generating AI and have the generating AI perform the determination of the estimation priority.

[0048] The estimation unit can adjust the order of estimation based on the relevance of the information used for sentiment estimation. For example, the estimation unit can prioritize using information with high relevance. For example, the estimation unit can then use information with moderate relevance. For example, the estimation unit can then use information with low relevance last. By adjusting the order of estimation based on the relevance of the information, efficient sentiment estimation becomes possible. Some or all of the above processing in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input relevance data of the information used for sentiment estimation into a generating AI and have the generating AI perform the adjustment of the estimation order.

[0049] The equipment control unit can estimate the emotions of the audience and adjust the equipment control method based on the estimated emotions. For example, if the audience is excited, the equipment control unit can brighten the lights and enhance the sound. For example, if the audience is relaxed, the equipment control unit can dim the lights and quiet the sound. For example, if the audience is feeling anxious, the equipment control unit can soften the lights and gentle the sound. This allows for a more appropriate viewing environment by controlling the equipment in accordance with the emotions of the audience. Some or all of the above processing in the equipment control unit may be performed using AI, for example, or without AI. For example, the equipment control unit can input audience emotion data into a generating AI and have the generating AI perform the adjustment of the equipment control method.

[0050] The device control unit can analyze past emotional data of the audience and select the optimal control method when controlling the device. For example, the device control unit can select the optimal lighting settings for a specific emotional state from past emotional data. For example, the device control unit can select the optimal sound settings for a specific emotional state based on past emotional data. For example, the device control unit can refer to past emotional data and select the optimal display settings for a specific emotional state. This enables more effective device control by analyzing past emotional data. Some or all of the above processing in the device control unit may be performed using AI, for example, or without AI. For example, the device control unit can input past emotional data into a generating AI and have the generating AI select the optimal control method.

[0051] The device control unit can customize the control means based on the audience's current emotional state when controlling the device. For example, if the audience is excited, the device control unit can change the color of the lighting to provide visual stimulation. For example, if the audience is relaxed, the device control unit can adjust the volume of the sound to provide a quiet environment. For example, if the audience is feeling anxious, the device control unit can change the display content to provide a sense of security. In this way, a more appropriate viewing environment can be provided by providing control means that correspond to the audience's current emotional state. Some or all of the above processing in the device control unit may be performed using AI, for example, or without AI. For example, the device control unit can input the audience's current emotional state data into a generating AI and have the generating AI perform the customization of the control means.

[0052] The equipment control unit can estimate the emotions of the audience and determine the priority of equipment control based on the estimated emotions. For example, if the audience is excited, the equipment control unit can prioritize lighting control. For example, if the audience is relaxed, the equipment control unit can prioritize sound control. For example, if the audience is feeling anxious, the equipment control unit can prioritize display control. By determining the priority of equipment control according to the emotions of the audience, a more effective viewing environment can be provided. Some or all of the above processing in the equipment control unit may be performed using AI, for example, or without AI. For example, the equipment control unit can input audience emotion data into a generating AI and have the generating AI perform the determination of equipment control priorities.

[0053] The equipment control unit can select the optimal control method when controlling equipment, taking into account the geographical location information of spectators. For example, if a spectator is in a specific area of ​​the stadium, the equipment control unit can select the optimal lighting settings for that area. For example, if a spectator is moving, the equipment control unit can select the optimal sound settings for their destination. For example, if a spectator is in a specific seat, the equipment control unit can select the optimal display settings from the viewpoint of that seat. By selecting the optimal control method based on the geographical location information of spectators, a more appropriate viewing environment can be provided. Some or all of the above processing in the equipment control unit may be performed using AI, for example, or without AI. For example, the equipment control unit can input the geographical location information of spectators into a generating AI and have the generating AI select the optimal control method.

[0054] The equipment control unit can analyze the audience's social media activity and propose control measures when controlling the equipment. For example, if an audience member mentions a particular player on social media, the equipment control unit can propose lighting settings related to that player. For example, if an audience member mentions a particular match on social media, the equipment control unit can propose sound settings related to that match. For example, if an audience member mentions a particular event on social media, the equipment control unit can propose display settings related to that event. This allows for a more appropriate viewing environment to be provided by proposing control measures based on the audience's social media activity. Some or all of the above processing in the equipment control unit may be performed using AI, for example, or without AI. For example, the equipment control unit can input audience social media activity data into a generating AI and have the generating AI execute the proposal of control measures.

[0055] The viewing position suggestion unit can estimate the emotions of the audience and adjust the method of suggesting viewing positions based on the estimated emotions. For example, if the audience is excited, the viewing position suggestion unit can suggest a visually stimulating position. For example, if the audience is relaxed, the viewing position suggestion unit can suggest a quiet position. For example, if the audience is feeling anxious, the viewing position suggestion unit can suggest a position that provides a sense of security. By providing a method of suggesting viewing positions that corresponds to the emotions of the audience, a more appropriate viewing environment can be provided. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input audience emotion data into a generating AI and have the generating AI perform adjustments to the viewing position suggestion method.

[0056] The viewing position suggestion unit can provide optimal suggestions by referring to the audience's past viewing history when suggesting viewing positions. For example, the viewing position suggestion unit can suggest the optimal viewing position based on the viewing positions the audience has preferred to view in the past. For example, the viewing position suggestion unit can suggest the optimal viewing position for a specific event based on the audience's past viewing history. For example, the viewing position suggestion unit can analyze the audience's past viewing history and suggest a viewing position that suits the audience's preferences. This allows for a more appropriate viewing environment by suggesting the optimal viewing position based on the audience's past viewing history. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input the audience's past viewing history data into a generating AI and have the generating AI perform the optimal viewing position suggestion.

[0057] The viewing position suggestion unit can customize its suggestions based on the audience's current emotional state when suggesting viewing positions. For example, if the audience is excited, the unit can suggest a visually stimulating position. If the audience is relaxed, the unit can suggest a quiet position. If the audience is feeling anxious, the unit can suggest a reassuring position. By providing viewing position suggestions that correspond to the audience's current emotional state, a more appropriate viewing environment can be provided. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input the audience's current emotional state data into a generating AI and have the generating AI customize the suggestions.

[0058] The viewing position suggestion unit can estimate the emotions of the audience and determine the priority of viewing positions based on the estimated emotions. For example, if the audience is excited, the viewing position suggestion unit can prioritize suggesting visually stimulating positions. For example, if the audience is relaxed, the viewing position suggestion unit can prioritize suggesting quiet positions. For example, if the audience is feeling anxious, the viewing position suggestion unit can prioritize suggesting positions that provide a sense of security. By determining the priority of viewing positions according to the emotions of the audience, a more appropriate viewing environment can be provided. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input audience emotion data into a generating AI and have the generating AI perform the determination of the priority of viewing positions.

[0059] The viewing position suggestion unit can provide optimal suggestions by considering the geographical location information of spectators when suggesting viewing positions. For example, if a spectator is in a specific area of ​​the stadium, the viewing position suggestion unit can suggest the optimal viewing position for that area. For example, if a spectator is moving, the viewing position suggestion unit can suggest the optimal viewing position for their destination. For example, if a spectator is in a specific seat, the viewing position suggestion unit can suggest the optimal viewing position from the viewpoint of that seat. By suggesting the optimal viewing position based on the geographical location information of spectators, a more appropriate viewing environment can be provided. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input the geographical location information of spectators into a generating AI and have the generating AI perform the optimal viewing position suggestion.

[0060] The viewing position suggestion unit can analyze the audience's social media activity when suggesting viewing positions and propose relevant viewing positions. For example, if an audience member mentions a particular player on social media, the viewing position suggestion unit can suggest a position where that player can be seen well. For example, if an audience member mentions a particular match on social media, the viewing position suggestion unit can suggest a position where that match can be seen well. For example, if an audience member mentions a particular event on social media, the viewing position suggestion unit can suggest a position where that event can be seen well. This allows for the provision of a more appropriate viewing environment by suggesting relevant viewing positions based on the audience member's social media activity. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input audience social media activity data into a generating AI and have the generating AI perform the suggestion of relevant viewing positions.

[0061] The travel route suggestion unit can estimate the emotions of the audience and adjust the method of suggesting travel routes based on the estimated emotions. For example, if the audience is excited, the travel route suggestion unit can suggest a visually stimulating route. For example, if the audience is relaxed, the travel route suggestion unit can suggest a quiet route. For example, if the audience is feeling anxious, the travel route suggestion unit can suggest a reassuring route. By providing a method of suggesting travel routes that responds to the emotions of the audience, a more appropriate viewing environment can be provided. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can input audience emotion data into a generating AI and have the generating AI adjust the method of suggesting travel routes.

[0062] The travel route suggestion unit can provide the optimal suggestion by referring to the spectator's past travel history when suggesting a travel route. For example, the travel route suggestion unit can suggest the optimal travel route based on routes that the spectator has preferred to use in the past. For example, the travel route suggestion unit can suggest the optimal travel route for a specific event based on the spectator's past travel history. For example, the travel route suggestion unit can analyze the spectator's past travel history and suggest a travel route that suits the spectator's preferences. This allows for a more appropriate viewing environment by suggesting the optimal travel route based on the spectator's past travel history. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can input the spectator's past travel history data into a generating AI and have the generating AI execute the suggestion of the optimal travel route.

[0063] The route suggestion unit can customize the suggested routes based on the audience's current emotional state. For example, if the audience is excited, the route suggestion unit can suggest a visually stimulating route. If the audience is relaxed, the route suggestion unit can suggest a calm route. If the audience is feeling anxious, the route suggestion unit can suggest a reassuring route. By providing route suggestions that correspond to the audience's current emotional state, a more appropriate viewing environment can be provided. Some or all of the above processing in the route suggestion unit may be performed using AI, for example, or without AI. For example, the route suggestion unit can input the audience's current emotional state data into a generating AI and have the generating AI customize the suggested routes.

[0064] The travel route suggestion unit can estimate the emotions of the audience and determine the priority of travel routes based on the estimated emotions. For example, if the audience is excited, the travel route suggestion unit can prioritize suggesting a visually stimulating route. For example, if the audience is relaxed, the travel route suggestion unit can prioritize suggesting a quiet route. For example, if the audience is feeling anxious, the travel route suggestion unit can prioritize suggesting a route that provides a sense of security. By determining the priority of travel routes according to the emotions of the audience, a more appropriate viewing environment can be provided. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can input audience emotion data into a generating AI and have the generating AI perform the determination of travel route priorities.

[0065] The travel route suggestion unit can provide optimal suggestions by considering the geographical location information of spectators when suggesting travel routes. For example, if a spectator is in a specific area of ​​the stadium, the travel route suggestion unit can suggest the optimal travel route for that area. For example, if a spectator is moving, the travel route suggestion unit can suggest the optimal travel route for their destination. For example, if a spectator is in a specific seat, the travel route suggestion unit can suggest the optimal travel route from the viewpoint of that seat. By suggesting the optimal travel route based on the geographical location information of spectators, a more appropriate viewing environment can be provided. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can input the geographical location information of spectators into a generating AI and have the generating AI perform the task of suggesting the optimal travel route.

[0066] The travel route suggestion unit can analyze the spectator's social media activity when suggesting travel routes and propose relevant routes. For example, if a spectator mentions a particular player on social media, the travel route suggestion unit can propose a route that allows for a good view of that player. For example, if a spectator mentions a particular match on social media, the travel route suggestion unit can propose a route that allows for a good view of that match. For example, if a spectator mentions a particular event on social media, the travel route suggestion unit can propose a route that allows for a good view of that event. This allows for the provision of a more appropriate viewing environment by suggesting relevant travel routes based on the spectator's social media activity. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can input the spectator's social media activity data into a generating AI and have the generating AI execute the suggestion of relevant travel routes.

[0067] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0068] The spectator support robot can analyze a spectator's past viewing history and suggest a personalized viewing plan based on their preferred viewing style. For example, if a spectator has previously preferred watching a particular team's games, the robot can prioritize suggesting games featuring that team. Similarly, if a spectator has previously preferred a specific seat location, the robot can suggest that location. Furthermore, by analyzing the spectator's preferences from their past viewing history, the robot can provide the optimal viewing plan. This allows for a personalized viewing experience based on the spectator's past viewing history.

[0069] The spectator support robot can suggest the optimal viewing spot within the venue based on the spectator's current location. For example, if a spectator is in a specific area of ​​the stadium, it can suggest the best viewing spot from that area. Furthermore, if a spectator is moving, it can suggest the optimal viewing spot for their next destination. Additionally, if a spectator is in a specific seat, it can suggest the optimal viewing spot based on the view from that seat. This allows for the provision of the best viewing spot based on the spectator's location.

[0070] The spectator support robot can analyze spectators' social media activity and provide real-time information about players and teams they are interested in. For example, if a spectator mentions a specific player on social media, it can provide that player's latest statistics and news. Similarly, if a spectator mentions a specific team, it can provide that team's match results and statistics. Furthermore, if a spectator mentions a specific event, it can provide detailed information about that event. This allows for the provision of relevant information based on spectators' social media activity.

[0071] The following briefly describes the processing flow for example form 1.

[0072] Step 1: The acquisition unit acquires emotion estimation information, which is information used to estimate the emotions of the audience in the venue. Emotion estimation information includes, for example, the audience's facial expressions, tone of voice, and heart rate. The acquisition unit can acquire the audience's facial expressions using a camera, for example. The acquisition unit can also acquire the audience's tone of voice using a microphone. Furthermore, the acquisition unit can acquire the audience's heart rate using a heart rate sensor. Step 2: The estimation unit estimates the audience's emotions based on the emotion estimation information acquired by the acquisition unit. For example, the estimation unit can analyze the audience's facial expression data to estimate emotions such as joy, excitement, or surprise. The estimation unit can also analyze the audience's voice tone data to estimate emotions. Furthermore, the estimation unit can analyze the audience's heart rate data to estimate emotions. Step 3: The equipment control unit controls the venue's equipment based on the emotions estimated by the estimation unit. For example, if the audience is excited, the equipment control unit can brighten the lights. It can also enhance the sound. Furthermore, the equipment control unit can display replay scenes using a display device. This can further increase the audience's excitement. If the audience is relaxed, the equipment control unit can dim the lights. It can also quiet the sound. This can create a relaxed atmosphere.

[0073] (Example of form 2) The spectator support robot according to an embodiment of the present invention is a system for improving the spectator experience at sporting events. This system uses an emotion engine to estimate the emotions of spectators and supports them in enjoying the spectator experience more. Specifically, it consists of the following steps. First, it acquires information used to estimate the emotions of spectators in the venue (e.g., facial expressions, tone of voice, heart rate, etc.). Next, it estimates the emotions of spectators based on the acquired information. Based on the estimated emotions, it controls the venue's equipment (lighting, sound, displays, etc.) and provides information and replay scenes to enhance the excitement of the spectators. It also suggests the optimal viewing position within the venue and a route to avoid congestion based on the spectator's emotion data. First, it acquires information used to estimate the emotions of spectators in the venue. This information is obtained from the spectators' facial expressions, tone of voice, heart rate, etc. For example, it collects information from spectators using cameras, microphones, heart rate sensors, etc. This provides data for estimating the emotions of spectators. Next, it estimates the emotions of spectators based on the acquired information. The emotion engine analyzes the collected data and estimates the emotions of the spectators. For example, it can estimate emotions such as joy, excitement, and surprise from the facial expressions of the audience. This allows the emotional state of the audience to be understood. Based on the estimated emotions, the venue's equipment is controlled. For example, if the audience is excited, the lighting can be brightened or the sound can be enhanced to further increase their excitement. Conversely, if the audience is relaxed, the lighting can be dimmed or the sound can be quieted to create a relaxed atmosphere. This provides an optimal viewing environment tailored to the audience's emotions. Furthermore, based on the audience's emotional data, the optimal viewing position within the venue and routes to avoid congestion can also be suggested. For example, if the audience is excited, a more exciting position can be suggested. Conversely, if the audience is relaxed, a quieter position can be suggested. This allows the audience to enjoy watching the event from an optimal viewing position that matches their emotional state. In this way, the spectator support robot of the present invention supports the audience in enjoying the viewing experience more by estimating their emotions and providing an optimal viewing environment tailored to those emotions.This allows the spectator support robot to control venue equipment based on the emotions of the spectators, thereby improving the viewing experience.

[0074] The spectator support robot according to this embodiment comprises an acquisition unit, an estimation unit, and an equipment control unit. The acquisition unit acquires emotion estimation information, which is information used to estimate the emotions of spectators in the venue. Emotion estimation information includes, for example, the spectators' facial expressions, tone of voice, and heart rate. The acquisition unit can acquire the spectators' facial expressions using a camera, for example. The acquisition unit can also acquire the spectators' tone of voice using a microphone. Furthermore, the acquisition unit can acquire the spectators' heart rate using a heart rate sensor. The estimation unit estimates the emotions of the spectators based on the emotion estimation information acquired by the acquisition unit. The estimation unit can, for example, analyze the spectators' facial expression data to estimate emotions such as joy, excitement, and surprise. Furthermore, the estimation unit can analyze the spectators' tone of voice data to estimate emotions. Furthermore, the estimation unit can analyze the spectators' heart rate data to estimate emotions. The equipment control unit controls the venue's equipment based on the emotions estimated by the estimation unit. For example, the equipment control unit can brighten the lighting if the spectators are excited. The equipment control unit can also enhance the sound. Furthermore, the equipment control unit can also display replay scenes using a display device. This can further enhance the excitement of the audience. The equipment control unit can dim the lights when the audience is relaxed. It can also lower the volume of sound. This can create a relaxed atmosphere. As a result, the spectator support robot according to the embodiment can control the venue's equipment based on the audience's emotions and improve the spectator experience.

[0075] The acquisition unit acquires emotion estimation information, which is used to estimate the emotions of the audience in the venue. Emotion estimation information includes, for example, the audience's facial expressions, tone of voice, and heart rate. The acquisition unit can acquire the audience's facial expressions using a camera, for example. The camera has high resolution and can capture even subtle changes in the audience's facial expressions. This makes it possible to understand the audience's emotions such as joy, surprise, and excitement in detail. The acquisition unit can also acquire the audience's tone of voice using a microphone. The microphone has high sensitivity and can accurately capture changes in the tone and volume of the audience's voice. This makes it possible to understand the audience's heightened or calmer emotions. Furthermore, the acquisition unit can acquire the audience's heart rate using a heart rate sensor. The heart rate sensor is attached to the audience's wrist or chest and monitors their heart rate in real time. This makes it possible to accurately understand the audience's level of excitement and tension. The acquisition unit collects this data centrally and transmits it to the analysis unit in real time. This makes it possible to quickly and accurately understand the emotional state of the audience. Furthermore, the data acquisition unit can adjust the frequency and accuracy of data collection, enabling flexible responses to specific situations and conditions. This allows the data acquisition unit to collect data efficiently and effectively, improving the overall system performance.

[0076] The estimation unit estimates the audience's emotions based on emotion estimation information acquired by the acquisition unit. For example, the estimation unit can analyze the audience's facial expression data to estimate emotions such as joy, excitement, and surprise. Specifically, it utilizes AI-based image recognition technology to extract feature points from the audience's faces and analyze changes in facial expressions. This allows it to capture subtle changes in the audience's facial expressions and estimate emotions with high accuracy. The estimation unit can also analyze the audience's voice tone data to estimate emotions. Using voice analysis technology, it extracts features such as voice pitch, volume, and rhythm to estimate the audience's emotional state. Furthermore, the estimation unit can analyze the audience's heart rate data to estimate emotions. It analyzes the fluctuation patterns of heart rate to evaluate the audience's level of excitement and tension. This allows the estimation unit to integrate multiple pieces of emotion estimation information and comprehensively evaluate the audience's emotional state. In addition, the estimation unit can utilize past data and statistical information to predict long-term emotional fluctuations. For example, based on past viewing data, the system can predict fluctuations in audience emotions during specific match developments or events, optimizing real-time responses. Furthermore, the estimation unit can use anomaly detection algorithms to detect unusual emotional states early and respond quickly. This allows the estimation unit to handle not only real-time emotion estimation but also long-term emotion management and anomaly detection, improving the overall reliability and security of the system.

[0077] The equipment control unit controls the venue's equipment based on the emotions estimated by the estimation unit. For example, if the audience is excited, the equipment control unit can brighten the lights. Specifically, it controls the lighting system in real time, adjusting the brightness and color of the lights according to the audience's level of excitement. This can further enhance the audience's excitement. The equipment control unit can also enhance the sound. It controls the sound system, adjusting the volume and sound quality according to the audience's emotions. This can further heighten the audience's excitement. Furthermore, the equipment control unit can display replay scenes using a display device. When the audience is excited, important scenes or replays can be displayed on a large screen to maintain their excitement. If the audience is relaxed, the equipment control unit can dim the lights. It adjusts the brightness of the lights to create a relaxed atmosphere. The equipment control unit can also quiet the sound. By lowering the volume and providing a quiet environment, it creates a space where the audience can relax. In this way, the equipment control unit can flexibly control the venue's equipment based on the audience's emotions and provide an optimal viewing experience. Furthermore, the equipment control unit can control multiple devices in coordination. For example, by integrating and controlling lighting, sound, and display devices, it is possible to create a comprehensive performance that responds to the emotions of the audience. This allows the equipment control unit to realize sophisticated performances based on the emotions of the audience, thereby improving the viewing experience.

[0078] The viewing position suggestion unit can suggest viewing positions in the venue based on emotion data estimated by the estimation unit. For example, if the audience is excited, the viewing position suggestion unit can suggest a visually stimulating position. For example, if the audience is relaxed, the viewing position suggestion unit can suggest a quiet position. For example, if the audience is feeling anxious, the viewing position suggestion unit can suggest a reassuring position. In this way, the viewing experience can be improved by suggesting the optimal viewing position based on the audience's emotions. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can suggest viewing positions using an AI model that takes emotion data estimated by the estimation unit as input and outputs viewing positions.

[0079] The travel route suggestion unit can suggest a travel route based on the emotion data estimated by the estimation unit. For example, if the audience is excited, the travel route suggestion unit can suggest a visually stimulating route. For example, if the audience is relaxed, the travel route suggestion unit can suggest a quiet route. For example, if the audience is feeling anxious, the travel route suggestion unit can suggest a reassuring route. This allows for avoiding congestion by suggesting the optimal travel route based on the audience's emotions. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can suggest a travel route using an AI model that takes emotion data estimated by the estimation unit as input and outputs a travel route.

[0080] The acquisition unit can acquire audience members' facial expressions, voice tone, and heart rate as information for emotion estimation. For example, the acquisition unit can acquire audience members' facial expressions using a camera. For example, the acquisition unit can acquire audience members' voice tone using a microphone. For example, the acquisition unit can acquire audience members' heart rate using a heart rate sensor. By acquiring diverse information about the audience, the accuracy of emotion estimation is improved. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input audience facial expression data acquired by the camera into the generative AI and have the generative AI perform the acquisition of emotion estimation information.

[0081] The equipment control unit can control venue equipment such as lighting, sound, and displays based on the emotions estimated by the estimation unit. For example, the equipment control unit can brighten the lights if the audience is excited. For example, the equipment control unit can enhance the sound. For example, the equipment control unit can display replay scenes using a display device. This can further increase the audience's excitement. For example, the equipment control unit can dim the lights if the audience is relaxed. For example, the equipment control unit can quiet the sound. This can create a relaxed atmosphere. This can provide an optimal viewing environment that responds to the audience's emotions. Some or all of the above processing in the equipment control unit may be performed using AI, for example, or without AI. For example, the equipment control unit can control equipment using an AI model that takes emotion data estimated by the estimation unit as input and outputs a method for controlling the equipment.

[0082] The acquisition unit can estimate the audience's emotions and adjust the timing of acquiring emotion estimation information based on the estimated audience emotions. For example, if the audience is excited, the acquisition unit can acquire emotion estimation information frequently in real time. For example, if the audience is relaxed, the acquisition unit can widen the interval between acquiring emotion estimation information. For example, if the audience is feeling anxious, the acquisition unit can periodically acquire emotion estimation information and monitor changes. This allows for the acquisition of more appropriate information by adjusting the timing of information acquisition according to the audience's emotions. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input audience emotion data into a generating AI and have the generating AI adjust the timing of acquiring emotion estimation information.

[0083] The data acquisition unit can analyze past emotional data of the audience and select an appropriate acquisition method. For example, the data acquisition unit can identify the timing of heightened emotions during a specific event from past emotional data and acquire information at that timing. For example, the data acquisition unit can analyze the emotional change patterns of a specific audience member based on past emotional data and select the optimal acquisition method. For example, the data acquisition unit can refer to past emotional data and adjust the frequency of information acquisition at specific emotional states. This makes it possible to acquire information more effectively by analyzing past emotional data. Some or all of the above processing in the data acquisition unit may be performed using AI, for example, or without AI. For example, the data acquisition unit can input past emotional data into a generating AI and have the generating AI select the optimal acquisition method.

[0084] The data acquisition unit can filter the information for sentiment estimation based on the audience's current activity status and areas of interest. For example, if an audience member is interested in a particular sport, the data acquisition unit can prioritize acquiring information related to that sport. For example, if an audience member is taking a break, the data acquisition unit can prioritize acquiring information that helps them relax. For example, if an audience member is paying attention to a particular player, the data acquisition unit can prioritize acquiring information about that player. This allows for the provision of more appropriate information by prioritizing the acquisition of information that matches the audience's interests. Some or all of the above processing in the data acquisition unit may be performed using AI, for example, or without AI. For example, the data acquisition unit can input data on the audience's activity status and areas of interest into a generating AI and have the generating AI perform the filtering.

[0085] The acquisition unit can estimate the audience's emotions and determine the priority of emotion estimation information to acquire based on the estimated audience emotions. For example, if the audience is excited, the acquisition unit can prioritize acquiring information that increases excitement. For example, if the audience is relaxed, the acquisition unit can prioritize acquiring information that promotes relaxation. For example, if the audience is feeling anxious, the acquisition unit can prioritize acquiring information that alleviates anxiety. By prioritizing information according to the audience's emotions, more effective information provision becomes possible. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input audience emotion data into a generating AI and have the generating AI perform the determination of information prioritization.

[0086] The acquisition unit can prioritize acquiring highly relevant information based on the geographical location information of spectators when acquiring information for emotion estimation. For example, if a spectator is in a specific area of ​​the stadium, the acquisition unit can prioritize acquiring information related to that area. For example, if a spectator is moving, the acquisition unit can prioritize acquiring information related to their destination. For example, if a spectator is in a specific seat, the acquisition unit can prioritize acquiring information related to the viewpoint from that seat. This makes it possible to provide more appropriate information by acquiring highly relevant information based on the spectator's location information. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input the geographical location information of spectators into a generating AI and have the generating AI perform the acquisition of highly relevant information.

[0087] The acquisition unit can analyze the audience's social media activity and acquire relevant information when acquiring information for sentiment estimation. For example, if an audience member mentions a specific player on social media, the acquisition unit can prioritize acquiring information about that player. For example, if an audience member mentions a specific match on social media, the acquisition unit can prioritize acquiring information about that match. For example, if an audience member mentions a specific event on social media, the acquisition unit can prioritize acquiring information about that event. This makes it possible to provide more appropriate information by acquiring relevant information based on the audience member's social media activity. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input audience social media activity data into a generating AI and have the generating AI acquire relevant information.

[0088] The estimation unit can estimate the audience's emotions and adjust the emotion estimation algorithm based on the estimated emotions. For example, if the audience is excited, the estimation unit can speed up the emotion estimation algorithm to estimate emotions in real time. For example, if the audience is relaxed, the estimation unit can loosen the emotion estimation algorithm to perform a more detailed emotional analysis. For example, if the audience is anxious, the estimation unit can make the emotion estimation algorithm more sensitive to capture subtle emotional changes. By adjusting the algorithm according to the audience's emotions, the accuracy of emotion estimation is improved. Some or all of the above processing in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input audience emotion data into a generating AI and have the generating AI adjust the emotion estimation algorithm.

[0089] The estimation unit can adjust the level of detail of the estimation based on the importance of the information used for sentiment estimation during the estimation process. For example, the estimation unit can perform detailed sentiment estimation based on information of high importance. For example, the estimation unit can perform simplified sentiment estimation based on information of low importance. For example, the estimation unit can perform sentiment estimation with a moderate level of detail based on information of moderate importance. By adjusting the level of detail of the estimation according to the importance of the information, efficient sentiment estimation becomes possible. Some or all of the above-described processes in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input importance data of the information used for sentiment estimation into a generating AI and have the generating AI perform the adjustment of the level of detail of the estimation.

[0090] The estimation unit can apply different estimation algorithms depending on the category of information used for emotion estimation during estimation. For example, the estimation unit can apply a facial expression recognition algorithm based on facial expression data. For example, the estimation unit can apply a voice emotion recognition algorithm based on voice tone data. For example, the estimation unit can apply a physiological emotion recognition algorithm based on heart rate data. By applying an algorithm according to the category of information, the accuracy of emotion estimation is improved. Some or all of the above processing in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input category data of information used for emotion estimation into a generating AI and cause the generating AI to apply different estimation algorithms.

[0091] The estimation unit can estimate the audience's emotions and adjust the display method of the estimation results based on the estimated emotions. For example, if the audience is excited, the estimation unit can provide a visually stimulating display method. For example, if the audience is relaxed, the estimation unit can provide a calming display method. For example, if the audience is feeling anxious, the estimation unit can provide a reassuring display method. By providing a display method that corresponds to the audience's emotions, it becomes possible to provide more appropriate information. Some or all of the above processing in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input audience emotion data into a generating AI and have the generating AI adjust the display method of the estimation results.

[0092] The estimation unit can determine the estimation priority based on the timing of acquisition of sentiment estimation information during estimation. For example, the estimation unit can perform estimation by prioritizing the use of the most recent sentiment estimation information. For example, the estimation unit can perform estimation by referring to past sentiment estimation information. For example, the estimation unit can perform estimation by prioritizing the use of sentiment estimation information acquired during a specific event. This enables efficient sentiment estimation by determining the estimation priority based on the timing of information acquisition. Some or all of the above processing in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input data on the timing of acquisition of sentiment estimation information into a generating AI and have the generating AI perform the determination of the estimation priority.

[0093] The estimation unit can adjust the order of estimation based on the relevance of the information used for sentiment estimation. For example, the estimation unit can prioritize using information with high relevance. For example, the estimation unit can then use information with moderate relevance. For example, the estimation unit can then use information with low relevance last. By adjusting the order of estimation based on the relevance of the information, efficient sentiment estimation becomes possible. Some or all of the above processing in the estimation unit may be performed using AI, for example, or without AI. For example, the estimation unit can input relevance data of the information used for sentiment estimation into a generating AI and have the generating AI perform the adjustment of the estimation order.

[0094] The equipment control unit can estimate the emotions of the audience and adjust the equipment control method based on the estimated emotions. For example, if the audience is excited, the equipment control unit can brighten the lights and enhance the sound. For example, if the audience is relaxed, the equipment control unit can dim the lights and quiet the sound. For example, if the audience is feeling anxious, the equipment control unit can soften the lights and gentle the sound. This allows for a more appropriate viewing environment by controlling the equipment in accordance with the emotions of the audience. Some or all of the above processing in the equipment control unit may be performed using AI, for example, or without AI. For example, the equipment control unit can input audience emotion data into a generating AI and have the generating AI perform the adjustment of the equipment control method.

[0095] The device control unit can analyze past emotional data of the audience and select the optimal control method when controlling the device. For example, the device control unit can select the optimal lighting settings for a specific emotional state from past emotional data. For example, the device control unit can select the optimal sound settings for a specific emotional state based on past emotional data. For example, the device control unit can refer to past emotional data and select the optimal display settings for a specific emotional state. This enables more effective device control by analyzing past emotional data. Some or all of the above processing in the device control unit may be performed using AI, for example, or without AI. For example, the device control unit can input past emotional data into a generating AI and have the generating AI select the optimal control method.

[0096] The device control unit can customize the control means based on the audience's current emotional state when controlling the device. For example, if the audience is excited, the device control unit can change the color of the lighting to provide visual stimulation. For example, if the audience is relaxed, the device control unit can adjust the volume of the sound to provide a quiet environment. For example, if the audience is feeling anxious, the device control unit can change the display content to provide a sense of security. In this way, a more appropriate viewing environment can be provided by providing control means that correspond to the audience's current emotional state. Some or all of the above processing in the device control unit may be performed using AI, for example, or without AI. For example, the device control unit can input the audience's current emotional state data into a generating AI and have the generating AI perform the customization of the control means.

[0097] The equipment control unit can estimate the emotions of the audience and determine the priority of equipment control based on the estimated emotions. For example, if the audience is excited, the equipment control unit can prioritize lighting control. For example, if the audience is relaxed, the equipment control unit can prioritize sound control. For example, if the audience is feeling anxious, the equipment control unit can prioritize display control. By determining the priority of equipment control according to the emotions of the audience, a more effective viewing environment can be provided. Some or all of the above processing in the equipment control unit may be performed using AI, for example, or without AI. For example, the equipment control unit can input audience emotion data into a generating AI and have the generating AI perform the determination of equipment control priorities.

[0098] The equipment control unit can select the optimal control method when controlling equipment, taking into account the geographical location information of spectators. For example, if a spectator is in a specific area of ​​the stadium, the equipment control unit can select the optimal lighting settings for that area. For example, if a spectator is moving, the equipment control unit can select the optimal sound settings for their destination. For example, if a spectator is in a specific seat, the equipment control unit can select the optimal display settings from the viewpoint of that seat. By selecting the optimal control method based on the geographical location information of spectators, a more appropriate viewing environment can be provided. Some or all of the above processing in the equipment control unit may be performed using AI, for example, or without AI. For example, the equipment control unit can input the geographical location information of spectators into a generating AI and have the generating AI select the optimal control method.

[0099] The equipment control unit can analyze the audience's social media activity and propose control measures when controlling the equipment. For example, if an audience member mentions a particular player on social media, the equipment control unit can propose lighting settings related to that player. For example, if an audience member mentions a particular match on social media, the equipment control unit can propose sound settings related to that match. For example, if an audience member mentions a particular event on social media, the equipment control unit can propose display settings related to that event. This allows for a more appropriate viewing environment to be provided by proposing control measures based on the audience's social media activity. Some or all of the above processing in the equipment control unit may be performed using AI, for example, or without AI. For example, the equipment control unit can input audience social media activity data into a generating AI and have the generating AI execute the proposal of control measures.

[0100] The viewing position suggestion unit can estimate the emotions of the audience and adjust the method of suggesting viewing positions based on the estimated emotions. For example, if the audience is excited, the viewing position suggestion unit can suggest a visually stimulating position. For example, if the audience is relaxed, the viewing position suggestion unit can suggest a quiet position. For example, if the audience is feeling anxious, the viewing position suggestion unit can suggest a position that provides a sense of security. By providing a method of suggesting viewing positions that corresponds to the emotions of the audience, a more appropriate viewing environment can be provided. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input audience emotion data into a generating AI and have the generating AI perform adjustments to the viewing position suggestion method.

[0101] The viewing position suggestion unit can provide optimal suggestions by referring to the audience's past viewing history when suggesting viewing positions. For example, the viewing position suggestion unit can suggest the optimal viewing position based on the viewing positions the audience has preferred to view in the past. For example, the viewing position suggestion unit can suggest the optimal viewing position for a specific event based on the audience's past viewing history. For example, the viewing position suggestion unit can analyze the audience's past viewing history and suggest a viewing position that suits the audience's preferences. This allows for a more appropriate viewing environment by suggesting the optimal viewing position based on the audience's past viewing history. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input the audience's past viewing history data into a generating AI and have the generating AI perform the optimal viewing position suggestion.

[0102] The viewing position suggestion unit can customize its suggestions based on the audience's current emotional state when suggesting viewing positions. For example, if the audience is excited, the unit can suggest a visually stimulating position. If the audience is relaxed, the unit can suggest a quiet position. If the audience is feeling anxious, the unit can suggest a reassuring position. By providing viewing position suggestions that correspond to the audience's current emotional state, a more appropriate viewing environment can be provided. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input the audience's current emotional state data into a generating AI and have the generating AI customize the suggestions.

[0103] The viewing position suggestion unit can estimate the emotions of the audience and determine the priority of viewing positions based on the estimated emotions. For example, if the audience is excited, the viewing position suggestion unit can prioritize suggesting visually stimulating positions. For example, if the audience is relaxed, the viewing position suggestion unit can prioritize suggesting quiet positions. For example, if the audience is feeling anxious, the viewing position suggestion unit can prioritize suggesting positions that provide a sense of security. By determining the priority of viewing positions according to the emotions of the audience, a more appropriate viewing environment can be provided. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input audience emotion data into a generating AI and have the generating AI perform the determination of the priority of viewing positions.

[0104] The viewing position suggestion unit can provide optimal suggestions by considering the geographical location information of spectators when suggesting viewing positions. For example, if a spectator is in a specific area of ​​the stadium, the viewing position suggestion unit can suggest the optimal viewing position for that area. For example, if a spectator is moving, the viewing position suggestion unit can suggest the optimal viewing position for their destination. For example, if a spectator is in a specific seat, the viewing position suggestion unit can suggest the optimal viewing position from the viewpoint of that seat. By suggesting the optimal viewing position based on the geographical location information of spectators, a more appropriate viewing environment can be provided. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input the geographical location information of spectators into a generating AI and have the generating AI perform the optimal viewing position suggestion.

[0105] The viewing position suggestion unit can analyze the audience's social media activity when suggesting viewing positions and propose relevant viewing positions. For example, if an audience member mentions a particular player on social media, the viewing position suggestion unit can suggest a position where that player can be seen well. For example, if an audience member mentions a particular match on social media, the viewing position suggestion unit can suggest a position where that match can be seen well. For example, if an audience member mentions a particular event on social media, the viewing position suggestion unit can suggest a position where that event can be seen well. This allows for the provision of a more appropriate viewing environment by suggesting relevant viewing positions based on the audience member's social media activity. Some or all of the above processing in the viewing position suggestion unit may be performed using AI, for example, or without AI. For example, the viewing position suggestion unit can input audience social media activity data into a generating AI and have the generating AI perform the suggestion of relevant viewing positions.

[0106] The travel route suggestion unit can estimate the emotions of the audience and adjust the method of suggesting travel routes based on the estimated emotions. For example, if the audience is excited, the travel route suggestion unit can suggest a visually stimulating route. For example, if the audience is relaxed, the travel route suggestion unit can suggest a quiet route. For example, if the audience is feeling anxious, the travel route suggestion unit can suggest a reassuring route. By providing a method of suggesting travel routes that responds to the emotions of the audience, a more appropriate viewing environment can be provided. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can input audience emotion data into a generating AI and have the generating AI adjust the method of suggesting travel routes.

[0107] The travel route suggestion unit can provide the optimal suggestion by referring to the spectator's past travel history when suggesting a travel route. For example, the travel route suggestion unit can suggest the optimal travel route based on routes that the spectator has preferred to use in the past. For example, the travel route suggestion unit can suggest the optimal travel route for a specific event based on the spectator's past travel history. For example, the travel route suggestion unit can analyze the spectator's past travel history and suggest a travel route that suits the spectator's preferences. This allows for a more appropriate viewing environment by suggesting the optimal travel route based on the spectator's past travel history. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can input the spectator's past travel history data into a generating AI and have the generating AI execute the suggestion of the optimal travel route.

[0108] The route suggestion unit can customize the suggested routes based on the audience's current emotional state. For example, if the audience is excited, the route suggestion unit can suggest a visually stimulating route. If the audience is relaxed, the route suggestion unit can suggest a calm route. If the audience is feeling anxious, the route suggestion unit can suggest a reassuring route. By providing route suggestions that correspond to the audience's current emotional state, a more appropriate viewing environment can be provided. Some or all of the above processing in the route suggestion unit may be performed using AI, for example, or without AI. For example, the route suggestion unit can input the audience's current emotional state data into a generating AI and have the generating AI customize the suggested routes.

[0109] The travel route suggestion unit can estimate the emotions of the audience and determine the priority of travel routes based on the estimated emotions. For example, if the audience is excited, the travel route suggestion unit can prioritize suggesting a visually stimulating route. For example, if the audience is relaxed, the travel route suggestion unit can prioritize suggesting a quiet route. For example, if the audience is feeling anxious, the travel route suggestion unit can prioritize suggesting a route that provides a sense of security. By determining the priority of travel routes according to the emotions of the audience, a more appropriate viewing environment can be provided. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can input audience emotion data into a generating AI and have the generating AI perform the determination of travel route priorities.

[0110] The travel route suggestion unit can provide optimal suggestions by considering the geographical location information of spectators when suggesting travel routes. For example, if a spectator is in a specific area of ​​the stadium, the travel route suggestion unit can suggest the optimal travel route for that area. For example, if a spectator is moving, the travel route suggestion unit can suggest the optimal travel route for their destination. For example, if a spectator is in a specific seat, the travel route suggestion unit can suggest the optimal travel route from the viewpoint of that seat. By suggesting the optimal travel route based on the geographical location information of spectators, a more appropriate viewing environment can be provided. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can input the geographical location information of spectators into a generating AI and have the generating AI perform the task of suggesting the optimal travel route.

[0111] The travel route suggestion unit can analyze the spectator's social media activity when suggesting travel routes and propose relevant routes. For example, if a spectator mentions a particular player on social media, the travel route suggestion unit can propose a route that allows for a good view of that player. For example, if a spectator mentions a particular match on social media, the travel route suggestion unit can propose a route that allows for a good view of that match. For example, if a spectator mentions a particular event on social media, the travel route suggestion unit can propose a route that allows for a good view of that event. This allows for the provision of a more appropriate viewing environment by suggesting relevant travel routes based on the spectator's social media activity. Some or all of the above processing in the travel route suggestion unit may be performed using AI, for example, or without AI. For example, the travel route suggestion unit can input the spectator's social media activity data into a generating AI and have the generating AI execute the suggestion of relevant travel routes.

[0112] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0113] The spectator support robot can provide personalized cheering messages in real time based on the audience's emotions. For example, if an audience member is excited, displaying a cheering message can further enhance their excitement. If an audience member is relaxed, it can provide messages to maintain that relaxed atmosphere. Furthermore, if an audience member is feeling anxious, it can display messages to provide reassurance. In this way, by providing cheering messages that match the audience's emotions, the spectator experience can be improved.

[0114] The spectator support robot can provide detailed statistics about specific players or teams based on the audience's emotions. For example, if the audience is excited, it can display past highlights and statistics of that player. If the audience is relaxed, it can provide background information and interview articles about the player. Furthermore, if the audience is feeling anxious, it can display success stories and encouraging messages from the player. In this way, by providing information tailored to the audience's emotions, the spectator experience can be made more enriching.

[0115] The spectator support robot can suggest food and drinks within the venue based on the emotions of the spectators. For example, if the spectators are excited, it can suggest drinks and snacks to replenish their energy. If the spectators are relaxed, it can suggest relaxing drinks and snacks. Furthermore, if the spectators are feeling anxious, it can suggest drinks and food that provide a sense of security. In this way, by suggesting food and drinks that match the emotions of the spectators, the spectator experience can be improved.

[0116] The spectator support robot can suggest events and activities in specific areas of the venue based on the emotions of the spectators. For example, if the spectators are excited, it can suggest active events or games. If the spectators are relaxed, it can suggest relaxing activities in quiet areas. Furthermore, if the spectators are feeling anxious, it can suggest areas or activities that provide a sense of security. In this way, by suggesting events and activities that match the emotions of the spectators, the spectator experience can be improved.

[0117] The spectator support robot can display the crowd situation in the venue in real time based on the emotions of the spectators and suggest the optimal route to move around. For example, if a spectator is excited, it can suggest a route that avoids crowds and allows for smooth movement. If a spectator is relaxed, it can suggest a quiet route. Furthermore, if a spectator is feeling anxious, it can suggest a route that provides a sense of security. In this way, by suggesting a route that responds to the emotions of the spectators, the spectator experience can be improved.

[0118] The spectator support robot can analyze a spectator's past viewing history and suggest a personalized viewing plan based on their preferred viewing style. For example, if a spectator has previously preferred watching a particular team's games, the robot can prioritize suggesting games featuring that team. Similarly, if a spectator has previously preferred a specific seat location, the robot can suggest that location. Furthermore, by analyzing the spectator's preferences from their past viewing history, the robot can provide the optimal viewing plan. This allows for a personalized viewing experience based on the spectator's past viewing history.

[0119] The spectator support robot can suggest the optimal viewing spot within the venue based on the spectator's current location. For example, if a spectator is in a specific area of ​​the stadium, it can suggest the best viewing spot from that area. Furthermore, if a spectator is moving, it can suggest the optimal viewing spot for their next destination. Additionally, if a spectator is in a specific seat, it can suggest the optimal viewing spot based on the view from that seat. This allows for the provision of the best viewing spot based on the spectator's location.

[0120] The spectator support robot can analyze spectators' social media activity and provide real-time information about players and teams they are interested in. For example, if a spectator mentions a specific player on social media, it can provide that player's latest statistics and news. Similarly, if a spectator mentions a specific team, it can provide that team's match results and statistics. Furthermore, if a spectator mentions a specific event, it can provide detailed information about that event. This allows for the provision of relevant information based on spectators' social media activity.

[0121] The spectator support robot can provide special offers and discounts in specific areas of the venue based on the emotions of the spectators. For example, if a spectator is excited, it can further enhance their excitement by providing special offers and discounts in that area. If a spectator is relaxed, it can provide special offers and discounts in areas where relaxation is possible. Furthermore, if a spectator is feeling anxious, it can provide special offers and discounts in areas where comfort is provided. In this way, the spectator experience can be improved by providing special offers and discounts tailored to the emotions of the spectators.

[0122] The spectator support robot can suggest photo spots in specific areas of the venue based on the emotions of the spectators. For example, if the spectators are excited, it can suggest exciting photo spots. If the spectators are relaxed, it can suggest relaxing photo spots. Furthermore, if the spectators are feeling anxious, it can suggest photo spots that provide a sense of security. In this way, by suggesting photo spots that match the emotions of the spectators, the spectator experience can be improved.

[0123] The following briefly describes the processing flow for example form 2.

[0124] Step 1: The acquisition unit acquires emotion estimation information, which is information used to estimate the emotions of the audience in the venue. Emotion estimation information includes, for example, the audience's facial expressions, tone of voice, and heart rate. The acquisition unit can acquire the audience's facial expressions using a camera, for example. The acquisition unit can also acquire the audience's tone of voice using a microphone. Furthermore, the acquisition unit can acquire the audience's heart rate using a heart rate sensor. Step 2: The estimation unit estimates the audience's emotions based on the emotion estimation information acquired by the acquisition unit. For example, the estimation unit can analyze the audience's facial expression data to estimate emotions such as joy, excitement, or surprise. The estimation unit can also analyze the audience's voice tone data to estimate emotions. Furthermore, the estimation unit can analyze the audience's heart rate data to estimate emotions. Step 3: The equipment control unit controls the venue's equipment based on the emotions estimated by the estimation unit. For example, if the audience is excited, the equipment control unit can brighten the lights. It can also enhance the sound. Furthermore, the equipment control unit can display replay scenes using a display device. This can further increase the audience's excitement. If the audience is relaxed, the equipment control unit can dim the lights. It can also quiet the sound. This can create a relaxed atmosphere.

[0125] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0126] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0127] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0128] For example, the acquisition unit can acquire the facial expressions and tone of voice of the audience using the camera 42 and microphone 38B of the smart device 14. The estimation unit is implemented by the specific processing unit 290 of the data processing device 12 and estimates the emotions of the audience by analyzing the acquired emotion estimation information. The equipment control unit is implemented by the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12 and controls the lighting and sound of the venue based on the estimated emotions. The correspondence between each unit and the equipment and control unit is not limited to the example described above and can be changed in various ways.

[0129] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0130] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0131] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0132] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0133] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0134] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0135] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0136] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0137] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0138] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0139] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0140] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0141] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0142] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0143] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0144] For example, the acquisition unit can acquire the facial expressions and tone of voice of the audience using the camera 42 and microphone 238 of the smart glasses 214. The estimation unit is implemented by the specific processing unit 290 of the data processing device 12, and analyzes the acquired emotion estimation information to estimate the emotions of the audience. The device control unit is implemented by the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12, and controls the lighting and sound of the venue based on the estimated emotions. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0145] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0146] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0147] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0148] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0149] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0150] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0151] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0152] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0153] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0154] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0155] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0156] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0157] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0158] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0159] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0160] For example, the acquisition unit can acquire the facial expressions and tone of voice of the audience using the camera 42 and microphone 238 of the headset terminal 314. The estimation unit is implemented by the specific processing unit 290 of the data processing device 12, and analyzes the acquired emotion estimation information to estimate the emotions of the audience. The equipment control unit is implemented by the control unit 46A of the headset terminal 314 or the specific processing unit 290 of the data processing device 12, and controls the lighting and sound of the venue based on the estimated emotions. The correspondence between each unit and the equipment and control unit is not limited to the example described above, and various modifications are possible.

[0161] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0162] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0163] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0164] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0165] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0166] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0167] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0168] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0169] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0170] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0171] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0172] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0173] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0174] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0175] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0176] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0177] For example, the acquisition unit can acquire the facial expressions and tone of voice of the audience using the camera 42 and microphone 238 of the robot 414. The estimation unit is implemented by the specific processing unit 290 of the data processing device 12, and analyzes the acquired emotion estimation information to estimate the emotions of the audience. The equipment control unit is implemented by the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12, and controls the lighting and sound of the venue based on the estimated emotions. The correspondence between each unit and the equipment and control unit is not limited to the example described above, and various modifications are possible.

[0178] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0179] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0180] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0181] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0182] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0183] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0184] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0185] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0186] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0187] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0188] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0189] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0190] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0191] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0192] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0193] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0194] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0195] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0196] (Note 1) An acquisition unit that acquires emotion estimation information, which is information used to estimate the emotions of the audience at the venue, An estimation unit estimates the emotions of the audience based on the emotion estimation information acquired by the acquisition unit, The system includes an equipment control unit that controls the equipment in the venue based on the emotions estimated by the estimation unit. A system characterized by the following features. (Note 2) The venue is equipped with a viewing position suggestion unit that suggests viewing positions based on the emotion data estimated by the estimation unit. The system described in Appendix 1, characterized by the features described herein. (Note 3) The system includes a travel route proposal unit that proposes a travel route based on the emotion data estimated by the estimation unit. The system described in Appendix 1, characterized by the features described herein. (Note 4) The acquisition unit is, Obtain information for emotion estimation from the audience's facial expressions, tone of voice, and heart rate. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned equipment control unit is Based on the emotions estimated by the estimation unit, the venue's lighting, sound, and display equipment are controlled. The system described in Appendix 1, characterized by the features described herein. (Note 6) The acquisition unit is, The system estimates the audience's emotions and adjusts the timing of acquiring emotion estimation information based on the estimated audience emotions. The system described in Appendix 1, characterized by the features described herein. (Note 7) The acquisition unit is, Analyze past audience sentiment data and select the appropriate acquisition method. The system described in Appendix 1, characterized by the features described herein. (Note 8) The acquisition unit is, When acquiring information for sentiment estimation, filtering is performed based on the audience's current activities and areas of interest. The system described in Appendix 1, characterized by the features described herein. (Note 9) The acquisition unit is, The system estimates the audience's emotions and prioritizes the emotion estimation information to be retrieved based on the estimated audience emotions. The system described in Appendix 1, characterized by the features described herein. (Note 10) The acquisition unit is, When acquiring information for emotion estimation, the system prioritizes acquiring highly relevant information based on the audience's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 11) The acquisition unit is, When acquiring information for sentiment estimation, the audience's social media activity is analyzed to obtain relevant information. The system described in Appendix 1, characterized by the features described herein. (Note 12) The estimation unit, The system estimates the audience's emotions and adjusts the emotion estimation algorithm based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 13) The estimation unit, During estimation, adjust the level of detail of the estimation based on the importance of the information used for sentiment estimation. The system described in Appendix 1, characterized by the features described herein. (Note 14) The estimation unit, During estimation, different estimation algorithms are applied depending on the category of information used for sentiment estimation. The system described in Appendix 1, characterized by the features described herein. (Note 15) The estimation unit, It estimates the audience's emotions and adjusts how the estimation results are displayed based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 16) The estimation unit, During estimation, the estimation priority is determined based on when the information used for sentiment estimation was acquired. The system described in Appendix 1, characterized by the features described herein. (Note 17) The estimation unit, During estimation, the order of estimations is adjusted based on the relevance of the information used for sentiment estimation. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned equipment control unit is The system estimates the audience's emotions and adjusts the equipment control methods based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned equipment control unit is During equipment control, the system analyzes past emotional data of the audience to select the optimal control method. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned equipment control unit is When controlling the equipment, the control methods are customized based on the audience's current emotional state. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned equipment control unit is The system estimates the audience's emotions and determines the priority of equipment control based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned equipment control unit is When controlling the equipment, the optimal control method is selected by considering the geographical location information of the audience. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned equipment control unit is When controlling equipment, we analyze the audience's social media activity and propose control methods. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned viewing position suggestion unit is, The system estimates the audience's emotions and adjusts the method of suggesting viewing positions based on those estimated emotions. The system described in Appendix 2, characterized by the features described herein. (Note 25) The aforementioned viewing position suggestion unit is, When suggesting viewing locations, the system provides optimal suggestions by referencing the audience's past viewing history. The system described in Appendix 2, characterized by the features described herein. (Note 26) The aforementioned viewing position suggestion unit is, When suggesting viewing locations, the suggestions are customized based on the audience's current emotional state. The system described in Appendix 2, characterized by the features described herein. (Note 27) The aforementioned viewing position suggestion unit is, The system estimates the audience's emotions and determines the priority of viewing positions based on those estimated emotions. The system described in Appendix 2, characterized by the features described herein. (Note 28) The aforementioned viewing position suggestion unit is, When suggesting viewing locations, we provide optimal suggestions by taking into account the geographical location information of the audience. The system described in Appendix 2, characterized by the features described herein. (Note 29) The aforementioned viewing position suggestion unit is, When suggesting viewing locations, we analyze the audience's social media activity and suggest relevant viewing locations. The system described in Appendix 2, characterized by the features described herein. (Note 30) The aforementioned travel route proposal unit is The system estimates the audience's emotions and adjusts the method of suggesting travel routes based on those estimated emotions. The system described in Appendix 3, characterized by the features described herein. (Note 31) The aforementioned travel route proposal unit is When suggesting travel routes, the system provides optimal suggestions by referencing the audience's past travel history. The system described in Appendix 3, characterized by the features described herein. (Note 32) The aforementioned travel route proposal unit is When suggesting travel routes, customize the suggestions based on the audience's current emotional state. The system described in Appendix 3, characterized by the features described herein. (Note 33) The aforementioned travel route proposal unit is The system estimates the audience's emotions and prioritizes travel routes based on those estimated emotions. The system described in Appendix 3, characterized by the features described herein. (Note 34) The aforementioned travel route proposal unit is When suggesting travel routes, we provide optimal suggestions by taking into account the geographical location information of the audience. The system described in Appendix 3, characterized by the features described herein. (Note 35) The aforementioned travel route proposal unit is When suggesting travel routes, the system analyzes the audience's social media activity and proposes relevant routes. The system described in Appendix 3, characterized by the features described herein. [Explanation of Symbols]

[0197] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. An acquisition unit that acquires emotion estimation information, which is information used to estimate the emotions of the audience at the venue, An estimation unit estimates the emotions of the audience based on the emotion estimation information acquired by the acquisition unit, The system includes an equipment control unit that controls the equipment in the venue based on the emotions estimated by the estimation unit. A system characterized by the following features.

2. The venue is equipped with a viewing position suggestion unit that suggests viewing positions based on the emotion data estimated by the estimation unit. The system according to feature 1.

3. The system includes a travel route proposal unit that proposes a travel route based on the emotion data estimated by the estimation unit. The system according to feature 1.

4. The acquisition unit is, The aforementioned audience members' facial expressions, tone of voice, and heart rate are used to obtain information for emotion estimation. The system according to feature 1.

5. The aforementioned equipment control unit is Based on the emotions estimated by the estimation unit, the lighting, sound, and displays of the venue are controlled. The system according to feature 1.

6. The acquisition unit is, The system estimates the emotions of the audience and adjusts the timing of acquiring emotion estimation information based on the estimated emotions of the audience. The system according to feature 1.

Images