system

Abstract

We provide the system. [Solution] A terminal that acquires travelers' personal data at the gate of a transportation facility, Means for controlling an inspection device for pre-processing travelers based on acquired personal data, A means of analyzing past and present data to predict the flow, A means of optimizing operations and dynamically allocating resources based on predictions, A means of updating the learning model to improve prediction accuracy based on monitoring results, A means to understand congestion levels and provide visitors with the most suitable routes and entrances, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present invention relates to a technology for efficiently managing the inspection and flow of passengers in transportation facilities such as airport facilities. In recent years, with the increase in the number of passengers, congestion at inspection points, the resulting stress on passengers, and the burden on airport staff have become serious problems. In particular, with the predicted shortage of personnel due to population decline, how to improve operational efficiency with limited resources has become an important issue.

Means for Solving the Problems

[0005] This invention constructs a system that acquires travelers' personal data using terminals installed at the gates of transportation facilities and controls pre-processing by 3D scanners and metal detection devices based on this data. Furthermore, it has a function to analyze past and present data to predict traveler flow. Based on these prediction results, it optimizes operations and dynamically allocates resources to reduce congestion. It also includes means for updating the learning model to continuously improve prediction accuracy based on monitoring results. This reduces passenger waiting times and the burden on airport staff, and enables efficient inspection operations.

[0006] "Transportation facilities" refers to all equipment and facilities related to the movement of people, such as airports and train stations.

[0007] A "gate" refers to an entrance or passageway within a transportation facility for passengers or travelers to pass through.

[0008] A "terminal" refers to an electronic device installed for the purpose of acquiring and processing data.

[0009] "Personal data" refers to identifying information related to an individual traveler, including passport information and reservation information.

[0010] "Inspection equipment" refers to devices used for security checks, including 3D scanners and metal detection devices.

[0011] A "3D scanner" refers to a device that reads the shape of an object in three dimensions.

[0012] A "metal detection device" refers to a device that detects metals contained in an object.

[0013] "Pre-processing" refers to preparatory processes performed before the main processing.

[0014] "Analysis" refers to the act of breaking down data and clarifying its meaning and structure.

[0015] "Prediction" refers to the act of estimating future states and trends based on the acquired data.

[0016] "Operation" refers to the overall operation and business conducted within the facility.

[0017] "Optimizing and dynamically allocating resources" means effectively and flexibly allocating limited resources (personnel and equipment).

[0018] "Monitoring" refers to the act of monitoring a system or situation and observing its status and changes.

[0019] "Learning model" refers to an algorithm or mathematical framework used for data analysis and prediction.

Brief Explanation of Drawings

[0020] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0021] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0022] First, the language used in the following description will be described.

[0023] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0024] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0025] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0026] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0028] [First Embodiment]

[0029] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0030] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0033] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0036] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0040] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0041] This invention relates to a system for efficiently managing passenger flow in transportation facilities such as airports and railway stations. This system mainly consists of terminals, servers, and users.

[0042] Terminal operation

[0043] The terminals are installed at gates within transportation facilities and collect personal data by scanning passports and boarding passes as passengers pass through. This allows for the collection of passenger flight information, reservation information, and other data.

[0044] Server Role

[0045] The server receives personal data transmitted from terminals and stores it in a database. Based on this data, the server controls 3D scanners and metal detection devices to perform pre-processing on travelers. From the data obtained through analysis, the server builds a predictive model of passenger flow and calculates the optimal allocation of resources. Furthermore, based on information obtained from monitoring, the machine learning model is updated to improve prediction accuracy.

[0046] User intervention

[0047] Users, i.e., airport staff, receive instructions from the server as needed and assist with gate inspections. For example, in cases where the server cannot make an automatic decision, users can perform inspections directly. Users can also input feedback into a terminal to help improve the overall system.

[0048] Specific example

[0049] For example, during the morning rush hour when many passengers arrive, terminals quickly collect passenger personal data, and servers use this data to perform rapid inspections using 3D scanners. Predictive analytics allows servers to proactively identify which gates will be congested and instruct users on the optimal allocation of resources. This speeds up gate passage, reduces passenger waiting times, and lessens the burden on airport staff.

[0050] Through such operations, the present invention enables efficient and safe management of traveler flow, achieving both passenger comfort and efficient facility operation.

[0051] The following describes the processing flow.

[0052] Step 1:

[0053] The terminal scans the passports and boarding passes of passengers passing through the gates of transportation facilities and obtains personal data. This personal data includes flight information and reservation information. The obtained data is transmitted to the server in real time.

[0054] Step 2:

[0055] The server stores the personal data received from the terminal in a dedicated database. The server then uses this data to send appropriate control commands to the 3D scanner and metal detector, instructing them to inspect the passenger's belongings.

[0056] Step 3:

[0057] The terminal collects scan results from the 3D scanner and metal detector and sends them back to the server. This includes information about the object's shape and metal detection.

[0058] Step 4:

[0059] The server analyzes the scanned data to check for any security issues. If a problem is detected, the server alerts the user and prompts them to investigate further.

[0060] Step 5:

[0061] The server runs a machine learning model that predicts passenger flow based on analysis results and historical data, and predicts congestion levels. The resulting predictions are used in real time to optimize resource allocation.

[0062] Step 6:

[0063] The server dynamically adjusts airport staff deployment and gate opening / closing times based on predictions and sends instructions to the user. The user then makes on-site adjustments according to these instructions.

[0064] Step 7:

[0065] The server monitors the system's operational status and inspection completion times, and uses this data in a feedback loop to improve the learning model. The model is retrained periodically to improve prediction accuracy.

[0066] By following these steps, the efficient operation of the entire system is maintained, and the smooth movement of passengers is ensured.

[0067] (Example 1)

[0068] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0069] Modern transportation facilities require users to pass through them smoothly and quickly. However, conventional systems have difficulty effectively predicting and managing user flow, resulting in congestion and long waiting times within facilities. This leads to decreased convenience and stress for users. Furthermore, facility operators face challenges in allocating resources appropriately and achieving efficient operation.

[0070] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0071] In this invention, the server includes a device for collecting user identification data at transit points of transportation facilities, means for adjusting an evaluation device for performing initial processing on users based on the collected identification data, means for predicting flow by utilizing historical and real-time information, means for optimizing operations based on the prediction results and allocating resources in response to changes, and means for modifying a learning model to improve the accuracy of predictions based on monitoring results. This makes it possible to efficiently manage the flow of users and alleviate congestion within facilities.

[0072] "Transportation facilities" refer to facilities such as airports and train stations where people gather and travel.

[0073] A "waypoint" refers to a location such as a gate or checkpoint that users must pass through.

[0074] "Identification data" refers to information that includes users' personal information and identification information.

[0075] The term "device" refers to a machine or mechanism designed to perform a specific function.

[0076] An "evaluation device" refers to a machine or equipment designed to perform initial processing for users.

[0077] "Flow" refers to the patterns of movement and behavior of users within a transportation facility.

[0078] "Prediction" refers to the act of estimating future events based on past information and real-time data.

[0079] "Operation" refers to the activities of efficiently managing and coordinating facilities and systems.

[0080] "Resources" refer to personnel, equipment, space, and other resources that can be used as such.

[0081] "Allocation" refers to the act of allocating resources to the appropriate places and times as needed.

[0082] "Monitoring results" refer to information and insights obtained through observation and data collection.

[0083] A "learning model" refers to an algorithm or mathematical model used to identify patterns based on data and perform predictions or classifications.

[0084] "Correction" refers to activities that involve making changes or adjustments to improve the current situation.

[0085] This invention is a system for efficiently managing the flow of users in transportation facilities. This system mainly consists of three elements: terminals, servers, and users.

[0086] Terminal operation

[0087] The terminals are installed at transit points in transportation facilities and acquire identification data by scanning passports or boarding passes as users pass through. The terminal hardware incorporates high-speed scanners and data transmission capabilities, transferring the acquired data to a server in real time.

[0088] Server Role

[0089] The server receives identification data transmitted from the terminal and securely stores it in a database. The server uses this data to control the evaluation device and perform initial user processing. The software used includes algorithms and machine learning models for data analysis, enabling prediction of user flow. Specifically, it analyzes historical and real-time data to predict user flow and identify potential congestion points. The server further refines the learning model using monitoring results to improve prediction accuracy. In this process, the server optimizes resources and improves operational efficiency.

[0090] User intervention

[0091] Users, i.e., facility staff, take appropriate actions based on information provided by the server. For example, at points where congestion is anticipated, additional staff are deployed based on instructions from the server. Furthermore, if the server cannot make a decision, users conduct inspections directly and contribute to system improvements by inputting necessary feedback into their terminals.

[0092] As a concrete example, consider an environment where many users arrive during the morning rush hour. Terminals quickly collect user identification data, and servers perform rapid inspections using 3D scanners. Through prediction, the servers identify where congestion will occur and notify users. Based on this information, users can allocate resources appropriately to avoid congestion.

[0093] An example of a prompt message would be, "Please advise on how to address the rapid changes in passenger density at the airport during specific times on weekends." This would allow for guidance on optimizing specific operations.

[0094] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0095] Step 1:

[0096] The terminal acquires user identification data at transit points within transportation facilities. This process involves scanning the user's passport or boarding pass. The input is the user's passport or boarding pass, and the terminal outputs digitized identification data through scanning. This data includes the user's ID and flight information.

[0097] Step 2:

[0098] The terminal transfers the acquired identification data to the server. The input here is the identification data generated in step 1. The terminal encrypts this data and sends it to the server over the network, and the server stores it in its database. The output is the securely stored identification data.

[0099] Step 3:

[0100] The server controls the evaluation equipment using the received identification data and performs initial processing for the user. The input is identification data transmitted from the terminal. Based on this data, the server activates 3D scanners and metal detectors and obtains output by analyzing the measurement results. The output is a determination of whether the user is allowed to pass and an alert if necessary.

[0101] Step 4:

[0102] The server analyzes historical and current data to predict user flow. The input for this step is historical and real-time identification data stored in a database. The server uses a machine learning model to analyze this data and predict the probability and location of congestion. The output is predicted information about future user flow.

[0103] Step 5:

[0104] The server optimizes operations and dynamically allocates resources based on predictive information. The input is the predictive information obtained in step 4. The server uses this to calculate how many staff members are needed at each transit point and notifies the user of this information to ensure optimal resource allocation. The output is the allocation instructions and related log information.

[0105] Step 6:

[0106] The user takes the necessary actions based on deployment instructions provided by the server. Here, the user receives instructions from the server as input. The user observes the situation in real time, changes staff deployments as needed, and provides feedback to the server. The output is a report of the improved deployment based on the feedback and its status.

[0107] (Application Example 1)

[0108] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0109] In large-scale facilities that attract large crowds, such as transportation facilities and event venues, efficiently managing the flow of visitors and travelers, avoiding congestion, and providing optimal traffic flow are crucial challenges. Conventional systems make it difficult to grasp congestion levels in real time, and there is a need for effective management that reduces visitor stress and improves the efficiency of facility operations.

[0110] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0111] In this invention, the server includes means for using a terminal to acquire travelers' personal data at the gates of transportation facilities, means for analyzing past and current data based on the acquired personal data to predict the flow, and means for understanding congestion levels and providing visitors with the optimal route and entrance / exit. This enables visitors and travelers to use the facilities efficiently, and allows operators to achieve optimized operations.

[0112] A "transportation facility" is a facility designed as a large-scale gathering point where users utilize means of transportation.

[0113] A "gate" is a passage point installed to manage entry and exit to a facility.

[0114] A "traveler" refers to an individual who travels using means of transportation.

[0115] "Personal data" refers to information used to identify a specific person, including passport information and boarding pass information.

[0116] A "terminal" is a device used to acquire personal data.

[0117] "Inspection equipment" refers to devices used to pre-process travelers, and may include 3D scanners and metal detectors.

[0118] "Flow" refers to the movement and flow of people within a facility.

[0119] "Prediction" refers to estimating future events using past and current data.

[0120] "Operations" refers to the operational management tasks carried out within a facility.

[0121] "Resources" refer to the human resources and physical materials necessary for the operation of a facility.

[0122] "Monitoring" is the act of observing the conditions within a facility and collecting data.

[0123] A "learning model" is a computational method used to make predictions and decisions based on accumulated data.

[0124] "Crowding status" refers to the degree of crowding and density of people within a facility.

[0125] A "route" refers to the path that visitors should take to move around within a facility.

[0126] "Entrances and exits" refer to the entrances and exits used for entering and leaving a facility.

[0127] To realize this invention, it is necessary to efficiently manage the flow of visitors and travelers using a system installed in transportation facilities and event venues. The server receives data from terminals installed at the gates of transportation facilities, analyzes personal data, and predicts flow. The terminals scan travelers' passports and boarding passes and collect personal data. As a result, the server can grasp congestion levels in real time and use a learning model to provide visitors with the optimal route and entrance / exit.

[0128] The hardware utilizes cameras and 3D scanners in smart glasses, while the software employs OpenCV and detection libraries. The device sends personal data to a cloud server, which analyzes it and generates congestion information and optimization data. The generated data is then displayed as instructions on the smart glasses or mobile devices worn by visitors.

[0129] As a concrete example, at a large music festival, visitors can enter the venue and, when moving to a designated stage while avoiding congestion, can be guided by a server to select the shortest route. An example of a prompt message would be: "Explain the effectiveness of a visitor management application within an event venue in a smart city. This application provides each visitor with the optimal route based on real-time congestion conditions and is implemented using smart glasses."

[0130] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0131] Step 1:

[0132] The terminal acquires visitors' personal data at the facility's gate. Using smart glasses or a camera, it scans the visitor's passport or ticket information and captures this information as digital data. The input is scanned personal information, and the output is personal information as data.

[0133] Step 2:

[0134] The device sends the acquired personal data to the server. This data includes the visitor's ID and flight information. The server receives this data, stores it in a database, and prepares it for subsequent processing. The input is personal data, and the output is information registered in the database on the server.

[0135] Step 3:

[0136] The server analyzes past and current visitor information based on the data. This allows it to predict flow patterns within the facility and generate instructions necessary for hardware control. A generative AI model is used for this analysis to efficiently predict visitor flow. The inputs are stored visitor data and the trained model, while the outputs are prediction results and control instructions.

[0137] Step 4:

[0138] The server monitors congestion in real time based on the analysis and generates instructions to provide visitors with the optimal route and exit. These instructions are sent to mobile devices and smart glasses, and visitors receive guidance visually or audibly. The input is the analysis result, and the output is the guidance instructions.

[0139] Step 5:

[0140] Users monitor congestion levels and guidance instructions provided by the server and provide support to visitors as needed. For example, if the guidance is not working properly, they may provide direct instruction. The input is the server's instructions, and the output is the actual support activity.

[0141] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0142] The present invention is a system for efficiently and appropriately managing passenger flow and inspection processes in transportation facilities, and in particular includes an emotion engine that recognizes the emotional state of passengers and users and provides feedback. The system mainly consists of terminals, servers, the emotion engine, and users.

[0143] Terminal operation

[0144] The terminal obtains personal data by scanning passengers' passports and boarding passes at the gate. This personal data is sent to a server and used for pre-processing by inspection equipment. In addition, the terminal has a built-in camera and microphone to capture passengers' facial expressions and voice data, which are then sent to an emotion engine.

[0145] Server Role

[0146] The server receives personal and emotional data transmitted from the terminal and stores it in a database. The server controls 3D scanners and metal detectors to check passengers' belongings. The server also dynamically optimizes operations and provides appropriate instructions to the user based on information from the emotional engine.

[0147] Functions of the Emotion Engine

[0148] The emotion engine analyzes facial expression and voice data received from the terminal to recognize the passenger's emotions. This recognition result is fed back to the server and used to implement measures to reduce passenger stress and anxiety. The emotion engine monitors changes in emotions in real time and incorporates them into a predictive model to improve the overall system performance.

[0149] User roles

[0150] Users manage airport inspections and operations based on information provided by the server. They can take appropriate action based on the emotional state of passengers. They also input feedback into the system, contributing to operational improvements.

[0151] Specific example

[0152] For example, if passengers are dissatisfied due to long wait times, the emotion engine can detect this dissatisfaction from their facial expressions. Based on this information, the server can allocate additional resources to avoid congestion and instruct users to take further action. This improves the passenger experience and optimizes operational efficiency.

[0153] In this way, the present invention realizes flexible and efficient operation of transportation facilities that utilize emotion recognition while ensuring passenger safety.

[0154] The following describes the processing flow.

[0155] Step 1:

[0156] The terminal scans the passenger's passport or boarding pass at the gate of a transportation facility and obtains personal data. This personal data includes the passenger's flight information and reservation information. The terminal sends this data to a server.

[0157] Step 2:

[0158] The camera and microphone on the device capture the passenger's facial expressions and voice data. This data is sent in real time to an emotion engine to analyze the passenger's emotional state.

[0159] Step 3:

[0160] The server records personal data transmitted from the terminal into a database and sends commands to control 3D scanners and metal detectors to inspect passengers and their belongings.

[0161] Step 4:

[0162] The emotion engine analyzes facial expressions and voice data received in real time to recognize the passenger's emotions. The recognition results identify states such as stress, anxiety, and relaxation.

[0163] Step 5:

[0164] The server integrates feedback from the emotion engine and inspection results to run a model for predicting passenger flow. Based on the prediction results, it calculates operational optimizations and dynamic resource allocation.

[0165] Step 6:

[0166] The server adjusts operations based on passengers' emotional states and provides users with necessary information to alleviate congestion. This includes deploying additional staff and adjusting gates.

[0167] Step 7:

[0168] Users will follow instructions from the server to take actions that streamline inspections and operations within the airport. They will also provide feedback via terminals, contributing to further improvements to the system.

[0169] Through these steps, the system maintains efficient operation while supporting comfortable travel for passengers.

[0170] (Example 2)

[0171] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0172] In transportation facilities, appropriately managing the flow and emotional state of users is crucial for ensuring user safety and comfort. However, conventional systems have struggled to accurately grasp the state of individual users and respond flexibly. In particular, when long waiting times occur, it is necessary to quickly detect and resolve user dissatisfaction and stress.

[0173] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0174] In this invention, the server includes a device for collecting user identification data at the entrance of a transportation facility, means for controlling an inspection device for pre-processing users based on the collected identification data and emotional state, and means for analyzing past and present data to predict the flow of people. This makes it possible to constantly understand the state of users and to allocate and respond to appropriate resources according to the situation.

[0175] "Transportation facilities" refer to places and equipment used by users of public transportation for travel and inspections.

[0176] The term "entrance" refers to the area within a transportation facility where users first pass through and where personal identification and security checks are performed.

[0177] "User" refers to an individual who uses transportation facilities, and in this context, includes persons who use means of transport.

[0178] "Identification data" refers to data that includes the user's personal information, such as passport information and information written on boarding passes.

[0179] "Emotional state" refers to the user's psychological and emotional state, including what can be inferred from data such as facial expressions and voice.

[0180] "Inspection equipment" refers to devices used to pre-process users, and includes three-dimensional scanners and metal detection devices.

[0181] "Predictive means" refers to methods and techniques for using past and present data to predict future user flows and conditions within a facility.

[0182] "Resource allocation" refers to methods for optimally allocating personnel and equipment within transportation facilities to ensure efficient user service.

[0183] This invention is a system for efficiently managing the flow and emotional state of users in transportation facilities. It mainly consists of terminals, servers, an emotion engine, and users.

[0184] The terminal collects user identification data at the entrance of transportation facilities. This identification data includes passport information and boarding pass data, and the terminal acquires this data using hardware such as a QR code reader or scanner. The terminal is also equipped with a camera and microphone to capture the user's facial expressions and voice, and transmits this data to the emotion engine.

[0185] The server receives identification and emotion data transmitted from terminals and stores them in a database. Simultaneously, the server controls 3D scanners and metal detectors, managing the actual inspection process. This control is performed in real time, supporting the smooth operation of the entire facility.

[0186] The emotion engine analyzes received facial expression and voice data to recognize the user's emotional state. This process utilizes facial expression recognition algorithms and voice analysis techniques. The recognition results are fed back to the server and used to reduce user dissatisfaction and stress.

[0187] Users will appropriately manage transportation facilities and respond to users based on information provided by the server. They will also play a role in dynamically improving facility operations by incorporating feedback within the system.

[0188] As a concrete example, if a user becomes dissatisfied due to a long waiting time, the emotion engine detects this dissatisfaction from the user's facial expression. Based on this information, the server can allocate additional resources and provide instructions to the user to alleviate congestion. This improves user comfort and enables the efficient operation of the entire facility.

[0189] For example, by inputting a prompt such as, "Please tell me the algorithm for detecting dissatisfaction and stress from passengers' facial expressions and voices," into the generating AI model, you can obtain detailed information about emotion recognition technology.

[0190] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0191] Step 1:

[0192] Collection of identification data by devices

[0193] The terminal scans the user's passport or boarding pass at the entrance of the transportation facility. For input, it receives the physical document (passport or boarding pass) and converts the data into a digital format using a QR code reader or scanner. For output, user identification data is generated and sent to the server.

[0194] Step 2:

[0195] Acquisition of emotional data

[0196] The device uses its camera and microphone to acquire facial expression and voice data from the first user it identifies. As input, it acquires real-time data of the user's face and voice, and converts this into digital signals. As output, facial expression and voice data are generated and sent to the emotion engine.

[0197] Step 3:

[0198] Data processing and storage by the server.

[0199] The server receives identification and sentiment data transmitted from the terminal. Identification and sentiment data are received as input and stored in the database. The data is classified and stored in storage, and as output, this data is prepared for use in subsequent processes.

[0200] Step 4:

[0201] Emotional analysis using an emotion engine

[0202] The emotion engine analyzes emotional data acquired from the device. It receives facial expression data and voice data as input, and applies facial expression recognition algorithms and voice analysis techniques. As output, the user's emotional state is quantified, and a recognition result is generated.

[0203] Step 5:

[0204] Server-based operation control

[0205] The server controls the testing device based on the recognition results from the emotion engine. It receives user emotional state information as input and generates signals for controlling the testing device. As output, it optimizes resource allocation to avoid congestion.

[0206] Step 6:

[0207] Providing and managing information to users

[0208] Users manage the operation of transportation facilities based on output from the server. Inputs include analysis results and operational recommendations provided by the server, while output involves efficient user support and adjustments to facility operations.

[0209] (Application Example 2)

[0210] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0211] In transportation facilities, it is difficult to recognize travelers' emotions in real time and efficiently optimize operations. Long waiting times and congestion often cause stress for travelers, and appropriate measures are needed to address this. Conventional systems have struggled to dynamically adjust based on travelers' emotions, resulting in decreased user satisfaction.

[0212] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0213] In this invention, the server includes means for analyzing past and present information to predict movement, means for recognizing the traveler's emotional state using facial expressions and voice data and making various suggestions, and means for updating a machine learning model to improve prediction accuracy based on monitoring results. This enables flexible and efficient operation of transportation facilities in accordance with the traveler's emotional state.

[0214] "Transportation facilities" refer to facilities such as train stations and airports that travelers use when moving around, and are places where efficient flow of people is required in their operation.

[0215] "Personal data" refers to information that enables the identification of travelers, including passport information and boarding pass data.

[0216] "Analytical device" refers to hardware or software used to analyze acquired data and perform pre-processing of travelers and baggage inspection.

[0217] "Predictive methods" refer to technologies that estimate traveler flow and congestion levels based on past and present information, and contribute to optimizing operations.

[0218] "Facial expression and voice data" refers to information necessary to understand the emotional state of travelers, and is data acquired using a camera and microphone.

[0219] "Emotional state" refers to the psychological and emotional state exhibited by a traveler, and is a concept that includes stress, anxiety, or pleasure.

[0220] A "machine learning model" is a technique for empirically improving a system by using algorithms that improve prediction accuracy based on data.

[0221] "Dynamic control" means flexibly adjusting system operation and resource allocation based on real-time information.

[0222] The system for realizing this invention includes a program that recognizes travelers' emotions in real time when using transportation facilities and optimizes operations accordingly. Its main components are a server, terminals, an emotion engine, and users. The operation method is described below.

[0223] The device is equipped with a camera and microphone to capture the traveler's facial expressions and voice data. This data is sent from the device to an emotion engine and used to evaluate their emotional state. The server receives the acquired emotion data and personal data (such as passport information and boarding pass information) and uses this to manage overall operations. For facial expression recognition, a platform with machine learning algorithms (e.g., Google® Cloud Vision API or Microsoft® Azure® Face API) is used.

[0224] The server analyzes historical and current data to predict traveler flow and congestion within facilities. For this purpose, a predictive model runs on the server, and machine learning models are updated in real time. The goal is to improve the traveler experience by dynamically adjusting resource allocation to avoid congestion. Furthermore, data obtained from emotional states, combined with monitoring results, contributes to improving prediction accuracy.

[0225] Users manage facility operations based on information provided by the server and offer optimal guidance to travelers. For example, by guiding travelers to routes with shorter wait times or crowded areas to avoid based on their stress levels as recognized by their devices, it is possible to make travel smoother and reduce dissatisfaction.

[0226] As a concrete example, if a traveler is dissatisfied due to long waiting times, facial expression data can be analyzed to detect this dissatisfaction. Based on this information, the server can reallocate resources and provide new guidance to the terminal to alleviate congestion. An example of a prompt using a generative AI model could be, "Tell us about your recent travel experience. What made you feel anxious?" This makes it possible to accurately capture the traveler's emotions and provide a more comfortable environment.

[0227] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0228] Step 1:

[0229] The terminals collect personal and emotional data of travelers within transportation facilities. Specifically, they collect personal data by scanning passport information and boarding passes, and facial expression and voice data using cameras and microphones. This data is necessary for identifying travelers and assessing their emotional state.

[0230] Step 2:

[0231] Personal data and emotional data acquired from the device are sent to the server. The server receives this data and stores it in a database. At this time, the emotional data is transferred to the emotional engine and used as input data to analyze the traveler's emotional state.

[0232] Step 3:

[0233] The server determines the traveler's emotional state based on the analysis results sent from the emotion engine. This result is recorded on the server as an emotional state (e.g., stress, anxiety, joy, etc.) and used in subsequent operational optimization processes.

[0234] Step 4:

[0235] The server uses accumulated historical data and current sentiment data to predict congestion levels and traveler flow within the facility. This prediction is performed using machine learning models. Based on the input data, it estimates congestion and waiting times and plans appropriate resource allocation.

[0236] Step 5:

[0237] The server dynamically optimizes operations based on prediction results. Specifically, it reallocates resources while considering emotional states and, if necessary, instructs terminals with guidance information to avoid congestion. This information is updated in real time to encourage travelers to take appropriate action.

[0238] Step 6:

[0239] Users manage facilities by utilizing emotional state and operational information provided by the server. They make adjustments as needed to ensure smooth operations on-site and provide appropriate guidance to travelers. By implementing operations that are sensitive to travelers' emotions, users aim to improve overall satisfaction.

[0240] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0241] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0242] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0243] [Second Embodiment]

[0244] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0245] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0246] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0247] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0248] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0249] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0250] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0251] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0252] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0253] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0254] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0255] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0256] This invention relates to a system for efficiently managing passenger flow in transportation facilities such as airports and railway stations. This system mainly consists of terminals, servers, and users.

[0257] Terminal operation

[0258] The terminals are installed at gates within transportation facilities and collect personal data by scanning passports and boarding passes as passengers pass through. This allows for the collection of passenger flight information, reservation information, and other data.

[0259] Server Role

[0260] The server receives personal data transmitted from terminals and stores it in a database. Based on this data, the server controls 3D scanners and metal detection devices to perform pre-processing on travelers. From the data obtained through analysis, the server builds a predictive model of passenger flow and calculates the optimal allocation of resources. Furthermore, based on information obtained from monitoring, the machine learning model is updated to improve prediction accuracy.

[0261] User intervention

[0262] Users, i.e., airport staff, receive instructions from the server as needed and assist with gate inspections. For example, in cases where the server cannot make an automatic decision, users can perform inspections directly. Users can also input feedback into a terminal to help improve the overall system.

[0263] Specific example

[0264] For example, during the morning rush hour when many passengers arrive, terminals quickly collect passenger personal data, and servers use this data to perform rapid inspections using 3D scanners. Predictive analytics allows servers to proactively identify which gates will be congested and instruct users on the optimal allocation of resources. This speeds up gate passage, reduces passenger waiting times, and lessens the burden on airport staff.

[0265] Through such operations, the present invention enables efficient and safe management of traveler flow, achieving both passenger comfort and efficient facility operation.

[0266] The following describes the processing flow.

[0267] Step 1:

[0268] The terminal scans the passports and boarding passes of passengers passing through the gates of transportation facilities and obtains personal data. This personal data includes flight information and reservation information. The obtained data is transmitted to the server in real time.

[0269] Step 2:

[0270] The server stores the personal data received from the terminal in a dedicated database. The server then uses this data to send appropriate control commands to the 3D scanner and metal detector, instructing them to inspect the passenger's belongings.

[0271] Step 3:

[0272] The terminal collects scan results from the 3D scanner and metal detector and sends them back to the server. This includes information about the object's shape and metal detection.

[0273] Step 4:

[0274] The server analyzes the scanned data to check for any security issues. If a problem is detected, the server alerts the user and prompts them to investigate further.

[0275] Step 5:

[0276] The server runs a machine learning model that predicts passenger flow based on analysis results and historical data, and predicts congestion levels. The resulting predictions are used in real time to optimize resource allocation.

[0277] Step 6:

[0278] The server dynamically adjusts airport staff deployment and gate opening / closing times based on predictions and sends instructions to the user. The user then makes on-site adjustments according to these instructions.

[0279] Step 7:

[0280] The server monitors the operating status of the system and the inspection passing time, and uses the data in a feedback loop to improve the learning model. To improve the prediction accuracy, the model is retrained regularly.

[0281] By going through these series of steps, the efficient operation of the entire system is maintained, and smooth movement of passengers is realized.

[0282] (Example 1)

[0283] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0284] In modern transportation facilities, it is required to enable users to pass through the facilities smoothly and quickly. However, in conventional systems, it is difficult to effectively predict and manage the flow of users, resulting in problems such as congestion in the facilities and waiting time for users. This has caused a decline in the convenience and stress of users. Also, for the facility operation side, there has been a problem that appropriate resources cannot be allocated and efficient operation is difficult.

[0285] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0286] In this invention, the server includes a device for collecting user identification data at the passing points of the transportation facility, means for adjusting an evaluation device for performing initial processing of the user based on the collected identification data, means for predicting the flow by utilizing past information and real-time information, means for optimizing the operation based on the prediction result and allocating resources according to changes, and means for correcting the learning model to improve the accuracy of the prediction from the monitoring results. Thereby, it becomes possible to efficiently manage the flow of users and relieve congestion in the facility.

[0287] "Transportation facilities" refer to facilities such as airports and train stations where people gather and travel.

[0288] A "waypoint" refers to a location such as a gate or checkpoint that users must pass through.

[0289] "Identification data" refers to information that includes users' personal information and identification information.

[0290] The term "device" refers to a machine or mechanism designed to perform a specific function.

[0291] An "evaluation device" refers to a machine or equipment designed to perform initial processing for users.

[0292] "Flow" refers to the patterns of movement and behavior of users within a transportation facility.

[0293] "Prediction" refers to the act of estimating future events based on past information and real-time data.

[0294] "Operation" refers to the activities of efficiently managing and coordinating facilities and systems.

[0295] "Resources" refer to personnel, equipment, space, and other resources that can be used as such.

[0296] "Allocation" refers to the act of allocating resources to the appropriate places and times as needed.

[0297] "Monitoring results" refer to information and insights obtained through observation and data collection.

[0298] A "learning model" refers to an algorithm or mathematical model used to identify patterns based on data and perform predictions or classifications.

[0299] "Correction" refers to activities that involve making changes or adjustments to improve the current situation.

[0300] This invention is a system for efficiently managing the flow of users in transportation facilities. This system mainly consists of three elements: terminals, servers, and users.

[0301] Terminal operation

[0302] The terminals are installed at transit points in transportation facilities and acquire identification data by scanning passports or boarding passes as users pass through. The terminal hardware incorporates high-speed scanners and data transmission capabilities, transferring the acquired data to a server in real time.

[0303] Server Role

[0304] The server receives identification data transmitted from the terminal and securely stores it in a database. The server uses this data to control the evaluation device and perform initial user processing. The software used includes algorithms and machine learning models for data analysis, enabling prediction of user flow. Specifically, it analyzes historical and real-time data to predict user flow and identify potential congestion points. The server further refines the learning model using monitoring results to improve prediction accuracy. In this process, the server optimizes resources and improves operational efficiency.

[0305] User intervention

[0306] Users, i.e., facility staff, take appropriate actions based on information provided by the server. For example, at points where congestion is anticipated, additional staff are deployed based on instructions from the server. Furthermore, if the server cannot make a decision, users conduct inspections directly and contribute to system improvements by inputting necessary feedback into their terminals.

[0307] As a specific example, consider an environment where many users arrive during the morning rush hour. The terminal quickly collects the identification data of the users, and the server uses a 3D scanner to perform a quick inspection. Through prediction, the server identifies at which passing points congestion will occur and notifies the users of this. Based on this information, the users perform appropriate resource allocation so that congestion does not occur.

[0308] As an example of a prompt sentence, "Please advise on how to cope with the sudden change in passenger density at the airport at a specific time on the weekend." can be considered. Thus, guidance for optimizing specific operations can be received.

[0309] The flow of the specific process in Example 1 will be described using FIG. 11.

[0310] Step 1:

[0311] The terminal acquires the identification data of the users at the passing points of the transportation facility. In this operation, the passport and boarding pass are read by the scanner of the terminal. The input is the user's passport or boarding pass, and the terminal outputs the digitized identification data by scanning. This data includes the user's ID, flight information, etc.

[0312] Step 2:

[0313] The terminal transfers the acquired identification data to the server. The input here is the identification data generated in Step 1. The terminal encrypts this data and transmits it to the server through the network, and the server stores this in the database. The output is the securely stored identification data.

[0314] Step 3:

[0315] The server controls the evaluation equipment using the received identification data and performs initial processing for the user. The input is identification data transmitted from the terminal. Based on this data, the server activates 3D scanners and metal detectors and obtains output by analyzing the measurement results. The output is a determination of whether the user is allowed to pass and an alert if necessary.

[0316] Step 4:

[0317] The server analyzes historical and current data to predict user flow. The input for this step is historical and real-time identification data stored in a database. The server uses a machine learning model to analyze this data and predict the probability and location of congestion. The output is predicted information about future user flow.

[0318] Step 5:

[0319] The server optimizes operations and dynamically allocates resources based on predictive information. The input is the predictive information obtained in step 4. The server uses this to calculate how many staff members are needed at each transit point and notifies the user of this information to ensure optimal resource allocation. The output is the allocation instructions and related log information.

[0320] Step 6:

[0321] The user takes the necessary actions based on deployment instructions provided by the server. Here, the user receives instructions from the server as input. The user observes the situation in real time, changes staff deployments as needed, and provides feedback to the server. The output is a report of the improved deployment based on the feedback and its status.

[0322] (Application Example 1)

[0323] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0324] In large-scale facilities that attract large crowds, such as transportation facilities and event venues, efficiently managing the flow of visitors and travelers, avoiding congestion, and providing optimal traffic flow are crucial challenges. Conventional systems make it difficult to grasp congestion levels in real time, and there is a need for effective management that reduces visitor stress and improves the efficiency of facility operations.

[0325] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0326] In this invention, the server includes means for using a terminal to acquire travelers' personal data at the gates of transportation facilities, means for analyzing past and current data based on the acquired personal data to predict the flow, and means for understanding congestion levels and providing visitors with the optimal route and entrance / exit. This enables visitors and travelers to use the facilities efficiently, and allows operators to achieve optimized operations.

[0327] A "transportation facility" is a facility designed as a large-scale gathering point where users utilize means of transportation.

[0328] A "gate" is a passage point installed to manage entry and exit to a facility.

[0329] A "traveler" refers to an individual who travels using means of transportation.

[0330] "Personal data" refers to information used to identify a specific person, including passport information and boarding pass information.

[0331] A "terminal" is a device used to acquire personal data.

[0332] "Inspection equipment" refers to devices used to pre-process travelers, and may include 3D scanners and metal detectors.

[0333] "Flow" refers to the movement and flow of people within a facility.

[0334] "Prediction" refers to estimating future events using past and current data.

[0335] "Operations" refers to the operational management tasks carried out within a facility.

[0336] "Resources" refer to the human resources and physical materials necessary for the operation of a facility.

[0337] "Monitoring" is the act of observing the conditions within a facility and collecting data.

[0338] A "learning model" is a computational method used to make predictions and decisions based on accumulated data.

[0339] "Crowding status" refers to the degree of crowding and density of people within a facility.

[0340] A "route" refers to the path that visitors should take to move around within a facility.

[0341] "Entrances and exits" refer to the entrances and exits used for entering and leaving a facility.

[0342] To realize this invention, it is necessary to efficiently manage the flow of visitors and travelers using a system installed in transportation facilities and event venues. The server receives data from terminals installed at the gates of transportation facilities, analyzes personal data, and predicts flow. The terminals scan travelers' passports and boarding passes and collect personal data. As a result, the server can grasp congestion levels in real time and use a learning model to provide visitors with the optimal route and entrance / exit.

[0343] The hardware utilizes cameras and 3D scanners in smart glasses, while the software employs OpenCV and detection libraries. The device sends personal data to a cloud server, which analyzes it and generates congestion information and optimization data. The generated data is then displayed as instructions on the smart glasses or mobile devices worn by visitors.

[0344] As a concrete example, at a large music festival, visitors can enter the venue and, when moving to a designated stage while avoiding congestion, can be guided by a server to select the shortest route. An example of a prompt message would be: "Explain the effectiveness of a visitor management application within an event venue in a smart city. This application provides each visitor with the optimal route based on real-time congestion conditions and is implemented using smart glasses."

[0345] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0346] Step 1:

[0347] The terminal acquires visitors' personal data at the facility's gate. Using smart glasses or a camera, it scans the visitor's passport or ticket information and captures this information as digital data. The input is scanned personal information, and the output is personal information as data.

[0348] Step 2:

[0349] The device sends the acquired personal data to the server. This data includes the visitor's ID and flight information. The server receives this data, stores it in a database, and prepares it for subsequent processing. The input is personal data, and the output is information registered in the database on the server.

[0350] Step 3:

[0351] The server analyzes past and current visitor information based on the data. This allows it to predict flow patterns within the facility and generate instructions necessary for hardware control. A generative AI model is used for this analysis to efficiently predict visitor flow. The inputs are stored visitor data and the trained model, while the outputs are prediction results and control instructions.

[0352] Step 4:

[0353] The server monitors congestion in real time based on the analysis and generates instructions to provide visitors with the optimal route and exit. These instructions are sent to mobile devices and smart glasses, and visitors receive guidance visually or audibly. The input is the analysis result, and the output is the guidance instructions.

[0354] Step 5:

[0355] Users monitor congestion levels and guidance instructions provided by the server and provide support to visitors as needed. For example, if the guidance is not working properly, they may provide direct instruction. The input is the server's instructions, and the output is the actual support activity.

[0356] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0357] The present invention is a system for efficiently and appropriately managing passenger flow and inspection processes in transportation facilities, and in particular includes an emotion engine that recognizes the emotional state of passengers and users and provides feedback. The system mainly consists of terminals, servers, the emotion engine, and users.

[0358] Terminal operation

[0359] The terminal obtains personal data by scanning passengers' passports and boarding passes at the gate. This personal data is sent to a server and used for pre-processing by inspection equipment. In addition, the terminal has a built-in camera and microphone to capture passengers' facial expressions and voice data, which are then sent to an emotion engine.

[0360] Server Role

[0361] The server receives personal and emotional data transmitted from the terminal and stores it in a database. The server controls 3D scanners and metal detectors to check passengers' belongings. The server also dynamically optimizes operations and provides appropriate instructions to the user based on information from the emotional engine.

[0362] Functions of the Emotion Engine

[0363] The emotion engine analyzes facial expression and voice data received from the terminal to recognize the passenger's emotions. This recognition result is fed back to the server and used to implement measures to reduce passenger stress and anxiety. The emotion engine monitors changes in emotions in real time and incorporates them into a predictive model to improve the overall system performance.

[0364] User roles

[0365] Users manage airport inspections and operations based on information provided by the server. They can take appropriate action based on the emotional state of passengers. They also input feedback into the system, contributing to operational improvements.

[0366] Specific example

[0367] For example, if passengers are dissatisfied due to long wait times, the emotion engine can detect this dissatisfaction from their facial expressions. Based on this information, the server can allocate additional resources to avoid congestion and instruct users to take further action. This improves the passenger experience and optimizes operational efficiency.

[0368] In this way, the present invention realizes flexible and efficient operation of transportation facilities that utilize emotion recognition while ensuring passenger safety.

[0369] The following describes the processing flow.

[0370] Step 1:

[0371] The terminal scans the passenger's passport or boarding pass at the gate of a transportation facility and obtains personal data. This personal data includes the passenger's flight information and reservation information. The terminal sends this data to a server.

[0372] Step 2:

[0373] The terminal's built-in camera and microphone capture passengers' facial expressions and voice data. This data is sent in real time to an emotion engine to analyze the passengers' emotional state.

[0374] Step 3:

[0375] The server records personal data transmitted from the terminal into a database and sends commands to control 3D scanners and metal detectors to inspect passengers and their belongings.

[0376] Step 4:

[0377] The emotion engine analyzes facial expressions and voice data received in real time to recognize the passenger's emotions. The recognition results identify states such as stress, anxiety, and relaxation.

[0378] Step 5:

[0379] The server integrates feedback from the emotion engine and inspection results to run a model for predicting passenger flow. Based on the prediction results, it calculates operational optimizations and dynamic resource allocation.

[0380] Step 6:

[0381] The server adjusts operations based on passengers' emotional states and provides users with necessary information to alleviate congestion. This includes deploying additional staff and adjusting gates.

[0382] Step 7:

[0383] Users will follow instructions from the server to take actions that streamline inspections and operations within the airport. They will also provide feedback via terminals, contributing to further improvements to the system.

[0384] Through these steps, the system maintains efficient operation while supporting comfortable travel for passengers.

[0385] (Example 2)

[0386] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0387] In transportation facilities, appropriately managing the flow and emotional state of users is crucial for ensuring user safety and comfort. However, conventional systems have struggled to accurately grasp the state of individual users and respond flexibly. In particular, when long waiting times occur, it is necessary to quickly detect and resolve user dissatisfaction and stress.

[0388] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0389] In this invention, the server includes a device for collecting user identification data at the entrance of a transportation facility, means for controlling an inspection device for pre-processing users based on the collected identification data and emotional state, and means for analyzing past and present data to predict the flow of people. This makes it possible to constantly understand the state of users and to allocate and respond to appropriate resources according to the situation.

[0390] "Transportation facilities" refer to places and equipment used by users of public transportation for travel and inspections.

[0391] The term "entrance" refers to the area within a transportation facility where users first pass through and where personal identification and security checks are performed.

[0392] "User" refers to an individual who uses transportation facilities, and in this context, includes persons who use means of transport.

[0393] "Identification data" refers to data that includes the user's personal information, such as passport information and information written on boarding passes.

[0394] "Emotional state" refers to the user's psychological and emotional state, including what can be inferred from data such as facial expressions and voice.

[0395] "Inspection equipment" refers to devices used to pre-process users, and includes three-dimensional scanners and metal detection devices.

[0396] "Predictive means" refers to methods and techniques for using past and present data to predict future user flows and conditions within a facility.

[0397] "Resource allocation" refers to methods for optimally allocating personnel and equipment within transportation facilities to ensure efficient user service.

[0398] This invention is a system for efficiently managing the flow and emotional state of users in transportation facilities. It mainly consists of terminals, servers, an emotion engine, and users.

[0399] The terminal collects user identification data at the entrance of transportation facilities. This identification data includes passport information and boarding pass data, and the terminal acquires this data using hardware such as QR code readers and scanners. The terminal is also equipped with a camera and microphone to capture the user's facial expressions and voice, and transmits this data to the emotion engine.

[0400] The server receives identification and emotion data transmitted from terminals and stores them in a database. Simultaneously, the server controls 3D scanners and metal detectors, managing the actual inspection process. This control is performed in real time, supporting the smooth operation of the entire facility.

[0401] The emotion engine analyzes received facial expression and voice data to recognize the user's emotional state. This process utilizes facial expression recognition algorithms and voice analysis techniques. The recognition results are fed back to the server and used to reduce user dissatisfaction and stress.

[0402] Users will appropriately manage transportation facilities and respond to users based on information provided by the server. They will also play a role in dynamically improving facility operations by incorporating feedback within the system.

[0403] As a concrete example, if a user becomes dissatisfied due to a long waiting time, the emotion engine detects this dissatisfaction from the user's facial expression. Based on this information, the server can allocate additional resources and provide instructions to the user to alleviate congestion. This improves user comfort and enables the efficient operation of the entire facility.

[0404] For example, by inputting a prompt such as, "Please tell me the algorithm for detecting dissatisfaction and stress from passengers' facial expressions and voices," into the generating AI model, you can obtain detailed information about emotion recognition technology.

[0405] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0406] Step 1:

[0407] Collection of identification data by devices

[0408] The terminal scans the user's passport or boarding pass at the entrance of the transportation facility. For input, it receives the physical document (passport or boarding pass) and converts the data into a digital format using a QR code reader or scanner. For output, user identification data is generated and sent to the server.

[0409] Step 2:

[0410] Acquisition of emotional data

[0411] The device uses its camera and microphone to acquire facial expression and voice data from the first user it identifies. As input, it acquires real-time data of the user's face and voice, and converts this into digital signals. As output, facial expression and voice data are generated and sent to the emotion engine.

[0412] Step 3:

[0413] Data processing and storage by the server.

[0414] The server receives identification and sentiment data transmitted from the terminal. Identification and sentiment data are received as input and stored in the database. The data is classified and stored, and as output, this data is prepared for use in subsequent processes.

[0415] Step 4:

[0416] Emotional analysis using an emotion engine

[0417] The emotion engine analyzes emotional data acquired from the device. It receives facial expression data and voice data as input, and applies facial expression recognition algorithms and voice analysis techniques. As output, the user's emotional state is quantified, and a recognition result is generated.

[0418] Step 5:

[0419] Server-based operation control

[0420] The server controls the testing device based on the recognition results from the emotion engine. It receives user emotional state information as input and generates signals for controlling the testing device. As output, it optimizes resource allocation to avoid congestion.

[0421] Step 6:

[0422] Providing and managing information to users

[0423] Users manage the operation of transportation facilities based on output from the server. Inputs include analysis results and operational recommendations provided by the server, while output involves efficient user support and adjustments to facility operations.

[0424] (Application Example 2)

[0425] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0426] In transportation facilities, it is difficult to recognize travelers' emotions in real time and efficiently optimize operations. Long waiting times and congestion often cause stress for travelers, and appropriate measures are needed to address this. Conventional systems have struggled to dynamically adjust based on travelers' emotions, resulting in decreased user satisfaction.

[0427] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0428] In this invention, the server includes means for analyzing past and present information to predict movement, means for recognizing the traveler's emotional state using facial expressions and voice data and making various suggestions, and means for updating a machine learning model to improve prediction accuracy based on monitoring results. This enables flexible and efficient operation of transportation facilities in accordance with the traveler's emotional state.

[0429] "Transportation facilities" refer to facilities such as train stations and airports that travelers use when moving around, and are places where efficient flow of people is required in their operation.

[0430] "Personal data" refers to information that enables the identification of travelers, including passport information and boarding pass data.

[0431] "Analytical device" refers to hardware or software used to analyze acquired data and perform pre-processing of travelers and baggage inspection.

[0432] "Predictive methods" refer to technologies that estimate traveler flow and congestion levels based on past and present information, and contribute to optimizing operations.

[0433] "Facial expression and voice data" refers to information necessary to understand the emotional state of travelers, and is data acquired using a camera and microphone.

[0434] "Emotional state" refers to the psychological and emotional state exhibited by a traveler, and is a concept that includes stress, anxiety, or pleasure.

[0435] A "machine learning model" is a technique for empirically improving a system by using algorithms that improve prediction accuracy based on data.

[0436] "Dynamic control" means flexibly adjusting system operation and resource allocation based on real-time information.

[0437] The system for realizing this invention includes a program that recognizes travelers' emotions in real time when using transportation facilities and optimizes operations accordingly. Its main components are a server, terminals, an emotion engine, and users. The operation method is described below.

[0438] The terminal is equipped with a camera and microphone to capture the traveler's facial expressions and voice data. This data is sent from the terminal to an emotion engine and used to evaluate their emotional state. The server receives the acquired emotion data and personal data (such as passport information and boarding pass information) and uses this to manage overall operations. A platform with machine learning algorithms (e.g., Google Cloud Vision API or Microsoft Azure Face API) is used for facial expression recognition.

[0439] The server analyzes historical and current data to predict traveler flow and congestion within facilities. For this purpose, a predictive model runs on the server, and machine learning models are updated in real time. The goal is to improve the traveler experience by dynamically adjusting resource allocation to avoid congestion. Furthermore, data obtained from emotional states, combined with monitoring results, contributes to improving prediction accuracy.

[0440] Users manage facility operations based on information provided by the server and offer optimal guidance to travelers. For example, by guiding travelers to routes with shorter wait times or crowded areas to avoid based on their stress levels as recognized by their devices, it is possible to make travel smoother and reduce dissatisfaction.

[0441] As a concrete example, if a traveler is dissatisfied due to long waiting times, facial expression data can be analyzed to detect this dissatisfaction. Based on this information, the server can reallocate resources and provide new guidance to the terminal to alleviate congestion. An example of a prompt using a generative AI model could be, "Tell us about your recent travel experience. What made you feel anxious?" This makes it possible to accurately capture the traveler's emotions and provide a more comfortable environment.

[0442] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0443] Step 1:

[0444] The terminals collect personal and emotional data of travelers within transportation facilities. Specifically, they collect personal data by scanning passport information and boarding passes, and facial expression and voice data using cameras and microphones. This data is necessary for identifying travelers and assessing their emotional state.

[0445] Step 2:

[0446] Personal data and emotional data acquired from the device are sent to the server. The server receives this data and stores it in a database. At this time, the emotional data is transferred to the emotional engine and used as input data to analyze the traveler's emotional state.

[0447] Step 3:

[0448] The server determines the traveler's emotional state based on the analysis results sent from the emotion engine. This result is recorded on the server as an emotional state (e.g., stress, anxiety, joy, etc.) and used in subsequent operational optimization processes.

[0449] Step 4:

[0450] The server uses accumulated historical data and current sentiment data to predict congestion levels and traveler flow within the facility. This prediction is performed using machine learning models. Based on the input data, it estimates congestion and waiting times and plans appropriate resource allocation.

[0451] Step 5:

[0452] The server dynamically optimizes operations based on prediction results. Specifically, it reallocates resources while considering emotional states and, if necessary, instructs terminals with guidance information to avoid congestion. This information is updated in real time to encourage travelers to take appropriate action.

[0453] Step 6:

[0454] Users manage facilities by utilizing emotional state and operational information provided by the server. They make adjustments as needed to ensure smooth operations on-site and provide appropriate guidance to travelers. By implementing operations that are sensitive to travelers' emotions, users aim to improve overall satisfaction.

[0455] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0456] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0457] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0458] [Third Embodiment]

[0459] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0460] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0461] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0462] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0463] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0464] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0465] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0466] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0467] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0468] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0469] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0470] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0471] This invention relates to a system for efficiently managing passenger flow in transportation facilities such as airports and railway stations. This system mainly consists of terminals, servers, and users.

[0472] Terminal operation

[0473] The terminals are installed at gates within transportation facilities and collect personal data by scanning passports and boarding passes as passengers pass through. This allows for the collection of passenger flight information, reservation information, and other data.

[0474] Server Role

[0475] The server receives personal data transmitted from terminals and stores it in a database. Based on this data, the server controls 3D scanners and metal detection devices to perform pre-processing on travelers. From the data obtained through analysis, the server builds a predictive model of passenger flow and calculates the optimal allocation of resources. Furthermore, based on information obtained from monitoring, the machine learning model is updated to improve prediction accuracy.

[0476] User intervention

[0477] Users, i.e., airport staff, receive instructions from the server as needed and assist with gate inspections. For example, in cases where the server cannot make an automatic decision, users can perform inspections directly. Users can also input feedback into a terminal to help improve the overall system.

[0478] Specific example

[0479] For example, during the morning rush hour when many passengers arrive, terminals quickly collect passenger personal data, and servers use this data to perform rapid inspections using 3D scanners. Predictive analytics allows servers to proactively identify which gates will be congested and instruct users on the optimal allocation of resources. This speeds up gate passage, reduces passenger waiting times, and lessens the burden on airport staff.

[0480] Through such operations, the present invention enables efficient and safe management of traveler flow, achieving both passenger comfort and efficient facility operation.

[0481] The following describes the processing flow.

[0482] Step 1:

[0483] The terminal scans the passports and boarding passes of passengers passing through the gates of transportation facilities and obtains personal data. This personal data includes flight information and reservation information. The obtained data is transmitted to the server in real time.

[0484] Step 2:

[0485] The server stores the personal data received from the terminal in a dedicated database. The server then uses this data to send appropriate control commands to the 3D scanner and metal detector, instructing them to inspect the passenger's belongings.

[0486] Step 3:

[0487] The terminal collects scan results from the 3D scanner and metal detector and sends them back to the server. This includes information about the object's shape and metal detection.

[0488] Step 4:

[0489] The server analyzes the scanned data to check for any security issues. If a problem is detected, the server alerts the user and prompts them to investigate further.

[0490] Step 5:

[0491] The server runs a machine learning model that predicts passenger flow based on analysis results and historical data, and predicts congestion levels. The resulting predictions are used in real time to optimize resource allocation.

[0492] Step 6:

[0493] The server dynamically adjusts airport staff deployment and gate opening / closing times based on predictions and sends instructions to the user. The user then makes on-site adjustments according to these instructions.

[0494] Step 7:

[0495] The server monitors the system's operational status and inspection completion times, and uses this data in a feedback loop to improve the learning model. The model is retrained periodically to improve prediction accuracy.

[0496] By following these steps, the efficient operation of the entire system is maintained, and the smooth movement of passengers is ensured.

[0497] (Example 1)

[0498] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0499] Modern transportation facilities require users to pass through them smoothly and quickly. However, conventional systems have difficulty effectively predicting and managing user flow, resulting in congestion and long waiting times within facilities. This leads to decreased convenience and stress for users. Furthermore, facility operators face challenges in allocating resources appropriately and achieving efficient operation.

[0500] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0501] In this invention, the server includes a device for collecting user identification data at transit points of transportation facilities, means for adjusting an evaluation device for performing initial processing on users based on the collected identification data, means for predicting flow by utilizing historical and real-time information, means for optimizing operations based on the prediction results and allocating resources in response to changes, and means for modifying a learning model to improve the accuracy of predictions based on monitoring results. This makes it possible to efficiently manage the flow of users and alleviate congestion within facilities.

[0502] "Transportation facilities" refer to facilities such as airports and train stations where people gather and travel.

[0503] A "waypoint" refers to a location such as a gate or checkpoint that users must pass through.

[0504] "Identification data" refers to information that includes users' personal information and identification information.

[0505] The term "device" refers to a machine or mechanism designed to perform a specific function.

[0506] An "evaluation device" refers to a machine or equipment designed to perform initial processing for users.

[0507] "Flow" refers to the patterns of movement and behavior of users within a transportation facility.

[0508] "Prediction" refers to the act of estimating future events based on past information and real-time data.

[0509] "Operation" refers to the activities of efficiently managing and coordinating facilities and systems.

[0510] "Resources" refer to personnel, equipment, space, and other resources that can be used as such.

[0511] "Allocation" refers to the act of allocating resources to the appropriate places and times as needed.

[0512] "Monitoring results" refer to information and insights obtained through observation and data collection.

[0513] A "learning model" refers to an algorithm or mathematical model used to identify patterns based on data and perform predictions or classifications.

[0514] "Correction" refers to activities that involve making changes or adjustments to improve the current situation.

[0515] This invention is a system for efficiently managing the flow of users in transportation facilities. This system mainly consists of three elements: terminals, servers, and users.

[0516] Terminal operation

[0517] The terminals are installed at transit points in transportation facilities and acquire identification data by scanning passports or boarding passes as users pass through. The terminal hardware incorporates high-speed scanners and data transmission capabilities, transferring the acquired data to a server in real time.

[0518] Server Role

[0519] The server receives identification data transmitted from the terminal and securely stores it in a database. The server uses this data to control the evaluation device and perform initial user processing. The software used includes algorithms and machine learning models for data analysis, enabling prediction of user flow. Specifically, it analyzes historical and real-time data to predict user flow and identify potential congestion points. The server further refines the learning model using monitoring results to improve prediction accuracy. In this process, the server optimizes resources and improves operational efficiency.

[0520] User intervention

[0521] Users, i.e., facility staff, take appropriate actions based on information provided by the server. For example, at points where congestion is anticipated, additional staff are deployed based on instructions from the server. Furthermore, if the server cannot make a decision, users conduct inspections directly and contribute to system improvements by inputting necessary feedback into their terminals.

[0522] As a concrete example, consider an environment where many users arrive during the morning rush hour. Terminals quickly collect user identification data, and servers perform rapid inspections using 3D scanners. Through prediction, the servers identify where congestion will occur and notify users. Based on this information, users can allocate resources appropriately to avoid congestion.

[0523] An example of a prompt message would be, "Please advise on how to address the rapid changes in passenger density at the airport during specific times on weekends." This would allow for guidance on optimizing specific operations.

[0524] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0525] Step 1:

[0526] The terminal acquires user identification data at transit points within transportation facilities. This process involves scanning the user's passport or boarding pass. The input is the user's passport or boarding pass, and the terminal outputs digitized identification data through scanning. This data includes the user's ID and flight information.

[0527] Step 2:

[0528] The terminal transfers the acquired identification data to the server. The input here is the identification data generated in step 1. The terminal encrypts this data and sends it to the server over the network, and the server stores it in its database. The output is the securely stored identification data.

[0529] Step 3:

[0530] The server controls the evaluation equipment using the received identification data and performs initial processing for the user. The input is identification data transmitted from the terminal. Based on this data, the server activates 3D scanners and metal detectors and obtains output by analyzing the measurement results. The output is a determination of whether the user is allowed to pass and an alert if necessary.

[0531] Step 4:

[0532] The server analyzes historical and current data to predict user flow. The input for this step is historical and real-time identification data stored in a database. The server uses a machine learning model to analyze this data and predict the probability and location of congestion. The output is predicted information about future user flow.

[0533] Step 5:

[0534] The server optimizes operations and dynamically allocates resources based on predictive information. The input is the predictive information obtained in step 4. The server uses this to calculate how many staff members are needed at each transit point and notifies the user of this information to ensure optimal resource allocation. The output is the allocation instructions and related log information.

[0535] Step 6:

[0536] The user takes the necessary actions based on deployment instructions provided by the server. Here, the user receives instructions from the server as input. The user observes the situation in real time, changes staff deployments as needed, and provides feedback to the server. The output is a report of the improved deployment based on the feedback and its status.

[0537] (Application Example 1)

[0538] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0539] In large-scale facilities that attract large crowds, such as transportation facilities and event venues, efficiently managing the flow of visitors and travelers, avoiding congestion, and providing optimal traffic flow are crucial challenges. Conventional systems make it difficult to grasp congestion levels in real time, and there is a need for effective management that reduces visitor stress and improves the efficiency of facility operations.

[0540] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0541] In this invention, the server includes means for using a terminal to acquire travelers' personal data at the gates of transportation facilities, means for analyzing past and current data based on the acquired personal data to predict the flow, and means for understanding congestion levels and providing visitors with the optimal route and entrance / exit. This enables visitors and travelers to use the facilities efficiently, and allows operators to achieve optimized operations.

[0542] A "transportation facility" is a facility designed as a large-scale gathering point where users utilize means of transportation.

[0543] A "gate" is a passage point installed to manage entry and exit to a facility.

[0544] A "traveler" refers to an individual who travels using means of transportation.

[0545] "Personal data" refers to information used to identify a specific person, including passport information and boarding pass information.

[0546] A "terminal" is a device used to acquire personal data.

[0547] "Inspection equipment" refers to devices used to pre-process travelers, and may include 3D scanners and metal detectors.

[0548] "Flow" refers to the movement and flow of people within a facility.

[0549] "Prediction" refers to estimating future events using past and current data.

[0550] "Operations" refers to the operational management tasks carried out within a facility.

[0551] "Resources" refer to the human resources and physical materials necessary for the operation of a facility.

[0552] "Monitoring" is the act of observing the conditions within a facility and collecting data.

[0553] A "learning model" is a computational method used to make predictions and decisions based on accumulated data.

[0554] "Crowding status" refers to the degree of crowding and density of people within a facility.

[0555] A "route" refers to the path that visitors should take to move around within a facility.

[0556] "Entrances and exits" refer to the entrances and exits used for entering and leaving a facility.

[0557] To realize this invention, it is necessary to efficiently manage the flow of visitors and travelers using a system installed in transportation facilities and event venues. The server receives data from terminals installed at the gates of transportation facilities, analyzes personal data, and predicts flow. The terminals scan travelers' passports and boarding passes and collect personal data. As a result, the server can grasp congestion levels in real time and use a learning model to provide visitors with the optimal route and entrance / exit.

[0558] The hardware utilizes cameras and 3D scanners in smart glasses, while the software employs OpenCV and detection libraries. The device sends personal data to a cloud server, which analyzes it and generates congestion information and optimization data. The generated data is then displayed as instructions on the smart glasses or mobile devices worn by visitors.

[0559] As a concrete example, at a large music festival, visitors can enter the venue and, when moving to a designated stage while avoiding congestion, can be guided by a server to select the shortest route. An example of a prompt message would be: "Explain the effectiveness of a visitor management application within an event venue in a smart city. This application provides each visitor with the optimal route based on real-time congestion conditions and is implemented using smart glasses."

[0560] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0561] Step 1:

[0562] The terminal acquires visitors' personal data at the facility's gate. Using smart glasses or a camera, it scans the visitor's passport or ticket information and captures this information as digital data. The input is scanned personal information, and the output is personal information as data.

[0563] Step 2:

[0564] The device sends the acquired personal data to the server. This data includes the visitor's ID and flight information. The server receives this data, stores it in a database, and prepares it for subsequent processing. The input is personal data, and the output is information registered in the database on the server.

[0565] Step 3:

[0566] The server analyzes past and current visitor information based on the data. This allows it to predict flow patterns within the facility and generate instructions necessary for hardware control. A generative AI model is used for this analysis to efficiently predict visitor flow. The inputs are stored visitor data and the trained model, while the outputs are prediction results and control instructions.

[0567] Step 4:

[0568] The server monitors congestion in real time based on the analysis and generates instructions to provide visitors with the optimal route and exit. These instructions are sent to mobile devices and smart glasses, and visitors receive guidance visually or audibly. The input is the analysis result, and the output is the guidance instructions.

[0569] Step 5:

[0570] Users monitor congestion levels and guidance instructions provided by the server and provide support to visitors as needed. For example, if the guidance is not working properly, they may provide direct instruction. The input is the server's instructions, and the output is the actual support activity.

[0571] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0572] The present invention is a system for efficiently and appropriately managing passenger flow and inspection processes in transportation facilities, and in particular includes an emotion engine that recognizes the emotional state of passengers and users and provides feedback. The system mainly consists of terminals, servers, the emotion engine, and users.

[0573] Terminal operation

[0574] The terminal obtains personal data by scanning passengers' passports and boarding passes at the gate. This personal data is sent to a server and used for pre-processing by inspection equipment. In addition, the terminal has a built-in camera and microphone to capture passengers' facial expressions and voice data, which are then sent to an emotion engine.

[0575] Server Role

[0576] The server receives personal and emotional data transmitted from the terminal and stores it in a database. The server controls 3D scanners and metal detectors to check passengers' belongings. The server also dynamically optimizes operations and provides appropriate instructions to the user based on information from the emotional engine.

[0577] Functions of the Emotion Engine

[0578] The emotion engine analyzes facial expression and voice data received from the terminal to recognize the passenger's emotions. This recognition result is fed back to the server and used to implement measures to reduce passenger stress and anxiety. The emotion engine monitors changes in emotions in real time and incorporates them into a predictive model to improve the overall system performance.

[0579] User roles

[0580] Users manage airport inspections and operations based on information provided by the server. They can take appropriate action based on the emotional state of passengers. They also input feedback into the system, contributing to operational improvements.

[0581] Specific example

[0582] For example, if passengers are dissatisfied due to long wait times, the emotion engine can detect this dissatisfaction from their facial expressions. Based on this information, the server can allocate additional resources to avoid congestion and instruct users to take further action. This improves the passenger experience and optimizes operational efficiency.

[0583] In this way, the present invention realizes flexible and efficient operation of transportation facilities that utilize emotion recognition while ensuring passenger safety.

[0584] The following describes the processing flow.

[0585] Step 1:

[0586] The terminal scans the passenger's passport or boarding pass at the gate of a transportation facility and obtains personal data. This personal data includes the passenger's flight information and reservation information. The terminal sends this data to a server.

[0587] Step 2:

[0588] The terminal's built-in camera and microphone capture passengers' facial expressions and voice data. This data is sent in real time to an emotion engine to analyze the passengers' emotional state.

[0589] Step 3:

[0590] The server records personal data transmitted from the terminal into a database and sends commands to control 3D scanners and metal detectors to inspect passengers and their belongings.

[0591] Step 4:

[0592] The emotion engine analyzes facial expressions and voice data received in real time to recognize the passenger's emotions. The recognition results identify states such as stress, anxiety, and relaxation.

[0593] Step 5:

[0594] The server integrates feedback from the emotion engine and inspection results to run a model for predicting passenger flow. Based on the prediction results, it calculates operational optimizations and dynamic resource allocation.

[0595] Step 6:

[0596] The server adjusts operations based on passengers' emotional states and provides users with necessary information to alleviate congestion. This includes deploying additional staff and adjusting gates.

[0597] Step 7:

[0598] Users will follow instructions from the server to take actions that streamline inspections and operations within the airport. They will also provide feedback via terminals, contributing to further improvements to the system.

[0599] Through these steps, the system maintains efficient operation while supporting comfortable travel for passengers.

[0600] (Example 2)

[0601] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0602] In transportation facilities, appropriately managing the flow and emotional state of users is crucial for ensuring user safety and comfort. However, conventional systems have struggled to accurately grasp the state of individual users and respond flexibly. In particular, when long waiting times occur, it is necessary to quickly detect and resolve user dissatisfaction and stress.

[0603] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0604] In this invention, the server includes a device for collecting user identification data at the entrance of a transportation facility, means for controlling an inspection device for pre-processing users based on the collected identification data and emotional state, and means for analyzing past and present data to predict the flow of people. This makes it possible to constantly understand the state of users and to allocate and respond to appropriate resources according to the situation.

[0605] "Transportation facilities" refer to places and equipment used by users of public transportation for travel and inspections.

[0606] The term "entrance" refers to the area within a transportation facility where users first pass through and where personal identification and security checks are performed.

[0607] "User" refers to an individual who uses transportation facilities, and in this context, includes persons who use means of transport.

[0608] "Identification data" refers to data that includes the user's personal information, such as passport information and information written on boarding passes.

[0609] "Emotional state" refers to the user's psychological and emotional state, including what can be inferred from data such as facial expressions and voice.

[0610] "Inspection equipment" refers to devices used to pre-process users, and includes three-dimensional scanners and metal detection devices.

[0611] "Predictive means" refers to methods and techniques for using past and present data to predict future user flows and conditions within a facility.

[0612] "Resource allocation" refers to methods for optimally allocating personnel and equipment within transportation facilities to ensure efficient user service.

[0613] This invention is a system for efficiently managing the flow and emotional state of users in transportation facilities. It mainly consists of terminals, servers, an emotion engine, and users.

[0614] The terminal collects user identification data at the entrance of transportation facilities. This identification data includes passport information and boarding pass data, and the terminal acquires this data using hardware such as QR code readers and scanners. The terminal is also equipped with a camera and microphone to capture the user's facial expressions and voice, and transmits this data to the emotion engine.

[0615] The server receives identification and emotion data transmitted from terminals and stores them in a database. Simultaneously, the server controls 3D scanners and metal detectors, managing the actual inspection process. This control is performed in real time, supporting the smooth operation of the entire facility.

[0616] The emotion engine analyzes received facial expression and voice data to recognize the user's emotional state. This process utilizes facial expression recognition algorithms and voice analysis techniques. The recognition results are fed back to the server and used to reduce user dissatisfaction and stress.

[0617] Users will appropriately manage transportation facilities and respond to users based on information provided by the server. They will also play a role in dynamically improving facility operations by incorporating feedback within the system.

[0618] As a concrete example, if a user becomes dissatisfied due to a long waiting time, the emotion engine detects this dissatisfaction from the user's facial expression. Based on this information, the server can allocate additional resources and provide instructions to the user to alleviate congestion. This improves user comfort and enables the efficient operation of the entire facility.

[0619] For example, by inputting a prompt such as, "Please tell me the algorithm for detecting dissatisfaction and stress from passengers' facial expressions and voices," into the generating AI model, you can obtain detailed information about emotion recognition technology.

[0620] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0621] Step 1:

[0622] Collection of identification data by devices

[0623] The terminal scans the user's passport or boarding pass at the entrance of the transportation facility. For input, it receives the physical document (passport or boarding pass) and converts the data into a digital format using a QR code reader or scanner. For output, user identification data is generated and sent to the server.

[0624] Step 2:

[0625] Acquisition of emotional data

[0626] The device uses its camera and microphone to acquire facial expression and voice data from the first user it identifies. As input, it acquires real-time data of the user's face and voice, and converts this into digital signals. As output, facial expression and voice data are generated and sent to the emotion engine.

[0627] Step 3:

[0628] Data processing and storage by the server.

[0629] The server receives identification and sentiment data transmitted from the terminal. Identification and sentiment data are received as input and stored in the database. The data is classified and stored, and as output, this data is prepared for use in subsequent processes.

[0630] Step 4:

[0631] Emotional analysis using an emotion engine

[0632] The emotion engine analyzes emotional data acquired from the device. It receives facial expression data and voice data as input, and applies facial expression recognition algorithms and voice analysis techniques. As output, the user's emotional state is quantified, and a recognition result is generated.

[0633] Step 5:

[0634] Server-based operation control

[0635] The server controls the testing device based on the recognition results from the emotion engine. It receives user emotional state information as input and generates signals for controlling the testing device. As output, it optimizes resource allocation to avoid congestion.

[0636] Step 6:

[0637] Providing and managing information to users

[0638] Users manage the operation of transportation facilities based on output from the server. Inputs include analysis results and operational recommendations provided by the server, while output involves efficient user support and adjustments to facility operations.

[0639] (Application Example 2)

[0640] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0641] In transportation facilities, it is difficult to recognize travelers' emotions in real time and efficiently optimize operations. Long waiting times and congestion often cause stress for travelers, and appropriate measures are needed to address this. Conventional systems have struggled to dynamically adjust based on travelers' emotions, resulting in decreased user satisfaction.

[0642] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0643] In this invention, the server includes means for analyzing past and present information to predict movement, means for recognizing the traveler's emotional state using facial expressions and voice data and making various suggestions, and means for updating a machine learning model to improve prediction accuracy based on monitoring results. This enables flexible and efficient operation of transportation facilities in accordance with the traveler's emotional state.

[0644] "Transportation facilities" refer to facilities such as train stations and airports that travelers use when moving around, and are places where efficient flow of people is required in their operation.

[0645] "Personal data" refers to information that enables the identification of travelers, including passport information and boarding pass data.

[0646] "Analytical device" refers to hardware or software used to analyze acquired data and perform pre-processing of travelers and baggage inspection.

[0647] "Predictive methods" refer to technologies that estimate traveler flow and congestion levels based on past and present information, and contribute to optimizing operations.

[0648] "Facial expression and voice data" refers to information necessary to understand the emotional state of travelers, and is data acquired using a camera and microphone.

[0649] "Emotional state" refers to the psychological and emotional state exhibited by a traveler, and is a concept that includes stress, anxiety, or pleasure.

[0650] A "machine learning model" is a technique for empirically improving a system by using algorithms that improve prediction accuracy based on data.

[0651] "Dynamic control" means flexibly adjusting system operation and resource allocation based on real-time information.

[0652] The system for realizing this invention includes a program that recognizes travelers' emotions in real time when using transportation facilities and optimizes operations accordingly. Its main components are a server, terminals, an emotion engine, and users. The operation method is described below.

[0653] The terminal is equipped with a camera and microphone to capture the traveler's facial expressions and voice data. This data is sent from the terminal to an emotion engine and used to evaluate their emotional state. The server receives the acquired emotion data and personal data (such as passport information and boarding pass information) and uses this to manage overall operations. A platform with machine learning algorithms (e.g., Google Cloud Vision API or Microsoft Azure Face API) is used for facial expression recognition.

[0654] The server analyzes historical and current data to predict traveler flow and congestion within facilities. For this purpose, a predictive model runs on the server, and machine learning models are updated in real time. The goal is to improve the traveler experience by dynamically adjusting resource allocation to avoid congestion. Furthermore, data obtained from emotional states, combined with monitoring results, contributes to improving prediction accuracy.

[0655] Users manage facility operations based on information provided by the server and offer optimal guidance to travelers. For example, by guiding travelers to routes with shorter wait times or crowded areas to avoid based on their stress levels as recognized by their devices, it is possible to make travel smoother and reduce dissatisfaction.

[0656] As a concrete example, if a traveler is dissatisfied due to long waiting times, facial expression data can be analyzed to detect this dissatisfaction. Based on this information, the server can reallocate resources and provide new guidance to the terminal to alleviate congestion. An example of a prompt using a generative AI model could be, "Tell us about your recent travel experience. What made you feel anxious?" This makes it possible to accurately capture the traveler's emotions and provide a more comfortable environment.

[0657] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0658] Step 1:

[0659] The terminals collect personal and emotional data of travelers within transportation facilities. Specifically, they collect personal data by scanning passport information and boarding passes, and facial expression and voice data using cameras and microphones. This data is necessary for identifying travelers and assessing their emotional state.

[0660] Step 2:

[0661] Personal data and emotional data acquired from the device are sent to the server. The server receives this data and stores it in a database. At this time, the emotional data is transferred to the emotional engine and used as input data to analyze the traveler's emotional state.

[0662] Step 3:

[0663] The server determines the traveler's emotional state based on the analysis results sent from the emotion engine. This result is recorded on the server as an emotional state (e.g., stress, anxiety, joy, etc.) and used in subsequent operational optimization processes.

[0664] Step 4:

[0665] The server uses accumulated historical data and current sentiment data to predict congestion levels and traveler flow within the facility. This prediction is performed using machine learning models. Based on the input data, it estimates congestion and waiting times and plans appropriate resource allocation.

[0666] Step 5:

[0667] The server dynamically optimizes operations based on prediction results. Specifically, it reallocates resources while considering emotional states and, if necessary, instructs terminals with guidance information to avoid congestion. This information is updated in real time to encourage travelers to take appropriate action.

[0668] Step 6:

[0669] Users manage facilities by utilizing emotional state and operational information provided by the server. They make adjustments as needed to ensure smooth operations on-site and provide appropriate guidance to travelers. By implementing operations that are sensitive to travelers' emotions, users aim to improve overall satisfaction.

[0670] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0671] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0672] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0673] [Fourth Embodiment]

[0674] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0675] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0676] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0677] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0678] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0679] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0680] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0681] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0682] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0683] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0684] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0685] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0686] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0687] This invention relates to a system for efficiently managing passenger flow in transportation facilities such as airports and railway stations. This system mainly consists of terminals, servers, and users.

[0688] Terminal operation

[0689] The terminals are installed at gates within transportation facilities and collect personal data by scanning passports and boarding passes as passengers pass through. This allows for the collection of passenger flight information, reservation information, and other data.

[0690] Server Role

[0691] The server receives personal data transmitted from terminals and stores it in a database. Based on this data, the server controls 3D scanners and metal detection devices to perform pre-processing on travelers. From the data obtained through analysis, the server builds a predictive model of passenger flow and calculates the optimal allocation of resources. Furthermore, based on information obtained from monitoring, the machine learning model is updated to improve prediction accuracy.

[0692] User intervention

[0693] Users, i.e., airport staff, receive instructions from the server as needed and assist with gate inspections. For example, in cases where the server cannot make an automatic decision, users can perform inspections directly. Users can also input feedback into a terminal to help improve the overall system.

[0694] Specific example

[0695] For example, during the morning rush hour when many passengers arrive, terminals quickly collect passenger personal data, and servers use this data to perform rapid inspections using 3D scanners. Predictive analytics allows servers to proactively identify which gates will be congested and instruct users on the optimal allocation of resources. This speeds up gate passage, reduces passenger waiting times, and lessens the burden on airport staff.

[0696] Through such operations, the present invention enables efficient and safe management of traveler flow, achieving both passenger comfort and efficient facility operation.

[0697] The following describes the processing flow.

[0698] Step 1:

[0699] The terminal scans the passports and boarding passes of passengers passing through the gates of transportation facilities and obtains personal data. This personal data includes flight information and reservation information. The obtained data is transmitted to the server in real time.

[0700] Step 2:

[0701] The server stores the personal data received from the terminal in a dedicated database. The server then uses this data to send appropriate control commands to the 3D scanner and metal detector, instructing them to inspect the passenger's belongings.

[0702] Step 3:

[0703] The terminal collects scan results from the 3D scanner and metal detector and sends them back to the server. This includes information about the object's shape and metal detection.

[0704] Step 4:

[0705] The server analyzes the scanned data to check for any security issues. If a problem is detected, the server alerts the user and prompts them to investigate further.

[0706] Step 5:

[0707] The server runs a machine learning model that predicts passenger flow based on analysis results and historical data, and predicts congestion levels. The resulting predictions are used in real time to optimize resource allocation.

[0708] Step 6:

[0709] The server dynamically adjusts airport staff deployment and gate opening / closing times based on predictions and sends instructions to the user. The user then makes on-site adjustments according to these instructions.

[0710] Step 7:

[0711] The server monitors the system's operational status and inspection completion times, and uses this data in a feedback loop to improve the learning model. The model is retrained periodically to improve prediction accuracy.

[0712] By following these steps, the efficient operation of the entire system is maintained, and the smooth movement of passengers is ensured.

[0713] (Example 1)

[0714] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0715] Modern transportation facilities require users to pass through them smoothly and quickly. However, conventional systems have difficulty effectively predicting and managing user flow, resulting in congestion and long waiting times within facilities. This leads to decreased convenience and stress for users. Furthermore, facility operators face challenges in allocating resources appropriately and achieving efficient operation.

[0716] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0717] In this invention, the server includes a device for collecting user identification data at transit points of transportation facilities, means for adjusting an evaluation device for performing initial processing on users based on the collected identification data, means for predicting flow by utilizing historical and real-time information, means for optimizing operations based on the prediction results and allocating resources in response to changes, and means for modifying a learning model to improve the accuracy of predictions based on monitoring results. This makes it possible to efficiently manage the flow of users and alleviate congestion within facilities.

[0718] "Transportation facilities" refer to facilities such as airports and train stations where people gather and travel.

[0719] A "waypoint" refers to a location such as a gate or checkpoint that users must pass through.

[0720] "Identification data" refers to information that includes users' personal information and identification information.

[0721] The term "device" refers to a machine or mechanism designed to perform a specific function.

[0722] An "evaluation device" refers to a machine or equipment designed to perform initial processing for users.

[0723] "Flow" refers to the patterns of movement and behavior of users within a transportation facility.

[0724] "Prediction" refers to the act of estimating future events based on past information and real-time data.

[0725] "Operation" refers to the activities of efficiently managing and coordinating facilities and systems.

[0726] "Resources" refer to personnel, equipment, space, and other resources that can be used as such.

[0727] "Allocation" refers to the act of allocating resources to the appropriate places and times as needed.

[0728] "Monitoring results" refer to information and insights obtained through observation and data collection.

[0729] A "learning model" refers to an algorithm or mathematical model used to identify patterns based on data and perform predictions or classifications.

[0730] "Correction" refers to activities that involve making changes or adjustments to improve the current situation.

[0731] This invention is a system for efficiently managing the flow of users in transportation facilities. This system mainly consists of three elements: terminals, servers, and users.

[0732] Terminal operation

[0733] The terminals are installed at transit points in transportation facilities and acquire identification data by scanning passports or boarding passes as users pass through. The terminal hardware incorporates high-speed scanners and data transmission capabilities, transferring the acquired data to a server in real time.

[0734] Server Role

[0735] The server receives identification data transmitted from the terminal and securely stores it in a database. The server uses this data to control the evaluation device and perform initial user processing. The software used includes algorithms and machine learning models for data analysis, enabling prediction of user flow. Specifically, it analyzes historical and real-time data to predict user flow and identify potential congestion points. The server further refines the learning model using monitoring results to improve prediction accuracy. In this process, the server optimizes resources and improves operational efficiency.

[0736] User intervention

[0737] Users, i.e., facility staff, take appropriate actions based on information provided by the server. For example, at points where congestion is anticipated, additional staff are deployed based on instructions from the server. Furthermore, if the server cannot make a decision, users conduct inspections directly and contribute to system improvements by inputting necessary feedback into their terminals.

[0738] As a concrete example, consider an environment where many users arrive during the morning rush hour. Terminals quickly collect user identification data, and servers perform rapid inspections using 3D scanners. Through prediction, the servers identify where congestion will occur and notify users. Based on this information, users can allocate resources appropriately to avoid congestion.

[0739] An example of a prompt message would be, "Please advise on how to address the rapid changes in passenger density at the airport during specific times on weekends." This would allow for guidance on optimizing specific operations.

[0740] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0741] Step 1:

[0742] The terminal acquires user identification data at transit points within transportation facilities. This process involves scanning the user's passport or boarding pass. The input is the user's passport or boarding pass, and the terminal outputs digitized identification data through scanning. This data includes the user's ID and flight information.

[0743] Step 2:

[0744] The terminal transfers the acquired identification data to the server. The input here is the identification data generated in step 1. The terminal encrypts this data and sends it to the server over the network, and the server stores it in its database. The output is the securely stored identification data.

[0745] Step 3:

[0746] The server controls the evaluation equipment using the received identification data and performs initial processing for the user. The input is identification data transmitted from the terminal. Based on this data, the server activates 3D scanners and metal detectors and obtains output by analyzing the measurement results. The output is a determination of whether the user is allowed to pass and an alert if necessary.

[0747] Step 4:

[0748] The server analyzes historical and current data to predict user flow. The input for this step is historical and real-time identification data stored in a database. The server uses a machine learning model to analyze this data and predict the probability and location of congestion. The output is predicted information about future user flow.

[0749] Step 5:

[0750] The server optimizes operations and dynamically allocates resources based on predictive information. The input is the predictive information obtained in step 4. The server uses this to calculate how many staff members are needed at each transit point and notifies the user of this information to ensure optimal resource allocation. The output is the allocation instructions and related log information.

[0751] Step 6:

[0752] The user takes the necessary actions based on deployment instructions provided by the server. Here, the user receives instructions from the server as input. The user observes the situation in real time, changes staff deployments as needed, and provides feedback to the server. The output is a report of the improved deployment based on the feedback and its status.

[0753] (Application Example 1)

[0754] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0755] In large-scale facilities that attract large crowds, such as transportation facilities and event venues, efficiently managing the flow of visitors and travelers, avoiding congestion, and providing optimal traffic flow are crucial challenges. Conventional systems make it difficult to grasp congestion levels in real time, and there is a need for effective management that reduces visitor stress and improves the efficiency of facility operations.

[0756] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0757] In this invention, the server includes means for using a terminal to acquire travelers' personal data at the gates of transportation facilities, means for analyzing past and current data based on the acquired personal data to predict the flow, and means for understanding congestion levels and providing visitors with the optimal route and entrance / exit. This enables visitors and travelers to use the facilities efficiently, and allows operators to achieve optimized operations.

[0758] A "transportation facility" is a facility designed as a large-scale gathering point where users utilize means of transportation.

[0759] A "gate" is a passage point installed to manage entry and exit to a facility.

[0760] A "traveler" refers to an individual who travels using means of transportation.

[0761] "Personal data" refers to information used to identify a specific person, including passport information and boarding pass information.

[0762] A "terminal" is a device used to acquire personal data.

[0763] "Inspection equipment" refers to devices used to pre-process travelers, and may include 3D scanners and metal detectors.

[0764] "Flow" refers to the movement and flow of people within a facility.

[0765] "Prediction" refers to estimating future events using past and current data.

[0766] "Operations" refers to the operational management tasks carried out within a facility.

[0767] "Resources" refer to the human resources and physical materials necessary for the operation of a facility.

[0768] "Monitoring" is the act of observing the conditions within a facility and collecting data.

[0769] A "learning model" is a computational method used to make predictions and decisions based on accumulated data.

[0770] "Crowding status" refers to the degree of crowding and density of people within a facility.

[0771] A "route" refers to the path that visitors should take to move around within a facility.

[0772] "Entrances and exits" refer to the entrances and exits used for entering and leaving a facility.

[0773] To realize this invention, it is necessary to efficiently manage the flow of visitors and travelers using a system installed in transportation facilities and event venues. The server receives data from terminals installed at the gates of transportation facilities, analyzes personal data, and predicts flow. The terminals scan travelers' passports and boarding passes and collect personal data. As a result, the server can grasp congestion levels in real time and use a learning model to provide visitors with the optimal route and entrance / exit.

[0774] The hardware utilizes cameras and 3D scanners in smart glasses, while the software employs OpenCV and detection libraries. The device sends personal data to a cloud server, which analyzes it and generates congestion information and optimization data. The generated data is then displayed as instructions on the smart glasses or mobile devices worn by visitors.

[0775] As a concrete example, at a large music festival, visitors can enter the venue and, when moving to a designated stage while avoiding congestion, can be guided by a server to select the shortest route. An example of a prompt message would be: "Explain the effectiveness of a visitor management application within an event venue in a smart city. This application provides each visitor with the optimal route based on real-time congestion conditions and is implemented using smart glasses."

[0776] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0777] Step 1:

[0778] The terminal acquires visitors' personal data at the facility's gate. Using smart glasses or a camera, it scans the visitor's passport or ticket information and captures this information as digital data. The input is scanned personal information, and the output is personal information as data.

[0779] Step 2:

[0780] The device sends the acquired personal data to the server. This data includes the visitor's ID and flight information. The server receives this data, stores it in a database, and prepares it for subsequent processing. The input is personal data, and the output is information registered in the database on the server.

[0781] Step 3:

[0782] The server analyzes past and current visitor information based on the data. This allows it to predict flow patterns within the facility and generate instructions necessary for hardware control. A generative AI model is used for this analysis to efficiently predict visitor flow. The inputs are stored visitor data and the trained model, while the outputs are prediction results and control instructions.

[0783] Step 4:

[0784] The server monitors congestion in real time based on the analysis and generates instructions to provide visitors with the optimal route and exit. These instructions are sent to mobile devices and smart glasses, and visitors receive guidance visually or audibly. The input is the analysis result, and the output is the guidance instructions.

[0785] Step 5:

[0786] Users monitor congestion levels and guidance instructions provided by the server and provide support to visitors as needed. For example, if the guidance is not working properly, they may provide direct instruction. The input is the server's instructions, and the output is the actual support activity.

[0787] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0788] The present invention is a system for efficiently and appropriately managing passenger flow and inspection processes in transportation facilities, and in particular includes an emotion engine that recognizes the emotional state of passengers and users and provides feedback. The system mainly consists of terminals, servers, the emotion engine, and users.

[0789] Terminal operation

[0790] The terminal obtains personal data by scanning passengers' passports and boarding passes at the gate. This personal data is sent to a server and used for pre-processing by inspection equipment. In addition, the terminal has a built-in camera and microphone to capture passengers' facial expressions and voice data, which are then sent to an emotion engine.

[0791] Server Role

[0792] The server receives personal and emotional data transmitted from the terminal and stores it in a database. The server controls 3D scanners and metal detectors to check passengers' belongings. The server also dynamically optimizes operations and provides appropriate instructions to the user based on information from the emotional engine.

[0793] Functions of the Emotion Engine

[0794] The emotion engine analyzes facial expression and voice data received from the terminal to recognize the passenger's emotions. This recognition result is fed back to the server and used to implement measures to reduce passenger stress and anxiety. The emotion engine monitors changes in emotions in real time and incorporates them into a predictive model to improve the overall system performance.

[0795] User roles

[0796] Users manage airport inspections and operations based on information provided by the server. They can take appropriate action based on the emotional state of passengers. They also input feedback into the system, contributing to operational improvements.

[0797] Specific example

[0798] For example, if passengers are dissatisfied due to long wait times, the emotion engine can detect this dissatisfaction from their facial expressions. Based on this information, the server can allocate additional resources to avoid congestion and instruct users to take further action. This improves the passenger experience and optimizes operational efficiency.

[0799] In this way, the present invention realizes flexible and efficient operation of transportation facilities that utilize emotion recognition while ensuring passenger safety.

[0800] The following describes the processing flow.

[0801] Step 1:

[0802] The terminal scans the passenger's passport or boarding pass at the gate of a transportation facility and obtains personal data. This personal data includes the passenger's flight information and reservation information. The terminal sends this data to a server.

[0803] Step 2:

[0804] The terminal's built-in camera and microphone capture passengers' facial expressions and voice data. This data is sent in real time to an emotion engine to analyze the passengers' emotional state.

[0805] Step 3:

[0806] The server records personal data transmitted from the terminal into a database and sends commands to control 3D scanners and metal detectors to inspect passengers and their belongings.

[0807] Step 4:

[0808] The emotion engine analyzes facial expressions and voice data received in real time to recognize the passenger's emotions. The recognition results identify states such as stress, anxiety, and relaxation.

[0809] Step 5:

[0810] The server integrates feedback from the emotion engine and inspection results to run a model for predicting passenger flow. Based on the prediction results, it calculates operational optimizations and dynamic resource allocation.

[0811] Step 6:

[0812] The server adjusts operations based on passengers' emotional states and provides users with necessary information to alleviate congestion. This includes deploying additional staff and adjusting gates.

[0813] Step 7:

[0814] Users will follow instructions from the server to take actions that streamline inspections and operations within the airport. They will also provide feedback via terminals, contributing to further improvements to the system.

[0815] Through these steps, the system maintains efficient operation while supporting comfortable travel for passengers.

[0816] (Example 2)

[0817] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0818] In transportation facilities, appropriately managing the flow and emotional state of users is crucial for ensuring user safety and comfort. However, conventional systems have struggled to accurately grasp the state of individual users and respond flexibly. In particular, when long waiting times occur, it is necessary to quickly detect and resolve user dissatisfaction and stress.

[0819] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0820] In this invention, the server includes a device for collecting user identification data at the entrance of a transportation facility, means for controlling an inspection device for pre-processing users based on the collected identification data and emotional state, and means for analyzing past and present data to predict the flow of people. This makes it possible to constantly understand the state of users and to allocate and respond to appropriate resources according to the situation.

[0821] "Transportation facilities" refer to places and equipment used by users of public transportation for travel and inspections.

[0822] The term "entrance" refers to the area within a transportation facility where users first pass through and where personal identification and security checks are performed.

[0823] "User" refers to an individual who uses transportation facilities, and in this context, includes persons who use means of transport.

[0824] "Identification data" refers to data that includes the user's personal information, such as passport information and information written on boarding passes.

[0825] "Emotional state" refers to the user's psychological and emotional state, including what can be inferred from data such as facial expressions and voice.

[0826] "Inspection equipment" refers to devices used to pre-process users, and includes three-dimensional scanners and metal detection devices.

[0827] "Predictive means" refers to methods and techniques for using past and present data to predict future user flows and conditions within a facility.

[0828] "Resource allocation" refers to methods for optimally allocating personnel and equipment within transportation facilities to ensure efficient user service.

[0829] This invention is a system for efficiently managing the flow and emotional state of users in transportation facilities. It mainly consists of terminals, servers, an emotion engine, and users.

[0830] The terminal collects user identification data at the entrance of transportation facilities. This identification data includes passport information and boarding pass data, and the terminal acquires this data using hardware such as QR code readers and scanners. The terminal is also equipped with a camera and microphone to capture the user's facial expressions and voice, and transmits this data to the emotion engine.

[0831] The server receives identification and emotion data transmitted from terminals and stores them in a database. Simultaneously, the server controls 3D scanners and metal detectors, managing the actual inspection process. This control is performed in real time, supporting the smooth operation of the entire facility.

[0832] The emotion engine analyzes received facial expression and voice data to recognize the user's emotional state. This process utilizes facial expression recognition algorithms and voice analysis techniques. The recognition results are fed back to the server and used to reduce user dissatisfaction and stress.

[0833] Users will appropriately manage transportation facilities and respond to users based on information provided by the server. They will also play a role in dynamically improving facility operations by incorporating feedback within the system.

[0834] As a concrete example, if a user becomes dissatisfied due to a long waiting time, the emotion engine detects this dissatisfaction from the user's facial expression. Based on this information, the server can allocate additional resources and provide instructions to the user to alleviate congestion. This improves user comfort and enables the efficient operation of the entire facility.

[0835] For example, by inputting a prompt such as, "Please tell me the algorithm for detecting dissatisfaction and stress from passengers' facial expressions and voices," into the generating AI model, you can obtain detailed information about emotion recognition technology.

[0836] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0837] Step 1:

[0838] Collection of identification data by devices

[0839] The terminal scans the user's passport or boarding pass at the entrance of the transportation facility. For input, it receives the physical document (passport or boarding pass) and converts the data into a digital format using a QR code reader or scanner. For output, user identification data is generated and sent to the server.

[0840] Step 2:

[0841] Acquisition of emotional data

[0842] The device uses its camera and microphone to acquire facial expression and voice data from the first user it identifies. As input, it acquires real-time data of the user's face and voice, and converts this into digital signals. As output, facial expression and voice data are generated and sent to the emotion engine.

[0843] Step 3:

[0844] Data processing and storage by the server.

[0845] The server receives identification and sentiment data transmitted from the terminal. Identification and sentiment data are received as input and stored in the database. The data is classified and stored, and as output, this data is prepared for use in subsequent processes.

[0846] Step 4:

[0847] Emotional analysis using an emotion engine

[0848] The emotion engine analyzes emotional data acquired from the device. It receives facial expression data and voice data as input, and applies facial expression recognition algorithms and voice analysis techniques. As output, the user's emotional state is quantified, and a recognition result is generated.

[0849] Step 5:

[0850] Server-based operation control

[0851] The server controls the testing device based on the recognition results from the emotion engine. It receives user emotional state information as input and generates signals for controlling the testing device. As output, it optimizes resource allocation to avoid congestion.

[0852] Step 6:

[0853] Providing and managing information to users

[0854] Users manage the operation of transportation facilities based on output from the server. Inputs include analysis results and operational recommendations provided by the server, while output involves efficient user support and adjustments to facility operations.

[0855] (Application Example 2)

[0856] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0857] In transportation facilities, it is difficult to recognize travelers' emotions in real time and efficiently optimize operations. Long waiting times and congestion often cause stress for travelers, and appropriate measures are needed to address this. Conventional systems have struggled to dynamically adjust based on travelers' emotions, resulting in decreased user satisfaction.

[0858] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0859] In this invention, the server includes means for analyzing past and present information to predict movement, means for recognizing the traveler's emotional state using facial expressions and voice data and making various suggestions, and means for updating a machine learning model to improve prediction accuracy based on monitoring results. This enables flexible and efficient operation of transportation facilities in accordance with the traveler's emotional state.

[0860] "Transportation facilities" refer to facilities such as train stations and airports that travelers use when moving around, and are places where efficient flow of people is required in their operation.

[0861] "Personal data" refers to information that enables the identification of travelers, including passport information and boarding pass data.

[0862] "Analytical device" refers to hardware or software used to analyze acquired data and perform pre-processing of travelers and baggage inspection.

[0863] "Predictive methods" refer to technologies that estimate traveler flow and congestion levels based on past and present information, and contribute to optimizing operations.

[0864] "Facial expression and voice data" refers to information necessary to understand the emotional state of travelers, and is data acquired using a camera and microphone.

[0865] "Emotional state" refers to the psychological and emotional state exhibited by a traveler, and is a concept that includes stress, anxiety, or pleasure.

[0866] A "machine learning model" is a technique for empirically improving a system by using algorithms that improve prediction accuracy based on data.

[0867] "Dynamic control" means flexibly adjusting system operation and resource allocation based on real-time information.

[0868] The system for realizing this invention includes a program that recognizes travelers' emotions in real time when using transportation facilities and optimizes operations accordingly. Its main components are a server, terminals, an emotion engine, and users. The operation method is described below.

[0869] The terminal is equipped with a camera and microphone to capture the traveler's facial expressions and voice data. This data is sent from the terminal to an emotion engine and used to evaluate their emotional state. The server receives the acquired emotion data and personal data (such as passport information and boarding pass information) and uses this to manage overall operations. A platform with machine learning algorithms (e.g., Google Cloud Vision API or Microsoft Azure Face API) is used for facial expression recognition.

[0870] The server analyzes historical and current data to predict traveler flow and congestion within facilities. For this purpose, a predictive model runs on the server, and machine learning models are updated in real time. The goal is to improve the traveler experience by dynamically adjusting resource allocation to avoid congestion. Furthermore, data obtained from emotional states, combined with monitoring results, contributes to improving prediction accuracy.

[0871] Users manage facility operations based on information provided by the server and offer optimal guidance to travelers. For example, by guiding travelers to routes with shorter wait times or crowded areas to avoid based on their stress levels as recognized by their devices, it is possible to make travel smoother and reduce dissatisfaction.

[0872] As a concrete example, if a traveler is dissatisfied due to long waiting times, facial expression data can be analyzed to detect this dissatisfaction. Based on this information, the server can reallocate resources and provide new guidance to the terminal to alleviate congestion. An example of a prompt using a generative AI model could be, "Tell us about your recent travel experience. What made you feel anxious?" This makes it possible to accurately capture the traveler's emotions and provide a more comfortable environment.

[0873] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0874] Step 1:

[0875] The terminals collect personal and emotional data of travelers within transportation facilities. Specifically, they collect personal data by scanning passport information and boarding passes, and facial expression and voice data using cameras and microphones. This data is necessary for identifying travelers and assessing their emotional state.

[0876] Step 2:

[0877] Personal data and emotional data acquired from the device are sent to the server. The server receives this data and stores it in a database. At this time, the emotional data is transferred to the emotional engine and used as input data to analyze the traveler's emotional state.

[0878] Step 3:

[0879] The server determines the traveler's emotional state based on the analysis results sent from the emotion engine. This result is recorded on the server as an emotional state (e.g., stress, anxiety, joy, etc.) and used in subsequent operational optimization processes.

[0880] Step 4:

[0881] The server uses accumulated historical data and current sentiment data to predict congestion levels and traveler flow within the facility. This prediction is performed using machine learning models. Based on the input data, it estimates congestion and waiting times and plans appropriate resource allocation.

[0882] Step 5:

[0883] The server dynamically optimizes operations based on prediction results. Specifically, it reallocates resources while considering emotional states and, if necessary, instructs terminals with guidance information to avoid congestion. This information is updated in real time to encourage travelers to take appropriate action.

[0884] Step 6:

[0885] Users manage facilities by utilizing emotional state and operational information provided by the server. They make adjustments as needed to ensure smooth operations on-site and provide appropriate guidance to travelers. By implementing operations that are sensitive to travelers' emotions, users aim to improve overall satisfaction.

[0886] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0887] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0888] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0889] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0890] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0891] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0892] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0893] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0894] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0895] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0896] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0897] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0898] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0899] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0900] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0901] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0902] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0903] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0904] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0905] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0906] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0907] The following is further disclosed regarding the embodiments described above.

[0908] (Claim 1)

[0909] A terminal that acquires travelers' personal data at the gate of a transportation facility,

[0910] Means for controlling an inspection device for pre-processing travelers based on acquired personal data,

[0911] A means of analyzing past and present data to predict the flow,

[0912] A means of optimizing operations and dynamically allocating resources based on predictions,

[0913] A means of updating the learning model to improve prediction accuracy based on monitoring results,

[0914] A system that includes this.

[0915] (Claim 2)

[0916] The system according to claim 1, further comprising means for automatically adjusting the schedule after obtaining the traveler's personal data.

[0917] (Claim 3)

[0918] The system according to claim 1, characterized in that the inspection device includes a 3D scanner and a metal detection device, and controls them based on data.

[0919] "Example 1"

[0920] (Claim 1)

[0921] A device for collecting user identification data at transit points of transportation facilities,

[0922] Means for adjusting an evaluation device for performing initial processing on a user based on collected identification data,

[0923] A means of predicting trends by utilizing historical and real-time information,

[0924] A means of optimizing operations based on prediction results and allocating resources in response to changes,

[0925] A means of modifying the learning model to improve the accuracy of predictions based on monitoring results,

[0926] A system that includes this.

[0927] (Claim 2)

[0928] The system according to claim 1, further comprising means for automatically adjusting the timetable after collecting user identification data.

[0929] (Claim 3)

[0930] The system according to claim 1, wherein the evaluation device includes a stereoscopic scanning device and a metal detection device, and these are adjusted based on the information.

[0931] "Application Example 1"

[0932] (Claim 1)

[0933] A terminal that acquires travelers' personal data at the gate of a transportation facility,

[0934] Means for controlling an inspection device for pre-processing travelers based on acquired personal data,

[0935] A means of analyzing past and present data to predict the flow,

[0936] A means of optimizing operations and dynamically allocating resources based on predictions,

[0937] A means of updating the learning model to improve prediction accuracy based on monitoring results,

[0938] A means to understand congestion levels and provide visitors with the most suitable routes and entrances,

[0939] A system that includes this.

[0940] (Claim 2)

[0941] The system according to claim 1, further comprising means for automatically adjusting the schedule after obtaining the traveler's personal data.

[0942] (Claim 3)

[0943] The system according to claim 1, characterized in that the inspection device includes a 3D scanner and a metal detection device, and controls them based on data.

[0944] "Example 2 of combining an emotion engine"

[0945] (Claim 1)

[0946] A device for collecting user identification data at the entrance of a transportation facility,

[0947] Means for controlling a testing device for pre-processing users based on collected identification data and emotional states,

[0948] A method for predicting the flow of people by analyzing past and present data,

[0949] A means of optimizing work and dynamically allocating resources based on predictions,

[0950] A means of updating the learning algorithm to improve prediction accuracy based on monitoring results,

[0951] A means of recognizing the user's state through emotion analysis and providing information to improve responses,

[0952] A system that includes this.

[0953] (Claim 2)

[0954] The system according to claim 1, further comprising means for automatically adjusting the schedule after obtaining identification data.

[0955] (Claim 3)

[0956] The system according to claim 1, characterized in that the inspection device includes a three-dimensional scanner and a metal detection device, and controls them based on data.

[0957] "Application example 2 when combining with an emotional engine"

[0958] (Claim 1)

[0959] A device that acquires travelers' personal data at the gates of transportation facilities,

[0960] Means for managing analytical equipment for pre-processing travelers based on acquired personal data,

[0961] A means of predicting movement by analyzing past and present information,

[0962] A means of optimizing operations and dynamically allocating resources based on predictions,

[0963] A means of recognizing the emotional state of travelers using facial expressions and voice data, and making various suggestions,

[0964] A means of updating machine learning models to improve prediction accuracy based on monitoring results,

[0965] A system that includes this.

[0966] (Claim 2)

[0967] The system according to claim 1, further comprising means for automatically adjusting the schedule after obtaining the traveler's personal data and providing optimal guidance based on the user's emotions.

[0968] (Claim 3)

[0969] The system according to claim 1, characterized in that the analysis device includes a three-dimensional scanner and a metal detection device, manages them based on data, and dynamically controls them based on the user's emotional state. [Explanation of Symbols]

[0970] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A terminal that acquires travelers' personal data at the gate of a transportation facility, Means for controlling an inspection device for pre-processing travelers based on acquired personal data, A means of analyzing past and present data to predict the flow, A means of optimizing operations and dynamically allocating resources based on predictions, A means of updating the learning model to improve prediction accuracy based on monitoring results, A means to understand congestion levels and provide visitors with the most suitable routes and entrances, A system that includes this.

2. The system according to claim 1, further comprising means for automatically adjusting the schedule after obtaining the traveler's personal data.

3. The system according to claim 1, characterized in that the inspection device includes a 3D scanner and a metal detection device, and controls them based on data.

Images