system

Abstract

We provide the system. [Solution] Means of collecting urban traffic data, A means for analyzing the aforementioned traffic data and predicting traffic congestion, A means for generating the optimal route based on the user's request, Means for providing the generated route to the user, A means of updating routes in response to changes in traffic conditions, A means of analyzing natural language questions from users and generating responses, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Traffic congestion in urban areas is a daily problem, which causes a great deal of time waste and stress. In addition, it is difficult to quickly respond to sudden changes in traffic conditions due to traffic accidents or construction. Furthermore, conventional traffic information services have not been able to sufficiently perform real-time traffic jam prediction and individual optimization for each user. As a result, users have been forced to have wasted waiting time and choose inefficient routes.

Means for Solving the Problems

[0005] To address the aforementioned challenges, the present invention provides a system that collects urban traffic data, analyzes this data using an AI algorithm, and predicts traffic congestion. Based on user requests, this system generates and provides the optimal route to the user. Furthermore, when traffic conditions change, it immediately updates the route, providing the user with the latest travel information. In addition, by utilizing natural language processing to respond to user inquiries, it is possible to appropriately address user needs.

[0006] "Traffic data" refers to information about traffic conditions within a city, including traffic volume, speed, accident occurrences, and construction work status.

[0007] "Analysis" is the act of analyzing collected data to derive meaningful information or predictions.

[0008] "Traffic congestion" refers to the degree of vehicle concentration and the reduction in travel speed in a specific road area.

[0009] A "user request" is a request sent by a user through their device for the provision of specific information or services.

[0010] A "route" refers to the points of passage or path taken when moving from one point to another.

[0011] "Updating" means modifying existing data and settings based on new information to bring them up to date.

[0012] "Natural language processing" refers to the techniques and methods by which computers understand, interpret, and generate human language.

[0013] "Response" refers to the information provided in response to questions or requests from users. [Brief explanation of the drawing]

[0014] [Figure 1]It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applicable to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system for reducing urban traffic congestion and providing users with efficient travel. The system primarily consists of three elements: a server, terminals, and users.

[0036] The server collects real-time traffic data through various sensors and communication infrastructure placed throughout the city. Data classifications include traffic volume, speed, accident information, and construction information. Based on this data, the server uses AI algorithms to analyze it and predict traffic congestion. To enhance user convenience, trend analysis based on historical data is also performed to improve the accuracy of future traffic predictions.

[0037] The user sends travel requests via their device, including their departure and destination points and departure time. The device then sends this information to the server, which initiates the calculation of the optimal route based on the user's specified conditions. The server generates routes combining various modes of transport, which can be customized to the user's preferences.

[0038] When traffic conditions change, the server detects the changes in real time and, if necessary, calculates a new route and sends it to the terminal. This allows users to travel based on the latest traffic information. In addition, the terminal uses natural language processing to generate responses to user questions in order to enable interaction with the user. For example, if a user asks, "What are the current traffic conditions like?", the terminal will quickly answer based on the latest information analyzed by the server.

[0039] As a concrete example, consider a user traveling to the city center by car during the morning rush hour. Since the usual route is likely to be congested, the server, based on traffic data, presents an alternative route combining train travel and walking. Even if a train delay occurs along the way, the server can quickly analyze the impact and provide the user with a new alternative. With these features, the user can reach their destination with minimal time and stress.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The server collects real-time traffic data through sensors and communication infrastructure installed within the city. This data, encompassing information on traffic volume, speed, accidents, and construction, forms the core of this invention.

[0043] Step 2:

[0044] The server analyzes the collected data using AI algorithms to understand the current traffic congestion situation and predict future congestion based on trends derived from past data. The accuracy of this data analysis significantly impacts the effectiveness of the system.

[0045] Step 3:

[0046] The user uses their device to input information about their travel, such as their current location, destination, and departure time, and sends a request to the device. This request is the starting point for achieving maximum performance.

[0047] Step 4:

[0048] The terminal immediately sends the user's request to the server. The server calculates the optimal route based on the received request. Here, route selection is performed based on the user's needs.

[0049] Step 5:

[0050] When calculating optimized routes, the server considers combinations of multiple modes of transport and suggests the most efficient way for the user to travel. If necessary, it also uses real-time traffic data for optimization.

[0051] Step 6:

[0052] The server sends the calculated optimal route to the terminal. The terminal receives this and provides the user with route information visually or audibly. The user then makes a travel decision based on this information.

[0053] Step 7:

[0054] If traffic conditions change, the server continuously analyzes the new data and updates the route as needed. This updated information is immediately sent to the terminal and reflected in the user's travel plan.

[0055] Step 8:

[0056] When a user sends a question in natural language through their device, the device forwards this request to the server. The server uses natural language processing to analyze the question, generates appropriate information, and immediately sends it back to the device. This two-way interaction further enhances user convenience.

[0057] (Example 1)

[0058] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0059] Traffic congestion in urban areas causes various problems, including increased travel time and higher energy consumption. Conventional traffic information systems have limitations in real-time analysis and flexible route suggestion, making it difficult to provide users with the optimal travel route. Therefore, there is a need for more accurate traffic information analysis and flexible route suggestions tailored to users.

[0060] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0061] In this invention, the server includes a device for aggregating traffic information, a device for analyzing the traffic information and predicting traffic congestion, and a device for constructing an optimal route based on user requests. This makes it possible to flexibly provide users with routes suitable for them based on real-time traffic information.

[0062] "Traffic information" refers to data related to roads and transportation systems, including vehicle flow, speed, accidents, and construction information.

[0063] "Congestion" refers to a state where traffic flow is slower or more stagnant than usual, and it is an indicator used for prediction.

[0064] A "user" refers to an individual who uses this system to obtain route information and select a mode of transportation.

[0065] A "route" is the recommended path for travel from a starting point to a destination, and can involve combining multiple modes of transportation.

[0066] A "device" refers to a collection of hardware and software designed to perform a specific function.

[0067] A "server" refers to a computer system that processes and analyzes data and plays a central role in the system.

[0068] "Real-time" means that data collection and processing are performed immediately, enabling the provision of information that responds promptly to the current situation.

[0069] "Natural language" refers to the forms of language that humans use on a daily basis, and it is used in communication with computers.

[0070] In this embodiment of the invention, a system is constructed to alleviate urban traffic congestion and provide users with an efficient means of transportation. The system consists of three main elements: a server, a terminal, and a user.

[0071] The server uses a high-performance computer to collect real-time traffic information from traffic sensors and related devices installed throughout the city. This data includes traffic flow, speed, accident locations, and road construction information. The server analyzes this data using AI algorithms to evaluate current traffic conditions and predict future congestion. Specifically, it uses machine learning libraries such as TENSORFLOW® and Python programs to extract significant patterns from large amounts of data.

[0072] Users input information such as their departure point, destination, and departure time using a device such as a smartphone or tablet. This information is sent from the device to the server, which calculates the most efficient travel route. Map information services such as Google® Maps API are used to generate the travel route, and the optimal combination of possible modes of transport, including vehicles, public transport, and walking, is suggested. The server updates the route in real time in response to requests, providing users with the latest information.

[0073] If traffic conditions suddenly change, the server immediately detects the situation and recalculates the route as needed. The terminal has the function to receive the new route information sent from the server and notify the user. This allows the user to respond quickly to changing traffic conditions.

[0074] By using natural language processing technology, the device can easily respond to user questions. For example, if a user asks, "What's the current traffic situation like?", an answer based on the latest information analyzed by the server will be provided. Generative AI models such as ChatGPT® are used for this purpose.

[0075] As a concrete example, consider a scenario where a user wants to commute by car on a weekday morning to avoid crowds. This system can suggest a route combining train and walking, rather than the usual route. If the train is delayed along the way, the server can quickly generate an alternative route. This allows the user to reach their destination with less stress.

[0076] An example of a prompt message is: "The user is driving towards the city center. Based on current traffic information, suggest the best route to avoid congestion."

[0077] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0078] Step 1:

[0079] The server collects traffic information from traffic sensors placed throughout the city. Specifically, it receives data transmitted from the sensors via the communication infrastructure and records it in a database in the form of traffic volume, speed, accident information, etc. The input to this process is raw data from the sensors, and the output is structured traffic information data.

[0080] Step 2:

[0081] The server analyzes the collected traffic information using an AI algorithm. During the data analysis process, it evaluates the current traffic situation using machine learning, referencing past traffic patterns. The input is the structured data obtained in step 1, and the output is the traffic congestion level prediction provided by the congestion prediction model.

[0082] Step 3:

[0083] The user enters the departure point, destination, and departure time via a terminal. The terminal sends this information to the server. The input is the travel conditions specified by the user, and the output is communication data sent to the server that receives this information and processes it in the next step.

[0084] Step 4:

[0085] The server calculates the optimal route using an AI algorithm, taking into account the user's input conditions. Based on the latest traffic forecast data, the server generates routes combining multiple modes of transport. The input consists of the user's travel conditions and analyzed traffic data, while the output is optimized route information.

[0086] Step 5:

[0087] The terminal receives optimal route information sent from the server and displays it to the user. Here, the terminal plays the role of visualizing the optimal route in an easy-to-understand way and presenting it to the user. The input is route information from the server, and the output is visualized navigation information provided to the user.

[0088] Step 6:

[0089] The server monitors changes in traffic conditions in real time and recalculates routes as needed. For example, when new information such as accidents or construction comes in, the server immediately processes that information and updates the routes for affected users. The input is the changed traffic conditions, and the output is the updated route information.

[0090] Step 7:

[0091] The device notifies the user of updated route information. This includes on-screen pop-ups and audio notifications. The input is the updated route information, and the output is an alert to the user and a presentation of the new route information.

[0092] Step 8:

[0093] The device uses generative AI models and natural language processing to respond to user questions. Based on the latest information received from the server, the device generates answers to questions such as "What is the current traffic situation?". The input is a natural language question from the user, and the output is an accurate response to that question.

[0094] (Application Example 1)

[0095] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0096] In modern cities, traffic congestion is a daily problem, and efficient travel is particularly difficult during rush hour. Furthermore, users are often stressed by the need to quickly adapt to changes in traffic conditions while traveling. This invention aims to solve these problems and provide users with a comfortable and efficient means of transportation.

[0097] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0098] In this invention, the server includes means for collecting movement information within a city, means for analyzing the movement information and predicting congestion levels, and means for displaying the prediction results to the user using a smartphone and suggesting a travel route. This allows the user to select the optimal route based on the latest movement information, enabling comfortable and efficient travel.

[0099] "Urban mobility information" refers to data on the dynamics and conditions of vehicles and pedestrians moving within an urban environment.

[0100] "Methods for predicting congestion" refer to methods that analyze data obtained from travel information to estimate current or future traffic density, speed, and other conditions.

[0101] "Means for generating the optimal route based on user requests" refers to a process that calculates an efficient travel route based on conditions such as the origin and destination specified by the user.

[0102] "Means of providing the generated route to the user" refers to a method of displaying or notifying the user of the calculated optimal travel route on their terminal.

[0103] "Means of updating routes in response to changes in travel conditions" refers to a process that recalculates and modifies routes based on real-time changing travel information so that users can reach their destinations more efficiently.

[0104] "A means of analyzing natural language questions from users and generating responses" refers to a method of understanding natural language questions entered by users into a terminal and creating appropriate answers to those questions.

[0105] "A method for displaying prediction results to users using smartphones and suggesting travel routes" refers to a method of displaying analyzed travel information and prediction results on a smartphone screen and guiding users to the most suitable travel route.

[0106] The system implementing this invention provides a platform for the integrated management of urban mobility information. A server collects mobility data from various sensors placed throughout the city, absorbing information such as traffic flow, speed, and traffic density. This data is integrated into the server in real time and stored in a database.

[0107] The software on the server is built using programming languages ​​such as Python and Java (registered trademark). Data from sensors is first cleaned, and then congestion levels are predicted using AI models such as TensorFlow. The prediction results are processed to suggest the optimal route based on the user's travel request.

[0108] Devices, especially smartphones, utilize the Google Maps API to provide users with an intuitive interface. Users enter their starting point and destination, and receive an optimized route from the server. Furthermore, the server updates the route to the smartphone in real time in response to changes in traffic conditions.

[0109] Furthermore, the device utilizes a generative AI model to enable natural language interaction. Specifically, when a user enters a prompt such as "Please tell me the shortest route to my destination," the server quickly generates an answer based on the analysis results.

[0110] When users travel to the city center during peak hours, the server predicts traffic conditions and suggests appropriate modes of transport or alternative routes, thereby reducing travel time and stress. This allows users to experience comfortable and efficient travel.

[0111] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0112] Step 1:

[0113] The server collects movement data in real time from sensors placed throughout the city. It receives raw movement data transmitted from the sensors (vehicle position, speed, traffic density, etc.) as input. It then performs data cleaning to correct for noise and missing data, outputting formatted movement information.

[0114] Step 2:

[0115] The server inputs formatted travel information into an AI model using TensorFlow to predict congestion levels. It utilizes historical data and current travel information as input. The prediction algorithm calculates future traffic density and the distribution of congestion points, and outputs the prediction results.

[0116] Step 3:

[0117] The user uses a terminal to specify the origin and destination and sends a request. This information is entered into the server and used as parameters to calculate the optimal route. Based on the prediction results and real-time travel information, the server calculates the optimal route and outputs the candidate route.

[0118] Step 4:

[0119] The device displays optimal route information sent from the server on a map using the Google Maps API. It receives route information from the server as input, visualizes it, and provides the user with intuitive route guidance.

[0120] Step 5:

[0121] The server monitors traffic conditions in real time and immediately recalculates routes if they are affected. When a change in data is detected, the AI ​​model is re-executed and outputs the corrected route. The terminal then notifies the user of this new route information.

[0122] Step 6:

[0123] The user inputs a natural language prompt, such as "Please tell me the shortest route to my destination," through the device. The device uses a generative AI model to analyze this prompt and generate and output a response based on data from the server. This response is then displayed to the user on the device.

[0124] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0125] This invention is an advanced mobility assistance technology that combines a system that analyzes urban traffic information and provides users with the optimal travel route with an emotion engine that recognizes the user's emotions. This system mainly consists of a server, terminals, and an emotion engine, and realizes personalized services tailored to the user's emotional state.

[0126] The server is equipped with conventional traffic data collection methods and acquires traffic data in real time from various sensors and traffic systems. In addition, it uses AI algorithms to predict traffic conditions based on past traffic data and constructs useful travel routes for the user at specific times. Furthermore, an emotion engine analyzes the user's facial expressions, voice, and behavioral patterns through sensors and cameras built into the terminal to recognize the user's emotional state.

[0127] Users operate their devices and receive route suggestions based on real-time traffic data. A new feature is added that uses the emotion engine's analysis to suggest routes tailored to the user's stress and fatigue levels. For example, a user experiencing stress might be offered less congested routes or alternative routes with scenic views. The emotion engine can also suggest music and relaxation messages to the user, providing a more comfortable travel experience.

[0128] For example, if a user wants to avoid congested roads during their morning commute, the server calculates multiple routes to their destination based on traffic data. Meanwhile, the emotion engine monitors the user's voice and camera footage to assess their motivation and stress levels. It then selects a route that offers a relatively relaxing environment and presents it through the device. This allows the user to have a more comfortable and less stressful commute.

[0129] In summary, this system achieves flexible route suggestions that take into account both user emotions and traffic conditions, thereby improving the quality of the user experience during travel.

[0130] The following describes the processing flow.

[0131] Step 1:

[0132] The server collects real-time traffic data from various sensors and digital infrastructure within the city. This data includes road traffic volume, average speed, accident information, and construction status.

[0133] Step 2:

[0134] The server analyzes collected traffic data using AI algorithms to identify current traffic congestion and predict future congestion based on past data. This analysis is a process that generates information that forms the basis for efficient route suggestions.

[0135] Step 3:

[0136] The emotion engine uses the device's built-in camera and microphone to analyze the user's facial expressions, voice tone, and other behavioral indicators to determine the user's current emotional state. This is done to recognize signs of stress and fatigue and plan countermeasures.

[0137] Step 4:

[0138] The user enters their destination and departure time into the terminal and sends a request to the server. This information is crucial for the server to calculate the optimal route for the user.

[0139] Step 5:

[0140] The server comprehensively evaluates user requests, traffic data, and the results of the emotion engine's analysis to generate the optimal route based on the user's emotional state. For example, if the emotion engine detects a high stress level, the server prioritizes calculating routes that avoid congestion and options that allow for relaxation.

[0141] Step 6:

[0142] The server sends the generated route to the terminal, and the terminal provides the route information to the user. Visual or auditory feedback allows the user to easily understand and select the proposed route.

[0143] Step 7:

[0144] If there are changes in traffic conditions or the user's emotional state, the server immediately updates the data and recalculates the route if necessary. This new information is quickly sent to the terminal, and the user is provided with refreshed suggestions.

[0145] Step 8:

[0146] When a user makes an individual question or request in natural language through their device, the device forwards it to the server. The server uses natural language processing to analyze the question, generate appropriate actions or information, and sends them back to the device. This response function allows users to communicate effectively with the system and enjoy greater convenience.

[0147] (Example 2)

[0148] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0149] The challenges people face during their daily commutes in urban environments include stress caused by congested traffic and increased travel times. Furthermore, there is a lack of services that consider the influence of emotional states on travel choices. In this context, there is a need for a system that can provide more comfortable and less stressful travel routes based on the user's emotional state.

[0150] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0151] In this invention, the server includes a device for collecting traffic data, a device for analyzing the traffic data to predict congestion levels, and a device for collecting emotional information to analyze the user's emotional state. This enables the integrated consideration of the user's emotional state and traffic conditions to generate optimal travel routes and provide personalized content.

[0152] "Traffic data" refers to a collection of information including the flow of vehicles within a city, the operating status of public transportation, the status of traffic signals, and related information.

[0153] "Congestion status" refers to the degree of traffic congestion or delays in a particular time period or area.

[0154] "Emotional information" refers to data that indicates a subjective psychological state, obtained from the user's facial expressions, voice, behavioral patterns, etc.

[0155] "Emotional state" refers to the user's psychological and emotional condition, such as stress, fatigue, and relaxation.

[0156] The "optimal route" is the route selected to enable the most comfortable and efficient travel, taking into account the user's emotional state and current traffic conditions.

[0157] "Personalized content" refers to music, messages, or other relaxation information provided in response to a user's individual emotional state.

[0158] This invention is a system that comprehensively analyzes urban traffic information and user sentiment information to provide optimal travel routes and personalized content. The main components of the system are a server, a terminal, and an sentiment engine.

[0159] The server functions as a device for collecting traffic information. It collects real-time traffic data from sensors and traffic management systems installed throughout the city. This data includes traffic volume, congestion levels, and public transport operation information. The server uses AI algorithms to analyze the collected data and predict congestion levels by comparing them with past traffic patterns. This prediction forms the basis for designing optimal travel routes for users.

[0160] The device is designed to acquire user emotional information. Using built-in sensors and a camera, it analyzes the user's facial expressions, voice tone, and behavioral patterns in real time. This analysis is processed by an emotion engine to identify the user's emotional state. This information reflects the user's stress level and fatigue level, and is used to suggest the optimal travel route.

[0161] Users receive suggested routes and content through their devices. For example, if a user is feeling stressed, the server predicts and provides routes that avoid congestion or offer scenic views. In addition, the device suggests relaxing music and messages to the user, making the travel experience more comfortable.

[0162] For example, if a user is feeling stressed during their morning commute, the server calculates a less congested route based on past traffic data. Simultaneously, the terminal senses changes in the user's facial expressions and voice, and provides a route that is effective in reducing stress. Specifically, this might involve selecting a detour route that offers more scenic views or playing relaxation music. Through this entire process, users can reduce stress during their commute and enjoy a more comfortable journey.

[0163] An example of a prompt message is the instruction, "Use the emotion engine to generate route suggestions that take the user's stress level into account." This enables the provision of services tailored to specific situations.

[0164] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0165] Step 1:

[0166] The server collects real-time traffic data from sensors and traffic management systems installed throughout the city. Inputs include traffic volume, signal status, and public transport operation status. This data is fed into an AI algorithm to organize and analyze traffic flow. The output is the analyzed traffic pattern. Specifically, the server accesses a database to obtain current traffic conditions and predicts congestion.

[0167] Step 2:

[0168] The server analyzes collected traffic data using AI algorithms and predicts congestion levels by comparing them with historical data. The input consists of organized and analyzed traffic patterns and historical traffic data, while the output is a prediction of future congestion. Specifically, the server uses statistical models to simulate future traffic conditions and generate useful information for users.

[0169] Step 3:

[0170] The device uses built-in sensors and cameras to acquire the user's facial expressions, voice, and behavioral patterns in real time. The input is emotional data obtained from the user. This data is analyzed by an emotion engine to identify the emotional state. The output is emotional state such as stress level and fatigue level. Specifically, the device uses voice analysis software and facial recognition technology to determine the user's psychological state.

[0171] Step 4:

[0172] The server integrates traffic forecast data and user sentiment data to generate the optimal travel route. The inputs are congestion forecasts and sentiment data, and the output is the recommended route. Specifically, the server uses a prioritization algorithm to evaluate multiple route options and select the most appropriate route for the user.

[0173] Step 5:

[0174] The user receives suggested routes through the terminal. The input is the recommended route sent from the server. The terminal displays this and plays it back as voice guidance. The output is visual and auditory route guidance. Specifically, the terminal uses a map application to provide real-time directions and updates the route as needed.

[0175] (Application Example 2)

[0176] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0177] In modern urban environments, traffic congestion is a daily problem, and planning efficient travel routes is crucial for users. However, conventional route guidance systems have limited ability to adapt to changing traffic conditions and struggle to provide personalized services that take into account users' emotions and stress levels. As a result, users cannot enjoy a comfortable and stress-free travel experience. Therefore, there is a need to provide personalized route suggestions and in-vehicle environments that take into account the emotional state of users.

[0178] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0179] In this invention, the server includes means for acquiring traffic information, means for analyzing traffic information and predicting traffic congestion, means for generating an optimal route based on user requests, means for analyzing the user's emotional state using an emotion recognition device, means for optimizing the route or service based on the user's emotional state, and means for providing in-vehicle entertainment. This makes it possible to provide flexible route suggestions adapted to traffic conditions while taking the user's emotional state into consideration, as well as a comfortable in-vehicle environment.

[0180] "Traffic information" refers to data about traffic flow and conditions within a city, collected through sensors and external databases.

[0181] "Traffic congestion" refers to the state of the road, indicating the density and flow of vehicles, and is information that is subject to prediction and analysis.

[0182] "User" refers to a person who uses the system to receive services such as travel and route guidance.

[0183] The "optimal route" refers to the most efficient travel route for the user, calculated considering the time and distance required to reach the destination.

[0184] An "emotion recognition device" is a device that analyzes a user's facial expressions and voice to determine their mental and emotional state.

[0185] A "personalized service" is a service that provides information and support tailored to the individual circumstances and needs of the user.

[0186] "Entertainment" refers to music, videos, or other content designed to provide relaxation and entertainment to travelers.

[0187] "Adapting to traffic conditions" refers to a dynamic process that responds to real-time, fluctuating traffic conditions and adjusts routes and services as needed.

[0188] The system in this invention mainly consists of a server, a terminal, and an emotion recognition device. The server collects and analyzes traffic information in real time from traffic sensors and an external database. This allows for the prediction of traffic congestion and the generation of efficient routes. This traffic information is processed using advanced algorithms executed on the server. Specifically, programming languages ​​such as Python and R are used, and machine learning algorithms are utilized in particular to predict traffic flow.

[0189] The terminal functions as the user's mobile device or as an information system within the vehicle, communicating with a server to provide the latest route information. The terminal is equipped with emotion recognition devices such as a camera and microphone, which are used to analyze the user's facial expressions and voice. Libraries such as OpenCV and TensorFlow are used for emotion recognition, identifying emotional states from facial expressions and voice tone.

[0190] Users receive real-time route guidance and traffic information through their devices, supporting their decision-making while traveling. The system provides personalized route suggestions based on the user's stress level and emotional state, as well as in-car entertainment. For example, if a user wants to relax, the system automatically selects and plays soothing music and suggests alternative routes that allow them to enjoy the scenery.

[0191] As an example of a prompt, if the user is relaxed, generate an action that selects the most relaxing music and guides them along a scenic route. By utilizing a generative AI model, it is possible to provide optimal services tailored to the user.

[0192] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0193] Step 1:

[0194] The server acquires real-time traffic information from traffic sensors and databases. Inputs include data from various traffic sensors and external traffic information feeds. This data is analyzed to understand the current traffic congestion situation and converted into a digital format. Outputs include traffic congestion maps and forecast data.

[0195] Step 2:

[0196] The server uses a prediction algorithm to forecast traffic congestion for the next few hours. This process uses historical and real-time traffic data as input. The server runs a machine learning model to predict peak congestion times and delay factors, generating a congestion prediction model. The output is data on predicted times and congestion patterns.

[0197] Step 3:

[0198] The terminal calculates the optimal route to the user's destination based on traffic congestion data received from the server. Inputs include the current location, destination, traffic congestion data, and user preferences. The terminal integrates this information to generate multiple route options. The output is an optimized route list.

[0199] Step 4:

[0200] The device uses a camera and microphone to analyze the user's facial expressions and voice, and evaluate their emotional state. Inputs include camera video and audio data. An emotion recognition model such as TensorFlow analyzes these to identify the user's emotional state (e.g., stress, relaxation). The output is the evaluation result of the emotional state.

[0201] Step 5:

[0202] The server generates personalized route and entertainment suggestions based on the emotional state and traffic congestion information provided by the terminal. Inputs include the user's emotional state, current traffic conditions, and destination information. Output is a set of recommended routes and entertainment options. Specifically, the server selects the most suitable music or audio content to help the user relax.

[0203] Step 6:

[0204] The terminal provides the user with optimal route information and entertainment suggestions received from the server. The input is recommendation data from the server. Specifically, the terminal's display and audio output device present route guidance and entertainment to the user. The output is the user's reaction and feedback.

[0205] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0206] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0207] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0208] [Second Embodiment]

[0209] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0210] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0211] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0212] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0213] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0214] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0215] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0216] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0217] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0218] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0219] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0220] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0221] This invention is a system for reducing urban traffic congestion and providing users with efficient travel. The system primarily consists of three elements: a server, terminals, and users.

[0222] The server collects real-time traffic data through various sensors and communication infrastructure placed throughout the city. Data classifications include traffic volume, speed, accident information, and construction information. Based on this data, the server uses AI algorithms to analyze it and predict traffic congestion. To enhance user convenience, trend analysis based on historical data is also performed to improve the accuracy of future traffic predictions.

[0223] The user sends travel requests via their device, including their departure and destination points and departure time. The device then sends this information to the server, which initiates the calculation of the optimal route based on the user's specified conditions. The server generates routes combining various modes of transport, which can be customized to the user's preferences.

[0224] When traffic conditions change, the server detects the changes in real time and, if necessary, calculates a new route and sends it to the terminal. This allows users to travel based on the latest traffic information. In addition, the terminal uses natural language processing to generate responses to user questions in order to enable interaction with the user. For example, if a user asks, "What are the current traffic conditions like?", the terminal will quickly answer based on the latest information analyzed by the server.

[0225] As a concrete example, consider a user traveling to the city center by car during the morning rush hour. Since the usual route is likely to be congested, the server, based on traffic data, presents an alternative route combining train travel and walking. Even if a train delay occurs along the way, the server can quickly analyze the impact and provide the user with a new alternative. With these features, the user can reach their destination with minimal time and stress.

[0226] The following describes the processing flow.

[0227] Step 1:

[0228] The server collects real-time traffic data through sensors and communication infrastructure installed within the city. This data, encompassing information on traffic volume, speed, accidents, and construction, forms the core of this invention.

[0229] Step 2:

[0230] The server analyzes the collected data using AI algorithms to understand the current traffic congestion situation and predict future congestion based on trends derived from past data. The accuracy of this data analysis significantly impacts the effectiveness of the system.

[0231] Step 3:

[0232] The user uses their device to input information about their travel, such as their current location, destination, and departure time, and sends a request to the device. This request is the starting point for achieving maximum performance.

[0233] Step 4:

[0234] The terminal immediately sends the user's request to the server. The server calculates the optimal route based on the received request. Here, route selection is performed based on the user's needs.

[0235] Step 5:

[0236] When calculating optimized routes, the server considers combinations of multiple modes of transport and suggests the most efficient way for the user to travel. If necessary, it also uses real-time traffic data for optimization.

[0237] Step 6:

[0238] The server sends the calculated optimal route to the terminal. The terminal receives this and provides the user with route information visually or audibly. The user then makes a travel decision based on this information.

[0239] Step 7:

[0240] If traffic conditions change, the server continuously analyzes the new data and updates the route as needed. This updated information is immediately sent to the terminal and reflected in the user's travel plan.

[0241] Step 8:

[0242] When a user sends a question in natural language through their device, the device forwards this request to the server. The server uses natural language processing to analyze the question, generates appropriate information, and immediately sends it back to the device. This two-way interaction further enhances user convenience.

[0243] (Example 1)

[0244] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0245] Traffic congestion in urban areas causes various problems, including increased travel time and higher energy consumption. Conventional traffic information systems have limitations in real-time analysis and flexible route suggestion, making it difficult to provide users with the optimal travel route. Therefore, there is a need for more accurate traffic information analysis and flexible route suggestions tailored to users.

[0246] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0247] In this invention, the server includes a device for aggregating traffic information, a device for analyzing the traffic information and predicting traffic congestion, and a device for constructing an optimal route based on user requests. This makes it possible to flexibly provide users with routes suitable for them based on real-time traffic information.

[0248] "Traffic information" refers to data related to roads and transportation systems, including vehicle flow, speed, accidents, and construction information.

[0249] "Congestion" refers to a state where traffic flow is slower or more stagnant than usual, and it is an indicator used for prediction.

[0250] A "user" refers to an individual who uses this system to obtain route information and select a mode of transportation.

[0251] A "route" is the recommended path for travel from a starting point to a destination, and can involve combining multiple modes of transportation.

[0252] A "device" refers to a collection of hardware and software designed to perform a specific function.

[0253] A "server" refers to a computer system that processes and analyzes data and plays a central role in the system.

[0254] "Real-time" means that data collection and processing are performed immediately, enabling the provision of information that responds promptly to the current situation.

[0255] "Natural language" refers to the forms of language that humans use on a daily basis, and it is used in communication with computers.

[0256] In this embodiment of the invention, a system is constructed to alleviate urban traffic congestion and provide users with an efficient means of transportation. The system consists of three main elements: a server, a terminal, and a user.

[0257] The server uses a high-performance computer to collect real-time traffic information from traffic sensors and related devices installed throughout the city. This data includes traffic volume, speed, accident locations, and road construction information. The server analyzes this data using AI algorithms to evaluate current traffic conditions and predict future congestion. Specifically, it uses machine learning libraries such as TensorFlow and Python programs to extract significant patterns from large amounts of data.

[0258] Users input information such as their departure point, destination, and departure time using a device such as a smartphone or tablet. This information is sent from the device to the server, which calculates the most efficient travel route. Map information services such as the Google Maps API are used to generate the travel route, and the optimal combination of possible modes of transport, including vehicles, public transport, and walking, is suggested. The server updates the route in real time in response to requests, providing users with the latest information.

[0259] If traffic conditions suddenly change, the server immediately detects the situation and recalculates the route as needed. The terminal has the function to receive the new route information sent from the server and notify the user. This allows the user to respond quickly to changing traffic conditions.

[0260] By using natural language processing technology, the device can easily respond to user questions. For example, if a user asks, "What's the current traffic situation like?", an answer based on the latest information analyzed by the server will be provided. Generative AI models such as ChatGPT are used for this purpose.

[0261] As a concrete example, consider a scenario where a user wants to commute by car on a weekday morning to avoid crowds. This system can suggest a route combining train and walking, rather than the usual route. If the train is delayed along the way, the server can quickly generate an alternative route. This allows the user to reach their destination with less stress.

[0262] An example of a prompt message is: "The user is driving towards the city center. Based on current traffic information, suggest the best route to avoid congestion."

[0263] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0264] Step 1:

[0265] The server collects traffic information from traffic sensors placed throughout the city. Specifically, it receives data transmitted from the sensors via the communication infrastructure and records it in a database in the form of traffic volume, speed, accident information, etc. The input to this process is raw data from the sensors, and the output is structured traffic information data.

[0266] Step 2:

[0267] The server analyzes the collected traffic information using an AI algorithm. During the data analysis process, it evaluates the current traffic situation using machine learning, referencing past traffic patterns. The input is the structured data obtained in step 1, and the output is the traffic congestion level prediction provided by the congestion prediction model.

[0268] Step 3:

[0269] The user enters the departure point, destination, and departure time via a terminal. The terminal sends this information to the server. The input is the travel conditions specified by the user, and the output is communication data sent to the server that receives this information and processes it in the next step.

[0270] Step 4:

[0271] The server calculates the optimal route using an AI algorithm, taking into account the user's input conditions. Based on the latest traffic forecast data, the server generates routes combining multiple modes of transport. The input consists of the user's travel conditions and analyzed traffic data, while the output is optimized route information.

[0272] Step 5:

[0273] The terminal receives optimal route information sent from the server and displays it to the user. Here, the terminal plays the role of visualizing the optimal route in an easy-to-understand way and presenting it to the user. The input is route information from the server, and the output is visualized navigation information provided to the user.

[0274] Step 6:

[0275] The server monitors changes in traffic conditions in real time and recalculates routes as needed. For example, when new information such as accidents or construction comes in, the server immediately processes that information and updates the routes for affected users. The input is the changed traffic conditions, and the output is the updated route information.

[0276] Step 7:

[0277] The device notifies the user of updated route information. This includes on-screen pop-ups and audio notifications. The input is the updated route information, and the output is an alert to the user and a presentation of the new route information.

[0278] Step 8:

[0279] The device uses generative AI models and natural language processing to respond to user questions. Based on the latest information received from the server, the device generates answers to questions such as "What is the current traffic situation?". The input is a natural language question from the user, and the output is an accurate response to that question.

[0280] (Application Example 1)

[0281] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0282] In modern cities, traffic congestion is a daily problem, and efficient travel is particularly difficult during rush hour. Furthermore, users are often stressed by the need to quickly adapt to changes in traffic conditions while traveling. This invention aims to solve these problems and provide users with a comfortable and efficient means of transportation.

[0283] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is realized by the following means.

[0284] In this invention, the server includes means for collecting movement information within a city, means for analyzing the movement information and predicting traffic congestion, and means for using a smartphone to display the prediction result to the user and propose a movement route. Thereby, the user can select an optimal route based on the latest movement information, enabling comfortable and efficient movement.

[0285] "Movement information within a city" refers to data regarding the dynamics and states when vehicles and pedestrians move in the urban environment.

[0286] "Means for predicting traffic congestion" is a method for analyzing data obtained from movement information and estimating states such as current or future traffic density and speed.

[0287] "Means for generating an optimal route based on the user's request" is a process for calculating an efficient movement route based on conditions such as the departure point and destination specified by the user.

[0288] "Means for providing the generated route to the user" is a method for displaying or notifying the calculated optimal movement route to the user's terminal.

[0289] "Means for updating the route according to changes in the movement situation" is a process for recalculating and correcting the route so that the user can reach the destination more efficiently based on real-time changing movement information.

[0290] "Means for analyzing a natural language question from the user and generating a response" is a method for understanding a natural language query input by the user to the terminal and creating an appropriate answer to the question.

[0291] "A method for displaying prediction results to users using smartphones and suggesting travel routes" refers to a method of displaying analyzed travel information and prediction results on a smartphone screen and guiding users to the most suitable travel route.

[0292] The system implementing this invention provides a platform for the integrated management of urban mobility information. A server collects mobility data from various sensors placed throughout the city, absorbing information such as traffic flow, speed, and traffic density. This data is integrated into the server in real time and stored in a database.

[0293] The software on the server is built using programming languages ​​such as Python and Java. Sensor data is first cleaned, and then congestion levels are predicted using AI models such as TensorFlow. The prediction results are then processed to suggest the optimal route based on the user's travel request.

[0294] Devices, especially smartphones, utilize the Google Maps API to provide users with an intuitive interface. Users enter their starting point and destination, and receive an optimized route from the server. Furthermore, the server updates the route to the smartphone in real time in response to changes in traffic conditions.

[0295] Furthermore, the device utilizes a generative AI model to enable natural language interaction. Specifically, when a user enters a prompt such as "Please tell me the shortest route to my destination," the server quickly generates an answer based on the analysis results.

[0296] When users travel to the city center during peak hours, the server predicts traffic conditions and suggests appropriate modes of transport or alternative routes, thereby reducing travel time and stress. This allows users to experience comfortable and efficient travel.

[0297] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0298] Step 1:

[0299] The server collects movement data in real time from sensors placed throughout the city. It receives raw movement data transmitted from the sensors (vehicle position, speed, traffic density, etc.) as input. It then performs data cleaning to correct for noise and missing data, outputting formatted movement information.

[0300] Step 2:

[0301] The server inputs formatted travel information into an AI model using TensorFlow to predict congestion levels. It utilizes historical data and current travel information as input. The prediction algorithm calculates future traffic density and the distribution of congestion points, and outputs the prediction results.

[0302] Step 3:

[0303] The user uses a terminal to specify the origin and destination and sends a request. This information is entered into the server and used as parameters to calculate the optimal route. Based on the prediction results and real-time travel information, the server calculates the optimal route and outputs the candidate route.

[0304] Step 4:

[0305] The device displays optimal route information sent from the server on a map using the Google Maps API. It receives route information from the server as input, visualizes it, and provides the user with intuitive route guidance.

[0306] Step 5:

[0307] The server monitors changes in traffic conditions in real time and immediately recalculates the route if it is affected. When a data change is detected, the AI model is re-executed to output the modified route. The terminal notifies the user of the new route information.

[0308] Step 6:

[0309] The user inputs an inquiry such as the natural language prompt sentence "Please tell me the shortest route to the destination" through the terminal. The terminal uses the generation AI model to analyze this prompt sentence and generate and output a response based on the data from the server. This response is displayed to the user on the terminal.

[0310] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion recognition model 59 and perform specific processing using the user's emotion.

[0311] The present invention is an advanced travel support technology that combines an emotion engine for recognizing the user's emotion with a system that analyzes traffic information in the city and provides the user with an optimal travel route. This system mainly includes a server, a terminal, and an emotion engine, and realizes a personalized service according to the user's emotional state.

[0312] The server has conventional traffic data collection means and obtains traffic data in real time from various sensors and traffic systems. In addition, using an AI algorithm, it predicts the traffic situation based on past traffic data and constructs a beneficial travel route for the user on the spot. In addition to this, the emotion engine analyzes the user's expression, voice, action pattern, etc. through sensors and cameras built into the terminal to recognize the user's emotional state.

[0313] Users operate their devices and receive route suggestions based on real-time traffic data. A new feature is added that uses the emotion engine's analysis to suggest routes tailored to the user's stress and fatigue levels. For example, a user experiencing stress might be offered less congested routes or alternative routes with scenic views. The emotion engine can also suggest music and relaxation messages to the user, providing a more comfortable travel experience.

[0314] For example, if a user wants to avoid congested roads during their morning commute, the server calculates multiple routes to their destination based on traffic data. Meanwhile, the emotion engine monitors the user's voice and camera footage to assess their motivation and stress levels. It then selects a route that offers a relatively relaxing environment and presents it through the device. This allows the user to have a more comfortable and less stressful commute.

[0315] In summary, this system achieves flexible route suggestions that take into account both user emotions and traffic conditions, thereby improving the quality of the user experience during travel.

[0316] The following describes the processing flow.

[0317] Step 1:

[0318] The server collects real-time traffic data from various sensors and digital infrastructure within the city. This data includes road traffic volume, average speed, accident information, and construction status.

[0319] Step 2:

[0320] The server analyzes collected traffic data using AI algorithms to identify current traffic congestion and predict future congestion based on past data. This analysis is a process that generates information that forms the basis for efficient route suggestions.

[0321] Step 3:

[0322] The emotion engine uses the device's built-in camera and microphone to analyze the user's facial expressions, voice tone, and other behavioral indicators to determine the user's current emotional state. This is done to recognize signs of stress and fatigue and plan countermeasures.

[0323] Step 4:

[0324] The user enters their destination and departure time into the terminal and sends a request to the server. This information is crucial for the server to calculate the optimal route for the user.

[0325] Step 5:

[0326] The server comprehensively evaluates user requests, traffic data, and the results of the emotion engine's analysis to generate the optimal route based on the user's emotional state. For example, if the emotion engine detects a high stress level, the server prioritizes calculating routes that avoid congestion and options that allow for relaxation.

[0327] Step 6:

[0328] The server sends the generated route to the terminal, and the terminal provides the route information to the user. Visual or auditory feedback allows the user to easily understand and select the proposed route.

[0329] Step 7:

[0330] If there are changes in traffic conditions or the user's emotional state, the server immediately updates the data and recalculates the route if necessary. This new information is quickly sent to the terminal, and the user is provided with refreshed suggestions.

[0331] Step 8:

[0332] When a user makes an individual question or request in natural language through their device, the device forwards it to the server. The server uses natural language processing to analyze the question, generate appropriate actions or information, and sends them back to the device. This response function allows users to communicate effectively with the system and enjoy greater convenience.

[0333] (Example 2)

[0334] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0335] The challenges people face during their daily commutes in urban environments include stress caused by congested traffic and increased travel times. Furthermore, there is a lack of services that consider the influence of emotional states on travel choices. In this context, there is a need for a system that can provide more comfortable and less stressful travel routes based on the user's emotional state.

[0336] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0337] In this invention, the server includes a device for collecting traffic data, a device for analyzing the traffic data to predict congestion levels, and a device for collecting emotional information to analyze the user's emotional state. This enables the integrated consideration of the user's emotional state and traffic conditions to generate optimal travel routes and provide personalized content.

[0338] "Traffic data" refers to a collection of information including the flow of vehicles within a city, the operating status of public transportation, the status of traffic signals, and related information.

[0339] "Congestion status" refers to the degree of traffic congestion or delays in a particular time period or area.

[0340] "Emotional information" refers to data that indicates a subjective psychological state, obtained from the user's facial expressions, voice, behavioral patterns, etc.

[0341] "Emotional state" refers to the user's psychological and emotional condition, such as stress, fatigue, and relaxation.

[0342] The "optimal route" is the route selected to enable the most comfortable and efficient travel, taking into account the user's emotional state and current traffic conditions.

[0343] "Personalized content" refers to music, messages, or other relaxation information provided in response to a user's individual emotional state.

[0344] This invention is a system that comprehensively analyzes urban traffic information and user sentiment information to provide optimal travel routes and personalized content. The main components of the system are a server, a terminal, and an sentiment engine.

[0345] The server functions as a device for collecting traffic information. It collects real-time traffic data from sensors and traffic management systems installed throughout the city. This data includes traffic volume, congestion levels, and public transport operation information. The server uses AI algorithms to analyze the collected data and predict congestion levels by comparing them with past traffic patterns. This prediction forms the basis for designing optimal travel routes for users.

[0346] The device is designed to acquire user emotional information. Using built-in sensors and a camera, it analyzes the user's facial expressions, voice tone, and behavioral patterns in real time. This analysis is processed by an emotion engine to identify the user's emotional state. This information reflects the user's stress level and fatigue level, and is used to suggest the optimal travel route.

[0347] Users receive suggested routes and content through their devices. For example, if a user is feeling stressed, the server predicts and provides routes that avoid congestion or offer scenic views. In addition, the device suggests relaxing music and messages to the user, making the travel experience more comfortable.

[0348] For example, if a user is feeling stressed during their morning commute, the server calculates a less congested route based on past traffic data. Simultaneously, the terminal senses changes in the user's facial expressions and voice, and provides a route that is effective in reducing stress. Specifically, this might involve selecting a detour route that offers more scenic views or playing relaxation music. Through this entire process, users can reduce stress during their commute and enjoy a more comfortable journey.

[0349] An example of a prompt message is the instruction, "Use the emotion engine to generate route suggestions that take the user's stress level into account." This enables the provision of services tailored to specific situations.

[0350] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0351] Step 1:

[0352] The server collects real-time traffic data from sensors and traffic management systems installed throughout the city. Inputs include traffic volume, signal status, and public transport operation status. This data is fed into an AI algorithm to organize and analyze traffic flow. The output is the analyzed traffic pattern. Specifically, the server accesses a database to obtain current traffic conditions and predicts congestion.

[0353] Step 2:

[0354] The server analyzes collected traffic data using AI algorithms and predicts congestion levels by comparing them with historical data. The input consists of organized and analyzed traffic patterns and historical traffic data, while the output is a prediction of future congestion. Specifically, the server uses statistical models to simulate future traffic conditions and generate useful information for users.

[0355] Step 3:

[0356] The device uses built-in sensors and cameras to acquire the user's facial expressions, voice, and behavioral patterns in real time. The input is emotional data obtained from the user. This data is analyzed by an emotion engine to identify the emotional state. The output is emotional state such as stress level and fatigue level. Specifically, the device uses voice analysis software and facial recognition technology to determine the user's psychological state.

[0357] Step 4:

[0358] The server integrates traffic forecast data and user sentiment data to generate the optimal travel route. The inputs are congestion forecasts and sentiment data, and the output is the recommended route. Specifically, the server uses a prioritization algorithm to evaluate multiple route options and select the most appropriate route for the user.

[0359] Step 5:

[0360] The user receives suggested routes through the terminal. The input is the recommended route sent from the server. The terminal displays this and plays it back as voice guidance. The output is visual and auditory route guidance. Specifically, the terminal uses a map application to provide real-time directions and updates the route as needed.

[0361] (Application Example 2)

[0362] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0363] In modern urban environments, traffic congestion is a daily problem, and planning efficient travel routes is crucial for users. However, conventional route guidance systems have limited ability to adapt to changing traffic conditions and struggle to provide personalized services that take into account users' emotions and stress levels. As a result, users cannot enjoy a comfortable and stress-free travel experience. Therefore, there is a need to provide personalized route suggestions and in-vehicle environments that take into account the emotional state of users.

[0364] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0365] In this invention, the server includes means for acquiring traffic information, means for analyzing traffic information and predicting traffic congestion, means for generating an optimal route based on user requests, means for analyzing the user's emotional state using an emotion recognition device, means for optimizing the route or service based on the user's emotional state, and means for providing in-vehicle entertainment. This makes it possible to provide flexible route suggestions adapted to traffic conditions while taking the user's emotional state into consideration, as well as a comfortable in-vehicle environment.

[0366] "Traffic information" refers to data about traffic flow and conditions within a city, collected through sensors and external databases.

[0367] "Traffic congestion" refers to the state of the road, indicating the density and flow of vehicles, and is information that is subject to prediction and analysis.

[0368] "User" refers to a person who uses the system to receive services such as travel and route guidance.

[0369] The "optimal route" refers to the most efficient travel route for the user, calculated considering the time and distance required to reach the destination.

[0370] An "emotion recognition device" is a device that analyzes a user's facial expressions and voice to determine their mental and emotional state.

[0371] A "personalized service" is a service that provides information and support tailored to the individual circumstances and needs of the user.

[0372] "Entertainment" refers to music, videos, or other content designed to provide relaxation and entertainment to travelers.

[0373] "Adapting to traffic conditions" refers to a dynamic process that responds to real-time, fluctuating traffic conditions and adjusts routes and services as needed.

[0374] The system in this invention mainly consists of a server, a terminal, and an emotion recognition device. The server collects and analyzes traffic information in real time from traffic sensors and an external database. This allows for the prediction of traffic congestion and the generation of efficient routes. This traffic information is processed using advanced algorithms executed on the server. Specifically, programming languages ​​such as Python and R are used, and machine learning algorithms are utilized in particular to predict traffic flow.

[0375] The terminal functions as the user's mobile device or as an information system within the vehicle, communicating with a server to provide the latest route information. The terminal is equipped with emotion recognition devices such as a camera and microphone, which are used to analyze the user's facial expressions and voice. Libraries such as OpenCV and TensorFlow are used for emotion recognition, identifying emotional states from facial expressions and voice tone.

[0376] Users receive real-time route guidance and traffic information through their devices, supporting their decision-making while traveling. The system provides personalized route suggestions based on the user's stress level and emotional state, as well as in-car entertainment. For example, if a user wants to relax, the system automatically selects and plays soothing music and suggests alternative routes that allow them to enjoy the scenery.

[0377] As an example of a prompt, if the user is relaxed, generate an action that selects the most relaxing music and guides them along a scenic route. By utilizing a generative AI model, it is possible to provide optimal services tailored to the user.

[0378] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0379] Step 1:

[0380] The server acquires real-time traffic information from traffic sensors and databases. Inputs include data from various traffic sensors and external traffic information feeds. This data is analyzed to understand the current traffic congestion situation and converted into a digital format. Outputs include traffic congestion maps and forecast data.

[0381] Step 2:

[0382] The server uses a prediction algorithm to forecast traffic congestion for the next few hours. This process uses historical and real-time traffic data as input. The server runs a machine learning model to predict peak congestion times and delay factors, generating a congestion prediction model. The output is data on predicted times and congestion patterns.

[0383] Step 3:

[0384] The terminal calculates the optimal route to the user's destination based on traffic congestion data received from the server. Inputs include the current location, destination, traffic congestion data, and user preferences. The terminal integrates this information to generate multiple route options. The output is an optimized route list.

[0385] Step 4:

[0386] The device uses a camera and microphone to analyze the user's facial expressions and voice, and evaluate their emotional state. Inputs include camera video and audio data. An emotion recognition model such as TensorFlow analyzes these to identify the user's emotional state (e.g., stress, relaxation). The output is the evaluation result of the emotional state.

[0387] Step 5:

[0388] The server generates personalized route and entertainment suggestions based on the emotional state and traffic congestion information provided by the terminal. Inputs include the user's emotional state, current traffic conditions, and destination information. Output is a set of recommended routes and entertainment options. Specifically, the server selects the most suitable music or audio content to help the user relax.

[0389] Step 6:

[0390] The terminal provides the user with optimal route information and entertainment suggestions received from the server. The input is recommendation data from the server. Specifically, the terminal's display and audio output device present route guidance and entertainment to the user. The output is the user's reaction and feedback.

[0391] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0392] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0393] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0394] [Third Embodiment]

[0395] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0396] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0397] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0398] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0399] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0400] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0401] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0402] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0403] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0404] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0405] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0406] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0407] This invention is a system for reducing urban traffic congestion and providing users with efficient travel. The system primarily consists of three elements: a server, terminals, and users.

[0408] The server collects real-time traffic data through various sensors and communication infrastructure placed throughout the city. Data classifications include traffic volume, speed, accident information, and construction information. Based on this data, the server uses AI algorithms to analyze it and predict traffic congestion. To enhance user convenience, trend analysis based on historical data is also performed to improve the accuracy of future traffic predictions.

[0409] The user sends travel requests via their device, including their departure and destination points and departure time. The device then sends this information to the server, which initiates the calculation of the optimal route based on the user's specified conditions. The server generates routes combining various modes of transport, which can be customized to the user's preferences.

[0410] When traffic conditions change, the server detects the changes in real time and, if necessary, calculates a new route and sends it to the terminal. This allows users to travel based on the latest traffic information. In addition, the terminal uses natural language processing to generate responses to user questions in order to enable interaction with the user. For example, if a user asks, "What are the current traffic conditions like?", the terminal will quickly answer based on the latest information analyzed by the server.

[0411] As a concrete example, consider a user traveling to the city center by car during the morning rush hour. Since the usual route is likely to be congested, the server, based on traffic data, presents an alternative route combining train travel and walking. Even if a train delay occurs along the way, the server can quickly analyze the impact and provide the user with a new alternative. With these features, the user can reach their destination with minimal time and stress.

[0412] The following describes the processing flow.

[0413] Step 1:

[0414] The server collects real-time traffic data through sensors and communication infrastructure installed within the city. This data, encompassing information on traffic volume, speed, accidents, and construction, forms the core of this invention.

[0415] Step 2:

[0416] The server analyzes the collected data using AI algorithms to understand the current traffic congestion situation and predict future congestion based on trends derived from past data. The accuracy of this data analysis significantly impacts the effectiveness of the system.

[0417] Step 3:

[0418] The user uses their device to input information about their travel, such as their current location, destination, and departure time, and sends a request to the device. This request is the starting point for achieving maximum performance.

[0419] Step 4:

[0420] The terminal immediately sends the user's request to the server. The server calculates the optimal route based on the received request. Here, route selection is performed based on the user's needs.

[0421] Step 5:

[0422] When calculating optimized routes, the server considers combinations of multiple modes of transport and suggests the most efficient way for the user to travel. If necessary, it also uses real-time traffic data for optimization.

[0423] Step 6:

[0424] The server sends the calculated optimal route to the terminal. The terminal receives this and provides the user with route information visually or audibly. The user then makes a travel decision based on this information.

[0425] Step 7:

[0426] If traffic conditions change, the server continuously analyzes the new data and updates the route as needed. This updated information is immediately sent to the terminal and reflected in the user's travel plan.

[0427] Step 8:

[0428] When a user sends a question in natural language through their device, the device forwards this request to the server. The server uses natural language processing to analyze the question, generates appropriate information, and immediately sends it back to the device. This two-way interaction further enhances user convenience.

[0429] (Example 1)

[0430] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0431] Traffic congestion in urban areas causes various problems, including increased travel time and higher energy consumption. Conventional traffic information systems have limitations in real-time analysis and flexible route suggestion, making it difficult to provide users with the optimal travel route. Therefore, there is a need for more accurate traffic information analysis and flexible route suggestions tailored to users.

[0432] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0433] In this invention, the server includes a device for aggregating traffic information, a device for analyzing the traffic information and predicting traffic congestion, and a device for constructing an optimal route based on user requests. This makes it possible to flexibly provide users with routes suitable for them based on real-time traffic information.

[0434] "Traffic information" refers to data related to roads and transportation systems, including vehicle flow, speed, accidents, and construction information.

[0435] "Congestion" refers to a state where traffic flow is slower or more stagnant than usual, and it is an indicator used for prediction.

[0436] A "user" refers to an individual who uses this system to obtain route information and select a mode of transportation.

[0437] A "route" is the recommended path for travel from a starting point to a destination, and can involve combining multiple modes of transportation.

[0438] A "device" refers to a collection of hardware and software designed to perform a specific function.

[0439] A "server" refers to a computer system that processes and analyzes data and plays a central role in the system.

[0440] "Real-time" means that data collection and processing are performed immediately, enabling the provision of information that responds promptly to the current situation.

[0441] "Natural language" refers to the forms of language that humans use on a daily basis, and it is used in communication with computers.

[0442] In this embodiment of the invention, a system is constructed to alleviate urban traffic congestion and provide users with an efficient means of transportation. The system consists of three main elements: a server, a terminal, and a user.

[0443] The server uses a high-performance computer to collect real-time traffic information from traffic sensors and related devices installed throughout the city. This data includes traffic volume, speed, accident locations, and road construction information. The server analyzes this data using AI algorithms to evaluate current traffic conditions and predict future congestion. Specifically, it uses machine learning libraries such as TensorFlow and Python programs to extract significant patterns from large amounts of data.

[0444] Users input information such as their departure point, destination, and departure time using a device such as a smartphone or tablet. This information is sent from the device to the server, which calculates the most efficient travel route. Map information services such as the Google Maps API are used to generate the travel route, and the optimal combination of possible modes of transport, including vehicles, public transport, and walking, is suggested. The server updates the route in real time in response to requests, providing users with the latest information.

[0445] If traffic conditions suddenly change, the server immediately detects the situation and recalculates the route as needed. The terminal has the function to receive the new route information sent from the server and notify the user. This allows the user to respond quickly to changing traffic conditions.

[0446] By using natural language processing technology, the device can easily respond to user questions. For example, if a user asks, "What's the current traffic situation like?", an answer based on the latest information analyzed by the server will be provided. Generative AI models such as ChatGPT are used for this purpose.

[0447] As a concrete example, consider a scenario where a user wants to commute by car on a weekday morning to avoid crowds. This system can suggest a route combining train and walking, rather than the usual route. If the train is delayed along the way, the server can quickly generate an alternative route. This allows the user to reach their destination with less stress.

[0448] An example of a prompt message is: "The user is driving towards the city center. Based on current traffic information, suggest the best route to avoid congestion."

[0449] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0450] Step 1:

[0451] The server collects traffic information from traffic sensors placed throughout the city. Specifically, it receives data transmitted from the sensors via the communication infrastructure and records it in a database in the form of traffic volume, speed, accident information, etc. The input to this process is raw data from the sensors, and the output is structured traffic information data.

[0452] Step 2:

[0453] The server analyzes the collected traffic information using an AI algorithm. During the data analysis process, it evaluates the current traffic situation using machine learning, referencing past traffic patterns. The input is the structured data obtained in step 1, and the output is the traffic congestion level prediction provided by the congestion prediction model.

[0454] Step 3:

[0455] The user enters the departure point, destination, and departure time via a terminal. The terminal sends this information to the server. The input is the travel conditions specified by the user, and the output is communication data sent to the server that receives this information and processes it in the next step.

[0456] Step 4:

[0457] The server calculates the optimal route using an AI algorithm, taking into account the user's input conditions. Based on the latest traffic forecast data, the server generates routes combining multiple modes of transport. The input consists of the user's travel conditions and analyzed traffic data, while the output is optimized route information.

[0458] Step 5:

[0459] The terminal receives optimal route information sent from the server and displays it to the user. Here, the terminal plays the role of visualizing the optimal route in an easy-to-understand way and presenting it to the user. The input is route information from the server, and the output is visualized navigation information provided to the user.

[0460] Step 6:

[0461] The server monitors changes in traffic conditions in real time and recalculates routes as needed. For example, when new information such as accidents or construction comes in, the server immediately processes that information and updates the routes for affected users. The input is the changed traffic conditions, and the output is the updated route information.

[0462] Step 7:

[0463] The device notifies the user of updated route information. This includes on-screen pop-ups and audio notifications. The input is the updated route information, and the output is an alert to the user and a presentation of the new route information.

[0464] Step 8:

[0465] The device uses generative AI models and natural language processing to respond to user questions. Based on the latest information received from the server, the device generates answers to questions such as "What is the current traffic situation?". The input is a natural language question from the user, and the output is an accurate response to that question.

[0466] (Application Example 1)

[0467] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0468] In modern cities, traffic congestion is a daily problem, and efficient travel is particularly difficult during rush hour. Furthermore, users are often stressed by the need to quickly adapt to changes in traffic conditions while traveling. This invention aims to solve these problems and provide users with a comfortable and efficient means of transportation.

[0469] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0470] In this invention, the server includes means for collecting movement information within a city, means for analyzing the movement information and predicting congestion levels, and means for displaying the prediction results to the user using a smartphone and suggesting a travel route. This allows the user to select the optimal route based on the latest movement information, enabling comfortable and efficient travel.

[0471] "Urban mobility information" refers to data on the dynamics and conditions of vehicles and pedestrians moving within an urban environment.

[0472] "Methods for predicting congestion" refer to methods that analyze data obtained from travel information to estimate current or future traffic density, speed, and other conditions.

[0473] "Means for generating the optimal route based on user requests" refers to a process that calculates an efficient travel route based on conditions such as the origin and destination specified by the user.

[0474] "Means of providing the generated route to the user" refers to a method of displaying or notifying the user of the calculated optimal travel route on their terminal.

[0475] "Means of updating routes in response to changes in travel conditions" refers to a process that recalculates and modifies routes based on real-time changing travel information so that users can reach their destinations more efficiently.

[0476] "A means of analyzing natural language questions from users and generating responses" refers to a method of understanding natural language questions entered by users into a terminal and creating appropriate answers to those questions.

[0477] "A method for displaying prediction results to users using smartphones and suggesting travel routes" refers to a method of displaying analyzed travel information and prediction results on a smartphone screen and guiding users to the most suitable travel route.

[0478] The system implementing this invention provides a platform for the integrated management of urban mobility information. A server collects mobility data from various sensors placed throughout the city, absorbing information such as traffic flow, speed, and traffic density. This data is integrated into the server in real time and stored in a database.

[0479] The software on the server is built using programming languages ​​such as Python and Java. Sensor data is first cleaned, and then congestion levels are predicted using AI models such as TensorFlow. The prediction results are then processed to suggest the optimal route based on the user's travel request.

[0480] Devices, especially smartphones, utilize the Google Maps API to provide users with an intuitive interface. Users enter their starting point and destination, and receive an optimized route from the server. Furthermore, the server updates the route to the smartphone in real time in response to changes in traffic conditions.

[0481] Furthermore, the device utilizes a generative AI model to enable natural language interaction. Specifically, when a user enters a prompt such as "Please tell me the shortest route to my destination," the server quickly generates an answer based on the analysis results.

[0482] When users travel to the city center during peak hours, the server predicts traffic conditions and suggests appropriate modes of transport or alternative routes, thereby reducing travel time and stress. This allows users to experience comfortable and efficient travel.

[0483] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0484] Step 1:

[0485] The server collects movement data in real time from sensors placed throughout the city. It receives raw movement data transmitted from the sensors (vehicle position, speed, traffic density, etc.) as input. It then performs data cleaning to correct for noise and missing data, outputting formatted movement information.

[0486] Step 2:

[0487] The server inputs formatted travel information into an AI model using TensorFlow to predict congestion levels. It utilizes historical data and current travel information as input. The prediction algorithm calculates future traffic density and the distribution of congestion points, and outputs the prediction results.

[0488] Step 3:

[0489] The user uses a terminal to specify the origin and destination and sends a request. This information is entered into the server and used as parameters to calculate the optimal route. Based on the prediction results and real-time travel information, the server calculates the optimal route and outputs the candidate route.

[0490] Step 4:

[0491] The device displays optimal route information sent from the server on a map using the Google Maps API. It receives route information from the server as input, visualizes it, and provides the user with intuitive route guidance.

[0492] Step 5:

[0493] The server monitors traffic conditions in real time and immediately recalculates routes if they are affected. When a change in data is detected, the AI ​​model is re-executed and outputs the corrected route. The terminal then notifies the user of this new route information.

[0494] Step 6:

[0495] The user inputs a natural language prompt, such as "Please tell me the shortest route to my destination," through the device. The device uses a generative AI model to analyze this prompt and generate and output a response based on data from the server. This response is then displayed to the user on the device.

[0496] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0497] This invention is an advanced mobility assistance technology that combines a system that analyzes urban traffic information and provides users with the optimal travel route with an emotion engine that recognizes the user's emotions. This system mainly consists of a server, terminals, and an emotion engine, and realizes personalized services tailored to the user's emotional state.

[0498] The server is equipped with conventional traffic data collection methods and acquires traffic data in real time from various sensors and traffic systems. In addition, it uses AI algorithms to predict traffic conditions based on past traffic data and constructs useful travel routes for the user at specific times. Furthermore, an emotion engine analyzes the user's facial expressions, voice, and behavioral patterns through sensors and cameras built into the terminal to recognize the user's emotional state.

[0499] Users operate their devices and receive route suggestions based on real-time traffic data. A new feature is added that uses the emotion engine's analysis to suggest routes tailored to the user's stress and fatigue levels. For example, a user experiencing stress might be offered less congested routes or alternative routes with scenic views. The emotion engine can also suggest music and relaxation messages to the user, providing a more comfortable travel experience.

[0500] For example, if a user wants to avoid congested roads during their morning commute, the server calculates multiple routes to their destination based on traffic data. Meanwhile, the emotion engine monitors the user's voice and camera footage to assess their motivation and stress levels. It then selects a route that offers a relatively relaxing environment and presents it through the device. This allows the user to have a more comfortable and less stressful commute.

[0501] In summary, this system achieves flexible route suggestions that take into account both user emotions and traffic conditions, thereby improving the quality of the user experience during travel.

[0502] The following describes the processing flow.

[0503] Step 1:

[0504] The server collects real-time traffic data from various sensors and digital infrastructure within the city. This data includes road traffic volume, average speed, accident information, and construction status.

[0505] Step 2:

[0506] The server analyzes collected traffic data using AI algorithms to identify current traffic congestion and predict future congestion based on past data. This analysis is a process that generates information that forms the basis for efficient route suggestions.

[0507] Step 3:

[0508] The emotion engine uses the device's built-in camera and microphone to analyze the user's facial expressions, voice tone, and other behavioral indicators to determine the user's current emotional state. This is done to recognize signs of stress and fatigue and plan countermeasures.

[0509] Step 4:

[0510] The user enters their destination and departure time into the terminal and sends a request to the server. This information is crucial for the server to calculate the optimal route for the user.

[0511] Step 5:

[0512] The server comprehensively evaluates user requests, traffic data, and the results of the emotion engine's analysis to generate the optimal route based on the user's emotional state. For example, if the emotion engine detects a high stress level, the server prioritizes calculating routes that avoid congestion and options that allow for relaxation.

[0513] Step 6:

[0514] The server sends the generated route to the terminal, and the terminal provides the route information to the user. Visual or auditory feedback allows the user to easily understand and select the proposed route.

[0515] Step 7:

[0516] If there are changes in traffic conditions or the user's emotional state, the server immediately updates the data and recalculates the route if necessary. This new information is quickly sent to the terminal, and the user is provided with refreshed suggestions.

[0517] Step 8:

[0518] When a user makes an individual question or request in natural language through their device, the device forwards it to the server. The server uses natural language processing to analyze the question, generate appropriate actions or information, and sends them back to the device. This response function allows users to communicate effectively with the system and enjoy greater convenience.

[0519] (Example 2)

[0520] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0521] The challenges people face during their daily commutes in urban environments include stress caused by congested traffic and increased travel times. Furthermore, there is a lack of services that consider the influence of emotional states on travel choices. In this context, there is a need for a system that can provide more comfortable and less stressful travel routes based on the user's emotional state.

[0522] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0523] In this invention, the server includes a device for collecting traffic data, a device for analyzing the traffic data to predict congestion levels, and a device for collecting emotional information to analyze the user's emotional state. This enables the integrated consideration of the user's emotional state and traffic conditions to generate optimal travel routes and provide personalized content.

[0524] "Traffic data" refers to a collection of information including the flow of vehicles within a city, the operating status of public transportation, the status of traffic signals, and related information.

[0525] "Congestion status" refers to the degree of traffic congestion or delays in a particular time period or area.

[0526] "Emotional information" refers to data that indicates a subjective psychological state, obtained from the user's facial expressions, voice, behavioral patterns, etc.

[0527] "Emotional state" refers to the user's psychological and emotional condition, such as stress, fatigue, and relaxation.

[0528] The "optimal route" is the route selected to enable the most comfortable and efficient travel, taking into account the user's emotional state and current traffic conditions.

[0529] "Personalized content" refers to music, messages, or other relaxation information provided in response to a user's individual emotional state.

[0530] This invention is a system that comprehensively analyzes urban traffic information and user sentiment information to provide optimal travel routes and personalized content. The main components of the system are a server, a terminal, and an sentiment engine.

[0531] The server functions as a device for collecting traffic information. It collects real-time traffic data from sensors and traffic management systems installed throughout the city. This data includes traffic volume, congestion levels, and public transport operation information. The server uses AI algorithms to analyze the collected data and predict congestion levels by comparing them with past traffic patterns. This prediction forms the basis for designing optimal travel routes for users.

[0532] The device is designed to acquire user emotional information. Using built-in sensors and a camera, it analyzes the user's facial expressions, voice tone, and behavioral patterns in real time. This analysis is processed by an emotion engine to identify the user's emotional state. This information reflects the user's stress level and fatigue level, and is used to suggest the optimal travel route.

[0533] Users receive suggested routes and content through their devices. For example, if a user is feeling stressed, the server predicts and provides routes that avoid congestion or offer scenic views. In addition, the device suggests relaxing music and messages to the user, making the travel experience more comfortable.

[0534] For example, if a user is feeling stressed during their morning commute, the server calculates a less congested route based on past traffic data. Simultaneously, the terminal senses changes in the user's facial expressions and voice, and provides a route that is effective in reducing stress. Specifically, this might involve selecting a detour route that offers more scenic views or playing relaxation music. Through this entire process, users can reduce stress during their commute and enjoy a more comfortable journey.

[0535] An example of a prompt message is the instruction, "Use the emotion engine to generate route suggestions that take the user's stress level into account." This enables the provision of services tailored to specific situations.

[0536] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0537] Step 1:

[0538] The server collects real-time traffic data from sensors and traffic management systems installed throughout the city. Inputs include traffic volume, signal status, and public transport operation status. This data is fed into an AI algorithm to organize and analyze traffic flow. The output is the analyzed traffic pattern. Specifically, the server accesses a database to obtain current traffic conditions and predicts congestion.

[0539] Step 2:

[0540] The server analyzes collected traffic data using AI algorithms and predicts congestion levels by comparing them with historical data. The input consists of organized and analyzed traffic patterns and historical traffic data, while the output is a prediction of future congestion. Specifically, the server uses statistical models to simulate future traffic conditions and generate useful information for users.

[0541] Step 3:

[0542] The device uses built-in sensors and cameras to acquire the user's facial expressions, voice, and behavioral patterns in real time. The input is emotional data obtained from the user. This data is analyzed by an emotion engine to identify the emotional state. The output is emotional state such as stress level and fatigue level. Specifically, the device uses voice analysis software and facial recognition technology to determine the user's psychological state.

[0543] Step 4:

[0544] The server integrates traffic forecast data and user sentiment data to generate the optimal travel route. The inputs are congestion forecasts and sentiment data, and the output is the recommended route. Specifically, the server uses a prioritization algorithm to evaluate multiple route options and select the most appropriate route for the user.

[0545] Step 5:

[0546] The user receives suggested routes through the terminal. The input is the recommended route sent from the server. The terminal displays this and plays it back as voice guidance. The output is visual and auditory route guidance. Specifically, the terminal uses a map application to provide real-time directions and updates the route as needed.

[0547] (Application Example 2)

[0548] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0549] In modern urban environments, traffic congestion is a daily problem, and planning efficient travel routes is crucial for users. However, conventional route guidance systems have limited ability to adapt to changing traffic conditions and struggle to provide personalized services that take into account users' emotions and stress levels. As a result, users cannot enjoy a comfortable and stress-free travel experience. Therefore, there is a need to provide personalized route suggestions and in-vehicle environments that take into account the emotional state of users.

[0550] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0551] In this invention, the server includes means for acquiring traffic information, means for analyzing traffic information and predicting traffic congestion, means for generating an optimal route based on user requests, means for analyzing the user's emotional state using an emotion recognition device, means for optimizing the route or service based on the user's emotional state, and means for providing in-vehicle entertainment. This makes it possible to provide flexible route suggestions adapted to traffic conditions while taking the user's emotional state into consideration, as well as a comfortable in-vehicle environment.

[0552] "Traffic information" refers to data about traffic flow and conditions within a city, collected through sensors and external databases.

[0553] "Traffic congestion" refers to the state of the road, indicating the density and flow of vehicles, and is information that is subject to prediction and analysis.

[0554] "User" refers to a person who uses the system to receive services such as travel and route guidance.

[0555] The "optimal route" refers to the most efficient travel route for the user, calculated considering the time and distance required to reach the destination.

[0556] An "emotion recognition device" is a device that analyzes a user's facial expressions and voice to determine their mental and emotional state.

[0557] A "personalized service" is a service that provides information and support tailored to the individual circumstances and needs of the user.

[0558] "Entertainment" refers to music, videos, or other content designed to provide relaxation and entertainment to travelers.

[0559] "Adapting to traffic conditions" refers to a dynamic process that responds to real-time, fluctuating traffic conditions and adjusts routes and services as needed.

[0560] The system in this invention mainly consists of a server, a terminal, and an emotion recognition device. The server collects and analyzes traffic information in real time from traffic sensors and an external database. This allows for the prediction of traffic congestion and the generation of efficient routes. This traffic information is processed using advanced algorithms executed on the server. Specifically, programming languages ​​such as Python and R are used, and machine learning algorithms are utilized in particular to predict traffic flow.

[0561] The terminal functions as the user's mobile device or as an information system within the vehicle, communicating with a server to provide the latest route information. The terminal is equipped with emotion recognition devices such as a camera and microphone, which are used to analyze the user's facial expressions and voice. Libraries such as OpenCV and TensorFlow are used for emotion recognition, identifying emotional states from facial expressions and voice tone.

[0562] Users receive real-time route guidance and traffic information through their devices, supporting their decision-making while traveling. The system provides personalized route suggestions based on the user's stress level and emotional state, as well as in-car entertainment. For example, if a user wants to relax, the system automatically selects and plays soothing music and suggests alternative routes that allow them to enjoy the scenery.

[0563] As an example of a prompt, if the user is relaxed, generate an action that selects the most relaxing music and guides them along a scenic route. By utilizing a generative AI model, it is possible to provide optimal services tailored to the user.

[0564] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0565] Step 1:

[0566] The server acquires real-time traffic information from traffic sensors and databases. Inputs include data from various traffic sensors and external traffic information feeds. This data is analyzed to understand the current traffic congestion situation and converted into a digital format. Outputs include traffic congestion maps and forecast data.

[0567] Step 2:

[0568] The server uses a prediction algorithm to forecast traffic congestion for the next few hours. This process uses historical and real-time traffic data as input. The server runs a machine learning model to predict peak congestion times and delay factors, generating a congestion prediction model. The output is data on predicted times and congestion patterns.

[0569] Step 3:

[0570] The terminal calculates the optimal route to the user's destination based on traffic congestion data received from the server. Inputs include the current location, destination, traffic congestion data, and user preferences. The terminal integrates this information to generate multiple route options. The output is an optimized route list.

[0571] Step 4:

[0572] The device uses a camera and microphone to analyze the user's facial expressions and voice, and evaluate their emotional state. Inputs include camera video and audio data. An emotion recognition model such as TensorFlow analyzes these to identify the user's emotional state (e.g., stress, relaxation). The output is the evaluation result of the emotional state.

[0573] Step 5:

[0574] The server generates personalized route and entertainment suggestions based on the emotional state and traffic congestion information provided by the terminal. Inputs include the user's emotional state, current traffic conditions, and destination information. Output is a set of recommended routes and entertainment options. Specifically, the server selects the most suitable music or audio content to help the user relax.

[0575] Step 6:

[0576] The terminal provides the user with optimal route information and entertainment suggestions received from the server. The input is recommendation data from the server. Specifically, the terminal's display and audio output device present route guidance and entertainment to the user. The output is the user's reaction and feedback.

[0577] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0578] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0579] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0580] [Fourth Embodiment]

[0581] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0582] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0583] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0584] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0585] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0586] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0587] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0588] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0589] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0590] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0591] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0592] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0593] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0594] This invention is a system for reducing urban traffic congestion and providing users with efficient travel. The system primarily consists of three elements: a server, terminals, and users.

[0595] The server collects real-time traffic data through various sensors and communication infrastructure placed throughout the city. Data classifications include traffic volume, speed, accident information, and construction information. Based on this data, the server uses AI algorithms to analyze it and predict traffic congestion. To enhance user convenience, trend analysis based on historical data is also performed to improve the accuracy of future traffic predictions.

[0596] The user sends travel requests via their device, including their departure and destination points and departure time. The device then sends this information to the server, which initiates the calculation of the optimal route based on the user's specified conditions. The server generates routes combining various modes of transport, which can be customized to the user's preferences.

[0597] When traffic conditions change, the server detects the changes in real time and, if necessary, calculates a new route and sends it to the terminal. This allows users to travel based on the latest traffic information. In addition, the terminal uses natural language processing to generate responses to user questions in order to enable interaction with the user. For example, if a user asks, "What are the current traffic conditions like?", the terminal will quickly answer based on the latest information analyzed by the server.

[0598] As a concrete example, consider a user traveling to the city center by car during the morning rush hour. Since the usual route is likely to be congested, the server, based on traffic data, presents an alternative route combining train travel and walking. Even if a train delay occurs along the way, the server can quickly analyze the impact and provide the user with a new alternative. With these features, the user can reach their destination with minimal time and stress.

[0599] The following describes the processing flow.

[0600] Step 1:

[0601] The server collects real-time traffic data through sensors and communication infrastructure installed within the city. This data, encompassing information on traffic volume, speed, accidents, and construction, forms the core of this invention.

[0602] Step 2:

[0603] The server analyzes the collected data using AI algorithms to understand the current traffic congestion situation and predict future congestion based on trends derived from past data. The accuracy of this data analysis significantly impacts the effectiveness of the system.

[0604] Step 3:

[0605] The user uses their device to input information about their travel, such as their current location, destination, and departure time, and sends a request to the device. This request is the starting point for achieving maximum performance.

[0606] Step 4:

[0607] The terminal immediately sends the user's request to the server. The server calculates the optimal route based on the received request. Here, route selection is performed based on the user's needs.

[0608] Step 5:

[0609] When calculating optimized routes, the server considers combinations of multiple modes of transport and suggests the most efficient way for the user to travel. If necessary, it also uses real-time traffic data for optimization.

[0610] Step 6:

[0611] The server sends the calculated optimal route to the terminal. The terminal receives this and provides the user with route information visually or audibly. The user then makes a travel decision based on this information.

[0612] Step 7:

[0613] If traffic conditions change, the server continuously analyzes the new data and updates the route as needed. This updated information is immediately sent to the terminal and reflected in the user's travel plan.

[0614] Step 8:

[0615] When a user sends a question in natural language through their device, the device forwards this request to the server. The server uses natural language processing to analyze the question, generates appropriate information, and immediately sends it back to the device. This two-way interaction further enhances user convenience.

[0616] (Example 1)

[0617] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0618] Traffic congestion in urban areas causes various problems, including increased travel time and higher energy consumption. Conventional traffic information systems have limitations in real-time analysis and flexible route suggestion, making it difficult to provide users with the optimal travel route. Therefore, there is a need for more accurate traffic information analysis and flexible route suggestions tailored to users.

[0619] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0620] In this invention, the server includes a device for aggregating traffic information, a device for analyzing the traffic information and predicting traffic congestion, and a device for constructing an optimal route based on user requests. This makes it possible to flexibly provide users with routes suitable for them based on real-time traffic information.

[0621] "Traffic information" refers to data related to roads and transportation systems, including vehicle flow, speed, accidents, and construction information.

[0622] "Congestion" refers to a state where traffic flow is slower or more stagnant than usual, and it is an indicator used for prediction.

[0623] A "user" refers to an individual who uses this system to obtain route information and select a mode of transportation.

[0624] A "route" is the recommended path for travel from a starting point to a destination, and can involve combining multiple modes of transportation.

[0625] A "device" refers to a collection of hardware and software designed to perform a specific function.

[0626] A "server" refers to a computer system that processes and analyzes data and plays a central role in the system.

[0627] "Real-time" means that data collection and processing are performed immediately, enabling the provision of information that responds promptly to the current situation.

[0628] "Natural language" refers to the forms of language that humans use on a daily basis, and it is used in communication with computers.

[0629] In this embodiment of the invention, a system is constructed to alleviate urban traffic congestion and provide users with an efficient means of transportation. The system consists of three main elements: a server, a terminal, and a user.

[0630] The server uses a high-performance computer to collect real-time traffic information from traffic sensors and related devices installed throughout the city. This data includes traffic volume, speed, accident locations, and road construction information. The server analyzes this data using AI algorithms to evaluate current traffic conditions and predict future congestion. Specifically, it uses machine learning libraries such as TensorFlow and Python programs to extract significant patterns from large amounts of data.

[0631] Users input information such as their departure point, destination, and departure time using a device such as a smartphone or tablet. This information is sent from the device to the server, which calculates the most efficient travel route. Map information services such as the Google Maps API are used to generate the travel route, and the optimal combination of possible modes of transport, including vehicles, public transport, and walking, is suggested. The server updates the route in real time in response to requests, providing users with the latest information.

[0632] If traffic conditions suddenly change, the server immediately detects the situation and recalculates the route as needed. The terminal has the function to receive the new route information sent from the server and notify the user. This allows the user to respond quickly to changing traffic conditions.

[0633] By using natural language processing technology, the device can easily respond to user questions. For example, if a user asks, "What's the current traffic situation like?", an answer based on the latest information analyzed by the server will be provided. Generative AI models such as ChatGPT are used for this purpose.

[0634] As a concrete example, consider a scenario where a user wants to commute by car on a weekday morning to avoid crowds. This system can suggest a route combining train and walking, rather than the usual route. If the train is delayed along the way, the server can quickly generate an alternative route. This allows the user to reach their destination with less stress.

[0635] An example of a prompt message is: "The user is driving towards the city center. Based on current traffic information, suggest the best route to avoid congestion."

[0636] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0637] Step 1:

[0638] The server collects traffic information from traffic sensors placed throughout the city. Specifically, it receives data transmitted from the sensors via the communication infrastructure and records it in a database in the form of traffic volume, speed, accident information, etc. The input to this process is raw data from the sensors, and the output is structured traffic information data.

[0639] Step 2:

[0640] The server analyzes the collected traffic information using an AI algorithm. During the data analysis process, it evaluates the current traffic situation using machine learning, referencing past traffic patterns. The input is the structured data obtained in step 1, and the output is the traffic congestion level prediction provided by the congestion prediction model.

[0641] Step 3:

[0642] The user enters the departure point, destination, and departure time via a terminal. The terminal sends this information to the server. The input is the travel conditions specified by the user, and the output is communication data sent to the server that receives this information and processes it in the next step.

[0643] Step 4:

[0644] The server calculates the optimal route using an AI algorithm, taking into account the user's input conditions. Based on the latest traffic forecast data, the server generates routes combining multiple modes of transport. The input consists of the user's travel conditions and analyzed traffic data, while the output is optimized route information.

[0645] Step 5:

[0646] The terminal receives optimal route information sent from the server and displays it to the user. Here, the terminal plays the role of visualizing the optimal route in an easy-to-understand way and presenting it to the user. The input is route information from the server, and the output is visualized navigation information provided to the user.

[0647] Step 6:

[0648] The server monitors changes in traffic conditions in real time and recalculates routes as needed. For example, when new information such as accidents or construction comes in, the server immediately processes that information and updates the routes for affected users. The input is the changed traffic conditions, and the output is the updated route information.

[0649] Step 7:

[0650] The device notifies the user of updated route information. This includes on-screen pop-ups and audio notifications. The input is the updated route information, and the output is an alert to the user and a presentation of the new route information.

[0651] Step 8:

[0652] The device uses generative AI models and natural language processing to respond to user questions. Based on the latest information received from the server, the device generates answers to questions such as "What is the current traffic situation?". The input is a natural language question from the user, and the output is an accurate response to that question.

[0653] (Application Example 1)

[0654] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0655] In modern cities, traffic congestion is a daily problem, and efficient travel is particularly difficult during rush hour. Furthermore, users are often stressed by the need to quickly adapt to changes in traffic conditions while traveling. This invention aims to solve these problems and provide users with a comfortable and efficient means of transportation.

[0656] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0657] In this invention, the server includes means for collecting movement information within a city, means for analyzing the movement information and predicting congestion levels, and means for displaying the prediction results to the user using a smartphone and suggesting a travel route. This allows the user to select the optimal route based on the latest movement information, enabling comfortable and efficient travel.

[0658] "Urban mobility information" refers to data on the dynamics and conditions of vehicles and pedestrians moving within an urban environment.

[0659] "Methods for predicting congestion" refer to methods that analyze data obtained from travel information to estimate current or future traffic density, speed, and other conditions.

[0660] "Means for generating the optimal route based on user requests" refers to a process that calculates an efficient travel route based on conditions such as the origin and destination specified by the user.

[0661] "Means of providing the generated route to the user" refers to a method of displaying or notifying the user of the calculated optimal travel route on their terminal.

[0662] "Means of updating routes in response to changes in travel conditions" refers to a process that recalculates and modifies routes based on real-time changing travel information so that users can reach their destinations more efficiently.

[0663] "A means of analyzing natural language questions from users and generating responses" refers to a method of understanding natural language questions entered by users into a terminal and creating appropriate answers to those questions.

[0664] "A method for displaying prediction results to users using smartphones and suggesting travel routes" refers to a method of displaying analyzed travel information and prediction results on a smartphone screen and guiding users to the most suitable travel route.

[0665] The system implementing this invention provides a platform for the integrated management of urban mobility information. A server collects mobility data from various sensors placed throughout the city, absorbing information such as traffic flow, speed, and traffic density. This data is integrated into the server in real time and stored in a database.

[0666] The software on the server is built using programming languages ​​such as Python and Java. Sensor data is first cleaned, and then congestion levels are predicted using AI models such as TensorFlow. The prediction results are then processed to suggest the optimal route based on the user's travel request.

[0667] Devices, especially smartphones, utilize the Google Maps API to provide users with an intuitive interface. Users enter their starting point and destination, and receive an optimized route from the server. Furthermore, the server updates the route to the smartphone in real time in response to changes in traffic conditions.

[0668] Furthermore, the device utilizes a generative AI model to enable natural language interaction. Specifically, when a user enters a prompt such as "Please tell me the shortest route to my destination," the server quickly generates an answer based on the analysis results.

[0669] When users travel to the city center during peak hours, the server predicts traffic conditions and suggests appropriate modes of transport or alternative routes, thereby reducing travel time and stress. This allows users to experience comfortable and efficient travel.

[0670] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0671] Step 1:

[0672] The server collects movement data in real time from sensors placed throughout the city. It receives raw movement data transmitted from the sensors (vehicle position, speed, traffic density, etc.) as input. It then performs data cleaning to correct for noise and missing data, outputting formatted movement information.

[0673] Step 2:

[0674] The server inputs formatted travel information into an AI model using TensorFlow to predict congestion levels. It utilizes historical data and current travel information as input. The prediction algorithm calculates future traffic density and the distribution of congestion points, and outputs the prediction results.

[0675] Step 3:

[0676] The user uses a terminal to specify the origin and destination and sends a request. This information is entered into the server and used as parameters to calculate the optimal route. Based on the prediction results and real-time travel information, the server calculates the optimal route and outputs the candidate route.

[0677] Step 4:

[0678] The device displays optimal route information sent from the server on a map using the Google Maps API. It receives route information from the server as input, visualizes it, and provides the user with intuitive route guidance.

[0679] Step 5:

[0680] The server monitors traffic conditions in real time and immediately recalculates routes if they are affected. When a change in data is detected, the AI ​​model is re-executed and outputs the corrected route. The terminal then notifies the user of this new route information.

[0681] Step 6:

[0682] The user inputs a natural language prompt, such as "Please tell me the shortest route to my destination," through the device. The device uses a generative AI model to analyze this prompt and generate and output a response based on data from the server. This response is then displayed to the user on the device.

[0683] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0684] This invention is an advanced mobility assistance technology that combines a system that analyzes urban traffic information and provides users with the optimal travel route with an emotion engine that recognizes the user's emotions. This system mainly consists of a server, terminals, and an emotion engine, and realizes personalized services tailored to the user's emotional state.

[0685] The server is equipped with conventional traffic data collection methods and acquires traffic data in real time from various sensors and traffic systems. In addition, it uses AI algorithms to predict traffic conditions based on past traffic data and constructs useful travel routes for the user at specific times. Furthermore, an emotion engine analyzes the user's facial expressions, voice, and behavioral patterns through sensors and cameras built into the terminal to recognize the user's emotional state.

[0686] Users operate their devices and receive route suggestions based on real-time traffic data. A new feature is added that uses the emotion engine's analysis to suggest routes tailored to the user's stress and fatigue levels. For example, a user experiencing stress might be offered less congested routes or alternative routes with scenic views. The emotion engine can also suggest music and relaxation messages to the user, providing a more comfortable travel experience.

[0687] For example, if a user wants to avoid congested roads during their morning commute, the server calculates multiple routes to their destination based on traffic data. Meanwhile, the emotion engine monitors the user's voice and camera footage to assess their motivation and stress levels. It then selects a route that offers a relatively relaxing environment and presents it through the device. This allows the user to have a more comfortable and less stressful commute.

[0688] In summary, this system achieves flexible route suggestions that take into account both user emotions and traffic conditions, thereby improving the quality of the user experience during travel.

[0689] The following describes the processing flow.

[0690] Step 1:

[0691] The server collects real-time traffic data from various sensors and digital infrastructure within the city. This data includes road traffic volume, average speed, accident information, and construction status.

[0692] Step 2:

[0693] The server analyzes collected traffic data using AI algorithms to identify current traffic congestion and predict future congestion based on past data. This analysis is a process that generates information that forms the basis for efficient route suggestions.

[0694] Step 3:

[0695] The emotion engine uses the device's built-in camera and microphone to analyze the user's facial expressions, voice tone, and other behavioral indicators to determine the user's current emotional state. This is done to recognize signs of stress and fatigue and plan countermeasures.

[0696] Step 4:

[0697] The user enters their destination and departure time into the terminal and sends a request to the server. This information is crucial for the server to calculate the optimal route for the user.

[0698] Step 5:

[0699] The server comprehensively evaluates user requests, traffic data, and the results of the emotion engine's analysis to generate the optimal route based on the user's emotional state. For example, if the emotion engine detects a high stress level, the server prioritizes calculating routes that avoid congestion and options that allow for relaxation.

[0700] Step 6:

[0701] The server sends the generated route to the terminal, and the terminal provides the route information to the user. Visual or auditory feedback allows the user to easily understand and select the proposed route.

[0702] Step 7:

[0703] If there are changes in traffic conditions or the user's emotional state, the server immediately updates the data and recalculates the route if necessary. This new information is quickly sent to the terminal, and the user is provided with refreshed suggestions.

[0704] Step 8:

[0705] When a user makes an individual question or request in natural language through their device, the device forwards it to the server. The server uses natural language processing to analyze the question, generate appropriate actions or information, and sends them back to the device. This response function allows users to communicate effectively with the system and enjoy greater convenience.

[0706] (Example 2)

[0707] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0708] The challenges people face during their daily commutes in urban environments include stress caused by congested traffic and increased travel times. Furthermore, there is a lack of services that consider the influence of emotional states on travel choices. In this context, there is a need for a system that can provide more comfortable and less stressful travel routes based on the user's emotional state.

[0709] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0710] In this invention, the server includes a device for collecting traffic data, a device for analyzing the traffic data to predict congestion levels, and a device for collecting emotional information to analyze the user's emotional state. This enables the integrated consideration of the user's emotional state and traffic conditions to generate optimal travel routes and provide personalized content.

[0711] "Traffic data" refers to a collection of information including the flow of vehicles within a city, the operating status of public transportation, the status of traffic signals, and related information.

[0712] "Congestion status" refers to the degree of traffic congestion or delays in a particular time period or area.

[0713] "Emotional information" refers to data that indicates a subjective psychological state, obtained from the user's facial expressions, voice, behavioral patterns, etc.

[0714] "Emotional state" refers to the user's psychological and emotional condition, such as stress, fatigue, and relaxation.

[0715] The "optimal route" is the route selected to enable the most comfortable and efficient travel, taking into account the user's emotional state and current traffic conditions.

[0716] "Personalized content" refers to music, messages, or other relaxation information provided in response to a user's individual emotional state.

[0717] This invention is a system that comprehensively analyzes urban traffic information and user sentiment information to provide optimal travel routes and personalized content. The main components of the system are a server, a terminal, and an sentiment engine.

[0718] The server functions as a device for collecting traffic information. It collects real-time traffic data from sensors and traffic management systems installed throughout the city. This data includes traffic volume, congestion levels, and public transport operation information. The server uses AI algorithms to analyze the collected data and predict congestion levels by comparing them with past traffic patterns. This prediction forms the basis for designing optimal travel routes for users.

[0719] The device is designed to acquire user emotional information. Using built-in sensors and a camera, it analyzes the user's facial expressions, voice tone, and behavioral patterns in real time. This analysis is processed by an emotion engine to identify the user's emotional state. This information reflects the user's stress level and fatigue level, and is used to suggest the optimal travel route.

[0720] Users receive suggested routes and content through their devices. For example, if a user is feeling stressed, the server predicts and provides routes that avoid congestion or offer scenic views. In addition, the device suggests relaxing music and messages to the user, making the travel experience more comfortable.

[0721] For example, if a user is feeling stressed during their morning commute, the server calculates a less congested route based on past traffic data. Simultaneously, the terminal senses changes in the user's facial expressions and voice, and provides a route that is effective in reducing stress. Specifically, this might involve selecting a detour route that offers more scenic views or playing relaxation music. Through this entire process, users can reduce stress during their commute and enjoy a more comfortable journey.

[0722] An example of a prompt message is the instruction, "Use the emotion engine to generate route suggestions that take the user's stress level into account." This enables the provision of services tailored to specific situations.

[0723] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0724] Step 1:

[0725] The server collects real-time traffic data from sensors and traffic management systems installed throughout the city. Inputs include traffic volume, signal status, and public transport operation status. This data is fed into an AI algorithm to organize and analyze traffic flow. The output is the analyzed traffic pattern. Specifically, the server accesses a database to obtain current traffic conditions and predicts congestion.

[0726] Step 2:

[0727] The server analyzes collected traffic data using AI algorithms and predicts congestion levels by comparing them with historical data. The input consists of organized and analyzed traffic patterns and historical traffic data, while the output is a prediction of future congestion. Specifically, the server uses statistical models to simulate future traffic conditions and generate useful information for users.

[0728] Step 3:

[0729] The device uses built-in sensors and cameras to acquire the user's facial expressions, voice, and behavioral patterns in real time. The input is emotional data obtained from the user. This data is analyzed by an emotion engine to identify the emotional state. The output is emotional state such as stress level and fatigue level. Specifically, the device uses voice analysis software and facial recognition technology to determine the user's psychological state.

[0730] Step 4:

[0731] The server integrates traffic forecast data and user sentiment data to generate the optimal travel route. The inputs are congestion forecasts and sentiment data, and the output is the recommended route. Specifically, the server uses a prioritization algorithm to evaluate multiple route options and select the most appropriate route for the user.

[0732] Step 5:

[0733] The user receives suggested routes through the terminal. The input is the recommended route sent from the server. The terminal displays this and plays it back as voice guidance. The output is visual and auditory route guidance. Specifically, the terminal uses a map application to provide real-time directions and updates the route as needed.

[0734] (Application Example 2)

[0735] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0736] In modern urban environments, traffic congestion is a daily problem, and planning efficient travel routes is crucial for users. However, conventional route guidance systems have limited ability to adapt to changing traffic conditions and struggle to provide personalized services that take into account users' emotions and stress levels. As a result, users cannot enjoy a comfortable and stress-free travel experience. Therefore, there is a need to provide personalized route suggestions and in-vehicle environments that take into account the emotional state of users.

[0737] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0738] In this invention, the server includes means for acquiring traffic information, means for analyzing traffic information and predicting traffic congestion, means for generating an optimal route based on user requests, means for analyzing the user's emotional state using an emotion recognition device, means for optimizing the route or service based on the user's emotional state, and means for providing in-vehicle entertainment. This makes it possible to provide flexible route suggestions adapted to traffic conditions while taking the user's emotional state into consideration, as well as a comfortable in-vehicle environment.

[0739] "Traffic information" refers to data about traffic flow and conditions within a city, collected through sensors and external databases.

[0740] "Traffic congestion" refers to the state of the road, indicating the density and flow of vehicles, and is information that is subject to prediction and analysis.

[0741] "User" refers to a person who uses the system to receive services such as travel and route guidance.

[0742] The "optimal route" refers to the most efficient travel route for the user, calculated considering the time and distance required to reach the destination.

[0743] An "emotion recognition device" is a device that analyzes a user's facial expressions and voice to determine their mental and emotional state.

[0744] A "personalized service" is a service that provides information and support tailored to the individual circumstances and needs of the user.

[0745] "Entertainment" refers to music, videos, or other content designed to provide relaxation and entertainment to travelers.

[0746] "Adapting to traffic conditions" refers to a dynamic process that responds to real-time, fluctuating traffic conditions and adjusts routes and services as needed.

[0747] The system in this invention mainly consists of a server, a terminal, and an emotion recognition device. The server collects and analyzes traffic information in real time from traffic sensors and an external database. This allows for the prediction of traffic congestion and the generation of efficient routes. This traffic information is processed using advanced algorithms executed on the server. Specifically, programming languages ​​such as Python and R are used, and machine learning algorithms are utilized in particular to predict traffic flow.

[0748] The terminal functions as the user's mobile device or as an information system within the vehicle, communicating with a server to provide the latest route information. The terminal is equipped with emotion recognition devices such as a camera and microphone, which are used to analyze the user's facial expressions and voice. Libraries such as OpenCV and TensorFlow are used for emotion recognition, identifying emotional states from facial expressions and voice tone.

[0749] Users receive real-time route guidance and traffic information through their devices, supporting their decision-making while traveling. The system provides personalized route suggestions based on the user's stress level and emotional state, as well as in-car entertainment. For example, if a user wants to relax, the system automatically selects and plays soothing music and suggests alternative routes that allow them to enjoy the scenery.

[0750] As an example of a prompt, if the user is relaxed, generate an action that selects the most relaxing music and guides them along a scenic route. By utilizing a generative AI model, it is possible to provide optimal services tailored to the user.

[0751] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0752] Step 1:

[0753] The server acquires real-time traffic information from traffic sensors and databases. Inputs include data from various traffic sensors and external traffic information feeds. This data is analyzed to understand the current traffic congestion situation and converted into a digital format. Outputs include traffic congestion maps and forecast data.

[0754] Step 2:

[0755] The server uses a prediction algorithm to forecast traffic congestion for the next few hours. This process uses historical and real-time traffic data as input. The server runs a machine learning model to predict peak congestion times and delay factors, generating a congestion prediction model. The output is data on predicted times and congestion patterns.

[0756] Step 3:

[0757] The terminal calculates the optimal route to the user's destination based on traffic congestion data received from the server. Inputs include the current location, destination, traffic congestion data, and user preferences. The terminal integrates this information to generate multiple route options. The output is an optimized route list.

[0758] Step 4:

[0759] The device uses a camera and microphone to analyze the user's facial expressions and voice, and evaluate their emotional state. Inputs include camera video and audio data. An emotion recognition model such as TensorFlow analyzes these to identify the user's emotional state (e.g., stress, relaxation). The output is the evaluation result of the emotional state.

[0760] Step 5:

[0761] The server generates personalized route and entertainment suggestions based on the emotional state and traffic congestion information provided by the terminal. Inputs include the user's emotional state, current traffic conditions, and destination information. Output is a set of recommended routes and entertainment options. Specifically, the server selects the most suitable music or audio content to help the user relax.

[0762] Step 6:

[0763] The terminal provides the user with optimal route information and entertainment suggestions received from the server. The input is recommendation data from the server. Specifically, the terminal's display and audio output device present route guidance and entertainment to the user. The output is the user's reaction and feedback.

[0764] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0765] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0766] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0767] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0768] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0769] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0770] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0771] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0772] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0773] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0774] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0775] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0776] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0777] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0778] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0779] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using this memory.

[0780] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0781] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0782] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0783] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0784] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0785] The following is further disclosed regarding the embodiments described above.

[0786] (Claim 1)

[0787] Means of collecting urban traffic data,

[0788] A means for analyzing the aforementioned traffic data and predicting traffic congestion,

[0789] A means for generating the optimal route based on the user's request,

[0790] Means for providing the generated route to the user,

[0791] A means of updating routes in response to changes in traffic conditions,

[0792] A means of analyzing natural language questions from users and generating responses,

[0793] A system that includes this.

[0794] (Claim 2)

[0795] The system according to claim 1, further comprising means for predicting traffic conditions using past traffic data.

[0796] (Claim 3)

[0797] The system according to claim 1, comprising means for integrating multiple modes of transport to propose an optimal route.

[0798] "Example 1"

[0799] (Claim 1)

[0800] A device for aggregating traffic information,

[0801] A device that analyzes the aforementioned traffic information and predicts the state of traffic congestion,

[0802] A device that constructs the optimal route based on user requests,

[0803] A device that supplies the constructed route to the user,

[0804] A device that updates the route in response to changes in traffic conditions,

[0805] A device that analyzes natural language inquiries from users and generates responses,

[0806] A device that reconstructs traffic information in real time and notifies users,

[0807] A system that includes this.

[0808] (Claim 2)

[0809] The system according to claim 1, further comprising a device for predicting traffic conditions using past traffic information.

[0810] (Claim 3)

[0811] The system according to claim 1, comprising a device that integrates multiple means of transportation to propose an optimal route.

[0812] "Application Example 1"

[0813] (Claim 1)

[0814] Means of collecting urban mobility information,

[0815] A means for analyzing the aforementioned movement information and predicting congestion levels,

[0816] A means for generating the optimal route based on the user's request,

[0817] Means for providing the generated route to the user,

[0818] A means of updating the route in response to changes in movement conditions,

[0819] A means for analyzing natural language questions from users and generating responses,

[0820] A method for displaying prediction results to users using smartphones and suggesting travel routes,

[0821] A system that includes this.

[0822] (Claim 2)

[0823] The system according to claim 1, further comprising means for predicting congestion levels using past travel information.

[0824] (Claim 3)

[0825] The system according to claim 1, comprising means for integrating multiple means of transportation to propose an optimal route.

[0826] "Example 2 of combining an emotion engine"

[0827] (Claim 1)

[0828] A device for collecting traffic data,

[0829] A device that analyzes the aforementioned traffic data to predict congestion levels,

[0830] A device that collects emotional information and analyzes the emotional state of users,

[0831] A device that generates an optimal route based on the aforementioned traffic conditions and emotional state,

[0832] A device that provides the generated route to the user,

[0833] A device that updates the route in response to changes in traffic conditions,

[0834] A device that provides personalized content according to emotional state,

[0835] A device that analyzes questions in natural language and generates responses,

[0836] A system that includes this.

[0837] (Claim 2)

[0838] The system according to claim 1, further comprising a device for predicting traffic conditions using past traffic data.

[0839] (Claim 3)

[0840] The system according to claim 1, comprising a device that integrates multiple modes of transportation to propose an optimal route.

[0841] "Application example 2 when combining with an emotional engine"

[0842] (Claim 1)

[0843] Means of obtaining traffic information,

[0844] A means for analyzing the aforementioned traffic information and predicting traffic congestion,

[0845] A means for generating the optimal route based on user requests,

[0846] Means for providing the generated route to the user,

[0847] A means of updating routes in response to changes in traffic conditions,

[0848] A means of analyzing the emotional state of a user using an emotion recognition device,

[0849] Means for optimizing a route or service based on the emotional state of the user,

[0850] Means for providing in-vehicle entertainment,

[0851] A system that includes this.

[0852] (Claim 2)

[0853] The system according to claim 1, further comprising means for predicting traffic conditions using past traffic information.

[0854] (Claim 3)

[0855] The system according to claim 1, comprising means for integrating multiple modes of transport to propose an optimal route. [Explanation of Symbols]

[0856] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of collecting urban traffic data, A means for analyzing the aforementioned traffic data and predicting traffic congestion, A means for generating the optimal route based on the user's request, Means for providing the generated route to the user, A means of updating routes in response to changes in traffic conditions, A means of analyzing natural language questions from users and generating responses, A system that includes this.

2. The system according to claim 1, further comprising means for predicting traffic conditions using past traffic data.

3. The system according to claim 1, comprising means for integrating multiple modes of transportation to propose an optimal route.

Images